ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-29 17:28:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
7,024 Commits 185 Branches 90 Tags 89 MiB
0abcbe3339
Commit Graph

4130 Commits

Author SHA1 Message Date
Daira Hopwood
12773ea034 Increase timeout for test_alice_bob to 5 minutes. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
504e0d02f3 refactor to use context managers 2016-07-21 12:35:59 -07:00
meejah
ec37d52942 more 2016-07-21 12:35:59 -07:00
meejah
d41477bd5f fix/change some assertions 2016-07-21 12:35:59 -07:00
meejah
575c00113a fix typo for .conflict files 2016-07-21 12:35:59 -07:00
Daira Hopwood
c978db4c90 more fixes 2016-07-21 12:35:59 -07:00
meejah
c087a87b4a fixes to test_alice_bob 2016-07-21 12:35:59 -07:00
Daira Hopwood
5c01d4781f Temporarily increase delay to make sure we receive notifications. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
874b4cf343 Debug prints in Windows inotify emulation. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
e845f7ff65 Filter out FILE_ACTION_MODIFIED for directories. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
23f4b9dbfd Repair tests after changing the semantics of objects_succeeded. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
8a83cff14d Make _process return a boolean to indicate whether processing succeeded, 
and only increment objects_succeeded if that is the case.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
0abe5baaf2 WIP: temporarily work around the AccessDenied errors after a test failure. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
afc7ab43a1 Fix a subtle typo. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
b949082f8e Change type of mtime and ctime stored in magicfolderdb to integer nanoseconds. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
meejah
7c85de813f (some) fixes for smoke_magicfolder 2016-07-21 12:35:59 -07:00
Daira Hopwood
237f4719ee Remove the flush_volume call in RealTest as well as in RealTestAliceBob. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
b05b701532 Try to avoid spurious failures when shutting down the INotify loop on Windows. 
This only partially solves the problem.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
23a1f311af Fix pyflakes errors, add a comment about a suspicious test. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
a78e9c750a WIP: add close method for MagicFolderDB and make MagicFolderDbTests use it. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
meejah
a50d0e5c8b WIP and debugging things 
Conflicts:
	src/allmydata/test/test_magic_folder.py
 2016-07-21 12:35:59 -07:00
meejah
58f9641179 Correct magic-folder use of StorageFarmBroker 
When landing the "connected enough" changes on master,
we changed the API; this fixes up usage of it in magic-folder.
 2016-07-21 12:35:59 -07:00
Daira Hopwood
e67d23b8fa Fix negative counts for object_queued. fixes ticket:2731 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
meejah
625c5a42cd daira points out bug 2016-07-21 12:35:59 -07:00
meejah
5f073c6987 more startup waiting 2016-07-21 12:35:59 -07:00
meejah
faed2574d6 possible less race-y startup 2016-07-21 12:35:59 -07:00
meejah
378ce27cbd Make retry_download test work, and some bonus comments 2016-07-21 12:35:59 -07:00
Daira Hopwood
9e5b01b3ae Swallow the error from Downloader._when_queue_is_empty. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
811d597cd8 Add WIP test_download_retry. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
meejah
e6104cd1a2 Replace _lazy_tail with iterative loop, cleanup tests (#2412) 
* uses @inlineCallbacks to turn the _lazy_tail recursion into
   a "real" looking loop;
 * remove the need for "immediate" vs delayed iteration of said loop;
 * make it easier for the unit-tests to control the behavior of the
   uploader/downloader;
 * consolidates (some) setup/teardown code into the setUp and tearDown
   hooks provided by unittest so unit-tests aren't doing that themselves
 * re-factors some of the unit-tests to use an @inlineCallbacks style
   so they're easier to follow and debug

This doesn't tackle the "how to know when our inotify events have arrived"
problem the unit-tests still have, nor does it eliminate the myriad bits
of state that get added to tests via all the MixIns.
 2016-07-21 12:35:59 -07:00
Daira Hopwood
e03e243c67 Expand test for Magic Folder statistics. 2016-07-21 12:35:59 -07:00
Daira Hopwood
37d22efaea Improve test for Magic Folder statistics and move it from test_system.py to test_magic_folder.py. 2016-07-21 12:35:59 -07:00
Daira Hopwood
dd0597f869 Work in progress to fix incorrect statistics output for Magic Folder. refs ticket:2709 2016-07-21 12:35:59 -07:00
Daira Hopwood
e3dfc8f3f4 Fix test_magic_folder.py to use absolute unicode paths when calling write_downloaded_file. 2016-07-21 12:35:59 -07:00
David Stainton
d2ae4255c8 Add daira's implementation of make_dirs_with_absolute_mode 2016-07-21 12:35:59 -07:00
David Stainton
2963e9e50c Use os.path.dirname instead of split and rename var to initial_path_u 2016-07-21 12:35:59 -07:00
David Stainton
cd390b90e4 remove superfluous trailing comma from make_dirs_with_absolute_mode def 2016-07-21 12:35:59 -07:00
David Stainton
ba35d7262b Add unit test and make corrections to make_dirs_with_absolute_mode 2016-07-21 12:35:59 -07:00
David Stainton
ee44732d03 Break out our chmod while loop into fielutils.py 2016-07-21 12:35:59 -07:00
David Stainton
51ccec7a33 Use chmod instead of changing umask 
Conflicts:
	src/allmydata/frontends/magic_folder.py
 2016-07-21 12:35:59 -07:00
David Stainton
3d909682a4 Add comment about FUDGE_SECONDS and refer to our design doc 2016-07-21 12:35:59 -07:00
Daira Hopwood
807cfbf0dc Fix pyflakes warnings. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
meejah
9cfcb1b3aa remove humanize, use internal method, teach internal method to understand timedelta 2016-07-21 12:35:59 -07:00
meejah
573a5481d1 add Bob leaving to smoke-tests 2016-07-21 12:35:59 -07:00
meejah
c217d1f8f3 fix return-valid of leave 2016-07-21 12:35:59 -07:00
meejah
bf3bf70814 add docstrings 2016-07-21 12:35:59 -07:00
meejah
8ba2912ff4 make history longer 2016-07-21 12:35:59 -07:00
meejah
7e84b0cf57 flesh out IQueuedItem 2016-07-21 12:35:59 -07:00
meejah
05ec62c4b1 humanize doesn't report a proper version 2016-07-21 12:35:59 -07:00
meejah
5ed24913d2 add humanize dependency 2016-07-21 12:35:59 -07:00
meejah
658470ba6e Add some magicfolder database tests 2016-07-21 12:35:59 -07:00
meejah
2573cf18ef Add simple auth-token to get JSON data 2016-07-21 12:35:59 -07:00
meejah
86abe56d91 Flesh out "tahoe magic-folder status" command 
Adds:

 - a JSON endpoint
 - CLI to display information
 - QueuedItem + IQueuedItem for uploader/downloader
 - IProgress interface + PercentProgress implementation
 - progress= args to many upload/download APIs
 2016-07-21 12:35:59 -07:00
David Stainton
3df0a82a38 Add cli stub for magic-folder status command 2016-07-21 12:35:59 -07:00
David Stainton
af10ab43ee Daira's fix during session with David and Meejah. 2016-07-21 12:35:59 -07:00
David Stainton
71e3164e92 Further work in progress refinements to unit tests 
wip from pairing with Daira and Meejah
 2016-07-21 12:35:59 -07:00
David Stainton
9483d24f67 Fix cleanup after test_periodic_full_scan. 2016-07-21 12:35:59 -07:00
David Stainton
38dd49ee5a Add logging for Downloader.stop. 2016-07-21 12:35:59 -07:00
David Stainton
2dabd8a10b Work in progress on fixing test_periodic_full_scan. 2016-07-21 12:35:59 -07:00
David Stainton
b4153659d3 Minor cleanup and logging fix. 2016-07-21 12:35:59 -07:00
David Stainton
5380ff8f5a Clean up queue loging and add a clock advance in unit test 
not yet working
 2016-07-21 12:35:59 -07:00
David Stainton
e1822c0518 WIP 2016-07-21 12:35:59 -07:00
David Stainton
96b846f762 Add ignore pending argument to _periodic_full_scan 2016-07-21 12:35:59 -07:00
David Stainton
cf462d8b1f Attempt to test periodic uploader full scan 2016-07-21 12:35:59 -07:00
David Stainton
a5163c32cc Only perform full scan if pending set is empty 2016-07-21 12:35:59 -07:00
David Stainton
e9e74c43e1 Naive periodic full scan 2016-07-21 12:35:59 -07:00
David Stainton
9b4b43cf19 Fix uploader's _process to extend queue after scan and count stats properly 2016-07-21 12:35:59 -07:00
David Stainton
d2971fbabe Daira's excellent fixes to the uploader from our pairing session 2016-07-21 12:35:59 -07:00
David Stainton
70f66677be Teach uploader's _scan to use the deque 2016-07-21 12:35:59 -07:00
David Stainton
6b32841b10 Remove UNIQUE constraint from magicfolder db schema 2016-07-21 12:35:59 -07:00
Daira Hopwood
9c9a4e7aa2 Add get_pathinfo. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
278762200d Teach magic-folder join to use configutil. 
Author: David Stainton <david@leastauthority.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
cc721505e7 Add magic-folder test_scan_once_on_startup 2016-07-21 12:35:59 -07:00
Daira Hopwood
cb6291e93c Various test cleanups. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
3074b687be Teach join twice test to test for proper stdout and stderr values 2016-07-21 12:35:59 -07:00
David Stainton
74ad656e39 More thorough checks to the join leave join test 2016-07-21 12:35:59 -07:00
David Stainton
966c556562 Fix test_join_leave_join 2016-07-21 12:35:59 -07:00
David Stainton
1c3c9df492 Add more magic-folder join leave tests 2016-07-21 12:35:59 -07:00
David Stainton
24aa74bf53 Add unit test for join twice failure 2016-07-21 12:35:59 -07:00
Daira Hopwood
2043bed3f1 Delay only the download scan not the turning of our event queue. 
Author: David Stainton <david@leastauthority.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
9d7785f57b Fix magic folder constructor signature in test 2016-07-21 12:35:59 -07:00
David Stainton
68539e3fe0 Fix tests usage of umask 2016-07-21 12:35:59 -07:00
David Stainton
0273034b0c Fix umask again 2016-07-21 12:35:59 -07:00
David Stainton
ae59b17e86 Fix umask usage bit fiddling and use try finally blocks 2016-07-21 12:35:59 -07:00
David Stainton
f2a7978b29 Add download.umask config option with default of 077 2016-07-21 12:35:59 -07:00
David Stainton
623ed3516e Add umask to Downloader 2016-07-21 12:35:59 -07:00
David Stainton
7b9c89c6f4 remove magic_folder from config in test 2016-07-21 12:35:59 -07:00
David Stainton
189b1b2dc2 Remove magic_folder subsection from default tahoe.cfg 2016-07-21 12:35:59 -07:00
David Stainton
79f3c5d81a Remove magic-folder exclude stat 2016-07-21 12:35:59 -07:00
Daira Hopwood
cd1c166790 Improve the error reporting for 'tahoe magic-folder join/leave'. refs #2568 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
b4b5590b00 teach leave to: remove magic_folder section of config 2016-07-21 12:35:59 -07:00
David Stainton
d6e68129ab Add simple magic-folder leave command 2016-07-21 12:35:59 -07:00
David Stainton
8ff7642fba Teach join to also check for existing magicfolder db file 2016-07-21 12:35:59 -07:00
David Stainton
aea2cf2890 Prevent magic-folder join if already joined. 2016-07-21 12:35:59 -07:00
Daira Hopwood
6b4d3b8c6d Refactor is_new_file. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
02dacbc067 Minor simplification. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
e990483750 Simply conflict detection by removing nested if statements 2016-07-21 12:35:59 -07:00
David Stainton
6ed4480eb9 Scan our own dmd upon start only 2016-07-21 12:35:59 -07:00
meejah
0aaabe0ebe remove print 2016-07-21 12:35:59 -07:00
meejah
ed4e4d053e Add a unit-test and correct the code for "already deleted" 
If a Downloader decides that it needs to delete a file, but that
file is already gone locally, the exeption is caugt and a log message
produced.
 2016-07-21 12:35:59 -07:00
David Stainton
49b5661347 Fix test: previously we accounted for the propagation of the conflict 
because alice scaned her own dmd... whereas now she does not.
 2016-07-21 12:35:59 -07:00
David Stainton
eab24d022b Fix test helper _check_version_in_local_db 2016-07-21 12:35:59 -07:00
Daira Hopwood
9d59ef493f WIP: Refactoring to get db fields in one query. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
bca4c87e47 WIP 2016-07-21 12:35:59 -07:00
Daira Hopwood
42f535edf9 Fix pending upload conflict detection. 
Author: David Stainton <david@leastauthority.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
6eeab03b70 Create test for last upload uri conflict 2016-07-21 12:35:59 -07:00
Daira Hopwood
e8b6b31373 Detect remote conflict by checking for pending upload. 
Author: David Stainton <david@leastauthority.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
c7f38a496d Add debugging print statement for timestamp comparison 2016-07-21 12:35:59 -07:00
David Stainton
edc671f577 Add last uploaded timestamp comparison for remote conflict detection 2016-07-21 12:35:59 -07:00
Daira Hopwood
6402159106 Fix an unused import. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
c7f298bce5 Fix some miscapture bugs. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
868c38d228 Workaround. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
b21a3dc070 Simplify _notify and improve logging. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
17384ac512 Simplify _scan_remote_* and remove Downloader._download_scan_batch attribute. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
d686575393 Cosmetics. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
4f41be0e70 Downloader doesn't need the pending set. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
017d63b47b Delete redundant is_ready attribute from MagicFolder. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
3fab5caa02 test_encodingutil: fixes for Unix. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
5f98d73216 Add precondition to Uploader._process. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
583066b364 Fix test_errors. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
9646a3ce3e Fix test_move_tree. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
0c12162bc8 Debugging WIP. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
b53efdbf8b Fix a corner case for to_filepath on Windows to make it consistent with Unix. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
b9eb8932b6 test_encodingutil: add tests for FilePath-related functions. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
b6b638efd1 Depend on FilePath.asTextMode(). 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
84af199d51 WIP: exclude own dirnode from scan. This is not quite right; we shouldn't exclude it on startup. refs #2553 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
ea14b1bebe Don't add subdirectory watches if the platform's notifier doesn't require them. refs #2559 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
6bb0963197 Improve reporting of assertion failures. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
a5b9cf36b8 Fix unused import. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
f77c593177 Add test for 'tahoe create-node/client/introducer' output. closes ticket:2556 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
d1eb64c01b bin\tahoe can't be run directly on Windows. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
3647ffc3a7 Correct type for Windows BOOL. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
9dafa2e759 Disable precondition that autoAdd == recursive. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
a6842946ca Fix a type error. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
0e19636146 Flush handling WIP. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
5e9792f8d7 Use fileutil.write for magic folder tests. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
f3803c058d Improve all of the Windows-specific error reporting. 
Also make the Windows function declarations more readable and consistent.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
c20cf843eb Fix fileutil tests. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
dacf6ef633 More path fixes. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
f12ac3d94e Fix a test broken by the last commit. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
a622109a87 Don't include [magic_folder]enabled and local.directory fields by default. 
Add a comment reminding to do the field modification properly.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
6fb7f86855 Don't use a long path for the [magic_folder]local.directory field. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
2a79252a72 Fix some path Unixisms. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
45adc30ed6 Fix test_alice_bob. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
e591c3aa81 Refactor _check_up/downloader_count. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
1abc2c39c8 Don't download the deletion marker file unnecessarily. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
c6191257ea Distinguish deletion of directories. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
d6bb49b1a6 Rename deleted files to .backup rather than unlinking them. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
a043a31860 Add file conflict unit test 2016-07-21 12:35:59 -07:00
David Stainton
99d1339d38 Add basic bob upload test and fix conflict detect 2016-07-21 12:35:59 -07:00
David Stainton
ee3d9b19be Fix bob's uploading test... 2016-07-21 12:35:59 -07:00
David Stainton
890e4622b9 Attempt to teach bob to upload a file 2016-07-21 12:35:59 -07:00
David Stainton
9ed61f1eb4 Count conflicted objects 2016-07-21 12:35:59 -07:00
Daira Hopwood
7a3b35e29c Basic remote conflict detection based on ancestor uri 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
f75dda7635 Avoid .format, since it is inconsistent between Python 2.6 and 2.7 (and the rest of Tahoe-LAFS doesn't use it). 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
5a2b2e61c7 Fix test_alice_bob. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
e4082a45d3 Add counter for uploader.objects_not_uploaded. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
95e2961a2e Advance Bob's clock after notifying. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
6e5e8ddd80 test_alice_bob: use magic= argument to notify, rather than self.magicfolder. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
meejah
0d466ec1f9 add excluded check 2016-07-21 12:35:59 -07:00
meejah
c883977eb6 add the 'spurious' notifies 2016-07-21 12:35:59 -07:00
Daira Hopwood
c156581b73 Fix a pyflakes warning and check existence of file in Bob's local dir. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
meejah
d61b8ed39e implement 'delete' functionality, with tests 2016-07-21 12:35:59 -07:00
meejah
946656b249 smoketest for magic-folder functionality 2016-07-21 12:35:59 -07:00
Daira Hopwood
cec315da49 Add test that we don't write files outside the magic folder directory. refs ticket:2506 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
567668eb8c Fix infinite loop in should_ignore_path for absolute paths. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
5d5d804719 More debug logging. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
57753a62bc Unicode fix for do_join. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
7d2f30643e Minor cleanups to tests. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
f754938cc9 Ensure that errors from Alice-and-Bob tests are reported correctly if setup fails. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
b56c3e06e4 Cosmetics. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
2a7bb6f444 Eliminate duplicate parsing of invite code. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
c28594268e Remaining test fixes. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
a5874070e3 Make sure that do_cli is only called with strs, and avoid unnecessary use of attributes in tests. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
ed2d27e712 Cosmetics. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
643a105991 URIs are strs. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
59eede8e88 Aliases and nicknames are Unicode. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
bb596b017d Fix call to argv_to_abspath. Also rename localdir to local_dir. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
b32b61e727 Fix magic-folder cli tests 
convert path to abs path when matching
strings in the generated config file.
 2016-07-21 12:35:59 -07:00
David Stainton
a1beef34a7 Attempt to fix cli tests 2016-07-21 12:35:59 -07:00
Daira Hopwood
b2d494837c Better but still broken tests. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
903c6f1d6c Fix check for initial '-' in argv_to_abspath. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
0789d295a6 Fix tests by submitting unicode args instead of str 2016-07-21 12:35:59 -07:00
David Stainton
16275cb8f6 Teach magic-folder join to use argv_to_abspath 
- also we modify argv_to_abspath to through a usage error
if the name starts with a '-'

- add a test
currently the tests fail
 2016-07-21 12:35:59 -07:00
David Stainton
941f67f3b0 Use argv_to_abspath for magic-folder join file path arg 2016-07-21 12:35:59 -07:00
Daira Hopwood
5f6783e382 Test creation of a subdirectory. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
8184ed4414 Watch for IN_CREATE events but filter them out for non-directories. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
ee5245ed2c Patch Downloader.REMOTE_SCAN_INTERVAL rather than setting it persistently. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
cbec939d86 Implement creating local directories in downloader. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
60ab5a057a Decode names in the scanned remote. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
a3fa1617c2 Refactoring to allow logging from _write_downloaded_file and _rename_conflicted_file. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
6cb41cadf5 Simplify and fix non-existent-file handling. 
Also make the existent and non-existent cases as similar as possible,
with a view to merging them.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
cb905d8d97 Logging/debugging improvements. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
fc3513d172 Refactor and fix race conditions in test_alice_bob. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
ac5730ea1c Correct a call to did_upload_version in the downloader. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
8a6dae6d59 Make sure that test_move_tree waits until files have been uploaded as well as downloaded. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
28587b5b2a Restore a call to increment files_uploaded that was mistakenly removed. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
2fcaa8c2b2 Teach uploader+downloader to use to db schema 
here we attempt to fix all the unit tests as well...
however two tests still fail
 2016-07-21 12:35:59 -07:00
Daira Hopwood
5754c01cb5 Add magicfolderdb.py. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
David Stainton
615859acc1 Remove magic-folder db code from backupdb.py 2016-07-21 12:35:59 -07:00
David Stainton
e2cee9ba2a WIP 2016-07-21 12:35:59 -07:00
David Stainton
3c50764486 Minor comment correction for get_all_relpaths 2016-07-21 12:35:59 -07:00
David Stainton
f8bb229ab7 For all downloaded files ensure parent dir exists 2016-07-21 12:35:59 -07:00
Daira Hopwood
c2d672e0b6 Simplify the cleanup_Alice_and_Bob callback. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
meejah
94a2ef0095 Make downloader delay a class-variable 
This gives the integration-style CLI-based tests a chance
to set the delay to 0 before the first 3-second delayed
call is queued to _lazy_tail in the Downloader
 2016-07-21 12:35:59 -07:00
meejah
9aecfed421 Teach unit-tests to time-warp 
1. Split alice/bob clocks to avoid races conditions
   in the tests
2. Wrap ._notify so we can advance the clock after inotify
   calls in the RealTest (since it takes >0ms to do the "real" notifies)
 2016-07-21 12:35:59 -07:00
meejah
4ed2fb725a Fix call to ready() 2016-07-21 12:35:59 -07:00
Daira Hopwood
31ab5cf4ed Correct a string-type error. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
e03a1e3361 WIP. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
262e238a7d Magic Folder file moves. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
3170cab0fc Prepare to move drop_upload.py to magic_folder.py. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
76a178969d Move backupdb.py to src/allmydata. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
976c89d2e4 Rename upload_ready_d to connected_enough_d. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
5f143f7635 Enable Windows inotify support. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
e7fb665e06 New code for Windows drop-upload support. refs #1431 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
61b3a9cef2 Add magic folder db. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Daira Hopwood
7b4067850f Unicode path fixes for drop-upload. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-07-21 12:35:59 -07:00
Brian Warner
b6dedc6223 clean up got_static_announcement() call 2016-07-19 17:31:43 -07:00
Brian Warner
2c5f7ed425 factor out yamlutil.py 
The yaml.SafeLoader.add_constructor() should probably only be done once,
and moving this all into a module gives us an opportunity to test it
directly.
 2016-07-19 17:31:41 -07:00
Brian Warner
9c1e22209f client.py: add missing import, fix wrong name 
Without these, clients with a non-empty connections.yaml would crash as
they start up. It's safe to say we need some tests for this :-).

pyflakes catches all of these, but it got accidentally disabled
recently, so travis wasn't running it. I'll fix that in the next commit.
 2016-07-19 17:30:45 -07:00
David Stainton
9063106f3a Change 'servers' sections of connections.yaml to support new design in #2788 2016-07-12 17:44:21 +01:00
David Stainton
a67e7c0db9 Remove old self.testing reference and conditional 2016-07-12 17:44:21 +01:00
David Stainton
e4d93a745c Load static server configuration from connections.yaml 2016-07-12 17:44:21 +01:00
Brian Warner
7d692d0afd Split control and log to two separate tubs, remove control knobs. 
This also removes the tahoe.cfg keys that would have configured the
control-port. And it deletes the logport.furl file before asking the Tub
to re-create it, because we're now using an ephemeral Tub (so we're not
persisting the private key, so the tubid will change each time).

closes ticket:2794
 2016-07-06 21:09:35 -07:00
David Stainton
e5ffbdbcdf Move control/logging to a separate Tub 2016-07-06 21:09:35 -07:00
Brian Warner
3402f75454 copy fixed allocate_tcp_port() from foolscap 
The old copy had a bug which occasionally returns a port that was
actually in use, causing intermittent test failures (when large numbers
of ports were allocated). I finally figured out how to fix it in
Foolscap, so this is just a copy of the updated function.

closes ticket:2795
 2016-07-06 19:20:32 -07:00
Brian Warner
a39270bd46 more v1-introducer cleanup: require sigs 
This removes a little bit of code that tolerated unsigned announcements.
 2016-07-05 16:41:15 -07:00
Brian Warner
6f1e01453e remove make_index() 
index is now always (service_name, key_id)
 2016-06-29 23:06:40 -07:00
Brian Warner
b2e5507e09 sign_to_foolscap: require key 2016-06-29 23:06:40 -07:00
Brian Warner
ae91fa9ffe require all outbound announcements to be signed 2016-06-29 23:06:40 -07:00
Brian Warner
c64ff7b310 more v1 removal cleanup 
Historical note: V2 introducers have been around for three years
now (released in 1.10.0), so it's time to drop v1. This branch removes a
lot of fallback code, and tests which exercised it. refs ticket:2784

This patch removes some now-unused code: v1-related support functions on
the client, "stub-client" handlers, and v1-tolerant remote methods on
the server. The unit tests have been cleaned up a bit too, now that
there are fewer cases to exercise.
 2016-06-29 22:58:14 -07:00
David Stainton
7feee8a25e Butcher unit tests until all test_introducer tests pass 2016-06-02 17:20:31 +00:00
David Stainton
ea35563b81 Remove v1 introducer code and fix tests 
Fixed many of the test_introducer tests.
Work-in-progress.
 2016-06-02 16:47:58 +00:00
Brian Warner
0ff00dff6f test loading the cache 2016-05-11 18:17:22 -07:00
Brian Warner
5bedca43e3 load-yaml-cache cleanups 
* use yaml.safe_load and yaml.safe_dump
  * configure SafeLoader to return unicode consistently, not str
* log+ignore bad cache, instead of throwing error, since we're already
  in the log+ignore chain from connect_failed()
* use a local exception type, instead of one from storage_client.py
* delegate delivery to self._deliver_announcements

Using yaml.safe_dump gives us:

- ann:
    my-version: tahoe-lafs/1.11.0.post96.dev0
    nickname: node-4

instead of:

- ann:
    !!python/unicode 'my-version': !!python/unicode 'tahoe-lafs/1.11.0.post96.dev0'
    !!python/unicode 'nickname': !!python/unicode 'node-4'

We want SafeLoader to consistently return unicode instead of sometimes
plain strings (for ASCII-safe values) and sometimes unicode
(for everything else). The data we write into the cache was all unicode
to start with (it came from a JSON parser), so it seems better to get
back unicode too.
 2016-05-11 18:15:50 -07:00
David Stainton
5508f751b6 Load announcement cache if failure to connect to introducer 2016-05-11 16:48:09 -07:00
Brian Warner
cfb939aa99 improve test_client_cache 2016-05-11 16:48:05 -07:00
Brian Warner
ecec58b339 test_introducer: factor out _load_cache 2016-05-11 15:03:32 -07:00
Brian Warner
1197b15106 introducer_client: split out _deliver_announcements 2016-05-11 12:54:11 -07:00
Brian Warner
155ca03174 test_introducer: cleanups 
* Use tempfile for cache to avoid collisions
* Fix pyflakes complaints
* Remove test_client_cache_2, which exercises unsigned announcements.
  These are scheduled to be removed soon (see ticket:2784) and don't
  need to be tested.
 2016-05-10 12:14:36 -07:00
Brian Warner
0c2dd8b86b IntroducerClient: small changes to cache handling 
* don't remove the cache at connection establishment, we can just wait
  for the first announcement to truncate the cache
* save announcements before notifying subscribers, so they can safely
  read it right away
* remove unused self._got_announcement_cb
 2016-05-10 12:14:04 -07:00
David Stainton
ae2b82a1f0 Add IntroducerClient write-only yaml cache file 
this change also includes unit tests
 2016-05-10 20:19:35 +00:00
Brian Warner
608719df91 tox: add "coverage" environment 
Run with "tox -e coverage". Uses a new helper
module (allmydata.test.run_trial) to let us import+execute trial without
knowing exactly where the "trial" binary lives, which helps with using
"coverage run" under tox.
 2016-05-09 14:32:21 -07:00
Brian Warner
93bb3e995a stats-gatherer: add --hostname/--location/--port 
Updates docs, tests, explains how to update an old gatherer.
 2016-05-04 17:58:45 -07:00
Brian Warner
d1d988410b remove "key-generator" node type and client support 
closes ticket:2783
 2016-05-04 16:53:04 -07:00
meejah
fa418a7809 appease windows, and skip test 2016-05-04 07:54:42 -07:00
Daira Hopwood
dd02f31a3b Fix missing import 2016-05-04 07:54:42 -07:00
Daira Hopwood
242f04818e Fix test to use arguments with absolute unicode paths 2016-05-04 07:54:42 -07:00
David Stainton
3524729fb1 Add make_dirs_with_absolute_mode to fileutils 2016-05-04 07:54:42 -07:00
Daira Hopwood
50747e2ca0 Strip any long path marker in the input to flush_volume. 
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
 2016-05-04 07:54:42 -07:00