136 Commits

Author SHA1 Message Date
Daira Hopwood
017d63b47b Delete redundant is_ready attribute from MagicFolder.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
5f98d73216 Add precondition to Uploader._process.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
0c12162bc8 Debugging WIP.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
84af199d51 WIP: exclude own dirnode from scan. This is not quite right; we shouldn't exclude it on startup. refs #2553
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
ea14b1bebe Don't add subdirectory watches if the platform's notifier doesn't require them. refs #2559
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
6bb0963197 Improve reporting of assertion failures.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
2a79252a72 Fix some path Unixisms.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
1abc2c39c8 Don't download the deletion marker file unnecessarily.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
c6191257ea Distinguish deletion of directories.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
d6bb49b1a6 Rename deleted files to .backup rather than unlinking them.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
David Stainton
99d1339d38 Add basic bob upload test and fix conflict detect 2016-07-21 12:35:59 -07:00
David Stainton
9ed61f1eb4 Count conflicted objects 2016-07-21 12:35:59 -07:00
Daira Hopwood
7a3b35e29c Basic remote conflict detection based on ancestor uri
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
f75dda7635 Avoid .format, since it is inconsistent between Python 2.6 and 2.7 (and the rest of Tahoe-LAFS doesn't use it).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
e4082a45d3 Add counter for uploader.objects_not_uploaded.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
meejah
c883977eb6 add the 'spurious' notifies 2016-07-21 12:35:59 -07:00
meejah
d61b8ed39e implement 'delete' functionality, with tests 2016-07-21 12:35:59 -07:00
Daira Hopwood
cec315da49 Add test that we don't write files outside the magic folder directory. refs ticket:2506
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
5d5d804719 More debug logging.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
8184ed4414 Watch for IN_CREATE events but filter them out for non-directories.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
cbec939d86 Implement creating local directories in downloader.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
60ab5a057a Decode names in the scanned remote.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
a3fa1617c2 Refactoring to allow logging from _write_downloaded_file and _rename_conflicted_file.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
6cb41cadf5 Simplify and fix non-existent-file handling.
Also make the existent and non-existent cases as similar as possible,
with a view to merging them.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
cb905d8d97 Logging/debugging improvements.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
ac5730ea1c Correct a call to did_upload_version in the downloader.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
28587b5b2a Restore a call to increment files_uploaded that was mistakenly removed.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
David Stainton
2fcaa8c2b2 Teach uploader+downloader to use to db schema
here we attempt to fix all the unit tests as well...
however two tests still fail
2016-07-21 12:35:59 -07:00
Daira Hopwood
5754c01cb5 Add magicfolderdb.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
David Stainton
e2cee9ba2a WIP 2016-07-21 12:35:59 -07:00
David Stainton
f8bb229ab7 For all downloaded files ensure parent dir exists 2016-07-21 12:35:59 -07:00
meejah
94a2ef0095 Make downloader delay a class-variable
This gives the integration-style CLI-based tests a chance
to set the delay to 0 before the first 3-second delayed
call is queued to _lazy_tail in the Downloader
2016-07-21 12:35:59 -07:00
Daira Hopwood
e03a1e3361 WIP.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
262e238a7d Magic Folder file moves.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
3170cab0fc Prepare to move drop_upload.py to magic_folder.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
76a178969d Move backupdb.py to src/allmydata.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
5f143f7635 Enable Windows inotify support.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
61b3a9cef2 Add magic folder db.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
7b4067850f Unicode path fixes for drop-upload.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
a56a3adaae Teach StorageFarmBroker to fire a deferred when a connection threshold is reached. refs #1449
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-04-26 12:26:39 -06:00
Daira Hopwood
5d046c597e drop-upload.py: fix error messages.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-05-13 14:21:17 +01:00
Brian Warner
eda00a5c58 typo 2015-03-31 11:07:21 -07:00
Brian Warner
cc68a81326 bump Twisted dep to 11.1.0, thus simplify IntishPermissions 2015-03-31 11:06:14 -07:00
Brian Warner
26c8abadec Fix ftp 'ls' to work with current Twisted-15.0.0
refs ticket:2394

It's kind of a hack, but Twisted changed the API and I couldn't find a
cleaner way to detect which form of "permissions" value the Twisted FTP
server wants.

I've manually tested it against 14.0.2 and 15.0.0.
2015-03-31 11:06:14 -07:00
Daira Hopwood
b6be693cbe Add a test, add missing imports. refs #2388
This tests ftpd, but not sftpd. Doing this sort of test on sftpd
requires the creation of a valid pubkey/privkey file pair, which is more
work than I want to do right now.

init_ftp/init_sftp were changed to interpret the configured
accounts.file as relative to the node's basedir, with
abspath_expanduser_unicode(accountfile, base=self.basedir).
This would happen naturally in a real node, since it os.chdir()s
to the basedir before doing anything. But tests don't do that.

Author: Brian Warner <warner@lothar.com>
Author: Daira Hopwood <daira@jacaranda.org>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-03-24 17:10:00 +00:00
Daira Hopwood
3066039f0f Ensure that path parameters to SFTPServer and FTPServer constructors are unicode. refs #2388
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-03-24 16:46:40 +00:00
Daira Hopwood
14f783086f Change uses of os.path.expanduser and os.path.abspath. refs #2235
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-01-30 00:50:18 +00:00
Daira Hopwood
0e9f0da51f Reserve all keytypes starting with "ssh-" in an accounts file, rather than only "ssh-rsa" and "ssh-dsa".
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-01-06 19:09:04 +00:00
Daira Hopwood
6c756ba3e9 Simplify key checking code by inlining _allowedKey and _correctSignature. refs #1141
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-01-06 18:10:41 +00:00
Jean-Paul Calderone
102d581a90 Give out FTPAvatarID objects instead. 2015-01-04 09:48:38 -05:00