3608 Commits

Author SHA1 Message Date
Daira Hopwood
8348913b8a Simplify and fix non-existent-file handling.
Also make the existent and non-existent cases as similar as possible,
with a view to merging them.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-15 23:02:30 +01:00
Daira Hopwood
1d0e2b5cda Logging/debugging improvements.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-15 23:02:07 +01:00
Daira Hopwood
768b478cb9 Refactor and fix race conditions in test_alice_bob.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-15 22:56:38 +01:00
Daira Hopwood
257da9f86d Correct a call to did_upload_version in the downloader.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-13 15:56:31 +01:00
Daira Hopwood
fd506bd550 Make sure that test_move_tree waits until files have been uploaded as well as downloaded.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-13 15:56:00 +01:00
Daira Hopwood
6945b85ce7 Restore a call to increment files_uploaded that was mistakenly removed.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-13 15:55:00 +01:00
David Stainton
37269740dd Teach uploader+downloader to use to db schema
here we attempt to fix all the unit tests as well...
however two tests still fail
2015-10-12 20:07:39 +02:00
Daira Hopwood
2342393d8a Add magicfolderdb.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-08 16:02:34 +01:00
David Stainton
c82f977fb7 Remove magic-folder db code from backupdb.py 2015-10-08 16:02:34 +01:00
David Stainton
f819aa5b5d WIP 2015-10-08 16:02:34 +01:00
David Stainton
5d7d2febc6 Minor comment correction for get_all_relpaths 2015-10-08 16:02:34 +01:00
David Stainton
531747303d For all downloaded files ensure parent dir exists 2015-10-08 16:02:34 +01:00
Daira Hopwood
27fb403dad Simplify the cleanup_Alice_and_Bob callback.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-08 14:54:39 +01:00
meejah
1377aecbd7 Make downloader delay a class-variable
This gives the integration-style CLI-based tests a chance
to set the delay to 0 before the first 3-second delayed
call is queued to _lazy_tail in the Downloader
2015-10-08 14:54:39 +01:00
meejah
906d3da819 Teach unit-tests to time-warp
1. Split alice/bob clocks to avoid races conditions
   in the tests
2. Wrap ._notify so we can advance the clock after inotify
   calls in the RealTest (since it takes >0ms to do the "real" notifies)
2015-10-08 14:54:39 +01:00
meejah
c398218c55 Fix call to ready() 2015-10-06 14:11:32 -06:00
Daira Hopwood
0b5039d253 Correct a string-type error.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-02 22:08:54 +01:00
Daira Hopwood
3180471ce8 WIP.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:40:10 +01:00
Daira Hopwood
27ed601cef Magic Folder file moves.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:36:18 +01:00
Daira Hopwood
df044f884c Prepare to move drop_upload.py to magic_folder.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:25:32 +01:00
Daira Hopwood
71ec32373a Move backupdb.py to src/allmydata.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:25:32 +01:00
Daira Hopwood
2b17d88644 Rename upload_ready_d to connected_enough_d.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:25:32 +01:00
Daira Hopwood
e7d4e3d4a9 Teach StorageFarmBroker to fire a deferred when a connection threshold is reached. refs #1449
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:25:32 +01:00
Daira Hopwood
dc49a1e511 Enable Windows inotify support.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:25:32 +01:00
Daira Hopwood
6e64ab563b New code for Windows drop-upload support. refs #1431
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:25:31 +01:00
Daira Hopwood
81e6a12779 Add magic folder db.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:25:31 +01:00
Daira Hopwood
30d7d8a501 Unicode path fixes for drop-upload.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-10-01 22:25:31 +01:00
Brian Warner
67dcedd8df introweb: fix connection hints for server announcements
A long time ago, the introducer's status web page would show the
advertised IP addresses for all published services, by parsing their
FURL's connection hints. This hasn't worked since about 12-Aug-2014 when
foolscap-0.6.5 changed the internal format of these hints (the column
has been empty this whole time).

This removes the "Advertised IPs" column from the Service Announcements
table. Instead, the service's full connection hints (not just the IP
address) is displayed in a tooltip/popup on the "Announced" timestamp
column.

The code that pulls these connection hints is now tolerant of all three
foolscap styles:

* foolscap<=0.6.4 : tuples of ("ipv4",host,port)
* 0.6.5 .. 0.8.0  : tuples of ("tcp",host,port)
* foolscap>=0.9.0 : strings

fixes ticket:2510
2015-09-22 17:19:52 -07:00
Brian Warner
8f0c417910 remove "announcement_distinct_hosts" from introweb JSON
The machine-parseable JSON output for the introducer status web page
used to include a key named "announcement_distinct_hosts", which counted
the number of distinct IP addresses advertised by all connected storage
servers. This hasn't worked since Aug-2014 when foolscap-0.6.5 change
the internal hints format.

This removes that field.
2015-09-22 17:19:47 -07:00
Brian Warner
71323fbd8f introducer: stop tracking hints for subscribed clients
A long time ago, the introducer's status web page would show the
advertised IP addresses for all subscribers, by parsing their
RemoteReference's FURL's connection hints. This hasn't worked since
about 12-Aug-2014 when foolscap-0.6.5 changed the internal format of
these hints.

This removes the feature: we no longer attempt to show advertised IP
addresses of subscribed clients. It also removes the code that looked
inside foolscap internals for this information.
2015-09-22 17:19:43 -07:00
Daira Hopwood
a4dfc31a19 Avoid spurious errors when an imported version is consistent with pkg_resources
but not parseable; also improve related error reporting. fixes ticket:2499

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-09-12 00:59:51 +01:00
Brian Warner
e5979f970f test_cli_cp: minor cleanup 2015-07-28 17:39:26 -07:00
Brian Warner
98ab848cda cp: error on target-filename collisions, rather than overwrite
Closes ticket:2447
2015-07-28 17:39:26 -07:00
Zooko
f0fd34d16c unit test for my desired behavior 2015-07-28 17:39:26 -07:00
Daira Hopwood
023f9fa4d3 test_version.py: minor cleanup.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-29 00:20:33 +01:00
Daira Hopwood
2c99294a4f Don't show scary diagnostic warnings from --version[-and-path] (corrected). refs ticket:2436
The previous version would incorrectly add to the output of
get_package_versions_string each time it was called.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-29 00:20:33 +01:00
Daira Hopwood
3239527929 Revert "Don't show scary diagnostic warnings from --version[-and-path]"
This reverts commit 431728f8f854e02d9ab2f731675f12ce18cda122.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-29 00:20:33 +01:00
Brian Warner
29ab496bd7 mutable/retrieve.py: remove all bare assert()s
Replace them with _assert() or precondition(). Closes ticket:2462.
2015-07-28 13:32:46 -07:00
Brian Warner
6252a72965 IReadable.read(): document out-of-range errors
closes ticket#2461
2015-07-28 13:01:15 -07:00
Brian Warner
27ea6e1a3c test_mutable: more enhancements 2015-07-28 10:27:24 -07:00
Brian Warner
89e9076c41 mutable/retrieve.py: rewrite partial-read handling
This should tolerate offset/size combinations that read the last byte of
the file, something which was broken before. It quits early in the case
of zero-byte reads, to simplify the resulting "which segments do I need"
logic. Probably addresses ticket:2459.
2015-07-28 10:13:32 -07:00
Brian Warner
46d7085dfe enhance SDMF/MDMF tests: zero-length files, size=None 2015-07-28 10:13:32 -07:00
Daira Hopwood
a7e1dac27f Add tests for SDMF partial reads. refs #2459
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-28 10:13:32 -07:00
Daira Hopwood
9076db137a Remove some bare asserts in retrieve.py (there are still quite a few left). refs #2462
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-28 10:13:32 -07:00
Daira Hopwood
19df54927d Minor code cleanup in __init__.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-21 18:55:47 +01:00
Daira Hopwood
431728f8f8 Don't show scary diagnostic warnings from --version[-and-path]
if the only problem is that extra packages are found by pkg_resources.
fixes ticket:2436

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-21 18:51:29 +01:00
Daira Hopwood
c830419e04 Eliminate mock dependency.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-17 22:12:25 +01:00
Daira Hopwood
3ae6ceb6a8 Add Namespace utility class. (This is already used in the leasedb/cloud backend branch.)
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-17 22:12:25 +01:00
Daira Hopwood
2d7c852ace Bump pyasn1 requirement to >= 0.1.8.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-16 15:38:06 +01:00
Daira Hopwood
baaf2a4a55 Blacklist mock > 1.0.1.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-07-16 15:24:45 +01:00