Commit Graph

3730 Commits

Author SHA1 Message Date
David Stainton
e2cee9ba2a WIP 2016-07-21 12:35:59 -07:00
David Stainton
3c50764486 Minor comment correction for get_all_relpaths 2016-07-21 12:35:59 -07:00
David Stainton
f8bb229ab7 For all downloaded files ensure parent dir exists 2016-07-21 12:35:59 -07:00
Daira Hopwood
c2d672e0b6 Simplify the cleanup_Alice_and_Bob callback.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
meejah
94a2ef0095 Make downloader delay a class-variable
This gives the integration-style CLI-based tests a chance
to set the delay to 0 before the first 3-second delayed
call is queued to _lazy_tail in the Downloader
2016-07-21 12:35:59 -07:00
meejah
9aecfed421 Teach unit-tests to time-warp
1. Split alice/bob clocks to avoid races conditions
   in the tests
2. Wrap ._notify so we can advance the clock after inotify
   calls in the RealTest (since it takes >0ms to do the "real" notifies)
2016-07-21 12:35:59 -07:00
meejah
4ed2fb725a Fix call to ready() 2016-07-21 12:35:59 -07:00
Daira Hopwood
31ab5cf4ed Correct a string-type error.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
e03a1e3361 WIP.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
262e238a7d Magic Folder file moves.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
3170cab0fc Prepare to move drop_upload.py to magic_folder.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
76a178969d Move backupdb.py to src/allmydata.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
976c89d2e4 Rename upload_ready_d to connected_enough_d.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
5f143f7635 Enable Windows inotify support.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
e7fb665e06 New code for Windows drop-upload support. refs #1431
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
61b3a9cef2 Add magic folder db.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Daira Hopwood
7b4067850f Unicode path fixes for drop-upload.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
Brian Warner
b6dedc6223 clean up got_static_announcement() call 2016-07-19 17:31:43 -07:00
Brian Warner
2c5f7ed425 factor out yamlutil.py
The yaml.SafeLoader.add_constructor() should probably only be done once,
and moving this all into a module gives us an opportunity to test it
directly.
2016-07-19 17:31:41 -07:00
Brian Warner
9c1e22209f client.py: add missing import, fix wrong name
Without these, clients with a non-empty connections.yaml would crash as
they start up. It's safe to say we need some tests for this :-).

pyflakes catches all of these, but it got accidentally disabled
recently, so travis wasn't running it. I'll fix that in the next commit.
2016-07-19 17:30:45 -07:00
David Stainton
9063106f3a Change 'servers' sections of connections.yaml to support new design in #2788 2016-07-12 17:44:21 +01:00
David Stainton
a67e7c0db9 Remove old self.testing reference and conditional 2016-07-12 17:44:21 +01:00
David Stainton
e4d93a745c Load static server configuration from connections.yaml 2016-07-12 17:44:21 +01:00
Brian Warner
7d692d0afd Split control and log to two separate tubs, remove control knobs.
This also removes the tahoe.cfg keys that would have configured the
control-port. And it deletes the logport.furl file before asking the Tub
to re-create it, because we're now using an ephemeral Tub (so we're not
persisting the private key, so the tubid will change each time).

closes ticket:2794
2016-07-06 21:09:35 -07:00
David Stainton
e5ffbdbcdf Move control/logging to a separate Tub 2016-07-06 21:09:35 -07:00
Brian Warner
3402f75454 copy fixed allocate_tcp_port() from foolscap
The old copy had a bug which occasionally returns a port that was
actually in use, causing intermittent test failures (when large numbers
of ports were allocated). I finally figured out how to fix it in
Foolscap, so this is just a copy of the updated function.

closes ticket:2795
2016-07-06 19:20:32 -07:00
Brian Warner
a39270bd46 more v1-introducer cleanup: require sigs
This removes a little bit of code that tolerated unsigned announcements.
2016-07-05 16:41:15 -07:00
Brian Warner
6f1e01453e remove make_index()
index is now always (service_name, key_id)
2016-06-29 23:06:40 -07:00
Brian Warner
b2e5507e09 sign_to_foolscap: require key 2016-06-29 23:06:40 -07:00
Brian Warner
ae91fa9ffe require all outbound announcements to be signed 2016-06-29 23:06:40 -07:00
Brian Warner
c64ff7b310 more v1 removal cleanup
Historical note: V2 introducers have been around for three years
now (released in 1.10.0), so it's time to drop v1. This branch removes a
lot of fallback code, and tests which exercised it. refs ticket:2784

This patch removes some now-unused code: v1-related support functions on
the client, "stub-client" handlers, and v1-tolerant remote methods on
the server. The unit tests have been cleaned up a bit too, now that
there are fewer cases to exercise.
2016-06-29 22:58:14 -07:00
David Stainton
7feee8a25e Butcher unit tests until all test_introducer tests pass 2016-06-02 17:20:31 +00:00
David Stainton
ea35563b81 Remove v1 introducer code and fix tests
Fixed many of the test_introducer tests.
Work-in-progress.
2016-06-02 16:47:58 +00:00
Brian Warner
0ff00dff6f test loading the cache 2016-05-11 18:17:22 -07:00
Brian Warner
5bedca43e3 load-yaml-cache cleanups
* use yaml.safe_load and yaml.safe_dump
  * configure SafeLoader to return unicode consistently, not str
* log+ignore bad cache, instead of throwing error, since we're already
  in the log+ignore chain from connect_failed()
* use a local exception type, instead of one from storage_client.py
* delegate delivery to self._deliver_announcements

Using yaml.safe_dump gives us:

- ann:
    my-version: tahoe-lafs/1.11.0.post96.dev0
    nickname: node-4

instead of:

- ann:
    !!python/unicode 'my-version': !!python/unicode 'tahoe-lafs/1.11.0.post96.dev0'
    !!python/unicode 'nickname': !!python/unicode 'node-4'

We want SafeLoader to consistently return unicode instead of sometimes
plain strings (for ASCII-safe values) and sometimes unicode
(for everything else). The data we write into the cache was all unicode
to start with (it came from a JSON parser), so it seems better to get
back unicode too.
2016-05-11 18:15:50 -07:00
David Stainton
5508f751b6 Load announcement cache if failure to connect to introducer 2016-05-11 16:48:09 -07:00
Brian Warner
cfb939aa99 improve test_client_cache 2016-05-11 16:48:05 -07:00
Brian Warner
ecec58b339 test_introducer: factor out _load_cache 2016-05-11 15:03:32 -07:00
Brian Warner
1197b15106 introducer_client: split out _deliver_announcements 2016-05-11 12:54:11 -07:00
Brian Warner
155ca03174 test_introducer: cleanups
* Use tempfile for cache to avoid collisions
* Fix pyflakes complaints
* Remove test_client_cache_2, which exercises unsigned announcements.
  These are scheduled to be removed soon (see ticket:2784) and don't
  need to be tested.
2016-05-10 12:14:36 -07:00
Brian Warner
0c2dd8b86b IntroducerClient: small changes to cache handling
* don't remove the cache at connection establishment, we can just wait
  for the first announcement to truncate the cache
* save announcements before notifying subscribers, so they can safely
  read it right away
* remove unused self._got_announcement_cb
2016-05-10 12:14:04 -07:00
David Stainton
ae2b82a1f0 Add IntroducerClient write-only yaml cache file
this change also includes unit tests
2016-05-10 20:19:35 +00:00
Brian Warner
608719df91 tox: add "coverage" environment
Run with "tox -e coverage". Uses a new helper
module (allmydata.test.run_trial) to let us import+execute trial without
knowing exactly where the "trial" binary lives, which helps with using
"coverage run" under tox.
2016-05-09 14:32:21 -07:00
Brian Warner
93bb3e995a stats-gatherer: add --hostname/--location/--port
Updates docs, tests, explains how to update an old gatherer.
2016-05-04 17:58:45 -07:00
Brian Warner
d1d988410b remove "key-generator" node type and client support
closes ticket:2783
2016-05-04 16:53:04 -07:00
meejah
fa418a7809 appease windows, and skip test 2016-05-04 07:54:42 -07:00
Daira Hopwood
dd02f31a3b Fix missing import 2016-05-04 07:54:42 -07:00
Daira Hopwood
242f04818e Fix test to use arguments with absolute unicode paths 2016-05-04 07:54:42 -07:00
David Stainton
3524729fb1 Add make_dirs_with_absolute_mode to fileutils 2016-05-04 07:54:42 -07:00
Daira Hopwood
50747e2ca0 Strip any long path marker in the input to flush_volume.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-05-04 07:54:42 -07:00