Commit Graph

6430 Commits

Author SHA1 Message Date
Brian Warner
7d692d0afd Split control and log to two separate tubs, remove control knobs.
This also removes the tahoe.cfg keys that would have configured the
control-port. And it deletes the logport.furl file before asking the Tub
to re-create it, because we're now using an ephemeral Tub (so we're not
persisting the private key, so the tubid will change each time).

closes ticket:2794
2016-07-06 21:09:35 -07:00
David Stainton
e5ffbdbcdf Move control/logging to a separate Tub 2016-07-06 21:09:35 -07:00
Brian Warner
3402f75454 copy fixed allocate_tcp_port() from foolscap
The old copy had a bug which occasionally returns a port that was
actually in use, causing intermittent test failures (when large numbers
of ports were allocated). I finally figured out how to fix it in
Foolscap, so this is just a copy of the updated function.

closes ticket:2795
2016-07-06 19:20:32 -07:00
Brian Warner
7407e7c8cf Merge branch 'pr285': add travis OS-X coverage 2016-07-06 02:08:27 -07:00
Brian Warner
5bf7fab99d travis: big cleanup, make it work on OS-X
Travis defaults to giving us an OS-X 10.9 box, which has an OpenSSL that
is too old for the current cryptography-1.4 (note that a previous
version of this branch worked, but only because the previous
cryptography-1.3.x didn't enforce the OpenSSL version).

On OS-X, this new .travis.yml does the following:

* set "osx_image: xcode7" to get us 10.10, with newer OpenSSL
* uses system python, not homebrew
* installs pip with get-pip.py, since system python doesn't have it
* adds the --user directory to $PATH, since OS-X python doesn't have it
  by default

On both linux and OS-X, this:

* installs tox and coveralls with --user, not to the system
* doesn't use sudo to run tox
* prints some extra debug info in case it's useful later

Closes #285
2016-07-06 02:05:43 -07:00
David Stainton
7e10b5d0e5 Corrections to .travis.yml from Brian's review 2016-07-06 02:00:41 -07:00
David Stainton
73c4aa727f wtf sudo tox for linux 2016-07-06 02:00:41 -07:00
David Stainton
8a374df8b1 Make travis test on osx and linux both 2016-07-06 02:00:41 -07:00
Brian Warner
a39270bd46 more v1-introducer cleanup: require sigs
This removes a little bit of code that tolerated unsigned announcements.
2016-07-05 16:41:15 -07:00
Brian Warner
78810d5851 Merge branch '2784-remove-v1-introducer'
Closes PR #289
Closes ticket:2784
2016-07-05 15:31:34 -07:00
Brian Warner
6f1e01453e remove make_index()
index is now always (service_name, key_id)
2016-06-29 23:06:40 -07:00
Brian Warner
b2e5507e09 sign_to_foolscap: require key 2016-06-29 23:06:40 -07:00
Brian Warner
ae91fa9ffe require all outbound announcements to be signed 2016-06-29 23:06:40 -07:00
Brian Warner
c64ff7b310 more v1 removal cleanup
Historical note: V2 introducers have been around for three years
now (released in 1.10.0), so it's time to drop v1. This branch removes a
lot of fallback code, and tests which exercised it. refs ticket:2784

This patch removes some now-unused code: v1-related support functions on
the client, "stub-client" handlers, and v1-tolerant remote methods on
the server. The unit tests have been cleaned up a bit too, now that
there are fewer cases to exercise.
2016-06-29 22:58:14 -07:00
David Stainton
7feee8a25e Butcher unit tests until all test_introducer tests pass 2016-06-02 17:20:31 +00:00
David Stainton
ea35563b81 Remove v1 introducer code and fix tests
Fixed many of the test_introducer tests.
Work-in-progress.
2016-06-02 16:47:58 +00:00
Brian Warner
73b08d2a54 Merge pr281: load YAML cache when connection fails
refs ticket:2788
2016-05-11 18:27:11 -07:00
Brian Warner
0ff00dff6f test loading the cache 2016-05-11 18:17:22 -07:00
Brian Warner
5bedca43e3 load-yaml-cache cleanups
* use yaml.safe_load and yaml.safe_dump
  * configure SafeLoader to return unicode consistently, not str
* log+ignore bad cache, instead of throwing error, since we're already
  in the log+ignore chain from connect_failed()
* use a local exception type, instead of one from storage_client.py
* delegate delivery to self._deliver_announcements

Using yaml.safe_dump gives us:

- ann:
    my-version: tahoe-lafs/1.11.0.post96.dev0
    nickname: node-4

instead of:

- ann:
    !!python/unicode 'my-version': !!python/unicode 'tahoe-lafs/1.11.0.post96.dev0'
    !!python/unicode 'nickname': !!python/unicode 'node-4'

We want SafeLoader to consistently return unicode instead of sometimes
plain strings (for ASCII-safe values) and sometimes unicode
(for everything else). The data we write into the cache was all unicode
to start with (it came from a JSON parser), so it seems better to get
back unicode too.
2016-05-11 18:15:50 -07:00
David Stainton
5508f751b6 Load announcement cache if failure to connect to introducer 2016-05-11 16:48:09 -07:00
Brian Warner
cfb939aa99 improve test_client_cache 2016-05-11 16:48:05 -07:00
Brian Warner
ecec58b339 test_introducer: factor out _load_cache 2016-05-11 15:03:32 -07:00
Brian Warner
1197b15106 introducer_client: split out _deliver_announcements 2016-05-11 12:54:11 -07:00
Brian Warner
b49b409c32 Merge branch 'pr278': write (but don't read) YAML cache
refs ticket:2759
2016-05-10 13:02:00 -07:00
Brian Warner
155ca03174 test_introducer: cleanups
* Use tempfile for cache to avoid collisions
* Fix pyflakes complaints
* Remove test_client_cache_2, which exercises unsigned announcements.
  These are scheduled to be removed soon (see ticket:2784) and don't
  need to be tested.
2016-05-10 12:14:36 -07:00
Brian Warner
0c2dd8b86b IntroducerClient: small changes to cache handling
* don't remove the cache at connection establishment, we can just wait
  for the first announcement to truncate the cache
* save announcements before notifying subscribers, so they can safely
  read it right away
* remove unused self._got_announcement_cb
2016-05-10 12:14:04 -07:00
David Stainton
ae2b82a1f0 Add IntroducerClient write-only yaml cache file
this change also includes unit tests
2016-05-10 20:19:35 +00:00
Brian Warner
e3aef2b966 travis.yml: use tox, with 'coverage' environment 2016-05-09 14:40:48 -07:00
Brian Warner
608719df91 tox: add "coverage" environment
Run with "tox -e coverage". Uses a new helper
module (allmydata.test.run_trial) to let us import+execute trial without
knowing exactly where the "trial" binary lives, which helps with using
"coverage run" under tox.
2016-05-09 14:32:21 -07:00
Brian Warner
1e1e86fe35 tox.ini: skip default install
We use "--deps = --editable=.[test]" to achieve three goals:

* make tahoe and it's dependencies available for tests
* use --editable, which is faster and allows "coverage run" to get the
  source filenames right
* use the [test] extra, which includes "mock"

Tox's default install command does the first, but doesn't use
--editable, so when the "deps" stage comes around, there's already a
non-editable install in place. It seems to get the [test] extra right,
but it doesn't wind up with an editable install.

So we disable the default install command and rely on the "deps" clause
instead.
2016-05-09 13:56:06 -07:00
Brian Warner
f57d1e9d3d Merge branch '2773-stats'
This changes "tahoe create-stats-gatherer" to take --hostname,
--location, and --port, according to (refs ticket:2773).
2016-05-04 18:37:23 -07:00
Brian Warner
93bb3e995a stats-gatherer: add --hostname/--location/--port
Updates docs, tests, explains how to update an old gatherer.
2016-05-04 17:58:45 -07:00
Brian Warner
d1d988410b remove "key-generator" node type and client support
closes ticket:2783
2016-05-04 16:53:04 -07:00
Brian Warner
c715e0d839 Merge pull request #265 from meejah/fileutil-improvements 2016-05-04 08:04:51 -07:00
meejah
fa418a7809 appease windows, and skip test 2016-05-04 07:54:42 -07:00
Daira Hopwood
dd02f31a3b Fix missing import 2016-05-04 07:54:42 -07:00
Daira Hopwood
242f04818e Fix test to use arguments with absolute unicode paths 2016-05-04 07:54:42 -07:00
David Stainton
3524729fb1 Add make_dirs_with_absolute_mode to fileutils 2016-05-04 07:54:42 -07:00
Daira Hopwood
50747e2ca0 Strip any long path marker in the input to flush_volume.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-05-04 07:54:42 -07:00
Daira Hopwood
d07c45bdf4 Improve all of the Windows-specific error reporting.
Also make the Windows function declarations more readable and consistent.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-05-04 07:54:42 -07:00
Daira Hopwood
b517e200aa replace_file should allow the replaced file not to exist on Windows.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-05-04 07:54:42 -07:00
Daira Hopwood
7015f5d826 Fix fileutil tests.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-05-04 07:54:42 -07:00
Daira Hopwood
07389571c2 Add long_path=False option to abspath_expanduser_unicode.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-05-04 07:54:42 -07:00
Daira Hopwood
62dc2f7ac1 Re-order some imports in fileutil
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-05-04 07:54:42 -07:00
Daira Hopwood
9bfcb72400 Windows fileutil improvements
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-05-04 07:54:42 -07:00
Brian Warner
9402b40da5 Merge branch 'storage_broker_tub.0' 2016-05-03 16:12:12 -07:00
Brian Warner
bbeb397a00 storage_client: remove unnecessary startService upcall 2016-05-03 15:48:08 -07:00
Brian Warner
76f6d2d21a create Tub inside NativeStorageServer, not broker
This makes IServer instances responsible for their own network
connections, which will help when we add HTTP-based servers in the
future. The StorageFarmBroker should not care about how the IServer uses
the network, it just provides the announcement (and local config).
2016-05-03 15:09:13 -07:00
Brian Warner
e147aa3da2 test: use patch() to control Tub creation
This reduces the dependency on the internals of storage_client.py
2016-05-03 14:49:31 -07:00
Brian Warner
e301ac88ea make test slightly easier to follow 2016-05-03 14:03:17 -07:00