Commit Graph

4176 Commits

Author SHA1 Message Date
meejah
a9521b89a6 --readonly-uri for 'tahoe list-aliases' 2017-01-19 15:40:09 -07:00
Brian Warner
be6e0a9c29 deps: we need foolscap-0.12.6 for the new I2P API 2017-01-17 20:13:05 -05:00
Brian Warner
6db7686a8f update tests to match 2017-01-17 20:12:55 -05:00
str4d
998af5c26e Pass I2P keyfile to foolscap
If no session management is performed, txi2p starts a process-wide session the
first time a connection (client or server) is opened; all subsequent connections
use that session and its configuration properties.

This commit ensures that the same properties are passed to both client and
server endpoints, so that the correct I2P Destination is started regardless of
whether the first connection made by Tahoe-LAFS is for a client or server.

Closes #2858.
2017-01-17 20:12:32 -05:00
Brian Warner
ee58637813 fix syntax (tcp=disabled, not =none), add test 2017-01-10 16:30:04 -08:00
David Stainton
3c22a3ce50 Set tcp = tor only if txtorcon is imported
for i2p this means tcp = none as long as txtorcon is not imported
2017-01-10 16:30:04 -08:00
Brian Warner
64e344b03a introducer.yaml: ascii-fy furls before use
This fixes the foolscap-rejects-unicode-furls problem which prevented a
correctly-formatted introducers.yaml from working.

closes ticket:2862
2017-01-08 22:59:51 -08:00
Brian Warner
582931b1c2 docs: recommend proper introducers.yaml syntax
add a test to exercise that this syntax is parseable, and another to point
out that the previously-recommended syntax was not

refs ticket:2862
2017-01-08 22:59:51 -08:00
Brian Warner
79ff7ae5cb move 'shutilwhich' dependency out to the [test] extra
This is only used by integration/conftest.py, so we don't need an
unconditional dependency on it.

closes ticket:2856
2016-12-25 14:15:52 -05:00
Brian Warner
a06cf2ea4c express pypiwin32 dep via a setup.py extra
Previously this looked at sys.platform to decide what dependencies to
include. The problem with that approach is that wheels built on a unix box
won't work on windows (and vice versa), when the (pure-python) Tahoe wheels
aren't supposed to be platform-dependent.

setup.py provides a syntax to express this properly, so wheels created on
either platform will include the pypiwin32 dependency in the metadata, but
marked as only being relevant when installing on a win32 platform.

closes ticket:2763
2016-12-24 18:00:13 -05:00
Stefan Lew
481a02aee7 made tests work with new attribute rel="noreferrer" 2016-12-23 23:20:52 -05:00
Stefan Lew
12844220ec use rel="noreferrer" to prevent leakage of dircap 2016-12-23 23:20:52 -05:00
Vladimir Rusinov
0d47e12e40 Add some docstrings. 2016-12-23 22:50:48 -05:00
Vladimir Rusinov
7faec2e10a Adjust formatting to be pep8-compliant. 2016-12-23 22:50:48 -05:00
Vladimir Rusinov
80fe154634 DeepStats: Initialize all class arguments in __init__ (good practice; makes pylint happy). 2016-12-23 22:50:48 -05:00
Vladimir Rusinov
a95f2743ef Add test for deep-stats api version. 2016-12-23 22:50:48 -05:00
Vladimir Rusinov
195d94cf2c Add api-version to DeepStats. 2016-12-23 22:50:48 -05:00
Vladimir Rusinov
8d63208edd Convert DeepStats to a new-style class. 2016-12-23 22:50:48 -05:00
Vladimir Rusinov
672efa7d17 Add a docsrting to dirnode.py. 2016-12-23 22:50:48 -05:00
Vladimir Rusinov
73fabaec19 Move DeepStats to separate file as dirnode is a bit too large to work with comfortably. 2016-12-23 22:50:48 -05:00
Brian Warner
5651994501 magic_folder_cli: remove unnecessary conditional
(this increases our branch coverage by one)
2016-12-14 21:20:45 -08:00
Brian Warner
9303d20ed7 test_magic_folder: improve coverage 2016-12-14 21:20:37 -08:00
meejah
e32b664b2b Adjust default poll_interval
Also adds a --poll-interval option to both 'magic-folder join'
and 'magic-folder create' so that the integration tests can pass
something "very short".
2016-12-14 20:29:41 -07:00
meejah
37c7d54bd3 keeps defaults in CreateClientOptions but makes shares-* required? 2016-12-13 19:50:19 -07:00
meejah
5b8be255d7 Set total/needed/happy from command-line 2016-12-13 18:52:45 -07:00
Brian Warner
5cbe580d90 improve ConnectionStatus and welcome-page display
* replace "last_details" with "non_connected_statuses" dict
* rename "last_connection_summary" to just "summary"
* for connected servers, show other hints in a tooltip
* for not-yet-connected servers, show all hints in a list
* build the list (in STAN) on the server side, not using IContainer
2016-12-09 18:41:38 -08:00
Brian Warner
70db0db5bd interface.py: cosmetic, rewrite docstrings as attributes 2016-12-09 18:41:38 -08:00
meejah
74209e643e Show the connection-status of all hints until we're connected to one 2016-12-09 18:41:38 -08:00
Brian Warner
8b35fe9efe tests: exercise connection_status.py 2016-12-09 11:46:26 -08:00
Brian Warner
70a616826c welcome.xhtml: update copyright year
Not sure why we have this in there, but it might as well be current.
2016-12-09 11:19:26 -08:00
Brian Warner
38935bbca6 remove old unused connection-status APIs 2016-12-08 16:25:00 -08:00
Brian Warner
77fd41b66e update WUI welcome page with new connection-status info
This shows current-connection info, and provides per-hint status details in a
tooltip.

The "Connection" section no longer shows seconds-since-loss when the server
was not connected (previously it showed seconds-since-connect when connected,
and flipped to seconds-since-loss when disconnected). We already have the
"Last RX" column, which is arguably more meaningful (and I can't think of a
good case when these would differ), so we don't really need
seconds-since-loss, and the new ConnectionStatus doesn't track it anyways.

So now the "Connection" timestamp for non-connected servers is just
"N/A" (both the main text and the tooltip). The "Introducers" section was
changed the same way.

This moves the per-server connection timestamp out of the nickname/serverid
box and over into the Connection box. It also right-floats all timestamps,
regardless of which box they're in, which makes them share the box with
connection_status more politely.

Internally, this adds code to create ConnectionStatus objects when necessary.
2016-12-08 16:24:32 -08:00
Brian Warner
48fc14bd30 add IConnectionStatus and implementation
also a function to build one from a foolscap.Reconnector
2016-12-08 15:41:37 -08:00
Brian Warner
8d008967e7 tor_provider: use new Foolscap API to provide better status 2016-12-08 15:41:37 -08:00
Brian Warner
de4295ae60 require foolscap >= 0.12.5, for ReconnectionInfo 2016-12-08 15:41:37 -08:00
Brian Warner
6879622894 create-node: avoid introducer.furl=None
Previously, "tahoe create-node" without an --introducer= argument would
result in the literal string "None" being written into tahoe.cfg:

 [client]
 introducer.furl = None

We were using config.get("introducer",""), but that didn't suffice because
the key was actually present: it just had a value of None, which then got
stringified into "None" when writing out tahoe.cfg.

This briefly caused test/cli/test_create to fail, as the startup code tried
to parse "None" as a FURL. This only happened against a development version
of Foolscap which accidentally became sensitive to unparseable FURLs in
started Reconnectors. I fixed that in the final foolscap-0.12.5 release, so
we shouldn't hit this bug, but I wanted to fix it properly in the tahoe-side
source.
2016-12-08 15:37:49 -08:00
Brian Warner
d5f3d47483 test/cli/test_create: remove errant print() 2016-12-08 14:59:11 -08:00
Brian Warner
5a5da0de2a magic_folder_cli.py: fix typo 2016-12-06 11:01:51 -08:00
Brian Warner
250c6f6054 test_magic_folder: mark failing test as .todo
As discussed in this week's meeting, since we don't yet know why some
flavors of linux have slightly different inotify behavior than others,
and since we believe the actual functionality is not significantly
impacted, and since the red buildbot is reducing our confidence that the
other tests are passing, and since we have a release coming up: we're
marking the one troublesome test as ".todo". We expect that the test
will be fixed soon (perhaps to accept either 3 or 4 events), but not
necessarily before the 1.12 release.

refs ticket:2834
2016-11-02 13:31:22 -07:00
meejah
783449b4f3 When tor or i2p options are present, confirm we have txi2p or txtorcon
We mock out some import methods and provide tests for providing
server-side or client-side tor/i2p options
2016-10-26 13:41:10 -06:00
meejah
0636d44cc1 add poll-interval config for magic-folder 2016-10-25 12:38:57 -07:00
meejah
da4e1589bb Rename _turn_delay and friends to make things more clear
This forces the Uploader and Downloader to implement a _scan_delay
method and makes the naming more consistent with what's actually
happening. Also, fix a few "bugs" in the names of args in the
mocks for some tests.
2016-10-25 12:38:57 -07:00
meejah
ddaa29ce39 refactor/white-space MagicFolder creation 2016-10-25 12:38:57 -07:00
Brian Warner
203067810d magic_folder: announce debug on/off to log, not stdout 2016-10-24 09:37:35 -07:00
Brian Warner
4d003c026e test_magic_folder: remove unused debug code 2016-10-24 09:23:26 -07:00
Brian Warner
ae5637dc9c test_magic_folder: turn off debugging when done 2016-10-24 09:23:16 -07:00
David Stainton
d6cbe44786 Enable debug logging for test_move_tree magic-folder unit test 2016-10-24 09:15:09 -07:00
str4d
6071c2b6f8
Implement i2p_provider and --listen=i2p
Closes ticket:2838
2016-10-23 20:30:01 -05:00
Lukas Pirl
1491e9fab1 added /sbin/ip as possible binary to detect local IP address
fixes IP detection for RHEL flavors

Signed-off-by: Lukas Pirl <git@lukas-pirl.de>
2016-10-12 12:46:12 +02:00
Brian Warner
6d3a20e78b test_tor_provider: accept windows pathname separators 2016-10-09 01:49:32 -04:00
Brian Warner
ea1c1d27de fix tests to match 2016-10-09 01:22:17 -04:00
meejah
2ecb8171eb flip remote/local around for Onion-service 'ports' lines 2016-10-09 01:21:52 -04:00
meejah
eb6c6bee9e tor's output, for debugging 2016-10-09 01:21:52 -04:00
meejah
646fad69ed ControlPort needs "unix:" prefix for Tor, and allocate SOCKSPort 2016-10-09 01:21:52 -04:00
Brian Warner
89eb86c5a4 node.py: use tor_provider at startup
We use it for two things: to create the foolscap connection handler, and
to possibly start an .onion listener at startup.

This also updates node._common_config_sections to accept the new tor
settings written by create-node/create-introducer.
2016-10-09 01:21:51 -04:00
Brian Warner
6b9218ff22 create_node.py: use tor_provider to handle --listen=tor
This adds tor-related CLI arguments to "create-node" and
"create-introducer", to control exactly how we should be using Tor.

* --tor-launch
* --tor-executable=
* --tor-control-port=

I went with "--tor-launch" instead of "--launch-tor" for consistency. I
don't particularly like the grammatical flow of it, and it doesn't
actually put all the tor-related arguments next to each other in the
--help output (the flags are put in one block, then the parameters in
the next). But it seems slightly more consistent to start all the
tor-related argument names with a "--tor*" prefix.
2016-10-09 01:21:51 -04:00
Brian Warner
a1741ce4dc rewrite tor_provider
This uses a unix-domain control port, and includes test coverage.

create_onion() displays pacifier messages, since the allocate-onion step
takes around 35 seconds
2016-10-09 01:21:47 -04:00
Brian Warner
8f9ad009bb move torutil.py to util/tor_provider.py 2016-10-09 00:57:02 -04:00
David Stainton
897079105a Code refactor TorProvider from Daira's code review 2016-10-09 00:35:03 -04:00
David Stainton
e02d21aed1 Add tor listening options to create-node 2016-10-09 00:34:57 -04:00
David Stainton
4a1d1a5541 Add helper function CreateOnion 2016-10-09 00:30:33 -04:00
David Stainton
b3b80d376b Use a TorProvider to get a Tor handler 2016-10-09 00:30:33 -04:00
David Stainton
993d91ec0d Implement TorProvider class as described by Brian Warner
https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2490#comment:10
2016-10-09 00:30:33 -04:00
Brian Warner
f549000121 test_create: get full coverage on create_node.py 2016-10-09 00:29:29 -04:00
Brian Warner
1e418ade83 create_node: slight refactoring
This prepares for various listeners to contribute values to tub.port and
tub.location, which will be helpful once Tor support is added.
2016-10-09 00:24:19 -04:00
Brian Warner
b5aaafbe68 create_node: add cosmetic newline in tahoe.cfg 2016-10-09 00:24:19 -04:00
Brian Warner
0eda544a73 create-node: introducer needs private/ too
All server-like nodes (storage servers and introducers both) will need
this for the tor state directory and .onion private key file, and it
needs to exist before the config is written, so tor onion-service
private keys can be placed there.

Also remove a redundant import.
2016-10-09 00:24:19 -04:00
Brian Warner
395f7cd4f0 create_node: prep for slow write_node_config()
This puts the right inlineCallbacks in place to allow
write_node_config() to return a Deferred. The upcoming Tor support will
need this (since it must wait for an .onion address to be allocated
before it can write tahoe.cfg's tub.port and tub.location lines).
2016-10-09 00:24:11 -04:00
Brian Warner
0e3225ac27 set up logging before init_connections 2016-10-09 00:11:17 -04:00
Daira Hopwood
4d18425ed4 Fix for Magic Folder tests if the I/O encoding is borked on Unicode platforms (OS X, Windows).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-09-27 18:05:09 +01:00
Daira Hopwood
64841e339d Remove redundant use of NonASCIIPathMixin.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-09-27 16:16:19 +01:00
Daira Hopwood
c9d73a936a Fix Unicode-related Magic Folder test failures. fixes #2807
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-09-27 16:09:15 +01:00
Brian Warner
5ef04ef59e add hashutil.permute_server_hash
which uses SHA1 to combine the file's storage index (known as "peer
selection index" in this context) and each server's "server permutation
seed". This is the only thing in tahoe that uses SHA1.

With this change, we stop importing sha1 from anywhere else.
2016-09-26 20:42:42 -07:00
Brian Warner
b00c2d21b7 test tub.port with multiple endpoints, add docs
I think the preferred way to listen on both IPv4 and IPv6 will be to use
"--port=tcp:PORT,tcp6:PORT". This is now reflected in the docs.

refs ticket:867
2016-09-20 13:04:06 -07:00
David Stainton
e6d7895865 Support comma separated endpoint list in tub.port
fixes ticket #867
2016-09-20 10:20:28 -07:00
Brian Warner
b9b731e3e2 update test to match new output 2016-09-20 10:14:35 -07:00
str4d
e82e2c31e2 Update error message to match implementation and docs
Closes ticket:2824
2016-09-20 10:02:05 -07:00
Brian Warner
a638a97806 implement connections:tcp=disabled
This enables an I2P-only node, which disables TCP entirely (instead of
mapping TCP to Tor, which was the only other option that
reveal-IP-address=False would allow).

closes ticket:2824
2016-09-14 16:27:12 -07:00
Brian Warner
02ba2a05c3 implement --listen=none, use it for create-client
Improve docs on server configuration to explain --listen options.
2016-09-14 16:12:32 -07:00
Brian Warner
aa162f5a90 close more filehandles with context managers 2016-09-13 22:17:22 -07:00
Brian Warner
a8f02d6684 enhance tests 2016-09-13 22:05:48 -07:00
David Stainton
9943a268ce Make the create-node --hide-ip option set [connections]tcp=tor 2016-09-13 10:15:45 +00:00
Brian Warner
a37b93ead9 create_node.py: refactor options 2016-09-13 01:14:48 -07:00
Brian Warner
d802135db1 test introducerless config
refs ticket:68
2016-09-12 17:45:17 -07:00
Brian Warner
4ae574cca0 test_multi_introducers: add more tests 2016-09-12 17:30:06 -07:00
David Stainton
4f717ba808 Add a multi-introducer implementation
this is based the previous work of warner, killyourtv and leif
 fixes ticket #68
2016-09-13 00:15:52 +00:00
Brian Warner
3b17289569 create_node: simplify validation, clean up tests 2016-09-09 18:53:22 -07:00
Brian Warner
229e306e9d test_runner: fix to use new conventions
create-node needs --hostname=, and we can't expect the node to write
client.port at startup (because tahoe.cfg now has tub.port= set
properly)
2016-09-09 18:53:22 -07:00
Brian Warner
38ebdfac20 create-node: reject --listen=tcp without --hostname= 2016-09-09 18:53:22 -07:00
Brian Warner
9a1a186197 simplify tests: use parse_cli, assertRaises, assertFailure
parse_cli() got added during the async-CLI-dispatch work

assertRaises/assertFailure have been in Twisted for a while, but I only
learned about them recently. Over time I'm looking forward to changing
all tahoe tests to use them (and getting rid of ShouldFailMixin/etc).
2016-09-09 18:52:01 -07:00
Brian Warner
7d3b4149ae remove leftover debug prints 2016-09-09 18:51:57 -07:00
Brian Warner
09ce7963c6 fixes 2016-09-09 17:29:20 -07:00
David Stainton
e9c1075ac5 Make corrections from Daira's code review 2016-09-09 17:29:16 -07:00
David Stainton
31dfee2dcd fix cli test create tests 2016-09-09 17:10:22 -07:00
David Stainton
83db7e8b43 Add create node args: listen, port, hostname, location
fixes ticket: 2773
2016-09-09 17:10:22 -07:00
Brian Warner
802cfc87fe CLI: allow dispatch functions to return Deferred
In addition, CLI functions are allowed to use sys.exit() instead of
always needing to return the exit code as an integer.

runner.py now knows about the blocking httplib calls in scripts/cli and
scripts/magic_folder, and uses deferToThread() to invoke them. Those
functions cannot return a Deferred: when rewrite them to use twisted.web
or treq, we'll remove this deferToThread call.

Option parsing was split out to a separate function for testing. We now
use twisted.internet.task.react() to start the reactor, which required
changing the way runner.py is tested.

closes ticket:2826
2016-09-09 15:52:42 -07:00
Brian Warner
97c29a3c0b test_runner: factor out parse_cli() helper 2016-09-09 15:41:00 -07:00
Brian Warner
7193bff48b tests: use shared run_cli()/do_cli()
A couple of test classes which defined their own flavors were changed to
use the common one.
2016-09-09 15:41:00 -07:00
Brian Warner
07e4c491f5 move run_cli() up to test/common_util.py 2016-09-09 15:40:01 -07:00
Brian Warner
442468f599 do_cli(): split out run_cli()
The main part of CLITestMixin.do_cli() was split into a standalone
function named run_cli(), leaving do_cli() as a method which includes a
nodedir in the arguments (for use by GridTestMixin tests which do a lot
of CLI operations against one of their client nodes, for which adding
the extra --nodedir argument would be ugly).
2016-09-09 15:40:01 -07:00