Commit Graph

444 Commits

Author SHA1 Message Date
ddaa29ce39 refactor/white-space MagicFolder creation 2016-10-25 12:38:57 -07:00
4f717ba808 Add a multi-introducer implementation
this is based the previous work of warner, killyourtv and leif
 fixes ticket #68
2016-09-13 00:15:52 +00:00
74dc7194b6 validate_config: improve tests
closes ticket:2809
2016-09-06 19:23:51 -07:00
2732c37941 Check for unknown config options
The list of valid sections + config-items came from
grep'ing the source for `.get_config`
2016-09-06 15:05:46 -06:00
fea8b627dd if tub.port is empty, don't listen
Updated config docs. Added errors if we're not listening but were told
to enable storage, helper, or if we're the Introducer server.

closes ticket:2816
2016-08-29 22:28:55 -07:00
cac99569e9 factor out "Tub maker"
This adds Node._create_tub(), which knows how to make a Tub with all the
right options and connection handlers that were specified in
tahoe.cfg (the connection handlers are disabled for now, but they'll get
implemented soon).

The new Node.create_main_tub() calls it. This main Tub is used:

* to connect to the Introducer
* to host the Helper (if enabled)
* to host the Storage Server (if enabled)

Node._create_tub() is also passed into the StorageFarmBroker, which
passes it into each NativeStorageServer, to create the (separate) Tub
for each server connection. _create_tub knows about the options, and
NativeStorageServer can override the connection handlers. This way we
don't need to pass tub options or default handlers into Client,
StorageFarmBroker, or NativeStorageServer.

A number of tests create NativeStorageServer objects: these were updated
to match the new arguments. test_storage_client was simplified because
we no longer need to mock out the Tub() constructor.
2016-08-27 17:36:55 -07:00
d75b9f822a Improve loading of static servers
This follows the latest comments in ticket:2788, moving the static
server definitions from "connections.yaml" to "servers.yaml". It removes
the "connections" and "introducers" blocks from that file, leaving it
responsible for just static servers (I think connections and introducers
can be configured from tahoe.cfg).

This feeds all the static server specs to the StorageFarmBroker in a
single call, rather than delivering them as simulated introducer
announcements. It cleans up the way handlers are specified too (the
handler dictionary is ignored, but that will change soon).
2016-08-26 18:05:03 -07:00
a970828b94 dicts don't have .haskey()
Apparently we don't have test coverage for this yet.
2016-08-26 13:56:50 -07:00
140c9a92cd Make StorageFarmBroker use specified or default handler 2016-08-26 11:17:56 +00:00
61eb839843 Add tub_handlers arg to NativeStorageServer constructor
Here we also define tub_handlers as a node attribute and pass it all the way
down to the NativeStorageServer via the Client and StorageFarmBroker
2016-08-22 09:15:25 -07:00
8a33fc612a Add magic-folder indicator to main welcome page
This doesn't reveal very much information, but does tell
you if magic-folder is currently working and if not it will
indicate when the last attempt to do a remote scan was.
2016-08-09 18:07:00 -06:00
1b64ab5e85 simplify when_connected_enough()
This seems happier as a method on StorageBroker, rather than a
completely separate helper class.
2016-07-21 17:23:22 -07:00
58f9641179 Correct magic-folder use of StorageFarmBroker
When landing the "connected enough" changes on master,
we changed the API; this fixes up usage of it in magic-folder.
2016-07-21 12:35:59 -07:00
86abe56d91 Flesh out "tahoe magic-folder status" command
Adds:

 - a JSON endpoint
 - CLI to display information
 - QueuedItem + IQueuedItem for uploader/downloader
 - IProgress interface + PercentProgress implementation
 - progress= args to many upload/download APIs
2016-07-21 12:35:59 -07:00
f2a7978b29 Add download.umask config option with default of 077 2016-07-21 12:35:59 -07:00
4ed2fb725a Fix call to ready() 2016-07-21 12:35:59 -07:00
e03a1e3361 WIP.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
3170cab0fc Prepare to move drop_upload.py to magic_folder.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
976c89d2e4 Rename upload_ready_d to connected_enough_d.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
61b3a9cef2 Add magic folder db.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
7b4067850f Unicode path fixes for drop-upload.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-07-21 12:35:59 -07:00
b6dedc6223 clean up got_static_announcement() call 2016-07-19 17:31:43 -07:00
2c5f7ed425 factor out yamlutil.py
The yaml.SafeLoader.add_constructor() should probably only be done once,
and moving this all into a module gives us an opportunity to test it
directly.
2016-07-19 17:31:41 -07:00
9c1e22209f client.py: add missing import, fix wrong name
Without these, clients with a non-empty connections.yaml would crash as
they start up. It's safe to say we need some tests for this :-).

pyflakes catches all of these, but it got accidentally disabled
recently, so travis wasn't running it. I'll fix that in the next commit.
2016-07-19 17:30:45 -07:00
9063106f3a Change 'servers' sections of connections.yaml to support new design in #2788 2016-07-12 17:44:21 +01:00
a67e7c0db9 Remove old self.testing reference and conditional 2016-07-12 17:44:21 +01:00
e4d93a745c Load static server configuration from connections.yaml 2016-07-12 17:44:21 +01:00
e5ffbdbcdf Move control/logging to a separate Tub 2016-07-06 21:09:35 -07:00
ae2b82a1f0 Add IntroducerClient write-only yaml cache file
this change also includes unit tests
2016-05-10 20:19:35 +00:00
d1d988410b remove "key-generator" node type and client support
closes ticket:2783
2016-05-04 16:53:04 -07:00
b9edccbeaa set the same options on all Tubs 2016-05-03 11:38:20 -07:00
6061b6fc3c WIP Storage broker client creates one tub per server
i was unable to get all the unit tests working;
this is my work in progress.
2016-05-02 15:23:07 +00:00
96c625920c Teach tub to start synchronously even with ip autodetect
* remove when_tub_ready() from all code
* synchronous-ify all node/client/introducer startup code

refs ticket:2491
2016-04-26 21:54:45 -07:00
1f6c04518e Client: init_helper() must run after init_client()
This has worked so far because everything waited for the Tub to be
ready. We'll soon be making Tub setup synchronous, so we won't have to
wait anymore, so the order will matter.
2016-04-26 18:35:03 -07:00
b834b71dac Change API to a listener-style, with helper 2016-04-26 13:40:02 -06:00
55898941da Get rid of the 'connected_threshold' argument, too
- use 'when_connected_to', which takes a threshold
2016-04-26 12:26:40 -06:00
4c619ac98d fix StorageFarmBroker constructors after connected_d gone 2016-04-26 12:26:40 -06:00
a56a3adaae Teach StorageFarmBroker to fire a deferred when a connection threshold is reached. refs #1449
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2016-04-26 12:26:39 -06:00
4a25573e2d Add simple auth-token to get JSON data 2016-04-12 12:58:27 -06:00
96eaca6a6d new feature: preferred storage servers
this includes a squash merge of dca1de6856 which
was previously seen in pull request #128, as well as daira's suggested changes
from pull request #204.
2015-12-01 19:35:41 +00:00
b6be693cbe Add a test, add missing imports. refs #2388
This tests ftpd, but not sftpd. Doing this sort of test on sftpd
requires the creation of a valid pubkey/privkey file pair, which is more
work than I want to do right now.

init_ftp/init_sftp were changed to interpret the configured
accounts.file as relative to the node's basedir, with
abspath_expanduser_unicode(accountfile, base=self.basedir).
This would happen naturally in a real node, since it os.chdir()s
to the basedir before doing anything. But tests don't do that.

Author: Brian Warner <warner@lothar.com>
Author: Daira Hopwood <daira@jacaranda.org>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-03-24 17:10:00 +00:00
3066039f0f Ensure that path parameters to SFTPServer and FTPServer constructors are unicode. refs #2388
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-03-24 16:46:40 +00:00
14f783086f Change uses of os.path.expanduser and os.path.abspath. refs #2235
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2015-01-30 00:50:18 +00:00
647ebce6b9 Better name for the file that causes a node to exit after a timeout when running unit tests. refs #1336
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-08-17 15:51:19 +01:00
4889129f37 Minor comment fix. refs #1847
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-21 22:41:23 +01:00
77767e9e12 Remove ugly shadowing of Client.DEFAULT_ENCODING_PARAMETERS. fixes #1847
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-04-21 22:41:23 +01:00
b989555d96 client.py: create node key even when storage is disabled. Closes #1945.
Thanks to Leif for the catch!
2013-04-18 00:06:55 -07:00
f14602cb34 welcome page: display new nodeid (ed25519-based) instead of old tubid
The old tubid is displayed in a tooltip. Also cleaned up the regexps in
the relevant tests.
2013-04-14 15:07:56 -07:00
e706156323 client.py: remove the old "server.privkey" fallback
For a brief while (in between releases 1.9 and 1.10, specifically from
revision bc21726 on 12-Mar-2012, until bf416af on 10-Jun-2012), the new
introducer code stored its node key in NODEDIR/private/server.privkey .
After that point, it was updated to store this key in
NODEDIR/private/node.privkey instead. Fallback code was added to read
from the old location if present (so that folks using development
versions could keep their node keys after the bf416af change).

This patch removes the fallback code. If you have a node which was run
under a version of Tahoe within this range, you need to manually update
your node by running:

 mv NODEDIR/private/server.privkey NODEDIR/private/node.privkey

and then restart the node. If you accidentally start an older node with
code after this patch, it will create a new key (and other peers will
think a new server has appeared). You can either stick with the new key,
or use the command above to switch back to the old key.

See docs/nodekeys.rst (not yet written) for details about the node key
and how it is used.
2013-04-14 14:03:34 -07:00
bda0e971f0 client.py: internal renaming, _server_key -> _node_key 2013-04-14 13:22:10 -07:00