ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2025-01-19 19:26:25 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch '2491-sync'

Browse Source 
Change the Node/Client startup process to set the Tub location during
`__init__`, rather than after the reactor starts up. This simplifies the
init process considerably, because components (like StorageServer and
IntroducerClient) can do their tub.registerReferences() immediately.

This addresses most of the runtime changes called for in ticket:2491,
but not the node-creation changes.
This commit is contained in:
Brian Warner 2016-04-27 14:29:22 -07:00
commit e187fba58e
13 changed files with 249 additions and 289 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

118
docs/configuration.rst
View File

@ -49,13 +49,22 @@ The item descriptions below use the following types:
not specified, Tahoe-LAFS will attempt to bind the port specified on all
interfaces.
``endpoint specification string``
a Twisted Endpoint specification string, like "``tcp:80``" or
"``tcp:3456:interface=127.0.0.1``". These are replacing strports strings.
For a full description of the format, see `the Twisted Endpoints
documentation`_. Please note, if interface= is not specified, Tahoe-LAFS
will attempt to bind the port specified on all interfaces. Also note that
``tub.port`` only works with TCP endpoints right now.
``FURL string``
a Foolscap endpoint identifier, like
``pb://soklj4y7eok5c3xkmjeqpw@192.168.69.247:44801/eqpwqtzm``
.. _the Twisted strports documentation: https://twistedmatrix.com/documents/current/api/twisted.application.strports.html
.. _the Twisted Endpoints documentation: http://twistedmatrix.com/documents/current/core/howto/endpoints.html#endpoint-types-included-with-twisted
Node Types
==========
@ -128,71 +137,96 @@ set the ``tub.location`` option described below.
``http://127.0.0.1:3456/static/foo.html`` will serve the contents of
``BASEDIR/public_html/foo.html`` .
``tub.port = (integer, optional)``
``tub.port = (endpoint specification string, optional)``
This controls which port the node uses to accept Foolscap connections
from other nodes. If not provided, the node will ask the kernel for any
available port. The port will be written to a separate file (named
``client.port`` or ``introducer.port``), so that subsequent runs will
re-use the same port.
from other nodes. It is parsed as a Twisted "server endpoint descriptor",
which accepts values like ``tcp:12345`` and
``tcp:23456:interface=127.0.0.1``.
For backwards compatibility, if this contains a simple integer, it will
be used as a TCP port number, like ``tcp:%d`` (which will accept
connections on all interfaces). However ``tub.port`` cannot be ``0`` or
``tcp:0`` (older versions accepted this, but the node is no longer
willing to ask Twisted to allocate port numbers in this way). To
automatically allocate a TCP port, leave ``tub.port`` blank.
If the ``tub.port`` config key is not provided, the node will look in
``BASEDIR/client.port`` (or ``BASEDIR/introducer.port``, for introducers)
for the descriptor that was used last time.
If neither is available, the node will ask the kernel for any available
port (the moral equivalent of ``tcp:0``). The allocated port number will
be written into a descriptor string in ``BASEDIR/client.port`` (or
``introducer.port``), so that subsequent runs will re-use the same port.
``tub.location = (string, optional)``
In addition to running as a client, each Tahoe-LAFS node also runs as a
server, listening for connections from other Tahoe-LAFS clients. The node
announces its location by publishing a "FURL" (a string with some
In addition to running as a client, each Tahoe-LAFS node can also run as
a server, listening for connections from other Tahoe-LAFS clients. The
node announces its location by publishing a "FURL" (a string with some
connection hints) to the Introducer. The string it publishes can be found
in ``BASEDIR/private/storage.furl`` . The ``tub.location`` configuration
controls what location is published in this announcement.
If your node is meant to run as a server, you should fill this in, using
a hostname or IP address that is reachable from your intended clients.
If you don't provide ``tub.location``, the node will try to figure out a
useful one by itself, by using tools like "``ifconfig``" to determine the
set of IP addresses on which it can be reached from nodes both near and
far. It will also include the TCP port number on which it is listening
far. It will also include the TCP port number on which it is listening
(either the one specified by ``tub.port``, or whichever port was assigned
by the kernel when ``tub.port`` is left unspecified).
by the kernel when ``tub.port`` is left unspecified). However this
automatic address-detection is discouraged, and will probably be removed
from a future release. It will include the ``127.0.0.1`` "localhost"
address (which is only useful to clients running on the same computer),
and RFC1918 private-network addresses like ``10.*.*.*`` and
``192.168.*.*`` (which are only useful to clients on the local LAN). In
general, the automatically-detected IP addresses will only be useful if
the node has a public IP address, such as a VPS or colo-hosted server.
You might want to override this value if your node lives behind a
firewall that is doing inbound port forwarding, or if you are using other
proxies such that the local IP address or port number is not the same one
that remote clients should use to connect. You might also want to control
this when using a Tor proxy to avoid revealing your actual IP address
through the Introducer announcement.
You will certainly need to set ``tub.location`` if your node lives behind
a firewall that is doing inbound port forwarding, or if you are using
other proxies such that the local IP address or port number is not the
same one that remote clients should use to connect. You might also want
to control this when using a Tor proxy to avoid revealing your actual IP
address through the Introducer announcement.
If ``tub.location`` is specified, by default it entirely replaces the
automatically determined set of IP addresses. To include the automatically
determined addresses as well as the specified ones, include the uppercase
string "``AUTO``" in the list.
The value is a comma-separated string of host:port location hints, like
this::
The value is a comma-separated string of method:host:port location hints,
like this::
123.45.67.89:8098,tahoe.example.com:8098,127.0.0.1:8098
tcp:123.45.67.89:8098,tcp:tahoe.example.com:8098,tcp:127.0.0.1:8098
A few examples:
* Emulate default behavior, assuming your host has IP address
123.45.67.89 and the kernel-allocated port number was 8098::
tub.port = 8098
tub.location = 123.45.67.89:8098,127.0.0.1:8098
* Use a DNS name so you can change the IP address more easily::
tub.port = 8098
tub.location = tahoe.example.com:8098
tub.port = tcp:8098
tub.location = tcp:tahoe.example.com:8098
* Run a node behind a firewall (which has an external IP address) that
has been configured to forward external port 7912 to our internal
node's port 8098::
tub.port = tcp:8098
tub.location = tcp:external-firewall.example.com:7912
* Emulate default behavior, assuming your host has public IP address of
123.45.67.89, and the kernel-allocated port number was 8098::
tub.port = tcp:8098
tub.location = tcp:123.45.67.89:8098,tcp:127.0.0.1:8098
* Use a DNS name but also include the default set of addresses::
tub.port = 8098
tub.location = tahoe.example.com:8098,AUTO
* Run a node behind a firewall (which has an external IP address) that
has been configured to forward port 7912 to our internal node's port
8098::
tub.port = 8098
tub.location = external-firewall.example.com:7912
tub.port = tcp:8098
tub.location = tcp:tahoe.example.com:8098,AUTO
* Run a node behind a Tor proxy (perhaps via ``torsocks``), in
client-only mode (i.e. we can make outbound connections, but other
@ -201,8 +235,8 @@ set the ``tub.location`` option described below.
reminder to human observers that this node cannot be reached. "Don't
call us.. we'll call you"::
tub.port = 8098
tub.location = unreachable.example.org:0
tub.port = tcp:8098
tub.location = tcp:unreachable.example.org:0
* Run a node behind a Tor proxy, and make the server available as a Tor
"hidden service". (This assumes that other clients are running their
@ -218,10 +252,8 @@ set the ``tub.location`` option described below.
``/var/lib/tor/hidden_services/tahoe/hostname``. Then set up your
``tahoe.cfg`` like::
tub.port = 8098
tub.location = ualhejtq2p7ohfbb.onion:29212
Most users will not need to set ``tub.location``.
tub.port = tcp:8098
tub.location = tor:ualhejtq2p7ohfbb.onion:29212
``log_gatherer.furl = (FURL, optional)``

84
src/allmydata/client.py
View File

@ -131,6 +131,8 @@ class Client(node.Node, pollmixin.PollMixin):
def __init__(self, basedir="."):
node.Node.__init__(self, basedir)
# All tub.registerReference must happen *after* we upcall, since
# that's what does tub.setLocation()
self.started_timestamp = time.time()
self.logSource="Client"
self.encoding_params = self.DEFAULT_ENCODING_PARAMETERS.copy()
@ -140,15 +142,14 @@ class Client(node.Node, pollmixin.PollMixin):
self.init_node_key()
self.init_storage()
self.init_control()
self.helper = None
if self.get_config("helper", "enabled", False, boolean=True):
self.init_helper()
self._key_generator = KeyGenerator()
key_gen_furl = self.get_config("client", "key_generator.furl", None)
if key_gen_furl:
self.init_key_gen(key_gen_furl)
self.init_client()
# ControlServer and Helper are attached after Tub startup
self.helper = None
if self.get_config("helper", "enabled", False, boolean=True):
self.init_helper()
self.init_ftp_server()
self.init_sftp_server()
self.init_drop_uploader()
@ -189,15 +190,7 @@ class Client(node.Node, pollmixin.PollMixin):
self.get_app_versions(),
self._sequencer)
self.introducer_client = ic
# hold off on starting the IntroducerClient until our tub has been
# started, so we'll have a useful address on our RemoteReference, so
# that the introducer's status page will show us.
d = self.when_tub_ready()
def _start_introducer_client(res):
ic.setServiceParent(self)
d.addCallback(_start_introducer_client)
d.addErrback(log.err, facility="tahoe.init",
level=log.BAD, umid="URyI5w")
ic.setServiceParent(self)
def init_stats_provider(self):
gatherer_furl = self.get_config("client", "stats_gatherer.furl", None)
@ -310,18 +303,12 @@ class Client(node.Node, pollmixin.PollMixin):
expiration_sharetypes=expiration_sharetypes)
self.add_service(ss)
d = self.when_tub_ready()
# we can't do registerReference until the Tub is ready
def _publish(res):
furl_file = os.path.join(self.basedir, "private", "storage.furl").encode(get_filesystem_encoding())
furl = self.tub.registerReference(ss, furlFile=furl_file)
ann = {"anonymous-storage-FURL": furl,
"permutation-seed-base32": self._init_permutation_seed(ss),
}
self.introducer_client.publish("storage", ann, self._node_key)
d.addCallback(_publish)
d.addErrback(log.err, facility="tahoe.init",
level=log.BAD, umid="aLGBKw")
furl_file = os.path.join(self.basedir, "private", "storage.furl").encode(get_filesystem_encoding())
furl = self.tub.registerReference(ss, furlFile=furl_file)
ann = {"anonymous-storage-FURL": furl,
"permutation-seed-base32": self._init_permutation_seed(ss),
}
self.introducer_client.publish("storage", ann, self._node_key)
def init_client(self):
helper_furl = self.get_config("client", "helper.furl", None)
@ -434,41 +421,26 @@ class Client(node.Node, pollmixin.PollMixin):
return self.history
def init_control(self):
d = self.when_tub_ready()
def _publish(res):
c = ControlServer()
c.setServiceParent(self)
control_url = self.tub.registerReference(c)
self.write_private_config("control.furl", control_url + "\n")
d.addCallback(_publish)
d.addErrback(log.err, facility="tahoe.init",
level=log.BAD, umid="d3tNXA")
c = ControlServer()
c.setServiceParent(self)
control_url = self.tub.registerReference(c)
self.write_private_config("control.furl", control_url + "\n")
def init_helper(self):
d = self.when_tub_ready()
def _publish(self):
self.helper = Helper(os.path.join(self.basedir, "helper"),
self.storage_broker, self._secret_holder,
self.stats_provider, self.history)
# TODO: this is confusing. BASEDIR/private/helper.furl is created
# by the helper. BASEDIR/helper.furl is consumed by the client
# who wants to use the helper. I like having the filename be the
# same, since that makes 'cp' work smoothly, but the difference
# between config inputs and generated outputs is hard to see.
helper_furlfile = os.path.join(self.basedir,
"private", "helper.furl").encode(get_filesystem_encoding())
self.tub.registerReference(self.helper, furlFile=helper_furlfile)
d.addCallback(_publish)
d.addErrback(log.err, facility="tahoe.init",
level=log.BAD, umid="K0mW5w")
self.helper = Helper(os.path.join(self.basedir, "helper"),
self.storage_broker, self._secret_holder,
self.stats_provider, self.history)
# TODO: this is confusing. BASEDIR/private/helper.furl is created by
# the helper. BASEDIR/helper.furl is consumed by the client who wants
# to use the helper. I like having the filename be the same, since
# that makes 'cp' work smoothly, but the difference between config
# inputs and generated outputs is hard to see.
helper_furlfile = os.path.join(self.basedir,
"private", "helper.furl").encode(get_filesystem_encoding())
self.tub.registerReference(self.helper, furlFile=helper_furlfile)
def init_key_gen(self, key_gen_furl):
d = self.when_tub_ready()
def _subscribe(self):
self.tub.connectTo(key_gen_furl, self._got_key_generator)
d.addCallback(_subscribe)
d.addErrback(log.err, facility="tahoe.init",
level=log.BAD, umid="z9DMzw")
self.tub.connectTo(key_gen_furl, self._got_key_generator)
def _got_key_generator(self, key_generator):
self._key_generator.set_remote_generator(key_generator)

13
src/allmydata/introducer/server.py
View File

@ -48,15 +48,10 @@ class IntroducerNode(node.Node):
"""
raise FurlFileConflictError(textwrap.dedent(msg))
os.rename(old_public_fn, private_fn)
d = self.when_tub_ready()
def _publish(res):
furl = self.tub.registerReference(introducerservice,
furlFile=private_fn)
self.log(" introducer is at %s" % furl, umid="qF2L9A")
self.introducer_url = furl # for tests
d.addCallback(_publish)
d.addErrback(log.err, facility="tahoe.init",
level=log.BAD, umid="UaNs9A")
furl = self.tub.registerReference(introducerservice,
furlFile=private_fn)
self.log(" introducer is at %s" % furl, umid="qF2L9A")
self.introducer_url = furl # for tests
def init_web(self, webport):
self.log("init_web(webport=%s)", args=(webport,), umid="2bUygA")

140
src/allmydata/node.py
View File

@ -3,13 +3,12 @@ from base64 import b32decode, b32encode
from twisted.python import log as twlog
from twisted.application import service
from twisted.internet import defer, reactor
from foolscap.api import Tub, eventually, app_versions
from foolscap.api import Tub, app_versions
import foolscap.logging.log
from allmydata import get_package_versions, get_package_versions_string
from allmydata.util import log
from allmydata.util import fileutil, iputil, observer
from allmydata.util.assertutil import precondition, _assert
from allmydata.util import fileutil, iputil
from allmydata.util.assertutil import _assert
from allmydata.util.fileutil import abspath_expanduser_unicode
from allmydata.util.encodingutil import get_filesystem_encoding, quote_output
from allmydata.util import configutil
@ -74,7 +73,6 @@ class Node(service.MultiService):
service.MultiService.__init__(self)
self.basedir = abspath_expanduser_unicode(unicode(basedir))
self._portnumfile = os.path.join(self.basedir, self.PORTNUMFILE)
self._tub_ready_observerlist = observer.OneShotObserverList()
fileutil.make_dirs(os.path.join(self.basedir, "private"), 0700)
open(os.path.join(self.basedir, "private", "README"), "w").write(PRIV_README)
@ -144,18 +142,6 @@ class Node(service.MultiService):
if os.path.exists(tahoe_cfg):
raise
cfg_tubport = self.get_config("node", "tub.port", "")
if not cfg_tubport:
# For 'tub.port', tahoe.cfg overrides the individual file on
# disk. So only read self._portnumfile if tahoe.cfg doesn't
# provide a value.
try:
file_tubport = fileutil.read(self._portnumfile).strip()
configutil.set_config(self.config, "node", "tub.port", file_tubport)
except EnvironmentError:
if os.path.exists(self._portnumfile):
raise
def error_about_old_config_files(self):
""" If any old configuration files are detected, raise OldConfigError. """
@ -175,6 +161,43 @@ class Node(service.MultiService):
twlog.msg(e)
raise e
def _convert_tub_port(self, s):
if re.search(r'^\d+$', s):
return "tcp:%d" % int(s)
return s
def get_tub_port(self):
# return a descriptor string
cfg_tubport = self.get_config("node", "tub.port", "")
if cfg_tubport:
return self._convert_tub_port(cfg_tubport)
# For 'tub.port', tahoe.cfg overrides the individual file on disk. So
# only read self._portnumfile if tahoe.cfg doesn't provide a value.
if os.path.exists(self._portnumfile):
file_tubport = fileutil.read(self._portnumfile).strip()
return self._convert_tub_port(file_tubport)
tubport = "tcp:%d" % iputil.allocate_tcp_port()
fileutil.write_atomically(self._portnumfile, tubport + "\n", mode="")
return tubport
def get_tub_location(self, tubport):
location = self.get_config("node", "tub.location", "AUTO")
# Replace the location "AUTO", if present, with the detected local
# addresses. Don't probe for local addresses unless necessary.
split_location = location.split(",")
if "AUTO" in split_location:
local_addresses = iputil.get_local_addresses_sync()
# tubport must be like "tcp:12345" or "tcp:12345:morestuff"
local_portnum = int(tubport.split(":")[1])
new_locations = []
for loc in split_location:
if loc == "AUTO":
new_locations.extend(["tcp:%s:%d" % (ip, local_portnum)
for ip in local_addresses])
else:
new_locations.append(loc)
return ",".join(new_locations)
def create_tub(self):
certfile = os.path.join(self.basedir, "private", self.CERTFILE)
self.tub = Tub(certFile=certfile)
@ -194,12 +217,16 @@ class Node(service.MultiService):
self.nodeid = b32decode(self.tub.tubID.upper()) # binary format
self.write_config("my_nodeid", b32encode(self.nodeid).lower() + "\n")
self.short_nodeid = b32encode(self.nodeid).lower()[:8] # ready for printing
tubport = self.get_config("node", "tub.port", "tcp:0")
tubport = self.get_tub_port()
if tubport in ("0", "tcp:0"):
raise ValueError("tub.port cannot be 0: you must choose")
self.tub.listenOn(tubport)
# we must wait until our service has started before we can find out
# our IP address and thus do tub.setLocation, and we can't register
# any services with the Tub until after that point
location = self.get_tub_location(tubport)
self.tub.setLocation(location)
self.log("Tub location set to %s" % (location,))
# the Tub is now ready for tub.registerReference()
self.tub.setServiceParent(self)
def get_app_versions(self):
@ -293,42 +320,13 @@ class Node(service.MultiService):
os.chmod("twistd.pid", 0644)
except EnvironmentError:
pass
# Delay until the reactor is running.
eventually(self._startService)
def _startService(self):
precondition(reactor.running)
self.log("Node._startService")
service.MultiService.startService(self)
d = defer.succeed(None)
d.addCallback(self._setup_tub)
def _ready(res):
self.log("%s running" % self.NODETYPE)
self._tub_ready_observerlist.fire(self)
return self
d.addCallback(_ready)
d.addErrback(self._service_startup_failed)
def _service_startup_failed(self, failure):
self.log('_startService() failed')
log.err(failure)
print "Node._startService failed, aborting"
print failure
#reactor.stop() # for unknown reasons, reactor.stop() isn't working. [ ] TODO
self.log('calling os.abort()')
twlog.msg('calling os.abort()') # make sure it gets into twistd.log
print "calling os.abort()"
os.abort()
self.log("%s running" % self.NODETYPE)
def stopService(self):
self.log("Node.stopService")
d = self._tub_ready_observerlist.when_fired()
def _really_stopService(ignored):
self.log("Node._really_stopService")
return service.MultiService.stopService(self)
d.addCallback(_really_stopService)
return d
return service.MultiService.stopService(self)
def shutdown(self):
"""Shut down the node. Returns a Deferred that fires (with None) when
@ -364,42 +362,6 @@ class Node(service.MultiService):
def log(self, *args, **kwargs):
return log.msg(*args, **kwargs)
def _setup_tub(self, ign):
# we can't get a dynamically-assigned portnum until our Tub is
# running, which means after startService.
l = self.tub.getListeners()[0]
portnum = l.getPortnum()
# record which port we're listening on, so we can grab the same one
# next time
fileutil.write_atomically(self._portnumfile, "%d\n" % portnum, mode="")
location = self.get_config("node", "tub.location", "AUTO")
# Replace the location "AUTO", if present, with the detected local addresses.
split_location = location.split(",")
if "AUTO" in split_location:
d = iputil.get_local_addresses_async()
def _add_local(local_addresses):
while "AUTO" in split_location:
split_location.remove("AUTO")
split_location.extend([ "%s:%d" % (addr, portnum)
for addr in local_addresses ])
return ",".join(split_location)
d.addCallback(_add_local)
else:
d = defer.succeed(location)
def _got_location(location):
self.log("Tub location set to %s" % (location,))
self.tub.setLocation(location)
return self.tub
d.addCallback(_got_location)
return d
def when_tub_ready(self):
return self._tub_ready_observerlist.when_fired()
def add_service(self, s):
s.setServiceParent(self)
return s

9
src/allmydata/stats.py
View File

@ -148,12 +148,9 @@ class StatsProvider(Referenceable, service.MultiService):
def startService(self):
if self.node and self.gatherer_furl:
d = self.node.when_tub_ready()
def connect(junk):
nickname_utf8 = self.node.nickname.encode("utf-8")
self.node.tub.connectTo(self.gatherer_furl,
self._connected, nickname_utf8)
d.addCallback(connect)
nickname_utf8 = self.node.nickname.encode("utf-8")
self.node.tub.connectTo(self.gatherer_furl,
self._connected, nickname_utf8)
service.MultiService.startService(self)
def count(self, name, delta=1):

13
src/allmydata/test/check_memory.py
View File

@ -116,10 +116,8 @@ class SystemFramework(pollmixin.PollMixin):
#print "STARTING"
self.stats = {}
self.statsfile = open(os.path.join(self.basedir, "stats.out"), "a")
d = self.make_introducer()
def _more(res):
return self.start_client()
d.addCallback(_more)
self.make_introducer()
d = self.start_client()
def _record_control_furl(control_furl):
self.control_furl = control_furl
#print "OBTAINING '%s'" % (control_furl,)
@ -174,12 +172,7 @@ class SystemFramework(pollmixin.PollMixin):
os.mkdir(iv_basedir)
iv = introducer.IntroducerNode(basedir=iv_basedir)
self.introducer = self.add_service(iv)
d = self.introducer.when_tub_ready()
def _introducer_ready(res):
q = self.introducer
self.introducer_furl = q.introducer_url
d.addCallback(_introducer_ready)
return d
self.introducer_furl = self.introducer.introducer_url
def make_nodes(self):
self.nodes = []

45
src/allmydata/test/common.py
View File

@ -481,8 +481,8 @@ class SystemTestMixin(pollmixin.PollMixin, testutil.StallMixin):
f.close()
iv = IntroducerNode(basedir=iv_dir)
self.introducer = self.add_service(iv)
d = self.introducer.when_tub_ready()
d.addCallback(self._get_introducer_web)
self._get_introducer_web()
d = defer.succeed(None)
if use_stats_gatherer:
d.addCallback(self._set_up_stats_gatherer)
if use_key_generator:
@ -492,7 +492,7 @@ class SystemTestMixin(pollmixin.PollMixin, testutil.StallMixin):
d.addCallback(self._grab_stats)
return d
def _get_introducer_web(self, res):
def _get_introducer_web(self):
f = open(os.path.join(self.getdir("introducer"), "node.url"), "r")
self.introweb_url = f.read().strip()
f.close()
@ -591,27 +591,25 @@ class SystemTestMixin(pollmixin.PollMixin, testutil.StallMixin):
c = self.add_service(client.Client(basedir=basedirs[0]))
self.clients.append(c)
c.set_default_mutable_keysize(TEST_RSA_KEY_SIZE)
d = c.when_tub_ready()
def _ready(res):
f = open(os.path.join(basedirs[0],"private","helper.furl"), "r")
helper_furl = f.read()
f.close()
self.helper_furl = helper_furl
if self.numclients >= 4:
f = open(os.path.join(basedirs[3], 'tahoe.cfg'), 'ab+')
f.write(
"[client]\n"
"helper.furl = %s\n" % helper_furl)
f.close()
# this starts the rest of the clients
for i in range(1, self.numclients):
c = self.add_service(client.Client(basedir=basedirs[i]))
self.clients.append(c)
c.set_default_mutable_keysize(TEST_RSA_KEY_SIZE)
log.msg("STARTING")
return self.wait_for_connections()
d.addCallback(_ready)
f = open(os.path.join(basedirs[0],"private","helper.furl"), "r")
helper_furl = f.read()
f.close()
self.helper_furl = helper_furl
if self.numclients >= 4:
f = open(os.path.join(basedirs[3], 'tahoe.cfg'), 'ab+')
f.write(
"[client]\n"
"helper.furl = %s\n" % helper_furl)
f.close()
# this starts the rest of the clients
for i in range(1, self.numclients):
c = self.add_service(client.Client(basedir=basedirs[i]))
self.clients.append(c)
c.set_default_mutable_keysize(TEST_RSA_KEY_SIZE)
log.msg("STARTING")
d = self.wait_for_connections()
def _connected(res):
log.msg("CONNECTED")
# now find out where the web port was
@ -643,7 +641,6 @@ class SystemTestMixin(pollmixin.PollMixin, testutil.StallMixin):
self.clients[num] = new_c
new_c.set_default_mutable_keysize(TEST_RSA_KEY_SIZE)
self.add_service(new_c)
return new_c.when_tub_ready()
d.addCallback(_stopped)
d.addCallback(lambda res: self.wait_for_connections())
def _maybe_get_webport(res):

2
src/allmydata/test/no_network.py
View File

@ -184,8 +184,6 @@ class NoNetworkClient(Client):
service.MultiService.startService(self)
def stopService(self):
service.MultiService.stopService(self)
def when_tub_ready(self):
raise NotImplementedError("NoNetworkClient has no Tub")
def init_control(self):
pass
def init_helper(self):

65
src/allmydata/test/test_introducer.py
View File

@ -34,45 +34,33 @@ class Node(testutil.SignalMixin, testutil.ReallyEqualMixin, unittest.TestCase):
os.mkdir(basedir)
public_fn = os.path.join(basedir, "introducer.furl")
private_fn = os.path.join(basedir, "private", "introducer.furl")
q1 = IntroducerNode(basedir)
d = fireEventually(None)
d.addCallback(lambda res: q1.startService())
d.addCallback(lambda res: q1.when_tub_ready())
d.addCallback(lambda res: q1.stopService())
d.addCallback(flushEventualQueue)
def _check_furl(res):
# new nodes create unguessable furls in private/introducer.furl
ifurl = fileutil.read(private_fn)
self.failUnless(ifurl)
ifurl = ifurl.strip()
self.failIf(ifurl.endswith("/introducer"), ifurl)
del q1
# new nodes create unguessable furls in private/introducer.furl
ifurl = fileutil.read(private_fn)
self.failUnless(ifurl)
ifurl = ifurl.strip()
self.failIf(ifurl.endswith("/introducer"), ifurl)
# old nodes created guessable furls in BASEDIR/introducer.furl
guessable = ifurl[:ifurl.rfind("/")] + "/introducer"
fileutil.write(public_fn, guessable+"\n", mode="w") # text
# old nodes created guessable furls in BASEDIR/introducer.furl
guessable = ifurl[:ifurl.rfind("/")] + "/introducer"
fileutil.write(public_fn, guessable+"\n", mode="w") # text
# if we see both files, throw an error
self.failUnlessRaises(FurlFileConflictError,
IntroducerNode, basedir)
# if we see both files, throw an error
self.failUnlessRaises(FurlFileConflictError,
IntroducerNode, basedir)
# when we see only the public one, move it to private/ and use
# the existing furl instead of creating a new one
os.unlink(private_fn)
q2 = IntroducerNode(basedir)
d2 = fireEventually(None)
d2.addCallback(lambda res: q2.startService())
d2.addCallback(lambda res: q2.when_tub_ready())
d2.addCallback(lambda res: q2.stopService())
d2.addCallback(flushEventualQueue)
def _check_furl2(res):
self.failIf(os.path.exists(public_fn))
ifurl2 = fileutil.read(private_fn)
self.failUnless(ifurl2)
self.failUnlessEqual(ifurl2.strip(), guessable)
d2.addCallback(_check_furl2)
return d2
d.addCallback(_check_furl)
return d
# when we see only the public one, move it to private/ and use
# the existing furl instead of creating a new one
os.unlink(private_fn)
q2 = IntroducerNode(basedir)
del q2
self.failIf(os.path.exists(public_fn))
ifurl2 = fileutil.read(private_fn)
self.failUnless(ifurl2)
self.failUnlessEqual(ifurl2.strip(), guessable)
def test_web_static(self):
basedir = u"introducer.Node.test_web_static"
@ -1014,10 +1002,17 @@ class ClientSeqnums(unittest.TestCase):
def test_client(self):
basedir = "introducer/ClientSeqnums/test_client"
fileutil.make_dirs(basedir)
# if storage is enabled, the Client will publish its storage server
# during startup (although the announcement will wait in a queue
# until the introducer connection is established). To avoid getting
# confused by this, disable storage.
f = open(os.path.join(basedir, "tahoe.cfg"), "w")
f.write("[client]\n")
f.write("introducer.furl = nope\n")
f.write("[storage]\n")
f.write("enabled = false\n")
f.close()
c = TahoeClient(basedir)
ic = c.introducer_client
outbound = ic._outbound_announcements

21
src/allmydata/test/test_node.py
View File

@ -47,19 +47,14 @@ class TestCase(testutil.SignalMixin, unittest.TestCase):
f.close()
if local_addresses:
self.patch(iputil, 'get_local_addresses_async', lambda target=None: defer.succeed(local_addresses))
self.patch(iputil, 'get_local_addresses_sync',
lambda: local_addresses)
n = TestNode(basedir)
n.setServiceParent(self.parent)
d = n.when_tub_ready()
def _check_addresses(ignored_result):
furl = n.tub.registerReference(n)
for address in expected_addresses:
self.failUnlessIn(address, furl)
d.addCallback(_check_addresses)
return d
furl = n.tub.registerReference(n)
for address in expected_addresses:
self.failUnlessIn(address, furl)
def test_location1(self):
return self._test_location(basedir="test_node/test_location1",
@ -96,10 +91,8 @@ class TestCase(testutil.SignalMixin, unittest.TestCase):
n = TestNode(basedir)
n.setServiceParent(self.parent)
d = n.when_tub_ready()
d.addCallback(lambda ign: self.failUnlessEqual(n.get_config("node", "nickname").decode('utf-8'),
u"\u2621"))
return d
self.failUnlessEqual(n.get_config("node", "nickname").decode('utf-8'),
u"\u2621")
def test_tahoe_cfg_hash_in_name(self):
basedir = "test_node/test_cfg_hash_in_name"

1
src/allmydata/test/test_web.py
View File

@ -4465,7 +4465,6 @@ class IntroducerWeb(unittest.TestCase):
d = fireEventually(None)
d.addCallback(lambda ign: self.node.startService())
d.addCallback(lambda ign: self.node.when_tub_ready())
d.addCallback(lambda ign: self.GET("/"))
def _check(res):

2
src/allmydata/util/iputil.py
View File

@ -71,6 +71,8 @@ except ImportError:
# since one might be shadowing the other. This hack appeases pyflakes.
increase_rlimits = _increase_rlimits
def get_local_addresses_sync():
return _synchronously_find_addresses_via_config()
def get_local_addresses_async(target="198.41.0.4"): # A.ROOT-SERVERS.NET
"""

25
topfiles/2491.config Normal file
View File

@ -0,0 +1,25 @@
* tub.port is now an Endpoint server specification string (which is pretty
much just like a strports string, but can be extended by plugins). It now
rejects "tcp:0" and "0". The tahoe.cfg value overrides anything stored on
disk (in client.port). This should have no effect on most nodes, which do
not set tub.port in tahoe.cfg, and wrote an allocated port number to
client.port the first time they launched. Folks who want to listen on a
specific port number typically set tub.port to "tcp:12345" or "12345", not
"0".
* The "portnumfile" (e.g. NODEDIR/client.port) is written as soon as the port
is allocated, before the tub is created, and only if "tub.port" was empty.
The old code wrote to it unconditionally, and after Tub startup. So if the
user allows NODEDIR/client.port to be written, then later modifies
tahoe.cfg to set "tub.port" to a different value, this difference will
persist (and the node will honor tahoe.cfg "tub.port" exclusively).
* We now encourage static allocation of tub.port, and pre-configuration of
the node's externally-reachable IP address or hostname (by setting
tub.location). Automatic IP-address detection is deprecated. Automatic port
allocation is merely discouraged. Eventually both will be managed by "tahoe
create-node", but for now we recommend users edit their tahoe.cfg after
node creation and before first launch.
* "tahoe start" now creates the Tub, and all primary software components,
before the child process daemonizes. Many configuration errors which would
previously have been reported in a logfile (after node startup), will now
be signalled immediately, via stderr. In these cases, the "tahoe start"
process will exit with a non-zero return code.