ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2025-02-21 02:01:31 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into 2916.grid-manager-proposal.5

Browse Source
This commit is contained in:
meejah 2020-12-01 14:32:31 -07:00
commit 4b5db76b53
12 changed files with 1316 additions and 793 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/specifications/index.rst
View File

@ -8,6 +8,7 @@ the data formats used by Tahoe.
:maxdepth: 2
outline
url
uri
file-encoding
URI-extension

165
docs/specifications/url.rst Normal file
View File

@ -0,0 +1,165 @@
URLs
====
The goal of this document is to completely specify the construction and use of the URLs by Tahoe-LAFS for service location.
This includes, but is not limited to, the original Foolscap-based URLs.
These are not to be confused with the URI-like capabilities Tahoe-LAFS uses to refer to stored data.
An attempt is also made to outline the rationale for certain choices about these URLs.
The intended audience for this document is Tahoe-LAFS maintainers and other developers interested in interoperating with Tahoe-LAFS or these URLs.
Background
----------
Tahoe-LAFS first used Foolscap_ for network communication.
Foolscap connection setup takes as an input a Foolscap URL or a *fURL*.
A fURL includes three components:
* the base32-encoded SHA1 hash of the DER form of an x509v3 certificate
* zero or more network addresses [1]_
* an object identifier
A Foolscap client tries to connect to each network address in turn.
If a connection is established then TLS is negotiated.
The server is authenticated by matching its certificate against the hash in the fURL.
A matching certificate serves as proof that the handshaking peer is the correct server.
This serves as the process by which the client authenticates the server.
The client can then exercise further Foolscap functionality using the fURL's object identifier.
If the object identifier is an unguessable, secret string then it serves as a capability.
This unguessable identifier is sometimes called a `swiss number`_ (or swissnum).
The client's use of the swissnum is what allows the server to authorize the client.
.. _`swiss number`: http://wiki.erights.org/wiki/Swiss_number
NURLs
-----
The authentication and authorization properties of fURLs are a good fit for Tahoe-LAFS' requirements.
These are not inherently tied to the Foolscap protocol itself.
In particular they are beneficial to :doc:`../proposed/http-storage-node-protocol` which uses HTTP instead of Foolscap.
It is conceivable they will also be used with WebSockets at some point as well.
Continuing to refer to these URLs as fURLs when they are being used for other protocols may cause confusion.
Therefore,
this document coins the name **NURL** for these URLs.
This can be considered to expand to "**N**\ ew URLs" or "Authe\ **N**\ ticating URLs" or "Authorizi\ **N**\ g URLs" as the reader prefers.
The anticipated use for a **NURL** will still be to establish a TLS connection to a peer.
The protocol run over that TLS connection could be Foolscap though it is more likely to be an HTTP-based protocol (such as GBS).
Syntax
------
The EBNF for a NURL is as follows::
nurl = scheme, hash, "@", net-loc-list, "/", swiss-number, [ version1 ]
scheme = "pb://"
hash = unreserved
net-loc-list = net-loc, [ { ",", net-loc } ]
net-loc = tcp-loc | tor-loc | i2p-loc
tcp-loc = [ "tcp:" ], hostname, [ ":" port ]
tor-loc = "tor:", hostname, [ ":" port ]
i2p-loc = "i2p:", i2p-addr, [ ":" port ]
i2p-addr = { unreserved }, ".i2p"
hostname = domain | IPv4address | IPv6address
swiss-number = segment
version1 = "#v=1"
See https://tools.ietf.org/html/rfc3986#section-3.3 for the definition of ``segment``.
See https://tools.ietf.org/html/rfc2396#appendix-A for the definition of ``unreserved``.
See https://tools.ietf.org/html/draft-main-ipaddr-text-rep-02#section-3.1 for the definition of ``IPv4address``.
See https://tools.ietf.org/html/draft-main-ipaddr-text-rep-02#section-3.2 for the definition of ``IPv6address``.
See https://tools.ietf.org/html/rfc1035#section-2.3.1 for the definition of ``domain``.
Versions
--------
Though all NURLs are syntactically compatible some semantic differences are allowed.
These differences are separated into distinct versions.
Version 0
---------
A Foolscap fURL is considered the canonical definition of a version 0 NURL.
Notably,
the hash component is defined as the base32-encoded SHA1 hash of the DER form of an x509v3 certificate.
A version 0 NURL is identified by the absence of the ``v=1`` fragment.
Examples
~~~~~~~~
* ``pb://sisi4zenj7cxncgvdog7szg3yxbrnamy@tcp:127.1:34399/xphmwz6lx24rh2nxlinni``
* ``pb://2uxmzoqqimpdwowxr24q6w5ekmxcymby@localhost:47877/riqhpojvzwxujhna5szkn``
Version 1
---------
The hash component of a version 1 NURL differs in three ways from the prior version.
1. The hash function used is SHA3-224 instead of SHA1.
The security of SHA1 `continues to be eroded`_.
Contrariwise SHA3 is currently the most recent addition to the SHA family by NIST.
The 224 bit instance is chosen to keep the output short and because it offers greater collision resistance than SHA1 was thought to offer even at its inception
(prior to security research showing actual collision resistance is lower).
2. The hash is computed over the certificate's SPKI instead of the whole certificate.
This allows certificate re-generation so long as the public key remains the same.
This is useful to allow contact information to be updated or extension of validity period.
Use of an SPKI hash has also been `explored by the web community`_ during its flirtation with using it for HTTPS certificate pinning
(though this is now largely abandoned).
.. note::
*Only* the certificate's keypair is pinned by the SPKI hash.
The freedom to change every other part of the certificate is coupled with the fact that all other parts of the certificate contain arbitrary information set by the private key holder.
It is neither guaranteed nor expected that a certificate-issuing authority has validated this information.
Therefore,
*all* certificate fields should be considered within the context of the relationship identified by the SPKI hash.
3. The hash is encoded using urlsafe-base64 (without padding) instead of base32.
This provides a more compact representation and minimizes the usability impacts of switching from a 160 bit hash to a 224 bit hash.
A version 1 NURL is identified by the presence of the ``v=1`` fragment.
Though the length of the hash string (38 bytes) could also be used to differentiate it from a version 0 NURL,
there is no guarantee that this will be effective in differentiating it from future versions so this approach should not be used.
It is possible for a client to unilaterally upgrade a version 0 NURL to a version 1 NURL.
After establishing and authenticating a connection the client will have received a copy of the server's certificate.
This is sufficient to compute the new hash and rewrite the NURL to upgrade it to version 1.
This provides stronger authentication assurances for future uses but it is not required.
Examples
~~~~~~~~
* ``pb://1WUX44xKjKdpGLohmFcBNuIRN-8rlv1Iij_7rQ@tcp:127.1:34399/jhjbc3bjbhk#v=1``
* ``pb://azEu8vlRpnEeYm0DySQDeNY3Z2iJXHC_bsbaAw@localhost:47877/64i4aokv4ej#v=1``
.. _`continues to be eroded`: https://en.wikipedia.org/wiki/SHA-1#Cryptanalysis_and_validation
.. _`explored by the web community`: https://www.imperialviolet.org/2011/05/04/pinning.html
.. _Foolscap: https://github.com/warner/foolscap
.. [1] ``foolscap.furl.decode_furl`` is taken as the canonical definition of the syntax of a fURL.
The **location hints** part of the fURL,
as it is referred to in Foolscap,
is matched by the regular expression fragment ``([^/]*)``.
Since this matches the empty string,
no network addresses are required to form a fURL.
The supporting code around the regular expression also takes extra steps to allow an empty string to match here.
Open Questions
--------------
1. Should we make a hard recommendation that all certificate fields are ignored?
The system makes no guarantees about validation of these fields.
Is it just an unnecessary risk to let a user see them?
2. Should the version specifier be a query-arg-alike or a fragment-alike?
The value is only necessary on the client side which makes it similar to an HTTP URL fragment.
The current Tahoe-LAFS configuration parsing code has special handling of the fragment character (``#``) which makes it unusable.
However,
the configuration parsing code is easily changed.

1440
integration/install-tor.sh
View File

File diff suppressed because it is too large Load Diff

5
integration/util.py
View File

@ -6,6 +6,9 @@ from os.path import exists, join
from six.moves import StringIO
from functools import partial
from twisted.python.filepath import (
FilePath,
)
from twisted.internet.defer import Deferred, succeed
from twisted.internet.protocol import ProcessProtocol
from twisted.internet.error import ProcessExitedAlready, ProcessDone
@ -263,7 +266,7 @@ def _create_node(reactor, request, temp_dir, introducer_furl, flog_gatherer, nam
u'log_gatherer.furl',
flog_gatherer.decode("utf-8"),
)
write_config(config_path, config)
write_config(FilePath(config_path), config)
created_d.addCallback(created)
d = Deferred()

1
newsfragments/3503.other Normal file
View File

@ -0,0 +1 @@
The specification section of the Tahoe-LAFS documentation now includes explicit discussion of the security properties of Foolscap "fURLs" on which it depends.

0
newsfragments/3511.minor Normal file
View File

0
newsfragments/3542.minor Normal file
View File

156
src/allmydata/node.py
View File

@ -22,9 +22,14 @@ import types
import errno
from base64 import b32decode, b32encode
import attr
# On Python 2 this will be the backported package.
import configparser
from twisted.python.filepath import (
FilePath,
)
from twisted.python import log as twlog
from twisted.application import service
from twisted.python.failure import Failure
@ -191,25 +196,27 @@ def read_config(basedir, portnumfile, generated_files=[], _valid_config=None):
# canonicalize the portnum file
portnumfile = os.path.join(basedir, portnumfile)
# (try to) read the main config file
config_fname = os.path.join(basedir, "tahoe.cfg")
config_path = FilePath(basedir).child("tahoe.cfg")
try:
parser = configutil.get_config(config_fname)
config_str = config_path.getContent()
except EnvironmentError as e:
if e.errno != errno.ENOENT:
raise
# The file is missing, just create empty ConfigParser.
parser = configutil.get_config_from_string(u"")
config_str = u""
else:
config_str = config_str.decode("utf-8-sig")
configutil.validate_config(config_fname, parser, _valid_config)
# make sure we have a private configuration area
fileutil.make_dirs(os.path.join(basedir, "private"), 0o700)
return _Config(parser, portnumfile, basedir, config_fname)
return config_from_string(
basedir,
portnumfile,
config_str,
_valid_config,
config_path,
)
def config_from_string(basedir, portnumfile, config_str, _valid_config=None):
def config_from_string(basedir, portnumfile, config_str, _valid_config=None, fpath=None):
"""
load and validate configuration from in-memory string
"""
@ -222,14 +229,19 @@ def config_from_string(basedir, portnumfile, config_str, _valid_config=None):
# load configuration from in-memory string
parser = configutil.get_config_from_string(config_str)
fname = "<in-memory>"
configutil.validate_config(fname, parser, _valid_config)
configutil.validate_config(
"<string>" if fpath is None else fpath.path,
parser,
_valid_config,
)
def write_new_config(cfg):
"""
We throw away any attempt to persist
"""
return _Config(parser, portnumfile, basedir, fname, write_new_config)
return _Config(
parser,
portnumfile,
basedir,
fpath,
_valid_config,
)
def _error_about_old_config_files(basedir, generated_files):
@ -257,6 +269,7 @@ def _error_about_old_config_files(basedir, generated_files):
raise e
@attr.s
class _Config(object):
"""
Manages configuration of a Tahoe 'node directory'.
@ -265,48 +278,47 @@ class _Config(object):
class; names and funtionality have been kept the same while moving
the code. It probably makes sense for several of these APIs to
have better names.
:ivar ConfigParser config: The actual configuration values.
:ivar str portnum_fname: filename to use for the port-number file (a
relative path inside basedir).
:ivar str _basedir: path to our "node directory", inside which all
configuration is managed.
:ivar (FilePath|NoneType) config_path: The path actually used to create
the configparser (might be ``None`` if using in-memory data).
:ivar ValidConfiguration valid_config_sections: The validator for the
values in this configuration.
"""
config = attr.ib(validator=attr.validators.instance_of(configparser.ConfigParser))
portnum_fname = attr.ib()
_basedir = attr.ib(
converter=lambda basedir: abspath_expanduser_unicode(ensure_text(basedir)),
)
config_path = attr.ib(
validator=attr.validators.optional(
attr.validators.instance_of(FilePath),
),
)
valid_config_sections = attr.ib(
default=configutil.ValidConfiguration.everything(),
validator=attr.validators.instance_of(configutil.ValidConfiguration),
)
def __init__(self, configparser, portnum_fname, basedir, config_fname,
write_new_tahoecfg=None):
"""
:param configparser: a ConfigParser instance
@property
def nickname(self):
nickname = self.get_config("node", "nickname", u"<unspecified>")
assert isinstance(nickname, str)
return nickname
:param portnum_fname: filename to use for the port-number file
(a relative path inside basedir)
:param basedir: path to our "node directory", inside which all
configuration is managed
:param config_fname: the pathname actually used to create the
configparser (might be 'fake' if using in-memory data)
:param write_new_tahoecfg: callable taking one argument which
is a ConfigParser instance
"""
self.portnum_fname = portnum_fname
self._basedir = abspath_expanduser_unicode(ensure_text(basedir))
self._config_fname = config_fname
self.config = configparser
if write_new_tahoecfg is None:
def write_new_tahoecfg(config):
"""
Write to the default place, <basedir>/tahoe.cfg
"""
fn = os.path.join(self._basedir, "tahoe.cfg")
with open(fn, "w") as f:
config.write(f)
self._write_config = write_new_tahoecfg
self.nickname = self.get_config("node", "nickname", u"<unspecified>")
assert isinstance(self.nickname, str)
def validate(self, valid_config_sections):
configutil.validate_config(self._config_fname, self.config, valid_config_sections)
@property
def _config_fname(self):
if self.config_path is None:
return "<string>"
return self.config_path.path
def write_config_file(self, name, value, mode="w"):
"""
@ -366,17 +378,31 @@ class _Config(object):
def set_config(self, section, option, value):
"""
Set a config options in a section and re-write the tahoe.cfg file
Set a config option in a section and re-write the tahoe.cfg file
:param str section: The name of the section in which to set the
option.
:param str option: The name of the option to set.
:param str value: The value of the option.
:raise UnescapedHashError: If the option holds a fURL and there is a
``#`` in the value.
"""
if option.endswith(".furl") and self._contains_unescaped_hash(value):
if option.endswith(".furl") and "#" in value:
raise UnescapedHashError(section, option, value)
try:
self.config.add_section(section)
except configparser.DuplicateSectionError:
pass
self.config.set(section, option, value)
self._write_config(self.config)
copied_config = configutil.copy_config(self.config)
configutil.set_config(copied_config, section, option, value)
configutil.validate_config(
self._config_fname,
copied_config,
self.valid_config_sections,
)
if self.config_path is not None:
configutil.write_config(self.config_path, copied_config)
self.config = copied_config
def get_config_from_file(self, name, required=False):
"""Get the (string) contents of a config file, or None if the file

7
src/allmydata/test/cli/test_create.py
View File

@ -52,13 +52,8 @@ class Config(unittest.TestCase):
create_node.write_node_config(f, opts)
create_node.write_client_config(f, opts)
config = configutil.get_config(fname)
# should succeed, no exceptions
configutil.validate_config(
fname,
config,
client._valid_config(),
)
client.read_config(d, "")
@defer.inlineCallbacks
def test_client(self):

186
src/allmydata/test/test_configutil.py
View File

@ -14,12 +14,89 @@ if PY2:
from builtins import filter, map, zip, ascii, chr, hex, input, next, oct, open, pow, round, super, bytes, list, object, range, str, max, min # noqa: F401
import os.path
from configparser import (
ConfigParser,
)
from functools import (
partial,
)
from hypothesis import (
given,
)
from hypothesis.strategies import (
dictionaries,
text,
characters,
)
from twisted.python.filepath import (
FilePath,
)
from twisted.trial import unittest
from allmydata.util import configutil
def arbitrary_config_dicts(
min_sections=0,
max_sections=3,
max_section_name_size=8,
max_items_per_section=3,
max_item_length=8,
max_value_length=8,
):
"""
Build ``dict[str, dict[str, str]]`` instances populated with arbitrary
configurations.
"""
identifier_text = partial(
text,
# Don't allow most control characters or spaces
alphabet=characters(
blacklist_categories=('Cc', 'Cs', 'Zs'),
),
)
return dictionaries(
identifier_text(
min_size=1,
max_size=max_section_name_size,
),
dictionaries(
identifier_text(
min_size=1,
max_size=max_item_length,
),
text(max_size=max_value_length),
max_size=max_items_per_section,
),
min_size=min_sections,
max_size=max_sections,
)
def to_configparser(dictconfig):
"""
Take a ``dict[str, dict[str, str]]`` and turn it into the corresponding
populated ``ConfigParser`` instance.
"""
cp = ConfigParser()
for section, items in dictconfig.items():
cp.add_section(section)
for k, v in items.items():
cp.set(
section,
k,
# ConfigParser has a feature that everyone knows and loves
# where it will use %-style interpolation to substitute
# values from one part of the config into another part of
# the config. Escape all our `%`s to avoid hitting this
# and complicating things.
v.replace("%", "%%"),
)
return cp
class ConfigUtilTests(unittest.TestCase):
def setUp(self):
super(ConfigUtilTests, self).setUp()
@ -55,7 +132,7 @@ enabled = false
# test that set_config can mutate an existing option
configutil.set_config(config, "node", "nickname", "Alice!")
configutil.write_config(tahoe_cfg, config)
configutil.write_config(FilePath(tahoe_cfg), config)
config = configutil.get_config(tahoe_cfg)
self.failUnlessEqual(config.get("node", "nickname"), "Alice!")
@ -63,19 +140,21 @@ enabled = false
# test that set_config can set a new option
descriptor = "Twas brillig, and the slithy toves Did gyre and gimble in the wabe"
configutil.set_config(config, "node", "descriptor", descriptor)
configutil.write_config(tahoe_cfg, config)
configutil.write_config(FilePath(tahoe_cfg), config)
config = configutil.get_config(tahoe_cfg)
self.failUnlessEqual(config.get("node", "descriptor"), descriptor)
def test_config_validation_success(self):
fname = self.create_tahoe_cfg('[node]\nvalid = foo\n')
config = configutil.get_config(fname)
"""
``configutil.validate_config`` returns ``None`` when the configuration it
is given has nothing more than the static sections and items defined
by the validator.
"""
# should succeed, no exceptions
configutil.validate_config(
fname,
config,
"<test_config_validation_success>",
to_configparser({"node": {"valid": "foo"}}),
self.static_valid_config,
)
@ -85,24 +164,20 @@ enabled = false
validation but are matched by the dynamic validation is considered
valid.
"""
fname = self.create_tahoe_cfg('[node]\nvalid = foo\n')
config = configutil.get_config(fname)
# should succeed, no exceptions
configutil.validate_config(
fname,
config,
"<test_config_dynamic_validation_success>",
to_configparser({"node": {"valid": "foo"}}),
self.dynamic_valid_config,
)
def test_config_validation_invalid_item(self):
fname = self.create_tahoe_cfg('[node]\nvalid = foo\ninvalid = foo\n')
config = configutil.get_config(fname)
config = to_configparser({"node": {"valid": "foo", "invalid": "foo"}})
e = self.assertRaises(
configutil.UnknownConfigError,
configutil.validate_config,
fname, config,
"<test_config_validation_invalid_item>",
config,
self.static_valid_config,
)
self.assertIn("section [node] contains unknown option 'invalid'", str(e))
@ -112,13 +187,12 @@ enabled = false
A configuration with a section that is matched by neither the static nor
dynamic validators is rejected.
"""
fname = self.create_tahoe_cfg('[node]\nvalid = foo\n[invalid]\n')
config = configutil.get_config(fname)
config = to_configparser({"node": {"valid": "foo"}, "invalid": {}})
e = self.assertRaises(
configutil.UnknownConfigError,
configutil.validate_config,
fname, config,
"<test_config_validation_invalid_section>",
config,
self.static_valid_config,
)
self.assertIn("contains unknown section [invalid]", str(e))
@ -128,13 +202,12 @@ enabled = false
A configuration with a section that is matched by neither the static nor
dynamic validators is rejected.
"""
fname = self.create_tahoe_cfg('[node]\nvalid = foo\n[invalid]\n')
config = configutil.get_config(fname)
config = to_configparser({"node": {"valid": "foo"}, "invalid": {}})
e = self.assertRaises(
configutil.UnknownConfigError,
configutil.validate_config,
fname, config,
"<test_config_dynamic_validation_invalid_section>",
config,
self.dynamic_valid_config,
)
self.assertIn("contains unknown section [invalid]", str(e))
@ -144,13 +217,12 @@ enabled = false
A configuration with a section, item pair that is matched by neither the
static nor dynamic validators is rejected.
"""
fname = self.create_tahoe_cfg('[node]\nvalid = foo\ninvalid = foo\n')
config = configutil.get_config(fname)
config = to_configparser({"node": {"valid": "foo", "invalid": "foo"}})
e = self.assertRaises(
configutil.UnknownConfigError,
configutil.validate_config,
fname, config,
"<test_config_dynamic_validation_invalid_item>",
config,
self.dynamic_valid_config,
)
self.assertIn("section [node] contains unknown option 'invalid'", str(e))
@ -163,3 +235,61 @@ enabled = false
config = configutil.get_config(fname)
self.assertEqual(config.get("node", "a"), "foo")
self.assertEqual(config.get("node", "b"), "bar")
@given(arbitrary_config_dicts())
def test_everything_valid(self, cfgdict):
"""
``validate_config`` returns ``None`` when the validator is
``ValidConfiguration.everything()``.
"""
cfg = to_configparser(cfgdict)
self.assertIs(
configutil.validate_config(
"<test_everything_valid>",
cfg,
configutil.ValidConfiguration.everything(),
),
None,
)
@given(arbitrary_config_dicts(min_sections=1))
def test_nothing_valid(self, cfgdict):
"""
``validate_config`` raises ``UnknownConfigError`` when the validator is
``ValidConfiguration.nothing()`` for all non-empty configurations.
"""
cfg = to_configparser(cfgdict)
with self.assertRaises(configutil.UnknownConfigError):
configutil.validate_config(
"<test_everything_valid>",
cfg,
configutil.ValidConfiguration.nothing(),
)
def test_nothing_empty_valid(self):
"""
``validate_config`` returns ``None`` when the validator is
``ValidConfiguration.nothing()`` if the configuration is empty.
"""
cfg = ConfigParser()
self.assertIs(
configutil.validate_config(
"<test_everything_valid>",
cfg,
configutil.ValidConfiguration.nothing(),
),
None,
)
@given(arbitrary_config_dicts())
def test_copy_config(self, cfgdict):
"""
``copy_config`` creates a new ``ConfigParser`` object containing the same
values as its input.
"""
cfg = to_configparser(cfgdict)
copied = configutil.copy_config(cfg)
# Should be equal
self.assertEqual(cfg, copied)
# But not because they're the same object.
self.assertIsNot(cfg, copied)

80
src/allmydata/test/test_node.py
View File

@ -29,6 +29,9 @@ from hypothesis.strategies import (
from unittest import skipIf
from twisted.python.filepath import (
FilePath,
)
from twisted.trial import unittest
from twisted.internet import defer
@ -52,7 +55,11 @@ from allmydata import client
from allmydata.util import fileutil, iputil
from allmydata.util.namespace import Namespace
from allmydata.util.configutil import UnknownConfigError
from allmydata.util.configutil import (
ValidConfiguration,
UnknownConfigError,
)
from allmydata.util.i2p_provider import create as create_i2p_provider
from allmydata.util.tor_provider import create as create_tor_provider
import allmydata.test.common_util as testutil
@ -431,19 +438,78 @@ class TestCase(testutil.SignalMixin, unittest.TestCase):
yield client.create_client(basedir)
self.failUnless(ns.called)
def test_set_config_unescaped_furl_hash(self):
"""
``_Config.set_config`` raises ``UnescapedHashError`` if the item being set
is a furl and the value includes ``"#"`` and does not set the value.
"""
basedir = self.mktemp()
new_config = config_from_string(basedir, "", "")
with self.assertRaises(UnescapedHashError):
new_config.set_config("foo", "bar.furl", "value#1")
with self.assertRaises(MissingConfigEntry):
new_config.get_config("foo", "bar.furl")
def test_set_config_new_section(self):
"""
set_config() can create a new config section
``_Config.set_config`` can be called with the name of a section that does
not already exist to create that section and set an item in it.
"""
basedir = "test_node/test_set_config_new_section"
config = config_from_string(basedir, "", "")
config.set_config("foo", "bar", "value1")
config.set_config("foo", "bar", "value2")
basedir = self.mktemp()
new_config = config_from_string(basedir, "", "", ValidConfiguration.everything())
new_config.set_config("foo", "bar", "value1")
self.assertEqual(
config.get_config("foo", "bar"),
new_config.get_config("foo", "bar"),
"value1"
)
def test_set_config_replace(self):
"""
``_Config.set_config`` can be called with a section and item that already
exists to change an existing value to a new one.
"""
basedir = self.mktemp()
new_config = config_from_string(basedir, "", "", ValidConfiguration.everything())
new_config.set_config("foo", "bar", "value1")
new_config.set_config("foo", "bar", "value2")
self.assertEqual(
new_config.get_config("foo", "bar"),
"value2"
)
def test_set_config_write(self):
"""
``_Config.set_config`` persists the configuration change so it can be
re-loaded later.
"""
# Let our nonsense config through
valid_config = ValidConfiguration.everything()
basedir = FilePath(self.mktemp())
basedir.makedirs()
cfg = basedir.child(b"tahoe.cfg")
cfg.setContent(b"")
new_config = read_config(basedir.path, "", [], valid_config)
new_config.set_config("foo", "bar", "value1")
loaded_config = read_config(basedir.path, "", [], valid_config)
self.assertEqual(
loaded_config.get_config("foo", "bar"),
"value1",
)
def test_set_config_rejects_invalid_config(self):
"""
``_Config.set_config`` raises ``UnknownConfigError`` if the section or
item is not recognized by the validation object and does not set the
value.
"""
# Make everything invalid.
valid_config = ValidConfiguration.nothing()
new_config = config_from_string(self.mktemp(), "", "", valid_config)
with self.assertRaises(UnknownConfigError):
new_config.set_config("foo", "bar", "baz")
with self.assertRaises(MissingConfigEntry):
new_config.get_config("foo", "bar")
class TestMissingPorts(unittest.TestCase):
"""

68
src/allmydata/util/configutil.py
View File

@ -20,6 +20,10 @@ from configparser import ConfigParser
import attr
from twisted.python.runtime import (
platform,
)
class UnknownConfigError(Exception):
"""
@ -59,8 +63,25 @@ def set_config(config, section, option, value):
assert config.get(section, option) == value
def write_config(tahoe_cfg, config):
with open(tahoe_cfg, "w") as f:
config.write(f)
"""
Write a configuration to a file.
:param FilePath tahoe_cfg: The path to which to write the config.
:param ConfigParser config: The configuration to write.
:return: ``None``
"""
tmp = tahoe_cfg.temporarySibling()
# FilePath.open can only open files in binary mode which does not work
# with ConfigParser.write.
with open(tmp.path, "wt") as fp:
config.write(fp)
# Windows doesn't have atomic overwrite semantics for moveTo. Thus we end
# up slightly less than atomic.
if platform.isWindows():
tahoe_cfg.remove()
tmp.moveTo(tahoe_cfg)
def validate_config(fname, cfg, valid_config):
"""
@ -102,10 +123,34 @@ class ValidConfiguration(object):
an item name as bytes and returns True if that section, item pair is
valid, False otherwise.
"""
_static_valid_sections = attr.ib()
_static_valid_sections = attr.ib(
validator=attr.validators.instance_of(dict)
)
_is_valid_section = attr.ib(default=lambda section_name: False)
_is_valid_item = attr.ib(default=lambda section_name, item_name: False)
@classmethod
def everything(cls):
"""
Create a validator which considers everything valid.
"""
return cls(
{},
lambda section_name: True,
lambda section_name, item_name: True,
)
@classmethod
def nothing(cls):
"""
Create a validator which considers nothing valid.
"""
return cls(
{},
lambda section_name: False,
lambda section_name, item_name: False,
)
def is_valid_section(self, section_name):
"""
:return: True if the given section name is valid, False otherwise.
@ -139,6 +184,23 @@ class ValidConfiguration(object):
)
def copy_config(old):
"""
Return a brand new ``ConfigParser`` containing the same values as
the given object.
:param ConfigParser old: The configuration to copy.
:return ConfigParser: The new object containing the same configuration.
"""
new = ConfigParser()
for section_name in old.sections():
new.add_section(section_name)
for k, v in old.items(section_name):
new.set(section_name, k, v.replace("%", "%%"))
return new
def _either(f, g):
"""
:return: A function which returns True if either f or g returns True.