ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-28 16:58:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
e3aef2b966
tahoe-lafs/src/allmydata/test/test_introducer.py

1153 lines
51 KiB
Python
Raw Normal View History

complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
import os, re, itertools
remove introducer/old.py, will create something similar when the RIIntroducer changes
2009-06-24 19:40:38 +00:00
from base64 import b32decode
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
import simplejson
decentralized directories: integration and testing * use new decentralized directories everywhere instead of old centralized directories * provide UI to them through the web server * provide UI to them through the CLI * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them * remove the notion of a "vdrive server" and a client thereof * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example) * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start. The default is 1. * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!) * fix a few bugs where we used to equate "mutable" with "not read-only". Nowadays all directories are mutable, but some might be read-only (to you). * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.
2007-12-03 21:52:42 +00:00
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
from twisted.trial import unittest
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
from twisted.internet import defer, address
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
from twisted.python import log
switch all foolscap imports to use foolscap.api or foolscap.logging
2009-05-22 00:38:23 +00:00
from foolscap.api import Tub, Referenceable, fireEventually, flushEventualQueue
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
from twisted.application import service
#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-22 03:07:27 +00:00
from allmydata.interfaces import InsufficientVersionError
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
from allmydata.introducer.client import IntroducerClient, \
WrapV2ClientInV1Interface
Make introducer.furl unguessable. Closes #1802. Previously, Introducers always used a swissnum of "introducer", so anyone who could learn the (public) tubid of the introducer would be able to connect to and use it. This changes new Introducers to use the same randomly-generated swissnum as clients and storage servers do, so that you absolutely must learn the introducer.furl from someone who knows it already before you can connect. This change also moves the location of the file that stores introducer.furl from BASEDIR/introducer.furl to BASEDIR/private/introducer.furl, since that's where we keep the private things. The first time an introducer is started with the new code, it will move any existing BASEDIR/introducer.furl into the new place. Note that this will not change the FURL of existing introducers: it will only affect newly created ones. When you change an introducer's FURL, you must also update all of the nodes (clients and storage servers) which connect to it, so upgrading it to an unguessable one isn't something we should do automatically.
2013-03-20 22:10:47 +00:00
from allmydata.introducer.server import IntroducerService, FurlFileConflictError
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
from allmydata.introducer.common import get_tubid_string_from_ann, \
get_tubid_string, sign_to_foolscap, unsign_from_foolscap, \
UnknownKeyError
from allmydata.introducer import old
break introducer up into separate modules in the new allmydata.introducer package
2008-06-18 19:24:16 +00:00
# test compatibility with old introducer .tac files
from allmydata.introducer import IntroducerNode
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
from allmydata.web import introweb
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
from allmydata.client import Client as TahoeClient
test_introducer: stop using tcp:0 This fixes some of the upcoming-deprecation warnings against Foolscap (>=0.11.0). There are still a bunch related to the key-generator and the stats gatherer.
2016-04-28 05:33:38 +00:00
from allmydata.util import pollmixin, keyutil, idlib, fileutil, iputil
Change relative imports to absolute
2010-02-26 08:14:33 +00:00
import allmydata.test.common_util as testutil
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
class LoggingMultiService(service.MultiService):
hierarchical logging: add numbered messages and parent= args
2007-11-20 01:23:18 +00:00
def log(self, msg, **kw):
log.msg(msg, **kw)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
test that web.static= is really treated as a relative path Also ssh.authorized_keys_file .
2015-02-04 06:09:40 +00:00
class Node(testutil.SignalMixin, testutil.ReallyEqualMixin, unittest.TestCase):
Make introducer.furl unguessable. Closes #1802. Previously, Introducers always used a swissnum of "introducer", so anyone who could learn the (public) tubid of the introducer would be able to connect to and use it. This changes new Introducers to use the same randomly-generated swissnum as clients and storage servers do, so that you absolutely must learn the introducer.furl from someone who knows it already before you can connect. This change also moves the location of the file that stores introducer.furl from BASEDIR/introducer.furl to BASEDIR/private/introducer.furl, since that's where we keep the private things. The first time an introducer is started with the new code, it will move any existing BASEDIR/introducer.furl into the new place. Note that this will not change the FURL of existing introducers: it will only affect newly created ones. When you change an introducer's FURL, you must also update all of the nodes (clients and storage servers) which connect to it, so upgrading it to an unguessable one isn't something we should do automatically.
2013-03-20 22:10:47 +00:00
def test_furl(self):
basedir = "introducer.IntroducerNode.test_furl"
decentralized directories: integration and testing * use new decentralized directories everywhere instead of old centralized directories * provide UI to them through the web server * provide UI to them through the CLI * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them * remove the notion of a "vdrive server" and a client thereof * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example) * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start. The default is 1. * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!) * fix a few bugs where we used to equate "mutable" with "not read-only". Nowadays all directories are mutable, but some might be read-only (to you). * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.
2007-12-03 21:52:42 +00:00
os.mkdir(basedir)
Make introducer.furl unguessable. Closes #1802. Previously, Introducers always used a swissnum of "introducer", so anyone who could learn the (public) tubid of the introducer would be able to connect to and use it. This changes new Introducers to use the same randomly-generated swissnum as clients and storage servers do, so that you absolutely must learn the introducer.furl from someone who knows it already before you can connect. This change also moves the location of the file that stores introducer.furl from BASEDIR/introducer.furl to BASEDIR/private/introducer.furl, since that's where we keep the private things. The first time an introducer is started with the new code, it will move any existing BASEDIR/introducer.furl into the new place. Note that this will not change the FURL of existing introducers: it will only affect newly created ones. When you change an introducer's FURL, you must also update all of the nodes (clients and storage servers) which connect to it, so upgrading it to an unguessable one isn't something we should do automatically.
2013-03-20 22:10:47 +00:00
public_fn = os.path.join(basedir, "introducer.furl")
private_fn = os.path.join(basedir, "private", "introducer.furl")
Teach tub to start synchronously even with ip autodetect * remove when_tub_ready() from all code * synchronous-ify all node/client/introducer startup code refs ticket:2491
2016-04-27 04:54:45 +00:00
Make introducer.furl unguessable. Closes #1802. Previously, Introducers always used a swissnum of "introducer", so anyone who could learn the (public) tubid of the introducer would be able to connect to and use it. This changes new Introducers to use the same randomly-generated swissnum as clients and storage servers do, so that you absolutely must learn the introducer.furl from someone who knows it already before you can connect. This change also moves the location of the file that stores introducer.furl from BASEDIR/introducer.furl to BASEDIR/private/introducer.furl, since that's where we keep the private things. The first time an introducer is started with the new code, it will move any existing BASEDIR/introducer.furl into the new place. Note that this will not change the FURL of existing introducers: it will only affect newly created ones. When you change an introducer's FURL, you must also update all of the nodes (clients and storage servers) which connect to it, so upgrading it to an unguessable one isn't something we should do automatically.
2013-03-20 22:10:47 +00:00
q1 = IntroducerNode(basedir)
Teach tub to start synchronously even with ip autodetect * remove when_tub_ready() from all code * synchronous-ify all node/client/introducer startup code refs ticket:2491
2016-04-27 04:54:45 +00:00
del q1
# new nodes create unguessable furls in private/introducer.furl
ifurl = fileutil.read(private_fn)
self.failUnless(ifurl)
ifurl = ifurl.strip()
self.failIf(ifurl.endswith("/introducer"), ifurl)
Make introducer.furl unguessable. Closes #1802. Previously, Introducers always used a swissnum of "introducer", so anyone who could learn the (public) tubid of the introducer would be able to connect to and use it. This changes new Introducers to use the same randomly-generated swissnum as clients and storage servers do, so that you absolutely must learn the introducer.furl from someone who knows it already before you can connect. This change also moves the location of the file that stores introducer.furl from BASEDIR/introducer.furl to BASEDIR/private/introducer.furl, since that's where we keep the private things. The first time an introducer is started with the new code, it will move any existing BASEDIR/introducer.furl into the new place. Note that this will not change the FURL of existing introducers: it will only affect newly created ones. When you change an introducer's FURL, you must also update all of the nodes (clients and storage servers) which connect to it, so upgrading it to an unguessable one isn't something we should do automatically.
2013-03-20 22:10:47 +00:00
Teach tub to start synchronously even with ip autodetect * remove when_tub_ready() from all code * synchronous-ify all node/client/introducer startup code refs ticket:2491
2016-04-27 04:54:45 +00:00
# old nodes created guessable furls in BASEDIR/introducer.furl
guessable = ifurl[:ifurl.rfind("/")] + "/introducer"
fileutil.write(public_fn, guessable+"\n", mode="w") # text
Make introducer.furl unguessable. Closes #1802. Previously, Introducers always used a swissnum of "introducer", so anyone who could learn the (public) tubid of the introducer would be able to connect to and use it. This changes new Introducers to use the same randomly-generated swissnum as clients and storage servers do, so that you absolutely must learn the introducer.furl from someone who knows it already before you can connect. This change also moves the location of the file that stores introducer.furl from BASEDIR/introducer.furl to BASEDIR/private/introducer.furl, since that's where we keep the private things. The first time an introducer is started with the new code, it will move any existing BASEDIR/introducer.furl into the new place. Note that this will not change the FURL of existing introducers: it will only affect newly created ones. When you change an introducer's FURL, you must also update all of the nodes (clients and storage servers) which connect to it, so upgrading it to an unguessable one isn't something we should do automatically.
2013-03-20 22:10:47 +00:00
Teach tub to start synchronously even with ip autodetect * remove when_tub_ready() from all code * synchronous-ify all node/client/introducer startup code refs ticket:2491
2016-04-27 04:54:45 +00:00
# if we see both files, throw an error
self.failUnlessRaises(FurlFileConflictError,
IntroducerNode, basedir)
Make introducer.furl unguessable. Closes #1802. Previously, Introducers always used a swissnum of "introducer", so anyone who could learn the (public) tubid of the introducer would be able to connect to and use it. This changes new Introducers to use the same randomly-generated swissnum as clients and storage servers do, so that you absolutely must learn the introducer.furl from someone who knows it already before you can connect. This change also moves the location of the file that stores introducer.furl from BASEDIR/introducer.furl to BASEDIR/private/introducer.furl, since that's where we keep the private things. The first time an introducer is started with the new code, it will move any existing BASEDIR/introducer.furl into the new place. Note that this will not change the FURL of existing introducers: it will only affect newly created ones. When you change an introducer's FURL, you must also update all of the nodes (clients and storage servers) which connect to it, so upgrading it to an unguessable one isn't something we should do automatically.
2013-03-20 22:10:47 +00:00
Teach tub to start synchronously even with ip autodetect * remove when_tub_ready() from all code * synchronous-ify all node/client/introducer startup code refs ticket:2491
2016-04-27 04:54:45 +00:00
# when we see only the public one, move it to private/ and use
# the existing furl instead of creating a new one
os.unlink(private_fn)
q2 = IntroducerNode(basedir)
del q2
self.failIf(os.path.exists(public_fn))
ifurl2 = fileutil.read(private_fn)
self.failUnless(ifurl2)
self.failUnlessEqual(ifurl2.strip(), guessable)
decentralized directories: integration and testing * use new decentralized directories everywhere instead of old centralized directories * provide UI to them through the web server * provide UI to them through the CLI * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them * remove the notion of a "vdrive server" and a client thereof * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example) * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start. The default is 1. * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!) * fix a few bugs where we used to equate "mutable" with "not read-only". Nowadays all directories are mutable, but some might be read-only (to you). * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.
2007-12-03 21:52:42 +00:00
test that web.static= is really treated as a relative path Also ssh.authorized_keys_file .
2015-02-04 06:09:40 +00:00
def test_web_static(self):
basedir = u"introducer.Node.test_web_static"
os.mkdir(basedir)
fileutil.write(os.path.join(basedir, "tahoe.cfg"),
"[node]\n" +
"web.port = tcp:0:interface=127.0.0.1\n" +
"web.static = relative\n")
c = IntroducerNode(basedir)
w = c.getServiceNamed("webish")
abs_basedir = fileutil.abspath_expanduser_unicode(basedir)
expected = fileutil.abspath_expanduser_unicode(u"relative", abs_basedir)
self.failUnlessReallyEqual(w.staticdir, expected)
introducer: add old (V1) introducer code, add test framework for compatibility testing
2008-06-18 23:58:34 +00:00
class ServiceMixin:
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
def setUp(self):
self.parent = LoggingMultiService()
self.parent.startService()
def tearDown(self):
log.msg("TestIntroducer.tearDown")
tests: clean up tearDown to use flushEventualQueue instead of hacking fixed-time delays
2007-04-04 23:09:13 +00:00
d = defer.succeed(None)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
d.addCallback(lambda res: self.parent.stopService())
test_introducer: flushEventualQueue at the end of the test run
2007-03-28 00:16:13 +00:00
d.addCallback(flushEventualQueue)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
return d
util: move PollMixin to a separate file (pollmixin.py), so testutil can be moved into test/
2008-10-29 04:15:48 +00:00
class Introducer(ServiceMixin, unittest.TestCase, pollmixin.PollMixin):
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
def test_create(self):
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
ic = IntroducerClient(None, "introducer.furl", u"my_nickname",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"my_version", "oldest_version", {}, fakeseq)
hush pyflakes-0.4.0 warnings: slightly less-trivial fixes. Closes #900. This includes one fix (in test_web) which was testing the wrong thing.
2010-01-14 22:17:19 +00:00
self.failUnless(isinstance(ic, IntroducerClient))
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
def test_listen(self):
decentralized directories: integration and testing * use new decentralized directories everywhere instead of old centralized directories * provide UI to them through the web server * provide UI to them through the CLI * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them * remove the notion of a "vdrive server" and a client thereof * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example) * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start. The default is 1. * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!) * fix a few bugs where we used to equate "mutable" with "not read-only". Nowadays all directories are mutable, but some might be read-only (to you). * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.
2007-12-03 21:52:42 +00:00
i = IntroducerService()
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
i.setServiceParent(self.parent)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def test_duplicate_publish(self):
introducer: only record one announcement per (tubid,service) tuple. Fixes #343.
2008-04-23 22:05:39 +00:00
i = IntroducerService()
self.failUnlessEqual(len(i.get_announcements()), 0)
self.failUnlessEqual(len(i.get_subscribers()), 0)
furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@192.168.69.247:36106,127.0.0.1:36106/gydnpigj2ja2qr2srq4ikjwnl7xfgbra"
furl2 = "pb://ttwwooyunnyhzs7r6vdonnm2hpi52w6y@192.168.69.247:36111,127.0.0.1:36106/ttwwoogj2ja2qr2srq4ikjwnl7xfgbra"
ann1 = (furl1, "storage", "RIStorage", "nick1", "ver23", "ver0")
ann1b = (furl1, "storage", "RIStorage", "nick1", "ver24", "ver0")
ann2 = (furl2, "storage", "RIStorage", "nick2", "ver30", "ver0")
i.remote_publish(ann1)
self.failUnlessEqual(len(i.get_announcements()), 1)
self.failUnlessEqual(len(i.get_subscribers()), 0)
i.remote_publish(ann2)
self.failUnlessEqual(len(i.get_announcements()), 2)
self.failUnlessEqual(len(i.get_subscribers()), 0)
i.remote_publish(ann1b)
self.failUnlessEqual(len(i.get_announcements()), 2)
self.failUnlessEqual(len(i.get_subscribers()), 0)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def test_id_collision(self):
# test replacement case where tubid equals a keyid (one should
# not replace the other)
i = IntroducerService()
ic = IntroducerClient(None,
"introducer.furl", u"my_nickname",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"my_version", "oldest_version", {}, fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
sk_s, vk_s = keyutil.make_keypair()
sk, _ignored = keyutil.parse_privkey(sk_s)
keyid = keyutil.remove_prefix(vk_s, "pub-v0-")
furl1 = "pb://onug64tu@127.0.0.1:123/short" # base32("short")
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
ann_t = make_ann_t(ic, furl1, sk, 1)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
i.remote_publish_v2(ann_t, Referenceable())
announcements = i.get_announcements()
self.failUnlessEqual(len(announcements), 1)
key1 = ("storage", "v0-"+keyid, None)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
self.failUnlessEqual(announcements[0].index, key1)
ann1_out = announcements[0].announcement
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(ann1_out["anonymous-storage-FURL"], furl1)
furl2 = "pb://%s@127.0.0.1:36106/swissnum" % keyid
ann2 = (furl2, "storage", "RIStorage", "nick1", "ver23", "ver0")
i.remote_publish(ann2)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
announcements = i.get_announcements()
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(len(announcements), 2)
key2 = ("storage", None, keyid)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
wanted = [ad for ad in announcements if ad.index == key2]
self.failUnlessEqual(len(wanted), 1)
ann2_out = wanted[0].announcement
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(ann2_out["anonymous-storage-FURL"], furl2)
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
def fakeseq():
return 1, "nonce"
seqnum_counter = itertools.count(1)
def realseq():
return seqnum_counter.next(), str(os.randint(1,100000))
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def make_ann(furl):
ann = { "anonymous-storage-FURL": furl,
"permutation-seed-base32": get_tubid_string(furl) }
return ann
introducer: add sequence-numbers to announcements, ignore replays This will support revocation of Accounting recommendation records, assuming the gossip-based broadcast channel isn't easily jammed.
2012-06-11 02:10:22 +00:00
def make_ann_t(ic, furl, privkey, seqnum):
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
ann_d = ic.create_announcement_dict("storage", make_ann(furl))
ann_d["seqnum"] = seqnum
ann_d["nonce"] = "nonce"
ann_t = sign_to_foolscap(ann_d, privkey)
return ann_t
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
class Client(unittest.TestCase):
def test_duplicate_receive_v1(self):
ic = IntroducerClient(None,
"introducer.furl", u"my_nickname",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"my_version", "oldest_version", {}, fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
announcements = []
ic.subscribe_to("storage",
lambda key_s,ann: announcements.append(ann))
furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:36106/gydnpigj2ja2qr2srq4ikjwnl7xfgbra"
ann1 = (furl1, "storage", "RIStorage", "nick1", "ver23", "ver0")
ann1b = (furl1, "storage", "RIStorage", "nick1", "ver24", "ver0")
ca = WrapV2ClientInV1Interface(ic)
ca.remote_announce([ann1])
d = fireEventually()
def _then(ign):
self.failUnlessEqual(len(announcements), 1)
self.failUnlessEqual(announcements[0]["nickname"], u"nick1")
self.failUnlessEqual(announcements[0]["my-version"], "ver23")
self.failUnlessEqual(ic._debug_counts["inbound_announcement"], 1)
self.failUnlessEqual(ic._debug_counts["new_announcement"], 1)
self.failUnlessEqual(ic._debug_counts["update"], 0)
self.failUnlessEqual(ic._debug_counts["duplicate_announcement"], 0)
# now send a duplicate announcement: this should not notify clients
ca.remote_announce([ann1])
return fireEventually()
d.addCallback(_then)
def _then2(ign):
self.failUnlessEqual(len(announcements), 1)
self.failUnlessEqual(ic._debug_counts["inbound_announcement"], 2)
self.failUnlessEqual(ic._debug_counts["new_announcement"], 1)
self.failUnlessEqual(ic._debug_counts["update"], 0)
self.failUnlessEqual(ic._debug_counts["duplicate_announcement"], 1)
# and a replacement announcement: same FURL, new other stuff.
# Clients should be notified.
ca.remote_announce([ann1b])
return fireEventually()
d.addCallback(_then2)
def _then3(ign):
self.failUnlessEqual(len(announcements), 2)
self.failUnlessEqual(ic._debug_counts["inbound_announcement"], 3)
self.failUnlessEqual(ic._debug_counts["new_announcement"], 1)
self.failUnlessEqual(ic._debug_counts["update"], 1)
self.failUnlessEqual(ic._debug_counts["duplicate_announcement"], 1)
# test that the other stuff changed
self.failUnlessEqual(announcements[-1]["nickname"], u"nick1")
self.failUnlessEqual(announcements[-1]["my-version"], "ver24")
d.addCallback(_then3)
return d
def test_duplicate_receive_v2(self):
ic1 = IntroducerClient(None,
"introducer.furl", u"my_nickname",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"ver23", "oldest_version", {}, fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# we use a second client just to create a different-looking
# announcement
ic2 = IntroducerClient(None,
"introducer.furl", u"my_nickname",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"ver24","oldest_version",{}, fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
announcements = []
def _received(key_s, ann):
announcements.append( (key_s, ann) )
ic1.subscribe_to("storage", _received)
furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:36106/gydnp"
furl1a = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:7777/gydnp"
furl2 = "pb://ttwwooyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:36106/ttwwoo"
privkey_s, pubkey_vs = keyutil.make_keypair()
privkey, _ignored = keyutil.parse_privkey(privkey_s)
pubkey_s = keyutil.remove_prefix(pubkey_vs, "pub-")
# ann1: ic1, furl1
# ann1a: ic1, furl1a (same SturdyRef, different connection hints)
# ann1b: ic2, furl1
# ann2: ic2, furl2
introducer: add sequence-numbers to announcements, ignore replays This will support revocation of Accounting recommendation records, assuming the gossip-based broadcast channel isn't easily jammed.
2012-06-11 02:10:22 +00:00
self.ann1 = make_ann_t(ic1, furl1, privkey, seqnum=10)
self.ann1old = make_ann_t(ic1, furl1, privkey, seqnum=9)
self.ann1noseqnum = make_ann_t(ic1, furl1, privkey, seqnum=None)
self.ann1b = make_ann_t(ic2, furl1, privkey, seqnum=11)
self.ann1a = make_ann_t(ic1, furl1a, privkey, seqnum=12)
self.ann2 = make_ann_t(ic2, furl2, privkey, seqnum=13)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
ic1.remote_announce_v2([self.ann1]) # queues eventual-send
d = fireEventually()
def _then1(ign):
self.failUnlessEqual(len(announcements), 1)
key_s,ann = announcements[0]
self.failUnlessEqual(key_s, pubkey_s)
self.failUnlessEqual(ann["anonymous-storage-FURL"], furl1)
self.failUnlessEqual(ann["my-version"], "ver23")
d.addCallback(_then1)
# now send a duplicate announcement. This should not fire the
# subscriber
d.addCallback(lambda ign: ic1.remote_announce_v2([self.ann1]))
d.addCallback(fireEventually)
def _then2(ign):
self.failUnlessEqual(len(announcements), 1)
d.addCallback(_then2)
introducer: add sequence-numbers to announcements, ignore replays This will support revocation of Accounting recommendation records, assuming the gossip-based broadcast channel isn't easily jammed.
2012-06-11 02:10:22 +00:00
# an older announcement shouldn't fire the subscriber either
d.addCallback(lambda ign: ic1.remote_announce_v2([self.ann1old]))
d.addCallback(fireEventually)
def _then2a(ign):
self.failUnlessEqual(len(announcements), 1)
d.addCallback(_then2a)
# announcement with no seqnum cannot replace one with-seqnum
d.addCallback(lambda ign: ic1.remote_announce_v2([self.ann1noseqnum]))
d.addCallback(fireEventually)
def _then2b(ign):
self.failUnlessEqual(len(announcements), 1)
d.addCallback(_then2b)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# and a replacement announcement: same FURL, new other stuff. The
# subscriber *should* be fired.
d.addCallback(lambda ign: ic1.remote_announce_v2([self.ann1b]))
d.addCallback(fireEventually)
def _then3(ign):
self.failUnlessEqual(len(announcements), 2)
key_s,ann = announcements[-1]
self.failUnlessEqual(key_s, pubkey_s)
self.failUnlessEqual(ann["anonymous-storage-FURL"], furl1)
self.failUnlessEqual(ann["my-version"], "ver24")
d.addCallback(_then3)
# and a replacement announcement with a different FURL (it uses
# different connection hints)
d.addCallback(lambda ign: ic1.remote_announce_v2([self.ann1a]))
d.addCallback(fireEventually)
def _then4(ign):
self.failUnlessEqual(len(announcements), 3)
key_s,ann = announcements[-1]
self.failUnlessEqual(key_s, pubkey_s)
self.failUnlessEqual(ann["anonymous-storage-FURL"], furl1a)
self.failUnlessEqual(ann["my-version"], "ver23")
d.addCallback(_then4)
# now add a new subscription, which should be called with the
# backlog. The introducer only records one announcement per index, so
# the backlog will only have the latest message.
announcements2 = []
def _received2(key_s, ann):
announcements2.append( (key_s, ann) )
d.addCallback(lambda ign: ic1.subscribe_to("storage", _received2))
d.addCallback(fireEventually)
def _then5(ign):
self.failUnlessEqual(len(announcements2), 1)
key_s,ann = announcements2[-1]
self.failUnlessEqual(key_s, pubkey_s)
self.failUnlessEqual(ann["anonymous-storage-FURL"], furl1a)
self.failUnlessEqual(ann["my-version"], "ver23")
d.addCallback(_then5)
return d
def test_id_collision(self):
# test replacement case where tubid equals a keyid (one should
# not replace the other)
ic = IntroducerClient(None,
"introducer.furl", u"my_nickname",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"my_version", "oldest_version", {}, fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
announcements = []
ic.subscribe_to("storage",
lambda key_s,ann: announcements.append(ann))
sk_s, vk_s = keyutil.make_keypair()
sk, _ignored = keyutil.parse_privkey(sk_s)
keyid = keyutil.remove_prefix(vk_s, "pub-v0-")
furl1 = "pb://onug64tu@127.0.0.1:123/short" # base32("short")
furl2 = "pb://%s@127.0.0.1:36106/swissnum" % keyid
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
ann_t = make_ann_t(ic, furl1, sk, 1)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
ic.remote_announce_v2([ann_t])
d = fireEventually()
def _then(ign):
# first announcement has been processed
self.failUnlessEqual(len(announcements), 1)
self.failUnlessEqual(announcements[0]["anonymous-storage-FURL"],
furl1)
# now submit a second one, with a tubid that happens to look just
# like the pubkey-based serverid we just processed. They should
# not overlap.
ann2 = (furl2, "storage", "RIStorage", "nick1", "ver23", "ver0")
ca = WrapV2ClientInV1Interface(ic)
ca.remote_announce([ann2])
return fireEventually()
d.addCallback(_then)
def _then2(ign):
# if they overlapped, the second announcement would be ignored
self.failUnlessEqual(len(announcements), 2)
self.failUnlessEqual(announcements[1]["anonymous-storage-FURL"],
furl2)
d.addCallback(_then2)
return d
introducer: add sequence-numbers to announcements, ignore replays This will support revocation of Accounting recommendation records, assuming the gossip-based broadcast channel isn't easily jammed.
2012-06-11 02:10:22 +00:00
class Server(unittest.TestCase):
def test_duplicate(self):
i = IntroducerService()
ic1 = IntroducerClient(None,
"introducer.furl", u"my_nickname",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"ver23", "oldest_version", {}, realseq)
introducer: add sequence-numbers to announcements, ignore replays This will support revocation of Accounting recommendation records, assuming the gossip-based broadcast channel isn't easily jammed.
2012-06-11 02:10:22 +00:00
furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:36106/gydnp"
privkey_s, _ = keyutil.make_keypair()
privkey, _ = keyutil.parse_privkey(privkey_s)
ann1 = make_ann_t(ic1, furl1, privkey, seqnum=10)
ann1_old = make_ann_t(ic1, furl1, privkey, seqnum=9)
ann1_new = make_ann_t(ic1, furl1, privkey, seqnum=11)
ann1_noseqnum = make_ann_t(ic1, furl1, privkey, seqnum=None)
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
ann1_badseqnum = make_ann_t(ic1, furl1, privkey, seqnum="not an int")
introducer: add sequence-numbers to announcements, ignore replays This will support revocation of Accounting recommendation records, assuming the gossip-based broadcast channel isn't easily jammed.
2012-06-11 02:10:22 +00:00
i.remote_publish_v2(ann1, None)
all = i.get_announcements()
self.failUnlessEqual(len(all), 1)
self.failUnlessEqual(all[0].announcement["seqnum"], 10)
self.failUnlessEqual(i._debug_counts["inbound_message"], 1)
self.failUnlessEqual(i._debug_counts["inbound_duplicate"], 0)
self.failUnlessEqual(i._debug_counts["inbound_no_seqnum"], 0)
self.failUnlessEqual(i._debug_counts["inbound_old_replay"], 0)
self.failUnlessEqual(i._debug_counts["inbound_update"], 0)
i.remote_publish_v2(ann1, None)
all = i.get_announcements()
self.failUnlessEqual(len(all), 1)
self.failUnlessEqual(all[0].announcement["seqnum"], 10)
self.failUnlessEqual(i._debug_counts["inbound_message"], 2)
self.failUnlessEqual(i._debug_counts["inbound_duplicate"], 1)
self.failUnlessEqual(i._debug_counts["inbound_no_seqnum"], 0)
self.failUnlessEqual(i._debug_counts["inbound_old_replay"], 0)
self.failUnlessEqual(i._debug_counts["inbound_update"], 0)
i.remote_publish_v2(ann1_old, None)
all = i.get_announcements()
self.failUnlessEqual(len(all), 1)
self.failUnlessEqual(all[0].announcement["seqnum"], 10)
self.failUnlessEqual(i._debug_counts["inbound_message"], 3)
self.failUnlessEqual(i._debug_counts["inbound_duplicate"], 1)
self.failUnlessEqual(i._debug_counts["inbound_no_seqnum"], 0)
self.failUnlessEqual(i._debug_counts["inbound_old_replay"], 1)
self.failUnlessEqual(i._debug_counts["inbound_update"], 0)
i.remote_publish_v2(ann1_new, None)
all = i.get_announcements()
self.failUnlessEqual(len(all), 1)
self.failUnlessEqual(all[0].announcement["seqnum"], 11)
self.failUnlessEqual(i._debug_counts["inbound_message"], 4)
self.failUnlessEqual(i._debug_counts["inbound_duplicate"], 1)
self.failUnlessEqual(i._debug_counts["inbound_no_seqnum"], 0)
self.failUnlessEqual(i._debug_counts["inbound_old_replay"], 1)
self.failUnlessEqual(i._debug_counts["inbound_update"], 1)
i.remote_publish_v2(ann1_noseqnum, None)
all = i.get_announcements()
self.failUnlessEqual(len(all), 1)
self.failUnlessEqual(all[0].announcement["seqnum"], 11)
self.failUnlessEqual(i._debug_counts["inbound_message"], 5)
self.failUnlessEqual(i._debug_counts["inbound_duplicate"], 1)
self.failUnlessEqual(i._debug_counts["inbound_no_seqnum"], 1)
self.failUnlessEqual(i._debug_counts["inbound_old_replay"], 1)
self.failUnlessEqual(i._debug_counts["inbound_update"], 1)
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
i.remote_publish_v2(ann1_badseqnum, None)
all = i.get_announcements()
self.failUnlessEqual(len(all), 1)
self.failUnlessEqual(all[0].announcement["seqnum"], 11)
self.failUnlessEqual(i._debug_counts["inbound_message"], 6)
self.failUnlessEqual(i._debug_counts["inbound_duplicate"], 1)
self.failUnlessEqual(i._debug_counts["inbound_no_seqnum"], 2)
self.failUnlessEqual(i._debug_counts["inbound_old_replay"], 1)
self.failUnlessEqual(i._debug_counts["inbound_update"], 1)
introducer: add sequence-numbers to announcements, ignore replays This will support revocation of Accounting recommendation records, assuming the gossip-based broadcast channel isn't easily jammed.
2012-06-11 02:10:22 +00:00
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
NICKNAME = u"n\u00EDickname-%s" # LATIN SMALL LETTER I WITH ACUTE
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
util: move PollMixin to a separate file (pollmixin.py), so testutil can be moved into test/
2008-10-29 04:15:48 +00:00
class SystemTestMixin(ServiceMixin, pollmixin.PollMixin):
introducer: only record one announcement per (tubid,service) tuple. Fixes #343.
2008-04-23 22:05:39 +00:00
test_introducer: stop using tcp:0 This fixes some of the upcoming-deprecation warnings against Foolscap (>=0.11.0). There are still a bunch related to the key-generator and the stats gatherer.
2016-04-28 05:33:38 +00:00
def create_tub(self, portnum=None):
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
tubfile = os.path.join(self.basedir, "tub.pem")
self.central_tub = tub = Tub(certFile=tubfile)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
#tub.setOption("logLocalFailures", True)
#tub.setOption("logRemoteFailures", True)
switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem
2009-05-22 00:46:32 +00:00
tub.setOption("expose-remote-exception-types", False)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
tub.setServiceParent(self.parent)
test_introducer: stop using tcp:0 This fixes some of the upcoming-deprecation warnings against Foolscap (>=0.11.0). There are still a bunch related to the key-generator and the stats gatherer.
2016-04-28 05:33:38 +00:00
if portnum is None:
portnum = iputil.allocate_tcp_port()
tub.listenOn("tcp:%d" % portnum)
self.central_portnum = portnum
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
tub.setLocation("localhost:%d" % self.central_portnum)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
class Queue(SystemTestMixin, unittest.TestCase):
def test_queue_until_connected(self):
self.basedir = "introducer/QueueUntilConnected/queued"
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
os.makedirs(self.basedir)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.create_tub()
introducer = IntroducerService()
introducer.setServiceParent(self.parent)
iff = os.path.join(self.basedir, "introducer.furl")
ifurl = self.central_tub.registerReference(introducer, furlFile=iff)
tub2 = Tub()
tub2.setServiceParent(self.parent)
c = IntroducerClient(tub2, ifurl,
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
u"nickname", "version", "oldest", {}, fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
furl1 = "pb://onug64tu@127.0.0.1:123/short" # base32("short")
sk_s, vk_s = keyutil.make_keypair()
sk, _ignored = keyutil.parse_privkey(sk_s)
d = introducer.disownServiceParent()
def _offline(ign):
# now that the introducer server is offline, create a client and
# publish some messages
c.setServiceParent(self.parent) # this starts the reconnector
c.publish("storage", make_ann(furl1), sk)
introducer.setServiceParent(self.parent) # restart the server
# now wait for the messages to be delivered
def _got_announcement():
return bool(introducer.get_announcements())
return self.poll(_got_announcement)
d.addCallback(_offline)
def _done(ign):
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
v = introducer.get_announcements()[0]
furl = v.announcement["anonymous-storage-FURL"]
self.failUnlessEqual(furl, furl1)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
d.addCallback(_done)
# now let the ack get back
def _wait_until_idle(ign):
def _idle():
if c._debug_outstanding:
return False
if introducer._debug_outstanding:
return False
return True
return self.poll(_idle)
d.addCallback(_wait_until_idle)
return d
introducer: add old (V1) introducer code, add test framework for compatibility testing
2008-06-18 23:58:34 +00:00
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
V1 = "v1"; V2 = "v2"
class SystemTest(SystemTestMixin, unittest.TestCase):
def do_system_test(self, server_version):
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
self.create_tub()
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
if server_version == V1:
introducer = old.IntroducerService_v1()
else:
introducer = IntroducerService()
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
introducer.setServiceParent(self.parent)
iff = os.path.join(self.basedir, "introducer.furl")
tub = self.central_tub
ifurl = self.central_tub.registerReference(introducer, furlFile=iff)
self.introducer_furl = ifurl
introducer: add old (V1) introducer code, add test framework for compatibility testing
2008-06-18 23:58:34 +00:00
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# we have 5 clients who publish themselves as storage servers, and a
# sixth which does which not. All 6 clients subscriber to hear about
# storage. When the connections are fully established, all six nodes
introducer: allow nodes to refrain from publishing themselves, by passing furl=None. This would be useful for clients who do not run storage servers.
2008-02-02 02:48:38 +00:00
# should have 5 connections each.
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
NUM_STORAGE = 5
NUM_CLIENTS = 6
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
clients = []
tubs = {}
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
received_announcements = {}
subscribing_clients = []
publishing_clients = []
introweb announcements: show serverid, not tubid 'serverid' is the pubkey (for V2 clients), falling back to the tubid (for V1 clients). This also required cleaning up the way the index is created for the old V1 introducer.
2012-04-24 05:37:28 +00:00
printable_serverids = {}
test_introducer.SystemTest: fix race condition SystemTest has a couple of different phases, separated by a poller which waits for everything to be idle (all messages delivered, none in flight). It does this by watching some internal "_debug_outstanding" counters in the server and in each client, and waiting for them to hit zero. Just before the last phase, we replace the server with a new one (to make sure clients re-send their messages properly). Unfortunately, the polling function closed over the variable holding the original server, and didn't see the replacement. It kept polling the old server, and failed to notice the outstanding messages for the new server. The last phase of the test (check3) was started too early, which failed (since some messages had not yet been delivered), and then exploded in a flurry of dirty-reactor errors (because some messages were delivered after test shutdown). This replaces the closed-over-variable with a "self.the_introducer", which seems to fix the race. One additional place to look at in the future: the client announcement-receive path (remote_announce) uses an eventually(). If the message has been received and the eventual-send posted (but not yet executed) when the poller sees it, the poller might erroneously conclude that the client is idle and cause the same problem as above. To fix this, the poller (probably all pollers) could be enhanced to do a flushEventualQueue before querying the are-we-done-yet predicate function.
2012-03-31 00:29:06 +00:00
self.the_introducer = introducer
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
privkeys = {}
expected_announcements = [0 for c in range(NUM_CLIENTS)]
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
for i in range(NUM_CLIENTS):
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
tub = Tub()
#tub.setOption("logLocalFailures", True)
#tub.setOption("logRemoteFailures", True)
switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem
2009-05-22 00:46:32 +00:00
tub.setOption("expose-remote-exception-types", False)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
tub.setServiceParent(self.parent)
test_introducer: stop using tcp:0 This fixes some of the upcoming-deprecation warnings against Foolscap (>=0.11.0). There are still a bunch related to the key-generator and the stats gatherer.
2016-04-28 05:33:38 +00:00
portnum = iputil.allocate_tcp_port()
tub.listenOn("tcp:%d" % portnum)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
tub.setLocation("localhost:%d" % portnum)
introducer: allow nodes to refrain from publishing themselves, by passing furl=None. This would be useful for clients who do not run storage servers.
2008-02-02 02:48:38 +00:00
log.msg("creating client %d: %s" % (i, tub.getShortTubID()))
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
if i == 0:
c = old.IntroducerClient_v1(tub, self.introducer_furl,
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
NICKNAME % str(i),
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
"version", "oldest")
else:
c = IntroducerClient(tub, self.introducer_furl,
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
NICKNAME % str(i),
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
"version", "oldest",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
{"component": "component-v1"}, fakeseq)
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
received_announcements[c] = {}
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def got(key_s_or_tubid, ann, announcements, i):
if i == 0:
index = get_tubid_string_from_ann(ann)
else:
index = key_s_or_tubid or get_tubid_string_from_ann(ann)
announcements[index] = ann
c.subscribe_to("storage", got, received_announcements[c], i)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
subscribing_clients.append(c)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
expected_announcements[i] += 1 # all expect a 'storage' announcement
node_furl = tub.registerReference(Referenceable())
if i < NUM_STORAGE:
if i == 0:
c.publish(node_furl, "storage", "ri_name")
introweb announcements: show serverid, not tubid 'serverid' is the pubkey (for V2 clients), falling back to the tubid (for V1 clients). This also required cleaning up the way the index is created for the old V1 introducer.
2012-04-24 05:37:28 +00:00
printable_serverids[i] = get_tubid_string(node_furl)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
elif i == 1:
# sign the announcement
privkey_s, pubkey_s = keyutil.make_keypair()
privkey, _ignored = keyutil.parse_privkey(privkey_s)
privkeys[c] = privkey
c.publish("storage", make_ann(node_furl), privkey)
introweb announcements: show serverid, not tubid 'serverid' is the pubkey (for V2 clients), falling back to the tubid (for V1 clients). This also required cleaning up the way the index is created for the old V1 introducer.
2012-04-24 05:37:28 +00:00
if server_version == V1:
printable_serverids[i] = get_tubid_string(node_furl)
else:
assert pubkey_s.startswith("pub-")
printable_serverids[i] = pubkey_s[len("pub-"):]
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
else:
c.publish("storage", make_ann(node_furl))
introweb announcements: show serverid, not tubid 'serverid' is the pubkey (for V2 clients), falling back to the tubid (for V1 clients). This also required cleaning up the way the index is created for the old V1 introducer.
2012-04-24 05:37:28 +00:00
printable_serverids[i] = get_tubid_string(node_furl)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
publishing_clients.append(c)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
else:
# the last one does not publish anything
pass
if i == 0:
# users of the V1 client were required to publish a
# 'stub_client' record (somewhat after they published the
# 'storage' record), so the introducer could see their
# version. Match that behavior.
c.publish(node_furl, "stub_client", "stub_ri_name")
if i == 2:
# also publish something that nobody cares about
boring_furl = tub.registerReference(Referenceable())
c.publish("boring", make_ann(boring_furl))
big introducer refactoring: separate publish+subscribe. Addresses #271.
2008-02-05 20:05:13 +00:00
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
c.setServiceParent(self.parent)
clients.append(c)
tubs[c] = tub
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def _wait_for_connected(ign):
def _connected():
for c in clients:
if not c.connected_to_introducer():
return False
return True
return self.poll(_connected)
# we watch the clients to determine when the system has settled down.
# Then we can look inside the server to assert things about its
# state.
def _wait_for_expected_announcements(ign):
def _got_expected_announcements():
for i,c in enumerate(subscribing_clients):
if len(received_announcements[c]) < expected_announcements[i]:
return False
return True
return self.poll(_got_expected_announcements)
# before shutting down any Tub, we'd like to know that there are no
# messages outstanding
def _wait_until_idle(ign):
def _idle():
for c in subscribing_clients + publishing_clients:
if c._debug_outstanding:
return False
test_introducer.SystemTest: fix race condition SystemTest has a couple of different phases, separated by a poller which waits for everything to be idle (all messages delivered, none in flight). It does this by watching some internal "_debug_outstanding" counters in the server and in each client, and waiting for them to hit zero. Just before the last phase, we replace the server with a new one (to make sure clients re-send their messages properly). Unfortunately, the polling function closed over the variable holding the original server, and didn't see the replacement. It kept polling the old server, and failed to notice the outstanding messages for the new server. The last phase of the test (check3) was started too early, which failed (since some messages had not yet been delivered), and then exploded in a flurry of dirty-reactor errors (because some messages were delivered after test shutdown). This replaces the closed-over-variable with a "self.the_introducer", which seems to fix the race. One additional place to look at in the future: the client announcement-receive path (remote_announce) uses an eventually(). If the message has been received and the eventual-send posted (but not yet executed) when the poller sees it, the poller might erroneously conclude that the client is idle and cause the same problem as above. To fix this, the poller (probably all pollers) could be enhanced to do a flushEventualQueue before querying the are-we-done-yet predicate function.
2012-03-31 00:29:06 +00:00
if self.the_introducer._debug_outstanding:
big introducer refactoring: separate publish+subscribe. Addresses #271.
2008-02-05 20:05:13 +00:00
return False
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
return True
return self.poll(_idle)
d = defer.succeed(None)
d.addCallback(_wait_for_connected)
d.addCallback(_wait_for_expected_announcements)
d.addCallback(_wait_until_idle)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
introducer.py: add test coverage of _disconnected()
2007-09-19 18:50:13 +00:00
def _check1(res):
log.msg("doing _check1")
test_introducer.SystemTest: fix race condition SystemTest has a couple of different phases, separated by a poller which waits for everything to be idle (all messages delivered, none in flight). It does this by watching some internal "_debug_outstanding" counters in the server and in each client, and waiting for them to hit zero. Just before the last phase, we replace the server with a new one (to make sure clients re-send their messages properly). Unfortunately, the polling function closed over the variable holding the original server, and didn't see the replacement. It kept polling the old server, and failed to notice the outstanding messages for the new server. The last phase of the test (check3) was started too early, which failed (since some messages had not yet been delivered), and then exploded in a flurry of dirty-reactor errors (because some messages were delivered after test shutdown). This replaces the closed-over-variable with a "self.the_introducer", which seems to fix the race. One additional place to look at in the future: the client announcement-receive path (remote_announce) uses an eventually(). If the message has been received and the eventual-send posted (but not yet executed) when the poller sees it, the poller might erroneously conclude that the client is idle and cause the same problem as above. To fix this, the poller (probably all pollers) could be enhanced to do a flushEventualQueue before querying the are-we-done-yet predicate function.
2012-03-31 00:29:06 +00:00
dc = self.the_introducer._debug_counts
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
if server_version == V1:
# each storage server publishes a record, and (after its
# 'subscribe' has been ACKed) also publishes a "stub_client".
# The non-storage client (which subscribes) also publishes a
# stub_client. There is also one "boring" service. The number
# of messages is higher, because the stub_clients aren't
# published until after we get the 'subscribe' ack (since we
# don't realize that we're dealing with a v1 server [which
# needs stub_clients] until then), and the act of publishing
# the stub_client causes us to re-send all previous
# announcements.
self.failUnlessEqual(dc["inbound_message"] - dc["inbound_duplicate"],
NUM_STORAGE + NUM_CLIENTS + 1)
else:
# each storage server publishes a record. There is also one
# "stub_client" and one "boring"
self.failUnlessEqual(dc["inbound_message"], NUM_STORAGE+2)
self.failUnlessEqual(dc["inbound_duplicate"], 0)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
self.failUnlessEqual(dc["inbound_update"], 0)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(dc["inbound_subscribe"], NUM_CLIENTS)
# the number of outbound messages is tricky.. I think it depends
# upon a race between the publish and the subscribe messages.
self.failUnless(dc["outbound_message"] > 0)
# each client subscribes to "storage", and each server publishes
self.failUnlessEqual(dc["outbound_announcements"],
NUM_STORAGE*NUM_CLIENTS)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
for c in subscribing_clients:
cdc = c._debug_counts
self.failUnless(cdc["inbound_message"])
self.failUnlessEqual(cdc["inbound_announcement"],
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
NUM_STORAGE)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
self.failUnlessEqual(cdc["wrong_service"], 0)
self.failUnlessEqual(cdc["duplicate_announcement"], 0)
self.failUnlessEqual(cdc["update"], 0)
self.failUnlessEqual(cdc["new_announcement"],
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
NUM_STORAGE)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
anns = received_announcements[c]
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(len(anns), NUM_STORAGE)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
nodeid0 = tubs[clients[0]].tubID
ann = anns[nodeid0]
nick = ann["nickname"]
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
self.failUnlessEqual(type(nick), unicode)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
self.failUnlessEqual(nick, NICKNAME % "0")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
if server_version == V1:
for c in publishing_clients:
cdc = c._debug_counts
expected = 1 # storage
if c is clients[2]:
expected += 1 # boring
if c is not clients[0]:
# the v2 client tries to call publish_v2, which fails
# because the server is v1. It then re-sends
# everything it has so far, plus a stub_client record
expected = 2*expected + 1
if c is clients[0]:
# we always tell v1 client to send stub_client
expected += 1
self.failUnlessEqual(cdc["outbound_message"], expected)
else:
for c in publishing_clients:
cdc = c._debug_counts
expected = 1
if c in [clients[0], # stub_client
clients[2], # boring
]:
expected = 2
self.failUnlessEqual(cdc["outbound_message"], expected)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
# now check the web status, make sure it renders without error
ir = introweb.IntroducerRoot(self.parent)
self.parent.nodeid = "NODEID"
text = ir.renderSynchronously().decode("utf-8")
self.failUnlessIn(NICKNAME % "0", text) # the v1 client
self.failUnlessIn(NICKNAME % "1", text) # a v2 client
introweb announcements: show serverid, not tubid 'serverid' is the pubkey (for V2 clients), falling back to the tubid (for V1 clients). This also required cleaning up the way the index is created for the old V1 introducer.
2012-04-24 05:37:28 +00:00
for i in range(NUM_STORAGE):
self.failUnlessIn(printable_serverids[i], text,
(i,printable_serverids[i],text))
# make sure there isn't a double-base32ed string too
self.failIfIn(idlib.nodeid_b2a(printable_serverids[i]), text,
(i,printable_serverids[i],text))
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
log.msg("_check1 done")
introducer.py: add test coverage of _disconnected()
2007-09-19 18:50:13 +00:00
d.addCallback(_check1)
big introducer refactoring: separate publish+subscribe. Addresses #271.
2008-02-05 20:05:13 +00:00
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
# force an introducer reconnect, by shutting down the Tub it's using
# and starting a new Tub (with the old introducer). Everybody should
# reconnect and republish, but the introducer should ignore the
# republishes as duplicates. However, because the server doesn't know
# what each client does and does not know, it will send them a copy
# of the current announcement table anyway.
d.addCallback(lambda _ign: log.msg("shutting down introducer's Tub"))
d.addCallback(lambda _ign: self.central_tub.disownServiceParent())
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def _wait_for_introducer_loss(ign):
def _introducer_lost():
for c in clients:
if c.connected_to_introducer():
return False
return True
return self.poll(_introducer_lost)
d.addCallback(_wait_for_introducer_loss)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
def _restart_introducer_tub(_ign):
log.msg("restarting introducer's Tub")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# reset counters
for i in range(NUM_CLIENTS):
c = subscribing_clients[i]
for k in c._debug_counts:
c._debug_counts[k] = 0
test_introducer.SystemTest: fix race condition SystemTest has a couple of different phases, separated by a poller which waits for everything to be idle (all messages delivered, none in flight). It does this by watching some internal "_debug_outstanding" counters in the server and in each client, and waiting for them to hit zero. Just before the last phase, we replace the server with a new one (to make sure clients re-send their messages properly). Unfortunately, the polling function closed over the variable holding the original server, and didn't see the replacement. It kept polling the old server, and failed to notice the outstanding messages for the new server. The last phase of the test (check3) was started too early, which failed (since some messages had not yet been delivered), and then exploded in a flurry of dirty-reactor errors (because some messages were delivered after test shutdown). This replaces the closed-over-variable with a "self.the_introducer", which seems to fix the race. One additional place to look at in the future: the client announcement-receive path (remote_announce) uses an eventually(). If the message has been received and the eventual-send posted (but not yet executed) when the poller sees it, the poller might erroneously conclude that the client is idle and cause the same problem as above. To fix this, the poller (probably all pollers) could be enhanced to do a flushEventualQueue before querying the are-we-done-yet predicate function.
2012-03-31 00:29:06 +00:00
for k in self.the_introducer._debug_counts:
self.the_introducer._debug_counts[k] = 0
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
expected_announcements[i] += 1 # new 'storage' for everyone
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
self.create_tub(self.central_portnum)
test_introducer.SystemTest: fix race condition SystemTest has a couple of different phases, separated by a poller which waits for everything to be idle (all messages delivered, none in flight). It does this by watching some internal "_debug_outstanding" counters in the server and in each client, and waiting for them to hit zero. Just before the last phase, we replace the server with a new one (to make sure clients re-send their messages properly). Unfortunately, the polling function closed over the variable holding the original server, and didn't see the replacement. It kept polling the old server, and failed to notice the outstanding messages for the new server. The last phase of the test (check3) was started too early, which failed (since some messages had not yet been delivered), and then exploded in a flurry of dirty-reactor errors (because some messages were delivered after test shutdown). This replaces the closed-over-variable with a "self.the_introducer", which seems to fix the race. One additional place to look at in the future: the client announcement-receive path (remote_announce) uses an eventually(). If the message has been received and the eventual-send posted (but not yet executed) when the poller sees it, the poller might erroneously conclude that the client is idle and cause the same problem as above. To fix this, the poller (probably all pollers) could be enhanced to do a flushEventualQueue before querying the are-we-done-yet predicate function.
2012-03-31 00:29:06 +00:00
newfurl = self.central_tub.registerReference(self.the_introducer,
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
furlFile=iff)
assert newfurl == self.introducer_furl
d.addCallback(_restart_introducer_tub)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
d.addCallback(_wait_for_connected)
d.addCallback(_wait_for_expected_announcements)
d.addCallback(_wait_until_idle)
d.addCallback(lambda _ign: log.msg(" reconnected"))
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# TODO: publish something while the introducer is offline, then
# confirm it gets delivered when the connection is reestablished
introducer.py: add test coverage of _disconnected()
2007-09-19 18:50:13 +00:00
def _check2(res):
log.msg("doing _check2")
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
# assert that the introducer sent out new messages, one per
# subscriber
test_introducer.SystemTest: fix race condition SystemTest has a couple of different phases, separated by a poller which waits for everything to be idle (all messages delivered, none in flight). It does this by watching some internal "_debug_outstanding" counters in the server and in each client, and waiting for them to hit zero. Just before the last phase, we replace the server with a new one (to make sure clients re-send their messages properly). Unfortunately, the polling function closed over the variable holding the original server, and didn't see the replacement. It kept polling the old server, and failed to notice the outstanding messages for the new server. The last phase of the test (check3) was started too early, which failed (since some messages had not yet been delivered), and then exploded in a flurry of dirty-reactor errors (because some messages were delivered after test shutdown). This replaces the closed-over-variable with a "self.the_introducer", which seems to fix the race. One additional place to look at in the future: the client announcement-receive path (remote_announce) uses an eventually(). If the message has been received and the eventual-send posted (but not yet executed) when the poller sees it, the poller might erroneously conclude that the client is idle and cause the same problem as above. To fix this, the poller (probably all pollers) could be enhanced to do a flushEventualQueue before querying the are-we-done-yet predicate function.
2012-03-31 00:29:06 +00:00
dc = self.the_introducer._debug_counts
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(dc["outbound_announcements"],
NUM_STORAGE*NUM_CLIENTS)
self.failUnless(dc["outbound_message"] > 0)
self.failUnlessEqual(dc["inbound_subscribe"], NUM_CLIENTS)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
for c in subscribing_clients:
cdc = c._debug_counts
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(cdc["inbound_message"], 1)
self.failUnlessEqual(cdc["inbound_announcement"], NUM_STORAGE)
self.failUnlessEqual(cdc["new_announcement"], 0)
self.failUnlessEqual(cdc["wrong_service"], 0)
self.failUnlessEqual(cdc["duplicate_announcement"], NUM_STORAGE)
introducer.py: add test coverage of _disconnected()
2007-09-19 18:50:13 +00:00
d.addCallback(_check2)
big introducer refactoring: separate publish+subscribe. Addresses #271.
2008-02-05 20:05:13 +00:00
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
# Then force an introducer restart, by shutting down the Tub,
# destroying the old introducer, and starting a new Tub+Introducer.
# Everybody should reconnect and republish, and the (new) introducer
# will distribute the new announcements, but the clients should
# ignore the republishes as duplicates.
d.addCallback(lambda _ign: log.msg("shutting down introducer"))
d.addCallback(lambda _ign: self.central_tub.disownServiceParent())
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
d.addCallback(_wait_for_introducer_loss)
d.addCallback(lambda _ign: log.msg("introducer lost"))
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
def _restart_introducer(_ign):
log.msg("restarting introducer")
self.create_tub(self.central_portnum)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# reset counters
for i in range(NUM_CLIENTS):
c = subscribing_clients[i]
for k in c._debug_counts:
c._debug_counts[k] = 0
expected_announcements[i] += 1 # new 'storage' for everyone
if server_version == V1:
introducer = old.IntroducerService_v1()
else:
introducer = IntroducerService()
test_introducer.SystemTest: fix race condition SystemTest has a couple of different phases, separated by a poller which waits for everything to be idle (all messages delivered, none in flight). It does this by watching some internal "_debug_outstanding" counters in the server and in each client, and waiting for them to hit zero. Just before the last phase, we replace the server with a new one (to make sure clients re-send their messages properly). Unfortunately, the polling function closed over the variable holding the original server, and didn't see the replacement. It kept polling the old server, and failed to notice the outstanding messages for the new server. The last phase of the test (check3) was started too early, which failed (since some messages had not yet been delivered), and then exploded in a flurry of dirty-reactor errors (because some messages were delivered after test shutdown). This replaces the closed-over-variable with a "self.the_introducer", which seems to fix the race. One additional place to look at in the future: the client announcement-receive path (remote_announce) uses an eventually(). If the message has been received and the eventual-send posted (but not yet executed) when the poller sees it, the poller might erroneously conclude that the client is idle and cause the same problem as above. To fix this, the poller (probably all pollers) could be enhanced to do a flushEventualQueue before querying the are-we-done-yet predicate function.
2012-03-31 00:29:06 +00:00
self.the_introducer = introducer
newfurl = self.central_tub.registerReference(self.the_introducer,
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
furlFile=iff)
assert newfurl == self.introducer_furl
d.addCallback(_restart_introducer)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
d.addCallback(_wait_for_connected)
d.addCallback(_wait_for_expected_announcements)
d.addCallback(_wait_until_idle)
big introducer refactoring: separate publish+subscribe. Addresses #271.
2008-02-05 20:05:13 +00:00
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
def _check3(res):
log.msg("doing _check3")
test_introducer.SystemTest: fix race condition SystemTest has a couple of different phases, separated by a poller which waits for everything to be idle (all messages delivered, none in flight). It does this by watching some internal "_debug_outstanding" counters in the server and in each client, and waiting for them to hit zero. Just before the last phase, we replace the server with a new one (to make sure clients re-send their messages properly). Unfortunately, the polling function closed over the variable holding the original server, and didn't see the replacement. It kept polling the old server, and failed to notice the outstanding messages for the new server. The last phase of the test (check3) was started too early, which failed (since some messages had not yet been delivered), and then exploded in a flurry of dirty-reactor errors (because some messages were delivered after test shutdown). This replaces the closed-over-variable with a "self.the_introducer", which seems to fix the race. One additional place to look at in the future: the client announcement-receive path (remote_announce) uses an eventually(). If the message has been received and the eventual-send posted (but not yet executed) when the poller sees it, the poller might erroneously conclude that the client is idle and cause the same problem as above. To fix this, the poller (probably all pollers) could be enhanced to do a flushEventualQueue before querying the are-we-done-yet predicate function.
2012-03-31 00:29:06 +00:00
dc = self.the_introducer._debug_counts
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(dc["outbound_announcements"],
NUM_STORAGE*NUM_CLIENTS)
self.failUnless(dc["outbound_message"] > 0)
self.failUnlessEqual(dc["inbound_subscribe"], NUM_CLIENTS)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
for c in subscribing_clients:
cdc = c._debug_counts
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnless(cdc["inbound_message"] > 0)
self.failUnlessEqual(cdc["inbound_announcement"], NUM_STORAGE)
self.failUnlessEqual(cdc["new_announcement"], 0)
self.failUnlessEqual(cdc["wrong_service"], 0)
self.failUnlessEqual(cdc["duplicate_announcement"], NUM_STORAGE)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
d.addCallback(_check3)
complete the Introducer changes, separate out vdrive access, make everything work again
2007-03-27 23:12:11 +00:00
return d
#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-22 03:07:27 +00:00
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def test_system_v2_server(self):
self.basedir = "introducer/SystemTest/system_v2_server"
os.makedirs(self.basedir)
return self.do_system_test(V2)
test_system_v2_server.timeout = 480
# occasionally takes longer than 350s on "draco"
def test_system_v1_server(self):
self.basedir = "introducer/SystemTest/system_v1_server"
os.makedirs(self.basedir)
return self.do_system_test(V1)
test_system_v1_server.timeout = 480
# occasionally takes longer than 350s on "draco"
class FakeRemoteReference:
def notifyOnDisconnect(self, *args, **kwargs): pass
def getRemoteTubID(self): return "62ubehyunnyhzs7r6vdonnm2hpi52w6y"
introweb: fix connection hints for server announcements A long time ago, the introducer's status web page would show the advertised IP addresses for all published services, by parsing their FURL's connection hints. This hasn't worked since about 12-Aug-2014 when foolscap-0.6.5 changed the internal format of these hints (the column has been empty this whole time). This removes the "Advertised IPs" column from the Service Announcements table. Instead, the service's full connection hints (not just the IP address) is displayed in a tooltip/popup on the "Announced" timestamp column. The code that pulls these connection hints is now tolerant of all three foolscap styles: * foolscap<=0.6.4 : tuples of ("ipv4",host,port) * 0.6.5 .. 0.8.0 : tuples of ("tcp",host,port) * foolscap>=0.9.0 : strings fixes ticket:2510
2015-09-22 23:37:12 +00:00
def getLocationHints(self): return ["tcp:here.example.com:1234",
"tcp:there.example.com2345"]
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
def getPeer(self): return address.IPv4Address("TCP", "remote.example.com",
3456)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
class ClientInfo(unittest.TestCase):
def test_client_v2(self):
introducer = IntroducerService()
tub = introducer_furl = None
app_versions = {"whizzy": "fizzy"}
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
client_v2 = IntroducerClient(tub, introducer_furl, NICKNAME % u"v2",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"my_version", "oldest", app_versions,
fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
#furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:0/swissnum"
introducer: add sequence-numbers to announcements, ignore replays This will support revocation of Accounting recommendation records, assuming the gossip-based broadcast channel isn't easily jammed.
2012-06-11 02:10:22 +00:00
#ann_s = make_ann_t(client_v2, furl1, None, 10)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
#introducer.remote_publish_v2(ann_s, Referenceable())
subscriber = FakeRemoteReference()
introducer.remote_subscribe_v2(subscriber, "storage",
client_v2._my_subscriber_info)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
subs = introducer.get_subscribers()
self.failUnlessEqual(len(subs), 1)
s0 = subs[0]
self.failUnlessEqual(s0.service_name, "storage")
self.failUnlessEqual(s0.app_versions, app_versions)
self.failUnlessEqual(s0.nickname, NICKNAME % u"v2")
self.failUnlessEqual(s0.version, "my_version")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def test_client_v1(self):
introducer = IntroducerService()
subscriber = FakeRemoteReference()
introducer.remote_subscribe(subscriber, "storage")
# the v1 subscribe interface had no subscriber_info: that was usually
# sent in a separate stub_client pseudo-announcement
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
subs = introducer.get_subscribers()
self.failUnlessEqual(len(subs), 1)
s0 = subs[0]
self.failUnlessEqual(s0.nickname, u"?") # not known yet
self.failUnlessEqual(s0.service_name, "storage")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# now submit the stub_client announcement
furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:0/swissnum"
ann = (furl1, "stub_client", "RIStubClient",
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
(NICKNAME % u"v1").encode("utf-8"), "my_version", "oldest")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
introducer.remote_publish(ann)
# the server should correlate the two
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
subs = introducer.get_subscribers()
self.failUnlessEqual(len(subs), 1)
s0 = subs[0]
self.failUnlessEqual(s0.service_name, "storage")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# v1 announcements do not contain app-versions
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
self.failUnlessEqual(s0.app_versions, {})
self.failUnlessEqual(s0.nickname, NICKNAME % u"v1")
self.failUnlessEqual(s0.version, "my_version")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# a subscription that arrives after the stub_client announcement
# should be correlated too
subscriber2 = FakeRemoteReference()
introducer.remote_subscribe(subscriber2, "thing2")
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
subs = introducer.get_subscribers()
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnlessEqual(len(subs), 2)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
s0 = [s for s in subs if s.service_name == "thing2"][0]
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
# v1 announcements do not contain app-versions
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
self.failUnlessEqual(s0.app_versions, {})
self.failUnlessEqual(s0.nickname, NICKNAME % u"v1")
self.failUnlessEqual(s0.version, "my_version")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
class Announcements(unittest.TestCase):
def test_client_v2_unsigned(self):
introducer = IntroducerService()
tub = introducer_furl = None
app_versions = {"whizzy": "fizzy"}
client_v2 = IntroducerClient(tub, introducer_furl, u"nick-v2",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"my_version", "oldest", app_versions,
fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:0/swissnum"
tubid = "62ubehyunnyhzs7r6vdonnm2hpi52w6y"
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
ann_s0 = make_ann_t(client_v2, furl1, None, 10)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
canary0 = Referenceable()
introducer.remote_publish_v2(ann_s0, canary0)
a = introducer.get_announcements()
self.failUnlessEqual(len(a), 1)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
self.failUnlessIdentical(a[0].canary, canary0)
self.failUnlessEqual(a[0].index, ("storage", None, tubid))
self.failUnlessEqual(a[0].announcement["app-versions"], app_versions)
self.failUnlessEqual(a[0].nickname, u"nick-v2")
self.failUnlessEqual(a[0].service_name, "storage")
self.failUnlessEqual(a[0].version, "my_version")
self.failUnlessEqual(a[0].announcement["anonymous-storage-FURL"], furl1)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def test_client_v2_signed(self):
introducer = IntroducerService()
tub = introducer_furl = None
app_versions = {"whizzy": "fizzy"}
client_v2 = IntroducerClient(tub, introducer_furl, u"nick-v2",
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
"my_version", "oldest", app_versions,
fakeseq)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:0/swissnum"
sk_s, vk_s = keyutil.make_keypair()
sk, _ignored = keyutil.parse_privkey(sk_s)
pks = keyutil.remove_prefix(vk_s, "pub-")
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
ann_t0 = make_ann_t(client_v2, furl1, sk, 10)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
canary0 = Referenceable()
introducer.remote_publish_v2(ann_t0, canary0)
a = introducer.get_announcements()
self.failUnlessEqual(len(a), 1)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
self.failUnlessIdentical(a[0].canary, canary0)
self.failUnlessEqual(a[0].index, ("storage", pks, None))
self.failUnlessEqual(a[0].announcement["app-versions"], app_versions)
self.failUnlessEqual(a[0].nickname, u"nick-v2")
self.failUnlessEqual(a[0].service_name, "storage")
self.failUnlessEqual(a[0].version, "my_version")
self.failUnlessEqual(a[0].announcement["anonymous-storage-FURL"], furl1)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def test_client_v1(self):
introducer = IntroducerService()
furl1 = "pb://62ubehyunnyhzs7r6vdonnm2hpi52w6y@127.0.0.1:0/swissnum"
tubid = "62ubehyunnyhzs7r6vdonnm2hpi52w6y"
ann = (furl1, "storage", "RIStorage",
u"nick-v1".encode("utf-8"), "my_version", "oldest")
introducer.remote_publish(ann)
a = introducer.get_announcements()
self.failUnlessEqual(len(a), 1)
Fix introweb display for mixed V1/V2 clients. Closes #1721. This significantly cleans up the IntroducerServer web-status renderers. Instead of poking around in the introducer's internals, now the web-status renderers get clean AnnouncementDescriptor and SubscriberDescriptor objects. They are still somewhat foolscap-centric, but will provide a clean abstraction boundary for future improvements. The specific #1721 bug was that old (V1) subscribers were handled by wrapping their RemoteReference in a special WrapV1SubscriberInV2Interface object, but the web-status display was trying to peek inside the object to learn what host+port it was associated with, and the wrapper did not proxy those extra attributes. A test was added to test_introducer to make sure the introweb page renders properly and at least contains the nicknames of both the V1 and V2 clients.
2012-04-23 22:02:22 +00:00
self.failUnlessEqual(a[0].index, ("storage", None, tubid))
self.failUnlessEqual(a[0].canary, None)
self.failUnlessEqual(a[0].announcement["app-versions"], {})
self.failUnlessEqual(a[0].nickname, u"nick-v1".encode("utf-8"))
self.failUnlessEqual(a[0].service_name, "storage")
self.failUnlessEqual(a[0].version, "my_version")
self.failUnlessEqual(a[0].announcement["anonymous-storage-FURL"], furl1)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
class ClientSeqnums(unittest.TestCase):
def test_client(self):
basedir = "introducer/ClientSeqnums/test_client"
fileutil.make_dirs(basedir)
test_introducer.ClientSeqnums: disable storage This test was depending upon the storage announcement happening *after* startup, but the upcoming synchronous-Tub-startup change will modify the ordering. Fix it in both cases by disabling storage in the client being tested.
2016-04-27 01:38:21 +00:00
# if storage is enabled, the Client will publish its storage server
# during startup (although the announcement will wait in a queue
# until the introducer connection is established). To avoid getting
# confused by this, disable storage.
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
f = open(os.path.join(basedir, "tahoe.cfg"), "w")
f.write("[client]\n")
f.write("introducer.furl = nope\n")
test_introducer.ClientSeqnums: disable storage This test was depending upon the storage announcement happening *after* startup, but the upcoming synchronous-Tub-startup change will modify the ordering. Fix it in both cases by disabling storage in the client being tested.
2016-04-27 01:38:21 +00:00
f.write("[storage]\n")
f.write("enabled = false\n")
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
f.close()
test_introducer.ClientSeqnums: disable storage This test was depending upon the storage announcement happening *after* startup, but the upcoming synchronous-Tub-startup change will modify the ordering. Fix it in both cases by disabling storage in the client being tested.
2016-04-27 01:38:21 +00:00
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
c = TahoeClient(basedir)
ic = c.introducer_client
outbound = ic._outbound_announcements
published = ic._published_announcements
def read_seqnum():
f = open(os.path.join(basedir, "announcement-seqnum"))
seqnum = f.read().strip()
f.close()
return int(seqnum)
client.py: internal renaming, _server_key -> _node_key
2013-04-14 20:22:10 +00:00
ic.publish("sA", {"key": "value1"}, c._node_key)
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
self.failUnlessEqual(read_seqnum(), 1)
self.failUnless("sA" in outbound)
self.failUnlessEqual(outbound["sA"]["seqnum"], 1)
nonce1 = outbound["sA"]["nonce"]
self.failUnless(isinstance(nonce1, str))
self.failUnlessEqual(simplejson.loads(published["sA"][0]),
outbound["sA"])
# [1] is the signature, [2] is the pubkey
# publishing a second service causes both services to be
# re-published, with the next higher sequence number
client.py: internal renaming, _server_key -> _node_key
2013-04-14 20:22:10 +00:00
ic.publish("sB", {"key": "value2"}, c._node_key)
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
self.failUnlessEqual(read_seqnum(), 2)
self.failUnless("sB" in outbound)
self.failUnlessEqual(outbound["sB"]["seqnum"], 2)
self.failUnless("sA" in outbound)
self.failUnlessEqual(outbound["sA"]["seqnum"], 2)
nonce2 = outbound["sA"]["nonce"]
self.failUnless(isinstance(nonce2, str))
self.failIfEqual(nonce1, nonce2)
self.failUnlessEqual(simplejson.loads(published["sA"][0]),
outbound["sA"])
self.failUnlessEqual(simplejson.loads(published["sB"][0]),
outbound["sB"])
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-22 03:07:27 +00:00
class TooNewServer(IntroducerService):
VERSION = { "http://allmydata.org/tahoe/protocols/introducer/v999":
{ },
"application-version": "greetings from the crazy future",
}
class NonV1Server(SystemTestMixin, unittest.TestCase):
# if the 1.3.0 client connects to a server that doesn't provide the 'v1'
# protocol, it is supposed to provide a useful error instead of a weird
# exception.
def test_failure(self):
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
self.basedir = "introducer/NonV1Server/failure"
os.makedirs(self.basedir)
self.create_tub()
#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-22 03:07:27 +00:00
i = TooNewServer()
i.setServiceParent(self.parent)
self.introducer_furl = self.central_tub.registerReference(i)
tub = Tub()
switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem
2009-05-22 00:46:32 +00:00
tub.setOption("expose-remote-exception-types", False)
#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-22 03:07:27 +00:00
tub.setServiceParent(self.parent)
test_introducer: stop using tcp:0 This fixes some of the upcoming-deprecation warnings against Foolscap (>=0.11.0). There are still a bunch related to the key-generator and the stats gatherer.
2016-04-28 05:33:38 +00:00
portnum = iputil.allocate_tcp_port()
tub.listenOn("tcp:%d" % portnum)
#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-22 03:07:27 +00:00
tub.setLocation("localhost:%d" % portnum)
c = IntroducerClient(tub, self.introducer_furl,
introducer.client: use integer seqnums, not time-based. Closes #1767. This stores the sequence number in BASEDIR/announcement-seqnum, and increments it each time any service is published (every service announcement is regenerated with the new sequence number). As everyone knows, time is an illusion, and occasionally goes backwards, so a counter is generally safer (and reveals less information about the node). Later, we'll improve the introducer client to tolerate rollbacks (where, perhaps due to a VM being restarted from an earlier checkpoint, the stored sequence number reverts to an earlier version).
2013-03-19 00:40:56 +00:00
u"nickname-client", "version", "oldest", {},
fakeseq)
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
announcements = {}
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
def got(key_s, ann):
announcements[key_s] = ann
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-23 02:10:47 +00:00
c.subscribe_to("storage", got)
#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-22 03:07:27 +00:00
c.setServiceParent(self.parent)
# now we wait for it to connect and notice the bad version
def _got_bad():
return bool(c._introducer_error) or bool(c._publisher)
d = self.poll(_got_bad)
def _done(res):
self.failUnless(c._introducer_error)
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
self.failUnless(c._introducer_error.check(InsufficientVersionError),
c._introducer_error)
#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-22 03:07:27 +00:00
d.addCallback(_done)
return d
remove introducer/old.py, will create something similar when the RIIntroducer changes
2009-06-24 19:40:38 +00:00
class DecodeFurl(unittest.TestCase):
def test_decode(self):
test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode
2009-05-19 03:41:01 +00:00
# make sure we have a working base64.b32decode. The one in
# python2.4.[01] was broken.
remove introducer/old.py, will create something similar when the RIIntroducer changes
2009-06-24 19:40:38 +00:00
furl = 'pb://t5g7egomnnktbpydbuijt6zgtmw4oqi5@127.0.0.1:51857/hfzv36i'
m = re.match(r'pb://(\w+)@', furl)
assert m
nodeid = b32decode(m.group(1).upper())
test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode
2009-05-19 03:41:01 +00:00
self.failUnlessEqual(nodeid, "\x9fM\xf2\x19\xcckU0\xbf\x03\r\x10\x99\xfb&\x9b-\xc7A\x1d")
new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers.
2011-11-20 10:21:32 +00:00
class Signatures(unittest.TestCase):
def test_sign(self):
ann = {"key1": "value1"}
sk_s,vk_s = keyutil.make_keypair()
sk,ignored = keyutil.parse_privkey(sk_s)
ann_t = sign_to_foolscap(ann, sk)
(msg, sig, key) = ann_t
self.failUnlessEqual(type(msg), type("".encode("utf-8"))) # bytes
self.failUnlessEqual(simplejson.loads(msg.decode("utf-8")), ann)
self.failUnless(sig.startswith("v0-"))
self.failUnless(key.startswith("v0-"))
(ann2,key2) = unsign_from_foolscap(ann_t)
self.failUnlessEqual(ann2, ann)
self.failUnlessEqual("pub-"+key2, vk_s)
# bad signature
bad_ann = {"key1": "value2"}
bad_msg = simplejson.dumps(bad_ann).encode("utf-8")
self.failUnlessRaises(keyutil.BadSignatureError,
unsign_from_foolscap, (bad_msg,sig,key))
# sneaky bad signature should be ignored
(ann2,key2) = unsign_from_foolscap( (bad_msg,None,key) )
self.failUnlessEqual(key2, None)
self.failUnlessEqual(ann2, bad_ann)
# unrecognized signatures
self.failUnlessRaises(UnknownKeyError,
unsign_from_foolscap, (bad_msg,"v999-sig",key))
self.failUnlessRaises(UnknownKeyError,
unsign_from_foolscap, (bad_msg,sig,"v999-key"))
# add tests of StorageFarmBroker: if it receives duplicate announcements, it
# should leave the Reconnector in place, also if it receives
# same-FURL-different-misc, but if it receives same-nodeid-different-FURL, it
# should tear down the Reconnector and make a new one. This behavior used to
# live in the IntroducerClient, and thus used to be tested by test_introducer
# copying more tests from old branch:
# then also add Upgrade test
Reference in New Issue Copy Permalink