ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-28 16:58:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
ca660a5fc6
tahoe-lafs/src/allmydata/test/test_dirnode.py

1699 lines
83 KiB
Python
Raw Normal View History

test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
import time
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
import unicodedata
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
from zope.interface import implements
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
from twisted.trial import unittest
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
from twisted.internet import defer
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
from twisted.internet.interfaces import IConsumer
move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected.
2008-07-16 20:14:39 +00:00
from allmydata import uri, dirnode
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
from allmydata.client import Client
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
from allmydata.immutable import upload
interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode The proper hierarchy is: IFilesystemNode +IFileNode ++IMutableFileNode ++IImmutableFileNode +IDirectoryNode Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.
2009-11-20 07:52:55 +00:00
from allmydata.interfaces import IImmutableFileNode, IMutableFileNode, \
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
ExistingChildError, NoSuchChildError, MustNotBeUnknownRWError, \
MustBeDeepImmutableError, MustBeReadonlyError, \
IDeepCheckResults, IDeepCheckAndRepairResults
mutable: merge renaming with test patches
2008-12-07 15:45:19 +00:00
from allmydata.mutable.filenode import MutableFileNode
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
from allmydata.mutable.common import UncoordinatedWriteError
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
from allmydata.util import hashutil, base32
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
from allmydata.util.netstring import split_netstring
more #514: pass a Monitor to all checker operations, make mutable-checker honor the cancel flag
2008-10-22 08:38:18 +00:00
from allmydata.monitor import Monitor
refactor web tests, and interfaces.IFileNode
2007-12-05 06:01:37 +00:00
from allmydata.test.common import make_chk_file_uri, make_mutable_file_uri, \
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
ErrorMixin
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-17 00:23:48 +00:00
from allmydata.test.no_network import GridTestMixin
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
from allmydata.unknown import UnknownNode, strip_prefix_for_ro
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
from allmydata.nodemaker import NodeMaker
Add tests for CachingDict, _pack_contents, _unpack_contents
2009-07-04 03:43:28 +00:00
from base64 import b32decode
Change relative imports to absolute
2010-02-26 08:14:33 +00:00
import allmydata.test.common_util as testutil
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
class MemAccum:
implements(IConsumer)
def registerProducer(self, producer, streaming):
self.producer = producer
self.producer.resumeProducing()
pass
def unregisterProducer(self):
pass
def write(self, data):
assert not hasattr(self, 'data')
self.data = data
self.producer.resumeProducing()
setup_py_uri = "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861"
one_uri = "URI:LIT:n5xgk" # LIT for "one"
mut_write_uri = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
empty_litdir_uri = "URI:DIR2-LIT:"
tiny_litdir_uri = "URI:DIR2-LIT:gqytunj2onug64tufqzdcosvkjetutcjkq5gw4tvm5vwszdgnz5hgyzufqydulbshj5x2lbm" # contains one child which is itself also LIT
mut_read_uri = "URI:SSK-RO:jf6wkflosyvntwxqcdo7a54jvm:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
future_write_uri = "x-tahoe-crazy://I_am_from_the_future."
future_read_uri = "x-tahoe-crazy-readonly://I_am_from_the_future."
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
# 'o' 'n' 'e-macron'
one_nfc = u"on\u0113"
one_nfd = u"one\u0304"
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class Dirnode(GridTestMixin, unittest.TestCase,
test_dirnode: add an explainError call
2008-11-19 23:02:12 +00:00
testutil.ShouldFailMixin, testutil.StallMixin, ErrorMixin):
tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel
2009-06-05 03:14:44 +00:00
timeout = 240 # It takes longer than 120 seconds on Francois's arm box.
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def test_basic(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_basic"
self.set_up_grid()
c = self.g.clients[0]
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _done(res):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.failUnless(isinstance(res, dirnode.DirectoryNode))
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnless(res.is_mutable())
self.failIf(res.is_readonly())
self.failIf(res.is_unknown())
self.failIf(res.is_allowed_in_immutable_directory())
res.raise_error()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
rep = str(res)
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
self.failUnless("RW-MUT" in rep)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_done)
return d
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
def test_initial_children(self):
self.basedir = "dirnode/Dirnode/test_initial_children"
self.set_up_grid()
c = self.g.clients[0]
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
nm = c.nodemaker
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
kids = {one_nfd: (nm.create_from_cap(one_uri), {}),
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
u"two": (nm.create_from_cap(setup_py_uri),
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
{"metakey": "metavalue"}),
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
u"mut": (nm.create_from_cap(mut_write_uri, mut_read_uri), {}),
u"fut": (nm.create_from_cap(future_write_uri, future_read_uri), {}),
u"fro": (nm.create_from_cap(None, future_read_uri), {}),
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
u"empty_litdir": (nm.create_from_cap(empty_litdir_uri), {}),
u"tiny_litdir": (nm.create_from_cap(tiny_litdir_uri), {}),
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
}
d = c.create_dirnode(kids)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
def _created(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnless(dn.is_mutable())
self.failIf(dn.is_readonly())
self.failIf(dn.is_unknown())
self.failIf(dn.is_allowed_in_immutable_directory())
dn.raise_error()
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
rep = str(dn)
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
self.failUnless("RW-MUT" in rep)
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
return dn.list()
d.addCallback(_created)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
def _check_kids(children):
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()),
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
set([one_nfc, u"two", u"mut", u"fut", u"fro", u"empty_litdir", u"tiny_litdir"]))
one_node, one_metadata = children[one_nfc]
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
two_node, two_metadata = children[u"two"]
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
mut_node, mut_metadata = children[u"mut"]
fut_node, fut_metadata = children[u"fut"]
fro_node, fro_metadata = children[u"fro"]
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
emptylit_node, emptylit_metadata = children[u"empty_litdir"]
tinylit_node, tinylit_metadata = children[u"tiny_litdir"]
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
self.failUnlessEqual(one_node.get_size(), 3)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(one_node.get_uri(), one_uri)
self.failUnlessEqual(one_node.get_readonly_uri(), one_uri)
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
self.failUnless(isinstance(one_metadata, dict), one_metadata)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(two_node.get_size(), 14861)
self.failUnlessEqual(two_node.get_uri(), setup_py_uri)
self.failUnlessEqual(two_node.get_readonly_uri(), setup_py_uri)
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
self.failUnlessEqual(two_metadata["metakey"], "metavalue")
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(mut_node.get_uri(), mut_write_uri)
self.failUnlessEqual(mut_node.get_readonly_uri(), mut_read_uri)
self.failUnless(isinstance(mut_metadata, dict), mut_metadata)
self.failUnless(fut_node.is_unknown())
self.failUnlessEqual(fut_node.get_uri(), future_write_uri)
self.failUnlessEqual(fut_node.get_readonly_uri(), "ro." + future_read_uri)
self.failUnless(isinstance(fut_metadata, dict), fut_metadata)
self.failUnless(fro_node.is_unknown())
self.failUnlessEqual(fro_node.get_uri(), "ro." + future_read_uri)
self.failUnlessEqual(fut_node.get_readonly_uri(), "ro." + future_read_uri)
self.failUnless(isinstance(fro_metadata, dict), fro_metadata)
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failIf(emptylit_node.is_unknown())
self.failUnlessEqual(emptylit_node.get_storage_index(), None)
self.failIf(tinylit_node.is_unknown())
self.failUnlessEqual(tinylit_node.get_storage_index(), None)
d2 = defer.succeed(None)
d2.addCallback(lambda ignored: emptylit_node.list())
d2.addCallback(lambda children: self.failUnlessEqual(children, {}))
d2.addCallback(lambda ignored: tinylit_node.list())
d2.addCallback(lambda children: self.failUnlessEqual(set(children.keys()),
set([u"short"])))
d2.addCallback(lambda ignored: tinylit_node.list())
d2.addCallback(lambda children: children[u"short"][0].read(MemAccum()))
d2.addCallback(lambda accum: self.failUnlessEqual(accum.data, "The end."))
return d2
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
d.addCallback(_check_kids)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
d.addCallback(lambda ign: nm.create_new_mutable_directory(kids))
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
d.addCallback(lambda dn: dn.list())
d.addCallback(_check_kids)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
bad_future_node = UnknownNode(future_write_uri, None)
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
bad_kids1 = {one_nfd: (bad_future_node, {})}
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
# This should fail because we don't know how to diminish the future_write_uri
# cap (given in a write slot and not prefixed with "ro." or "imm.") to a readcap.
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
d.addCallback(lambda ign:
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(MustNotBeUnknownRWError, "bad_kids1",
"cannot attach unknown",
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
nm.create_new_mutable_directory,
bad_kids1))
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
bad_kids2 = {one_nfd: (nm.create_from_cap(one_uri), None)}
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
d.addCallback(lambda ign:
self.shouldFail(AssertionError, "bad_kids2",
"requires metadata to be a dict",
nm.create_new_mutable_directory,
bad_kids2))
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
return d
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
def test_immutable(self):
self.basedir = "dirnode/Dirnode/test_immutable"
self.set_up_grid()
c = self.g.clients[0]
nm = c.nodemaker
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
kids = {one_nfd: (nm.create_from_cap(one_uri), {}),
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
u"two": (nm.create_from_cap(setup_py_uri),
{"metakey": "metavalue"}),
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
u"fut": (nm.create_from_cap(None, future_read_uri), {}),
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
u"empty_litdir": (nm.create_from_cap(empty_litdir_uri), {}),
u"tiny_litdir": (nm.create_from_cap(tiny_litdir_uri), {}),
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
}
d = c.create_immutable_dirnode(kids)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
def _created(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
self.failIf(dn.is_mutable())
self.failUnless(dn.is_readonly())
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failIf(dn.is_unknown())
self.failUnless(dn.is_allowed_in_immutable_directory())
dn.raise_error()
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
rep = str(dn)
self.failUnless("RO-IMM" in rep)
cap = dn.get_cap()
self.failUnlessIn("CHK", cap.to_string())
self.cap = cap
return dn.list()
d.addCallback(_created)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
def _check_kids(children):
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()),
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
set([one_nfc, u"two", u"fut", u"empty_litdir", u"tiny_litdir"]))
one_node, one_metadata = children[one_nfc]
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
two_node, two_metadata = children[u"two"]
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
fut_node, fut_metadata = children[u"fut"]
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
emptylit_node, emptylit_metadata = children[u"empty_litdir"]
tinylit_node, tinylit_metadata = children[u"tiny_litdir"]
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
self.failUnlessEqual(one_node.get_size(), 3)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(one_node.get_uri(), one_uri)
self.failUnlessEqual(one_node.get_readonly_uri(), one_uri)
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
self.failUnless(isinstance(one_metadata, dict), one_metadata)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(two_node.get_size(), 14861)
self.failUnlessEqual(two_node.get_uri(), setup_py_uri)
self.failUnlessEqual(two_node.get_readonly_uri(), setup_py_uri)
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
self.failUnlessEqual(two_metadata["metakey"], "metavalue")
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnless(fut_node.is_unknown())
self.failUnlessEqual(fut_node.get_uri(), "imm." + future_read_uri)
self.failUnlessEqual(fut_node.get_readonly_uri(), "imm." + future_read_uri)
self.failUnless(isinstance(fut_metadata, dict), fut_metadata)
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failIf(emptylit_node.is_unknown())
self.failUnlessEqual(emptylit_node.get_storage_index(), None)
self.failIf(tinylit_node.is_unknown())
self.failUnlessEqual(tinylit_node.get_storage_index(), None)
d2 = defer.succeed(None)
d2.addCallback(lambda ignored: emptylit_node.list())
d2.addCallback(lambda children: self.failUnlessEqual(children, {}))
d2.addCallback(lambda ignored: tinylit_node.list())
d2.addCallback(lambda children: self.failUnlessEqual(set(children.keys()),
set([u"short"])))
d2.addCallback(lambda ignored: tinylit_node.list())
d2.addCallback(lambda children: children[u"short"][0].read(MemAccum()))
d2.addCallback(lambda accum: self.failUnlessEqual(accum.data, "The end."))
return d2
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
d.addCallback(_check_kids)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
d.addCallback(lambda ign: nm.create_from_cap(self.cap.to_string()))
d.addCallback(lambda dn: dn.list())
d.addCallback(_check_kids)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
bad_future_node1 = UnknownNode(future_write_uri, None)
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
bad_kids1 = {one_nfd: (bad_future_node1, {})}
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
d.addCallback(lambda ign:
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(MustNotBeUnknownRWError, "bad_kids1",
"cannot attach unknown",
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
c.create_immutable_dirnode,
bad_kids1))
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
bad_future_node2 = UnknownNode(future_write_uri, future_read_uri)
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
bad_kids2 = {one_nfd: (bad_future_node2, {})}
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
d.addCallback(lambda ign:
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(MustBeDeepImmutableError, "bad_kids2",
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
"is not allowed in an immutable directory",
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
c.create_immutable_dirnode,
bad_kids2))
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
bad_kids3 = {one_nfd: (nm.create_from_cap(one_uri), None)}
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
d.addCallback(lambda ign:
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(AssertionError, "bad_kids3",
"requires metadata to be a dict",
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
c.create_immutable_dirnode,
bad_kids3))
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
bad_kids4 = {one_nfd: (nm.create_from_cap(mut_write_uri), {})}
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
d.addCallback(lambda ign:
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(MustBeDeepImmutableError, "bad_kids4",
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
"is not allowed in an immutable directory",
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
c.create_immutable_dirnode,
bad_kids4))
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
bad_kids5 = {one_nfd: (nm.create_from_cap(mut_read_uri), {})}
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
d.addCallback(lambda ign:
self.shouldFail(MustBeDeepImmutableError, "bad_kids5",
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
"is not allowed in an immutable directory",
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
c.create_immutable_dirnode,
bad_kids5))
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
d.addCallback(lambda ign: c.create_immutable_dirnode({}))
def _created_empty(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
self.failIf(dn.is_mutable())
self.failUnless(dn.is_readonly())
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failIf(dn.is_unknown())
self.failUnless(dn.is_allowed_in_immutable_directory())
dn.raise_error()
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
rep = str(dn)
self.failUnless("RO-IMM" in rep)
cap = dn.get_cap()
self.failUnlessIn("LIT", cap.to_string())
self.failUnlessEqual(cap.to_string(), "URI:DIR2-LIT:")
self.cap = cap
return dn.list()
d.addCallback(_created_empty)
d.addCallback(lambda kids: self.failUnlessEqual(kids, {}))
smallkids = {u"o": (nm.create_from_cap(one_uri), {})}
d.addCallback(lambda ign: c.create_immutable_dirnode(smallkids))
def _created_small(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
self.failIf(dn.is_mutable())
self.failUnless(dn.is_readonly())
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failIf(dn.is_unknown())
self.failUnless(dn.is_allowed_in_immutable_directory())
dn.raise_error()
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
rep = str(dn)
self.failUnless("RO-IMM" in rep)
cap = dn.get_cap()
self.failUnlessIn("LIT", cap.to_string())
self.failUnlessEqual(cap.to_string(),
"URI:DIR2-LIT:gi4tumj2n4wdcmz2kvjesosmjfkdu3rvpbtwwlbqhiwdeot3puwcy")
self.cap = cap
return dn.list()
d.addCallback(_created_small)
d.addCallback(lambda kids: self.failUnlessEqual(kids.keys(), [u"o"]))
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
# now test n.create_subdirectory(mutable=False)
d.addCallback(lambda ign: c.create_dirnode())
def _made_parent(n):
d = n.create_subdirectory(u"subdir", kids, mutable=False)
d.addCallback(lambda sd: sd.list())
d.addCallback(_check_kids)
d.addCallback(lambda ign: n.list())
d.addCallback(lambda children:
self.failUnlessEqual(children.keys(), [u"subdir"]))
d.addCallback(lambda ign: n.get(u"subdir"))
d.addCallback(lambda sd: sd.list())
d.addCallback(_check_kids)
d.addCallback(lambda ign: n.get(u"subdir"))
d.addCallback(lambda sd: self.failIf(sd.is_mutable()))
test_dirnode.py: partial tests for normalization changes.
2010-06-17 03:40:25 +00:00
bad_kids = {one_nfd: (nm.create_from_cap(mut_write_uri), {})}
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
d.addCallback(lambda ign:
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(MustBeDeepImmutableError, "YZ",
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
"is not allowed in an immutable directory",
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
n.create_subdirectory,
u"sub2", bad_kids, mutable=False))
return d
d.addCallback(_made_parent)
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
return d
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
def test_directory_representation(self):
self.basedir = "dirnode/Dirnode/test_directory_representation"
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
self.set_up_grid()
c = self.g.clients[0]
nm = c.nodemaker
# This test checks that any trailing spaces in URIs are retained in the
# encoded directory, but stripped when we get them out of the directory.
# See ticket #925 for why we want that.
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
# It also tests that we store child names as UTF-8 NFC, and normalize
# them again when retrieving them.
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
stripped_write_uri = "lafs://from_the_future\t"
stripped_read_uri = "lafs://readonly_from_the_future\t"
spacedout_write_uri = stripped_write_uri + " "
spacedout_read_uri = stripped_read_uri + " "
child = nm.create_from_cap(spacedout_write_uri, spacedout_read_uri)
self.failUnlessEqual(child.get_write_uri(), spacedout_write_uri)
self.failUnlessEqual(child.get_readonly_uri(), "ro." + spacedout_read_uri)
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
child_dottedi = u"ch\u0131\u0307ld"
kids_in = {child_dottedi: (child, {}), one_nfd: (child, {})}
kids_out = {child_dottedi: (child, {}), one_nfc: (child, {})}
kids_norm = {u"child": (child, {}), one_nfc: (child, {})}
d = c.create_dirnode(kids_in)
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
def _created(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
self.failUnless(dn.is_mutable())
self.failIf(dn.is_readonly())
dn.raise_error()
self.cap = dn.get_cap()
self.rootnode = dn
return dn._node.download_best_version()
d.addCallback(_created)
def _check_data(data):
# Decode the netstring representation of the directory to check that the
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
# spaces are retained when the URIs are stored, and that the names are stored
# as NFC.
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
position = 0
numkids = 0
while position < len(data):
entries, position = split_netstring(data, 1, position)
entry = entries[0]
(name_utf8, ro_uri, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
name = name_utf8.decode("utf-8")
rw_uri = self.rootnode._decrypt_rwcapdata(rwcapdata)
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
self.failUnlessIn(name, kids_out)
(expected_child, ign) = kids_out[name]
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
self.failUnlessEqual(rw_uri, expected_child.get_write_uri())
self.failUnlessEqual("ro." + ro_uri, expected_child.get_readonly_uri())
numkids += 1
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
self.failUnlessEqual(numkids, len(kids_out))
return self.rootnode
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
d.addCallback(_check_data)
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
# Mock up a hypothetical future version of Unicode that adds a canonical equivalence
# between dotless-i + dot-above, and 'i'. That would actually be prohibited by the
# stability rules, but similar additions involving currently-unassigned characters
# would not be.
old_normalize = unicodedata.normalize
def future_normalize(form, s):
assert form == 'NFC', form
return old_normalize(form, s).replace(u"\u0131\u0307", u"i")
def _list(node):
unicodedata.normalize = future_normalize
d2 = node.list()
def _undo_mock(res):
unicodedata.normalize = old_normalize
return res
d2.addBoth(_undo_mock)
return d2
d.addCallback(_list)
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
def _check_kids(children):
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
# Now when we use the real directory listing code, the trailing spaces
# should have been stripped (and "ro." should have been prepended to the
# ro_uri, since it's unknown). Also the dotless-i + dot-above should have been
# normalized to 'i'.
self.failUnlessEqual(set(children.keys()), set(kids_norm.keys()))
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
child_node, child_metadata = children[u"child"]
self.failUnlessEqual(child_node.get_write_uri(), stripped_write_uri)
self.failUnlessEqual(child_node.get_readonly_uri(), "ro." + stripped_read_uri)
d.addCallback(_check_kids)
d.addCallback(lambda ign: nm.create_from_cap(self.cap.to_string()))
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-18 00:02:49 +00:00
d.addCallback(_list)
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
d.addCallback(_check_kids) # again with dirnode recreated from cap
return d
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def test_check(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_check"
self.set_up_grid()
c = self.g.clients[0]
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
more #514: pass a Monitor to all checker operations, make mutable-checker honor the cancel flag
2008-10-22 08:38:18 +00:00
d.addCallback(lambda dn: dn.check(Monitor()))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _done(res):
overhaul checker invocation Removed the Checker service, removed checker results storage (both in-memory and the tiny stub of sqlite-based storage). Added ICheckable, all check/verify is now done by calling the check() method on filenodes and dirnodes (immutable files, literal files, mutable files, and directory instances). Checker results are returned in a Results instance, with an html() method for display. Checker results have been temporarily removed from the wui directory listing until we make some other fixes. Also fixed client.create_node_from_uri() to create LiteralFileNodes properly, since they have different checking behavior. Previously we were creating full FileNodes with LIT uris inside, which were downloadable but not checkable.
2008-07-16 00:23:25 +00:00
self.failUnless(res.is_healthy())
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_done)
return d
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
def _test_deepcheck_create(self):
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
# create a small tree with a loop, and some non-directories
# root/
# root/subdir/
# root/subdir/file1
# root/subdir/link -> root
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
# root/rodir
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
c = self.g.clients[0]
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
def _created_root(rootnode):
self._rootnode = rootnode
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
return rootnode.create_subdirectory(u"subdir")
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
d.addCallback(_created_root)
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
def _created_subdir(subdir):
self._subdir = subdir
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
d = subdir.add_file(u"file1", upload.Data("data"*100, None))
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
d.addCallback(lambda res: subdir.set_node(u"link", self._rootnode))
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d.addCallback(lambda res: c.create_dirnode())
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
d.addCallback(lambda dn:
self._rootnode.set_uri(u"rodir",
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
dn.get_uri(),
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
dn.get_readonly_uri()))
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
return d
d.addCallback(_created_subdir)
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
def _done(res):
return self._rootnode
d.addCallback(_done)
return d
def test_deepcheck(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_deepcheck"
self.set_up_grid()
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
d = self._test_deepcheck_create()
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
d.addCallback(lambda rootnode: rootnode.start_deep_check().when_done())
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
def _check_results(r):
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
self.failUnless(IDeepCheckResults.providedBy(r))
c = r.get_counters()
self.failUnlessEqual(c,
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
{"count-objects-checked": 4,
"count-objects-healthy": 4,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-objects-unhealthy": 0,
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-07 05:35:47 +00:00
"count-objects-unrecoverable": 0,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-corrupt-shares": 0,
})
self.failIf(r.get_corrupt_shares())
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
self.failUnlessEqual(len(r.get_all_results()), 4)
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
d.addCallback(_check_results)
return d
def test_deepcheck_and_repair(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_deepcheck_and_repair"
self.set_up_grid()
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
d = self._test_deepcheck_create()
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
d.addCallback(lambda rootnode:
rootnode.start_deep_check_and_repair().when_done())
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
def _check_results(r):
self.failUnless(IDeepCheckAndRepairResults.providedBy(r))
c = r.get_counters()
self.failUnlessEqual(c,
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
{"count-objects-checked": 4,
"count-objects-healthy-pre-repair": 4,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-objects-unhealthy-pre-repair": 0,
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-07 05:35:47 +00:00
"count-objects-unrecoverable-pre-repair": 0,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-corrupt-shares-pre-repair": 0,
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
"count-objects-healthy-post-repair": 4,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-objects-unhealthy-post-repair": 0,
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-07 05:35:47 +00:00
"count-objects-unrecoverable-post-repair": 0,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-corrupt-shares-post-repair": 0,
"count-repairs-attempted": 0,
"count-repairs-successful": 0,
"count-repairs-unsuccessful": 0,
})
self.failIf(r.get_corrupt_shares())
self.failIf(r.get_remaining_corrupt_shares())
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
self.failUnlessEqual(len(r.get_all_results()), 4)
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
d.addCallback(_check_results)
return d
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
def _mark_file_bad(self, rootnode):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.delete_shares_numbered(rootnode.get_uri(), [0])
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
return rootnode
def test_deepcheck_problems(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_deepcheck_problems"
self.set_up_grid()
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
d = self._test_deepcheck_create()
d.addCallback(lambda rootnode: self._mark_file_bad(rootnode))
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
d.addCallback(lambda rootnode: rootnode.start_deep_check().when_done())
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
def _check_results(r):
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
c = r.get_counters()
self.failUnlessEqual(c,
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
{"count-objects-checked": 4,
"count-objects-healthy": 3,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-objects-unhealthy": 1,
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-07 05:35:47 +00:00
"count-objects-unrecoverable": 0,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-corrupt-shares": 0,
})
#self.failUnlessEqual(len(r.get_problems()), 1) # TODO
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
d.addCallback(_check_results)
return d
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def test_readonly(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_readonly"
self.set_up_grid()
c = self.g.clients[0]
nm = c.nodemaker
filecap = make_chk_file_uri(1234)
filenode = nm.create_from_cap(filecap)
use added secret to protect convergent encryption Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key. If you want traditional convergent encryption behavior, set the added secret to be the empty string. This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names. (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.) This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 16:46:06 +00:00
uploadable = upload.Data("some data", convergence="some convergence string")
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _created(rw_dn):
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d2 = rw_dn.set_uri(u"child", filecap, filecap)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d2.addCallback(lambda res: rw_dn)
return d2
d.addCallback(_created)
def _ready(rw_dn):
ro_uri = rw_dn.get_readonly_uri()
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
ro_dn = c.create_node_from_uri(ro_uri)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.failUnless(ro_dn.is_readonly())
self.failUnless(ro_dn.is_mutable())
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failIf(ro_dn.is_unknown())
self.failIf(ro_dn.is_allowed_in_immutable_directory())
ro_dn.raise_error()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
ro_dn.set_uri, u"newchild", filecap, filecap)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
ro_dn.set_node, u"newchild", filenode)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_nodes ro", None,
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
ro_dn.set_nodes, { u"newchild": (filenode, None) })
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
ro_dn.add_file, u"newchild", uploadable)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
ro_dn.delete, u"child")
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
ro_dn.create_subdirectory, u"newchild")
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_metadata_for ro", None,
test_dirnode: improve coverage of not-mutable-error a bit
2008-05-08 23:53:35 +00:00
ro_dn.set_metadata_for, u"child", {})
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
ro_dn.move_child_to, u"child", rw_dn)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
rw_dn.move_child_to, u"child", ro_dn)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
return ro_dn.list()
d.addCallback(_ready)
def _listed(children):
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
self.failUnless(u"child" in children)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_listed)
return d
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
def failUnlessGreaterThan(self, a, b):
test_dirnode.py: improve error messages in case timestamp tests fail
2008-02-12 00:37:51 +00:00
self.failUnless(a > b, "%r should be > %r" % (a, b))
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
test_dirnode.py: add diag output to test-ctime/mtime tests
2008-02-11 21:13:07 +00:00
def failUnlessGreaterOrEqualThan(self, a, b):
test_dirnode.py: improve error messages in case timestamp tests fail
2008-02-12 00:37:51 +00:00
self.failUnless(a >= b, "%r should be >= %r" % (a, b))
test_dirnode.py: add diag output to test-ctime/mtime tests
2008-02-11 21:13:07 +00:00
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def test_create(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_create"
self.set_up_grid()
c = self.g.clients[0]
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.expected_manifest = []
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.expected_verifycaps = set()
self.expected_storage_indexes = set()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _then(n):
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
# /
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.rootnode = n
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.failUnless(n.is_mutable())
u = n.get_uri()
self.failUnless(u)
self.failUnless(u.startswith("URI:DIR2:"), u)
u_ro = n.get_readonly_uri()
self.failUnless(u_ro.startswith("URI:DIR2-RO:"), u_ro)
rename "get_verifier()" to "get_verify_cap()"
2008-12-08 19:44:11 +00:00
u_v = n.get_verify_cap().to_string()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.failUnless(u_v.startswith("URI:DIR2-Verifier:"), u_v)
dirnode: add get_repair_cap()
2009-01-23 04:44:49 +00:00
u_r = n.get_repair_cap().to_string()
self.failUnlessEqual(u_r, u)
web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.
2008-10-07 04:36:18 +00:00
self.expected_manifest.append( ((), u) )
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.expected_verifycaps.add(u_v)
si = n.get_storage_index()
self.expected_storage_indexes.add(base32.b2a(si))
Change direct accesses to an_uri.storage_index to calls to .get_storage_index() (fixes #948)
2010-02-22 02:45:04 +00:00
expected_si = n._uri.get_storage_index()
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.failUnlessEqual(si, expected_si)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d = n.list()
d.addCallback(lambda res: self.failUnlessEqual(res, {}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.has_child(u"missing"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: self.failIf(res))
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
fake_file_uri = make_mutable_file_uri()
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
other_file_uri = make_mutable_file_uri()
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
m = c.nodemaker.create_from_cap(fake_file_uri)
rename "get_verifier()" to "get_verify_cap()"
2008-12-08 19:44:11 +00:00
ffu_v = m.get_verify_cap().to_string()
web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.
2008-10-07 04:36:18 +00:00
self.expected_manifest.append( ((u"child",) , m.get_uri()) )
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.expected_verifycaps.add(ffu_v)
self.expected_storage_indexes.add(base32.b2a(m.get_storage_index()))
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"child",
fake_file_uri, fake_file_uri))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_uri-no",
"child 'child' already exists",
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
n.set_uri, u"child",
other_file_uri, other_file_uri,
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
# /
# /child = mutable
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
d.addCallback(lambda res: n.create_subdirectory(u"subdir"))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
# /
# /child = mutable
# /subdir = directory
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _created(subdir):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.failUnless(isinstance(subdir, dirnode.DirectoryNode))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.subdir = subdir
rename "get_verifier()" to "get_verify_cap()"
2008-12-08 19:44:11 +00:00
new_v = subdir.get_verify_cap().to_string()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
assert isinstance(new_v, str)
web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.
2008-10-07 04:36:18 +00:00
self.expected_manifest.append( ((u"subdir",), subdir.get_uri()) )
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.expected_verifycaps.add(new_v)
si = subdir.get_storage_index()
self.expected_storage_indexes.add(base32.b2a(si))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_created)
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "mkdir-no",
"child 'subdir' already exists",
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
n.create_subdirectory, u"subdir",
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()),
set([u"child", u"subdir"])))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
d.addCallback(lambda res: n.start_deep_stats().when_done())
dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too.
2008-05-08 20:21:14 +00:00
def _check_deepstats(stats):
self.failUnless(isinstance(stats, dict))
expected = {"count-immutable-files": 0,
"count-mutable-files": 1,
"count-literal-files": 0,
"count-files": 1,
"count-directories": 2,
"size-immutable-files": 0,
"size-literal-files": 0,
#"size-directories": 616, # varies
#"largest-directory": 616,
"largest-directory-children": 2,
"largest-immutable-file": 0,
}
for k,v in expected.iteritems():
self.failUnlessEqual(stats[k], v,
"stats[%s] was %s, not %s" %
(k, stats[k], v))
test_dirnode: the size of the directories varies, tolerate it as low as 500 bytes
2008-05-08 20:42:42 +00:00
self.failUnless(stats["size-directories"] > 500,
dirnode: refactor deep-stats a bit
2008-05-08 20:33:07 +00:00
stats["size-directories"])
test_dirnode: the size of the directories varies, tolerate it as low as 500 bytes
2008-05-08 20:42:42 +00:00
self.failUnless(stats["largest-directory"] > 500,
dirnode: refactor deep-stats a bit
2008-05-08 20:33:07 +00:00
stats["largest-directory"])
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
self.failUnlessEqual(stats["size-files-histogram"], [])
dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too.
2008-05-08 20:21:14 +00:00
d.addCallback(_check_deepstats)
manifest: include stats in results. webapi is unchanged.
2008-11-19 22:03:47 +00:00
d.addCallback(lambda res: n.build_manifest().when_done())
def _check_manifest(res):
manifest = res["manifest"]
self.failUnlessEqual(sorted(manifest),
sorted(self.expected_manifest))
stats = res["stats"]
_check_deepstats(stats)
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.failUnlessEqual(self.expected_verifycaps,
res["verifycaps"])
self.failUnlessEqual(self.expected_storage_indexes,
res["storage-index"])
manifest: include stats in results. webapi is unchanged.
2008-11-19 22:03:47 +00:00
d.addCallback(_check_manifest)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _add_subsubdir(res):
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
return self.subdir.create_subdirectory(u"subsubdir")
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_add_subsubdir)
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
# /
# /child = mutable
# /subdir = directory
# /subdir/subsubdir = directory
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_child_at_path(u"subdir/subsubdir"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda subsubdir:
self.failUnless(isinstance(subsubdir,
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
dirnode.DirectoryNode)))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_child_at_path(u""))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: self.failUnlessEqual(res.get_uri(),
n.get_uri()))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"child"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
set(["tahoe"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
d.addCallback(lambda res:
dirnode lookup: use distinct NoSuchChildError instead of the generic KeyError when a child can't be found
2008-10-27 20:15:25 +00:00
self.shouldFail(NoSuchChildError, "gcamap-no",
"nope",
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-03 01:08:45 +00:00
n.get_child_and_metadata_at_path,
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
u"subdir/nope"))
d.addCallback(lambda res:
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-03 01:08:45 +00:00
n.get_child_and_metadata_at_path(u""))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
def _check_child_and_metadata1(res):
child, metadata = res
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.failUnless(isinstance(child, dirnode.DirectoryNode))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
# edge-metadata needs at least one path segment
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(metadata.keys()), set([]))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
d.addCallback(_check_child_and_metadata1)
d.addCallback(lambda res:
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-03 01:08:45 +00:00
n.get_child_and_metadata_at_path(u"child"))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
def _check_child_and_metadata2(res):
child, metadata = res
self.failUnlessEqual(child.get_uri(),
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
fake_file_uri)
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"]))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
d.addCallback(_check_child_and_metadata2)
d.addCallback(lambda res:
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-03 01:08:45 +00:00
n.get_child_and_metadata_at_path(u"subdir/subsubdir"))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
def _check_child_and_metadata3(res):
child, metadata = res
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.failUnless(isinstance(child, dirnode.DirectoryNode))
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"]))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
d.addCallback(_check_child_and_metadata3)
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# set_uri + metadata
# it should be possible to add a child without any metadata
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"c2",
fake_file_uri, fake_file_uri,
{}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"c2"))
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
# You can't override the link timestamps.
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"c2",
fake_file_uri, fake_file_uri,
{ 'tahoe': {'linkcrtime': "bogus"}}))
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"c2"))
def _has_good_linkcrtime(metadata):
self.failUnless(metadata.has_key('tahoe'))
self.failUnless(metadata['tahoe'].has_key('linkcrtime'))
self.failIfEqual(metadata['tahoe']['linkcrtime'], 'bogus')
d.addCallback(_has_good_linkcrtime)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# if we don't set any defaults, the child should get timestamps
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"c3",
fake_file_uri, fake_file_uri))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"c3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.
2010-06-02 03:26:41 +00:00
# we can also add specific metadata at set_uri() time
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"c4",
fake_file_uri, fake_file_uri,
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
{"key": "value"}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"c4"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
(metadata['key'] == "value"), metadata))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"c2"))
d.addCallback(lambda res: n.delete(u"c3"))
d.addCallback(lambda res: n.delete(u"c4"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# set_node + metadata
dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.
2010-06-02 03:26:41 +00:00
# it should be possible to add a child without any metadata except for timestamps
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"d2", n, {}))
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d.addCallback(lambda res: c.create_dirnode())
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda n2:
self.shouldFail(ExistingChildError, "set_node-no",
"child 'd2' already exists",
n.set_node, u"d2", n2,
overwrite=False))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"d2"))
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# if we don't set any defaults, the child should get timestamps
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"d3", n))
d.addCallback(lambda res: n.get_metadata_for(u"d3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.
2010-06-02 03:26:41 +00:00
# we can also add specific metadata at set_node() time
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"d4", n,
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
{"key": "value"}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"d4"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
(metadata["key"] == "value"), metadata))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"d2"))
d.addCallback(lambda res: n.delete(u"d3"))
d.addCallback(lambda res: n.delete(u"d4"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
wapi: add POST /uri/$DIRECTORY?t=set_children Unfinished bits: doc in webapi.txt, test handling of badly formed JSON, return reasonable HTTP response, examination of the effect of this patch on code coverage -- but I'm committing it anyway because MikeB can use it and I'm being called to dinner...
2008-03-01 01:40:27 +00:00
# metadata through set_children()
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res:
dirnode.set_children: take a dict, not a list
2009-10-13 00:24:40 +00:00
n.set_children({
u"e1": (fake_file_uri, fake_file_uri),
u"e2": (fake_file_uri, fake_file_uri, {}),
u"e3": (fake_file_uri, fake_file_uri,
{"key": "value"}),
}))
dirnode.set_children: change return value: fire with self instead of None
2009-10-13 01:50:26 +00:00
d.addCallback(lambda n2: self.failUnlessIdentical(n2, n))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_children-no",
"child 'e1' already exists",
n.set_children,
dirnode.set_children: take a dict, not a list
2009-10-13 00:24:40 +00:00
{ u"e1": (other_file_uri,
other_file_uri),
u"new": (other_file_uri,
other_file_uri),
},
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
# and 'new' should not have been created
d.addCallback(lambda res: n.list())
d.addCallback(lambda children: self.failIf(u"new" in children))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"e1"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"e2"))
trivial: remove trailing whitespace and unused import
2009-04-12 02:17:42 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"e3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
(metadata["key"] == "value"), metadata))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"e1"))
d.addCallback(lambda res: n.delete(u"e2"))
d.addCallback(lambda res: n.delete(u"e3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# metadata through set_nodes()
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
d.addCallback(lambda res:
n.set_nodes({ u"f1": (n, None),
u"f2": (n, {}),
u"f3": (n, {"key": "value"}),
}))
dirnode.set_nodes: change return value: fire with self instead of None
2009-10-13 01:45:46 +00:00
d.addCallback(lambda n2: self.failUnlessIdentical(n2, n))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_nodes-no",
"child 'f1' already exists",
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
n.set_nodes, { u"f1": (n, None),
u"new": (n, None), },
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
# and 'new' should not have been created
d.addCallback(lambda res: n.list())
d.addCallback(lambda children: self.failIf(u"new" in children))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"f1"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"f2"))
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"f3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
(metadata["key"] == "value"), metadata))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"f1"))
d.addCallback(lambda res: n.delete(u"f2"))
d.addCallback(lambda res: n.delete(u"f3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
d.addCallback(lambda res:
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
n.set_metadata_for(u"child",
dirnode.py: Fix #1034 (MetadataSetter does not enforce restriction on setting 'tahoe' subkeys), and expose the metadata updater for use by SFTP. Also, support diminishing a child cap to read-only if 'no-write' is set in the metadata.
2010-06-01 04:54:28 +00:00
{"tags": ["web2.0-compatible"], "tahoe": {"bad": "mojo"}}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda n1: n1.get_metadata_for(u"child"))
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnless((set(metadata.keys()) == set(["tags", "tahoe"])) and
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
metadata["tags"] == ["web2.0-compatible"] and
"bad" not in metadata["tahoe"], metadata))
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.
2010-06-02 03:26:41 +00:00
d.addCallback(lambda res:
self.shouldFail(NoSuchChildError, "set_metadata_for-nosuch", "",
n.set_metadata_for, u"nosuch", {}))
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
def _start(res):
self._start_timestamp = time.time()
d.addCallback(_start)
test_dirnode.py: simplejson-1.7.1 incorrectly rounds floats to two decimal places. Don't let this bug flunk the timestamp test.
2008-02-12 00:35:17 +00:00
# simplejson-1.7.1 (as shipped on Ubuntu 'gutsy') rounds all
# floats to hundredeths (it uses str(num) instead of repr(num)).
# simplejson-1.7.3 does not have this bug. To prevent this bug
# from causing the test to fail, stall for more than a few
# hundrededths of a second.
d.addCallback(self.stall, 0.1)
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.add_file(u"timestamps",
use added secret to protect convergent encryption Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key. If you want traditional convergent encryption behavior, set the added secret to be the empty string. This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names. (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.) This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 16:46:06 +00:00
upload.Data("stamp me", convergence="some convergence string")))
test_dirnode.py: simplejson-1.7.1 incorrectly rounds floats to two decimal places. Don't let this bug flunk the timestamp test.
2008-02-12 00:35:17 +00:00
d.addCallback(self.stall, 0.1)
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
def _stop(res):
self._stop_timestamp = time.time()
d.addCallback(_stop)
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"timestamps"))
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
def _check_timestamp1(metadata):
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"]))
tahoe_md = metadata["tahoe"]
self.failUnlessEqual(set(tahoe_md.keys()), set(["linkcrtime", "linkmotime"]))
self.failUnlessGreaterOrEqualThan(tahoe_md["linkcrtime"],
test_dirnode.py: add diag output to test-ctime/mtime tests
2008-02-11 21:13:07 +00:00
self._start_timestamp)
self.failUnlessGreaterOrEqualThan(self._stop_timestamp,
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
tahoe_md["linkcrtime"])
self.failUnlessGreaterOrEqualThan(tahoe_md["linkmotime"],
test_dirnode.py: add diag output to test-ctime/mtime tests
2008-02-11 21:13:07 +00:00
self._start_timestamp)
self.failUnlessGreaterOrEqualThan(self._stop_timestamp,
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
tahoe_md["linkmotime"])
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
# Our current timestamp rules say that replacing an existing
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
# child should preserve the 'linkcrtime' but update the
# 'linkmotime'
self._old_linkcrtime = tahoe_md["linkcrtime"]
self._old_linkmotime = tahoe_md["linkmotime"]
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
d.addCallback(_check_timestamp1)
d.addCallback(self.stall, 2.0) # accomodate low-res timestamps
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"timestamps", n))
d.addCallback(lambda res: n.get_metadata_for(u"timestamps"))
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
def _check_timestamp2(metadata):
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessIn("tahoe", metadata)
tahoe_md = metadata["tahoe"]
self.failUnlessEqual(set(tahoe_md.keys()), set(["linkcrtime", "linkmotime"]))
self.failUnlessEqual(tahoe_md["linkcrtime"], self._old_linkcrtime)
self.failUnlessGreaterThan(tahoe_md["linkmotime"], self._old_linkmotime)
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
return n.delete(u"timestamps")
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
d.addCallback(_check_timestamp2)
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"subdir"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda old_child:
self.failUnlessEqual(old_child.get_uri(),
self.subdir.get_uri()))
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()),
set([u"child"])))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable1 = upload.Data("some data", convergence="converge")
d.addCallback(lambda res: n.add_file(u"newfile", uploadable1))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda newnode:
interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode The proper hierarchy is: IFilesystemNode +IFileNode ++IMutableFileNode ++IImmutableFileNode +IDirectoryNode Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.
2009-11-20 07:52:55 +00:00
self.failUnless(IImmutableFileNode.providedBy(newnode)))
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable2 = upload.Data("some data", convergence="stuff")
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "add_file-no",
"child 'newfile' already exists",
n.add_file, u"newfile",
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable2,
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()),
set([u"child", u"newfile"])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"newfile"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(set(metadata.keys()), set(["tahoe"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable3 = upload.Data("some data", convergence="converge")
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.add_file(u"newfile-metadata",
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable3,
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
{"key": "value"}))
d.addCallback(lambda newnode:
interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode The proper hierarchy is: IFilesystemNode +IFileNode ++IMutableFileNode ++IImmutableFileNode +IDirectoryNode Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.
2009-11-20 07:52:55 +00:00
self.failUnless(IImmutableFileNode.providedBy(newnode)))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"newfile-metadata"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
(metadata['key'] == "value"), metadata))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"newfile-metadata"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
d.addCallback(lambda res: n.create_subdirectory(u"subdir2"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _created2(subdir2):
self.subdir2 = subdir2
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
# put something in the way, to make sure it gets overwritten
return subdir2.add_file(u"child", upload.Data("overwrite me",
"converge"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_created2)
d.addCallback(lambda res:
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
n.move_child_to(u"child", self.subdir2))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()),
set([u"newfile", u"subdir2"])))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: self.subdir2.list())
d.addCallback(lambda children:
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()),
set([u"child"])))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res: self.subdir2.get(u"child"))
d.addCallback(lambda child:
self.failUnlessEqual(child.get_uri(),
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
fake_file_uri))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
# move it back, using new_child_name=
d.addCallback(lambda res:
self.subdir2.move_child_to(u"child", n, u"newchild"))
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()),
set([u"newchild", u"newfile",
u"subdir2"])))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res: self.subdir2.list())
d.addCallback(lambda children:
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(children.keys()), set([])))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
# now make sure that we honor overwrite=False
d.addCallback(lambda res:
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
self.subdir2.set_uri(u"newchild",
other_file_uri, other_file_uri))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "move_child_to-no",
"child 'newchild' already exists",
n.move_child_to, u"newchild",
self.subdir2,
overwrite=False))
d.addCallback(lambda res: self.subdir2.get(u"newchild"))
d.addCallback(lambda child:
self.failUnlessEqual(child.get_uri(),
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
other_file_uri))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.
2010-06-02 03:26:41 +00:00
# Setting the no-write field should diminish a mutable cap to read-only
# (for both files and directories).
d.addCallback(lambda ign: n.set_uri(u"mutable", other_file_uri, other_file_uri))
d.addCallback(lambda ign: n.get(u"mutable"))
d.addCallback(lambda mutable: self.failIf(mutable.is_readonly(), mutable))
d.addCallback(lambda ign: n.set_metadata_for(u"mutable", {"no-write": True}))
d.addCallback(lambda ign: n.get(u"mutable"))
d.addCallback(lambda mutable: self.failUnless(mutable.is_readonly(), mutable))
d.addCallback(lambda ign: n.set_metadata_for(u"mutable", {"no-write": True}))
d.addCallback(lambda ign: n.get(u"mutable"))
d.addCallback(lambda mutable: self.failUnless(mutable.is_readonly(), mutable))
d.addCallback(lambda ign: n.get(u"subdir2"))
d.addCallback(lambda subdir2: self.failIf(subdir2.is_readonly()))
d.addCallback(lambda ign: n.set_metadata_for(u"subdir2", {"no-write": True}))
d.addCallback(lambda ign: n.get(u"subdir2"))
d.addCallback(lambda subdir2: self.failUnless(subdir2.is_readonly(), subdir2))
d.addCallback(lambda ign: n.set_uri(u"mutable_ro", other_file_uri, other_file_uri,
metadata={"no-write": True}))
d.addCallback(lambda ign: n.get(u"mutable_ro"))
d.addCallback(lambda mutable_ro: self.failUnless(mutable_ro.is_readonly(), mutable_ro))
d.addCallback(lambda ign: n.create_subdirectory(u"subdir_ro", metadata={"no-write": True}))
d.addCallback(lambda ign: n.get(u"subdir_ro"))
d.addCallback(lambda subdir_ro: self.failUnless(subdir_ro.is_readonly(), subdir_ro))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
return d
d.addCallback(_then)
test_dirnode: add an explainError call
2008-11-19 23:02:12 +00:00
d.addErrback(self.explain_error)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
return d
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
def test_update_metadata(self):
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
(t1, t2, t3) = (626644800.0, 634745640.0, 892226160.0)
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
md1 = dirnode.update_metadata({"ctime": t1}, {}, t2)
self.failUnlessEqual(md1, {"tahoe":{"linkcrtime": t1, "linkmotime": t2}})
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
md2 = dirnode.update_metadata(md1, {"key": "value", "tahoe": {"bad": "mojo"}}, t3)
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(md2, {"key": "value",
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
"tahoe":{"linkcrtime": t1, "linkmotime": t3}})
md3 = dirnode.update_metadata({}, None, t3)
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 23:01:19 +00:00
self.failUnlessEqual(md3, {"tahoe":{"linkcrtime": t3, "linkmotime": t3}})
md4 = dirnode.update_metadata({}, {"bool": True, "number": 42}, t1)
self.failUnlessEqual(md4, {"bool": True, "number": 42,
"tahoe":{"linkcrtime": t1, "linkmotime": t1}})
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-02 01:46:44 +00:00
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
def test_create_subdirectory(self):
self.basedir = "dirnode/Dirnode/test_create_subdirectory"
self.set_up_grid()
c = self.g.clients[0]
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
nm = c.nodemaker
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
d = c.create_dirnode()
def _then(n):
# /
self.rootnode = n
fake_file_uri = make_mutable_file_uri()
other_file_uri = make_mutable_file_uri()
md = {"metakey": "metavalue"}
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
kids = {u"kid1": (nm.create_from_cap(fake_file_uri), {}),
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
u"kid2": (nm.create_from_cap(other_file_uri), md),
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
}
d = n.create_subdirectory(u"subdir", kids)
def _check(sub):
d = n.get_child_at_path(u"subdir")
d.addCallback(lambda sub2: self.failUnlessEqual(sub2.get_uri(),
sub.get_uri()))
d.addCallback(lambda ign: sub.list())
return d
d.addCallback(_check)
def _check_kids(kids2):
dirnode: add tests of literal dirnodes (current and fix for #948)
2010-02-24 04:33:45 +00:00
self.failUnlessEqual(set(kids.keys()), set(kids2.keys()))
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
self.failUnlessEqual(kids2[u"kid2"][1]["metakey"], "metavalue")
d.addCallback(_check_kids)
return d
d.addCallback(_then)
return d
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
class MinimalFakeMutableFile:
def get_writekey(self):
return "writekey"
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class Packing(unittest.TestCase):
# This is a base32-encoded representation of the directory tree
# root/file1
# root/file2
# root/file3
# as represented after being fed to _pack_contents.
# We have it here so we can decode it, feed it to
# _unpack_contents, and verify that _unpack_contents
# works correctly.
known_tree = "GM4TOORVHJTGS3DFGEWDSNJ2KVJESOSDJBFTU33MPB2GS3LZNVYG6N3GGI3WU5TIORTXC3DOMJ2G4NB2MVWXUZDONBVTE5LNGRZWK2LYN55GY23XGNYXQMTOMZUWU5TENN4DG23ZG5UTO2L2NQ2DO6LFMRWDMZJWGRQTUMZ2GEYDUMJQFQYTIMZ22XZKZORX5XS7CAQCSK3URR6QOHISHRCMGER5LRFSZRNAS5ZSALCS6TWFQAE754IVOIKJVK73WZPP3VUUEDTX3WHTBBZ5YX3CEKHCPG3ZWQLYA4QM6LDRCF7TJQYWLIZHKGN5ROA3AUZPXESBNLQQ6JTC2DBJU2D47IZJTLR3PKZ4RVF57XLPWY7FX7SZV3T6IJ3ORFW37FXUPGOE3ROPFNUX5DCGMAQJ3PGGULBRGM3TU6ZCMN2GS3LFEI5CAMJSGQ3DMNRTHA4TOLRUGI3TKNRWGEWCAITUMFUG6ZJCHIQHWITMNFXGW3LPORUW2ZJCHIQDCMRUGY3DMMZYHE3S4NBSG42TMNRRFQQCE3DJNZVWG4TUNFWWKIR2EAYTENBWGY3DGOBZG4XDIMRXGU3DMML5FQQCE3LUNFWWKIR2EAYTENBWGY3DGOBZG4XDIMRXGU3DMML5FQWDGOJRHI2TUZTJNRSTELBZGQ5FKUSJHJBUQSZ2MFYGKZ3SOBSWQ43IO52WO23CNAZWU3DUGVSWSNTIOE5DK33POVTW4ZLNMNWDK6DHPA2GS2THNF2W25DEN5VGY2LQNFRGG5DKNNRHO5TZPFTWI6LNMRYGQ2LCGJTHM4J2GM5DCMB2GQWDCNBSHKVVQBGRYMACKJ27CVQ6O6B4QPR72RFVTGOZUI76XUSWAX73JRV5PYRHMIFYZIA25MXDPGUGML6M2NMRSG4YD4W4K37ZDYSXHMJ3IUVT4F64YTQQVBJFFFOUC7J7LAB2VFCL5UKKGMR2D3F4EPOYC7UYWQZNR5KXHBSNXLCNBX2SNF22DCXJIHSMEKWEWOG5XCJEVVZ7UW5IB6I64XXQSJ34B5CAYZGZIIMR6LBRGMZTU6ZCMN2GS3LFEI5CAMJSGQ3DMNRTHA4TOLRUGMYDEMJYFQQCE5DBNBXWKIR2EB5SE3DJNZVW233UNFWWKIR2EAYTENBWGY3DGOBZG4XDIMZQGIYTQLBAEJWGS3TLMNZHI2LNMURDUIBRGI2DMNRWGM4DSNZOGQZTAMRRHB6SYIBCNV2GS3LFEI5CAMJSGQ3DMNRTHA4TOLRUGMYDEMJYPUWCYMZZGU5DKOTGNFWGKMZMHE2DUVKSJE5EGSCLHJRW25DDPBYTO2DXPB3GM6DBNYZTI6LJMV3DM2LWNB4TU4LWMNSWW3LKORXWK5DEMN3TI23NNE3WEM3SORRGY5THPA3TKNBUMNZG453BOF2GSZLXMVWWI3DJOFZW623RHIZTUMJQHI2SYMJUGI5BOSHWDPG3WKPAVXCF3XMKA7QVIWPRMWJHDTQHD27AHDCPJWDQENQ5H5ZZILTXQNIXXCIW4LKQABU2GCFRG5FHQN7CHD7HF4EKNRZFIV2ZYQIBM7IQU7F4RGB3XCX3FREPBKQ7UCICHVWPCYFGA6OLH3J45LXQ6GWWICJ3PGWJNLZ7PCRNLAPNYUGU6BENS7OXMBEOOFRIZV3PF2FFWZ5WHDPKXERYP7GNHKRMGEZTOOT3EJRXI2LNMURDUIBRGI2DMNRWGM4DSNZOGQZTGNRSGY4SYIBCORQWQ33FEI5CA6ZCNRUW423NN52GS3LFEI5CAMJSGQ3DMNRTHA4TOLRUGMZTMMRWHEWCAITMNFXGWY3SORUW2ZJCHIQDCMRUGY3DMMZYHE3S4NBTGM3DENRZPUWCAITNORUW2ZJCHIQDCMRUGY3DMMZYHE3S4NBTGM3DENRZPUWCY==="
Add tests for CachingDict, _pack_contents, _unpack_contents
2009-07-04 03:43:28 +00:00
def test_unpack_and_pack_behavior(self):
known_tree = b32decode(self.known_tree)
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
nodemaker = NodeMaker(None, None, None,
None, None, None,
{"k": 3, "n": 10}, None)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
write_uri = "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q"
filenode = nodemaker.create_from_cap(write_uri)
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
node = dirnode.DirectoryNode(filenode, nodemaker, None)
children = node._unpack_contents(known_tree)
self._check_children(children)
packed_children = node._pack_contents(children)
children = node._unpack_contents(packed_children)
self._check_children(children)
def _check_children(self, children):
# Are all the expected child nodes there?
self.failUnless(children.has_key(u'file1'))
self.failUnless(children.has_key(u'file2'))
self.failUnless(children.has_key(u'file3'))
# Are the metadata for child 3 right?
file3_rocap = "URI:CHK:cmtcxq7hwxvfxan34yiev6ivhy:qvcekmjtoetdcw4kmi7b3rtblvgx7544crnwaqtiewemdliqsokq:3:10:5"
file3_rwcap = "URI:CHK:cmtcxq7hwxvfxan34yiev6ivhy:qvcekmjtoetdcw4kmi7b3rtblvgx7544crnwaqtiewemdliqsokq:3:10:5"
file3_metadata = {'ctime': 1246663897.4336269, 'tahoe': {'linkmotime': 1246663897.4336269, 'linkcrtime': 1246663897.4336269}, 'mtime': 1246663897.4336269}
self.failUnlessEqual(file3_metadata, children[u'file3'][1])
self.failUnlessEqual(file3_rocap,
children[u'file3'][0].get_readonly_uri())
self.failUnlessEqual(file3_rwcap,
children[u'file3'][0].get_uri())
# Are the metadata for child 2 right?
file2_rocap = "URI:CHK:apegrpehshwugkbh3jlt5ei6hq:5oougnemcl5xgx4ijgiumtdojlipibctjkbwvyygdymdphib2fvq:3:10:4"
file2_rwcap = "URI:CHK:apegrpehshwugkbh3jlt5ei6hq:5oougnemcl5xgx4ijgiumtdojlipibctjkbwvyygdymdphib2fvq:3:10:4"
file2_metadata = {'ctime': 1246663897.430218, 'tahoe': {'linkmotime': 1246663897.430218, 'linkcrtime': 1246663897.430218}, 'mtime': 1246663897.430218}
self.failUnlessEqual(file2_metadata, children[u'file2'][1])
self.failUnlessEqual(file2_rocap,
children[u'file2'][0].get_readonly_uri())
self.failUnlessEqual(file2_rwcap,
children[u'file2'][0].get_uri())
# Are the metadata for child 1 right?
file1_rocap = "URI:CHK:olxtimympo7f27jvhtgqlnbtn4:emzdnhk2um4seixozlkw3qx2nfijvdkx3ky7i7izl47yedl6e64a:3:10:10"
file1_rwcap = "URI:CHK:olxtimympo7f27jvhtgqlnbtn4:emzdnhk2um4seixozlkw3qx2nfijvdkx3ky7i7izl47yedl6e64a:3:10:10"
file1_metadata = {'ctime': 1246663897.4275661, 'tahoe': {'linkmotime': 1246663897.4275661, 'linkcrtime': 1246663897.4275661}, 'mtime': 1246663897.4275661}
self.failUnlessEqual(file1_metadata, children[u'file1'][1])
self.failUnlessEqual(file1_rocap,
children[u'file1'][0].get_readonly_uri())
self.failUnlessEqual(file1_rwcap,
children[u'file1'][0].get_uri())
Add tests for CachingDict, _pack_contents, _unpack_contents
2009-07-04 03:43:28 +00:00
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
def _make_kids(self, nm, which):
caps = {"imm": "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861",
"lit": "URI:LIT:n5xgk", # LIT for "one"
"write": "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq",
"read": "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q",
"dirwrite": "URI:DIR2:n6x24zd3seu725yluj75q5boaa:mm6yoqjhl6ueh7iereldqxue4nene4wl7rqfjfybqrehdqmqskvq",
"dirread": "URI:DIR2-RO:b7sr5qsifnicca7cbk3rhrhbvq:mm6yoqjhl6ueh7iereldqxue4nene4wl7rqfjfybqrehdqmqskvq",
}
kids = {}
for name in which:
kids[unicode(name)] = (nm.create_from_cap(caps[name]), {})
return kids
def test_deep_immutable(self):
nm = NodeMaker(None, None, None, None, None, None, {"k": 3, "n": 10},
None)
fn = MinimalFakeMutableFile()
kids = self._make_kids(nm, ["imm", "lit", "write", "read",
"dirwrite", "dirread"])
minor code clean-up in dirnode.py Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring. fixes #967
2010-07-14 06:02:55 +00:00
packed = dirnode.pack_children(kids, fn.get_writekey(), deep_immutable=False)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
self.failUnlessIn("lit", packed)
kids = self._make_kids(nm, ["imm", "lit"])
minor code clean-up in dirnode.py Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring. fixes #967
2010-07-14 06:02:55 +00:00
packed = dirnode.pack_children(kids, fn.get_writekey(), deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
self.failUnlessIn("lit", packed)
kids = self._make_kids(nm, ["imm", "lit", "write"])
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessRaises(dirnode.MustBeDeepImmutableError,
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
dirnode.pack_children,
minor code clean-up in dirnode.py Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring. fixes #967
2010-07-14 06:02:55 +00:00
kids, fn.get_writekey(), deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
# read-only is not enough: all children must be immutable
kids = self._make_kids(nm, ["imm", "lit", "read"])
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessRaises(dirnode.MustBeDeepImmutableError,
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
dirnode.pack_children,
minor code clean-up in dirnode.py Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring. fixes #967
2010-07-14 06:02:55 +00:00
kids, fn.get_writekey(), deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
kids = self._make_kids(nm, ["imm", "lit", "dirwrite"])
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessRaises(dirnode.MustBeDeepImmutableError,
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
dirnode.pack_children,
minor code clean-up in dirnode.py Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring. fixes #967
2010-07-14 06:02:55 +00:00
kids, fn.get_writekey(), deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
kids = self._make_kids(nm, ["imm", "lit", "dirread"])
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessRaises(dirnode.MustBeDeepImmutableError,
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
dirnode.pack_children,
minor code clean-up in dirnode.py Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring. fixes #967
2010-07-14 06:02:55 +00:00
kids, fn.get_writekey(), deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
class FakeMutableFile:
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
implements(IMutableFileNode)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
counter = 0
def __init__(self, initial_contents=""):
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
self.data = self._get_initial_contents(initial_contents)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
counter = FakeMutableFile.counter
FakeMutableFile.counter += 1
writekey = hashutil.ssk_writekey_hash(str(counter))
fingerprint = hashutil.ssk_pubkey_fingerprint_hash(str(counter))
self.uri = uri.WriteableSSKFileURI(writekey, fingerprint)
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
def _get_initial_contents(self, contents):
if isinstance(contents, str):
return contents
if contents is None:
return ""
assert callable(contents), "%s should be callable, not %s" % \
(contents, type(contents))
return contents(self)
stop using IURI()/etc as an adapter
2009-11-11 22:45:42 +00:00
def get_cap(self):
return self.uri
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def get_uri(self):
return self.uri.to_string()
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
def get_write_uri(self):
return self.uri.to_string()
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def download_best_version(self):
return defer.succeed(self.data)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def get_writekey(self):
return "writekey"
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def is_readonly(self):
return False
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def is_mutable(self):
return True
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
def is_unknown(self):
return False
def is_allowed_in_immutable_directory(self):
return False
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 23:06:42 +00:00
def raise_error(self):
pass
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def modify(self, modifier):
self.data = modifier(self.data, None, True)
return defer.succeed(None)
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class FakeNodeMaker(NodeMaker):
def create_mutable_file(self, contents="", keysize=None):
return defer.succeed(FakeMutableFile(contents))
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
class FakeClient2(Client):
def __init__(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.nodemaker = FakeNodeMaker(None, None, None,
None, None, None,
{"k":3,"n":10}, None)
def create_node_from_uri(self, rwcap, rocap):
return self.nodemaker.create_from_cap(rwcap, rocap)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
class Dirnode2(unittest.TestCase, testutil.ShouldFailMixin):
def setUp(self):
test_dirnode.py: reduce scope of a Client instance, suggested by Kevan.
2010-01-15 06:27:13 +00:00
client = FakeClient2()
self.nodemaker = client.nodemaker
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def test_from_future(self):
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
# Create a mutable directory that contains unknown URI types, and make sure
# we tolerate them properly.
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
d = self.nodemaker.create_new_mutable_directory()
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
future_write_uri = "x-tahoe-crazy://I_am_from_the_future."
future_read_uri = "x-tahoe-crazy-readonly://I_am_from_the_future."
future_imm_uri = "x-tahoe-crazy-immutable://I_am_from_the_future."
future_node = UnknownNode(future_write_uri, future_read_uri)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def _then(n):
self._node = n
return n.set_node(u"future", future_node)
d.addCallback(_then)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
# We should be prohibited from adding an unknown URI to a directory
# just in the rw_uri slot, since we don't know how to diminish the cap
# to a readcap (for the ro_uri slot).
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
d.addCallback(lambda ign:
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.shouldFail(MustNotBeUnknownRWError,
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
"copy unknown",
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
"cannot attach unknown rw cap as child",
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
self._node.set_uri, u"add",
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
future_write_uri, None))
# However, we should be able to add both rw_uri and ro_uri as a pair of
# unknown URIs.
d.addCallback(lambda ign: self._node.set_uri(u"add-pair",
future_write_uri, future_read_uri))
# and to add an URI prefixed with "ro." or "imm." when it is given in a
# write slot (or URL parameter).
d.addCallback(lambda ign: self._node.set_uri(u"add-ro",
"ro." + future_read_uri, None))
d.addCallback(lambda ign: self._node.set_uri(u"add-imm",
"imm." + future_imm_uri, None))
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
d.addCallback(lambda ign: self._node.list())
def _check(children):
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(len(children), 4)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
(fn, metadata) = children[u"future"]
self.failUnless(isinstance(fn, UnknownNode), fn)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(fn.get_uri(), future_write_uri)
self.failUnlessEqual(fn.get_write_uri(), future_write_uri)
self.failUnlessEqual(fn.get_readonly_uri(), "ro." + future_read_uri)
(fn2, metadata2) = children[u"add-pair"]
self.failUnless(isinstance(fn2, UnknownNode), fn2)
self.failUnlessEqual(fn2.get_uri(), future_write_uri)
self.failUnlessEqual(fn2.get_write_uri(), future_write_uri)
self.failUnlessEqual(fn2.get_readonly_uri(), "ro." + future_read_uri)
(fn3, metadata3) = children[u"add-ro"]
self.failUnless(isinstance(fn3, UnknownNode), fn3)
self.failUnlessEqual(fn3.get_uri(), "ro." + future_read_uri)
self.failUnlessEqual(fn3.get_write_uri(), None)
self.failUnlessEqual(fn3.get_readonly_uri(), "ro." + future_read_uri)
(fn4, metadata4) = children[u"add-imm"]
self.failUnless(isinstance(fn4, UnknownNode), fn4)
self.failUnlessEqual(fn4.get_uri(), "imm." + future_imm_uri)
self.failUnlessEqual(fn4.get_write_uri(), None)
self.failUnlessEqual(fn4.get_readonly_uri(), "imm." + future_imm_uri)
# We should also be allowed to copy the "future" UnknownNode, because
# it contains all the information that was in the original directory
# (readcap and writecap), so we're preserving everything.
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
return self._node.set_node(u"copy", fn)
d.addCallback(_check)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
d.addCallback(lambda ign: self._node.list())
def _check2(children):
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(len(children), 5)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
(fn, metadata) = children[u"copy"]
self.failUnless(isinstance(fn, UnknownNode), fn)
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
self.failUnlessEqual(fn.get_uri(), future_write_uri)
self.failUnlessEqual(fn.get_write_uri(), future_write_uri)
self.failUnlessEqual(fn.get_readonly_uri(), "ro." + future_read_uri)
d.addCallback(_check2)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
return d
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
def test_unknown_strip_prefix_for_ro(self):
self.failUnlessEqual(strip_prefix_for_ro("foo", False), "foo")
self.failUnlessEqual(strip_prefix_for_ro("ro.foo", False), "foo")
self.failUnlessEqual(strip_prefix_for_ro("imm.foo", False), "imm.foo")
self.failUnlessEqual(strip_prefix_for_ro("foo", True), "foo")
self.failUnlessEqual(strip_prefix_for_ro("ro.foo", True), "foo")
self.failUnlessEqual(strip_prefix_for_ro("imm.foo", True), "foo")
def test_unknownnode(self):
mut_write_uri = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
mut_read_uri = "URI:SSK-RO:jf6wkflosyvntwxqcdo7a54jvm:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
lit_uri = "URI:LIT:n5xgk"
# This does not attempt to be exhaustive.
no_no = [# Opaque node, but not an error.
( 0, UnknownNode(None, None)),
( 1, UnknownNode(None, None, deep_immutable=True)),
]
unknown_rw = [# These are errors because we're only given a rw_uri, and we can't
# diminish it.
( 2, UnknownNode("foo", None)),
( 3, UnknownNode("foo", None, deep_immutable=True)),
( 4, UnknownNode("ro.foo", None, deep_immutable=True)),
( 5, UnknownNode("ro." + mut_read_uri, None, deep_immutable=True)),
( 6, UnknownNode("URI:SSK-RO:foo", None, deep_immutable=True)),
( 7, UnknownNode("URI:SSK:foo", None)),
]
must_be_ro = [# These are errors because a readonly constraint is not met.
( 8, UnknownNode("ro." + mut_write_uri, None)),
( 9, UnknownNode(None, "ro." + mut_write_uri)),
]
must_be_imm = [# These are errors because an immutable constraint is not met.
(10, UnknownNode(None, "ro.URI:SSK-RO:foo", deep_immutable=True)),
(11, UnknownNode(None, "imm.URI:SSK:foo")),
(12, UnknownNode(None, "imm.URI:SSK-RO:foo")),
(13, UnknownNode("bar", "ro.foo", deep_immutable=True)),
(14, UnknownNode("bar", "imm.foo", deep_immutable=True)),
(15, UnknownNode("bar", "imm." + lit_uri, deep_immutable=True)),
(16, UnknownNode("imm." + mut_write_uri, None)),
(17, UnknownNode("imm." + mut_read_uri, None)),
(18, UnknownNode("bar", "imm.foo")),
]
bad_uri = [# These are errors because the URI is bad once we've stripped the prefix.
(19, UnknownNode("ro.URI:SSK-RO:foo", None)),
(20, UnknownNode("imm.URI:CHK:foo", None, deep_immutable=True)),
Improvements to test_unknownnode to cover invalid cap URIs with known prefixes
2010-01-30 06:39:08 +00:00
(21, UnknownNode(None, "URI:CHK:foo")),
(22, UnknownNode(None, "URI:CHK:foo", deep_immutable=True)),
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
]
ro_prefixed = [# These are valid, and the readcap should end up with a ro. prefix.
Improvements to test_unknownnode to cover invalid cap URIs with known prefixes
2010-01-30 06:39:08 +00:00
(23, UnknownNode(None, "foo")),
(24, UnknownNode(None, "ro.foo")),
(25, UnknownNode(None, "ro." + lit_uri)),
(26, UnknownNode("bar", "foo")),
(27, UnknownNode("bar", "ro.foo")),
(28, UnknownNode("bar", "ro." + lit_uri)),
(29, UnknownNode("ro.foo", None)),
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
(30, UnknownNode("ro." + lit_uri, None)),
]
imm_prefixed = [# These are valid, and the readcap should end up with an imm. prefix.
Improvements to test_unknownnode to cover invalid cap URIs with known prefixes
2010-01-30 06:39:08 +00:00
(31, UnknownNode(None, "foo", deep_immutable=True)),
(32, UnknownNode(None, "ro.foo", deep_immutable=True)),
(33, UnknownNode(None, "imm.foo")),
(34, UnknownNode(None, "imm.foo", deep_immutable=True)),
(35, UnknownNode("imm." + lit_uri, None)),
(36, UnknownNode("imm." + lit_uri, None, deep_immutable=True)),
(37, UnknownNode(None, "imm." + lit_uri)),
(38, UnknownNode(None, "imm." + lit_uri, deep_immutable=True)),
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-27 06:44:30 +00:00
]
error = unknown_rw + must_be_ro + must_be_imm + bad_uri
ok = ro_prefixed + imm_prefixed
for (i, n) in no_no + error + ok:
self.failUnless(n.is_unknown(), i)
for (i, n) in no_no + error:
self.failUnless(n.get_uri() is None, i)
self.failUnless(n.get_write_uri() is None, i)
self.failUnless(n.get_readonly_uri() is None, i)
for (i, n) in no_no + ok:
n.raise_error()
for (i, n) in unknown_rw:
self.failUnlessRaises(MustNotBeUnknownRWError, lambda: n.raise_error())
for (i, n) in must_be_ro:
self.failUnlessRaises(MustBeReadonlyError, lambda: n.raise_error())
for (i, n) in must_be_imm:
self.failUnlessRaises(MustBeDeepImmutableError, lambda: n.raise_error())
for (i, n) in bad_uri:
self.failUnlessRaises(uri.BadURIError, lambda: n.raise_error())
for (i, n) in ok:
self.failIf(n.get_readonly_uri() is None, i)
for (i, n) in ro_prefixed:
self.failUnless(n.get_readonly_uri().startswith("ro."), i)
for (i, n) in imm_prefixed:
self.failUnless(n.get_readonly_uri().startswith("imm."), i)
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
class DeepStats(unittest.TestCase):
tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel
2009-06-05 03:14:44 +00:00
timeout = 240 # It takes longer than 120 seconds on Francois's arm box.
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
def test_stats(self):
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
ds = dirnode.DeepStats(None)
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
ds.add("count-files")
ds.add("size-immutable-files", 123)
ds.histogram("size-files-histogram", 123)
ds.max("largest-directory", 444)
s = ds.get_results()
self.failUnlessEqual(s["count-files"], 1)
self.failUnlessEqual(s["size-immutable-files"], 123)
self.failUnlessEqual(s["largest-directory"], 444)
self.failUnlessEqual(s["count-literal-files"], 0)
ds.add("count-files")
ds.add("size-immutable-files", 321)
ds.histogram("size-files-histogram", 321)
ds.max("largest-directory", 2)
s = ds.get_results()
self.failUnlessEqual(s["count-files"], 2)
self.failUnlessEqual(s["size-immutable-files"], 444)
self.failUnlessEqual(s["largest-directory"], 444)
self.failUnlessEqual(s["count-literal-files"], 0)
self.failUnlessEqual(s["size-files-histogram"],
[ (101, 316, 1), (317, 1000, 1) ])
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
ds = dirnode.DeepStats(None)
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
for i in range(1, 1100):
ds.histogram("size-files-histogram", i)
ds.histogram("size-files-histogram", 4*1000*1000*1000*1000) # 4TB
s = ds.get_results()
self.failUnlessEqual(s["size-files-histogram"],
[ (1, 3, 3),
(4, 10, 7),
(11, 31, 21),
(32, 100, 69),
(101, 316, 216),
(317, 1000, 684),
(1001, 3162, 99),
(3162277660169L, 10000000000000L, 1),
])
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
class UCWEingMutableFileNode(MutableFileNode):
please_ucwe_after_next_upload = False
def _upload(self, new_contents, servermap):
d = MutableFileNode._upload(self, new_contents, servermap)
def _ucwe(res):
mutable.modify(): after UCWE, publish even if the second invocation of the modifier didn't modify anything. For #551.
2008-12-06 05:49:23 +00:00
if self.please_ucwe_after_next_upload:
self.please_ucwe_after_next_upload = False
raise UncoordinatedWriteError()
return res
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
d.addCallback(_ucwe)
return d
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class UCWEingNodeMaker(NodeMaker):
def _create_mutable(self, cap):
n = UCWEingMutableFileNode(self.storage_broker, self.secret_holder,
self.default_encoding_parameters,
self.history)
clean up uri-vs-cap terminology, emphasize cap instances instead of URI strings * "cap" means a python instance which encapsulates a filecap/dircap (uri.py) * "uri" means a string with a "URI:" prefix * FileNode instances are created with (and retain) a cap instance, and generate uri strings on demand * .get_cap/get_readcap/get_verifycap/get_repaircap return cap instances * .get_uri/get_readonly_uri return uri strings * add filenode.download_to_filename() for control.py, should find a better way * use MutableFileNode.init_from_cap, not .init_from_uri * directory URI instances: use get_filenode_cap, not get_filenode_uri * update/cleanup bench_dirnode.py to match, add Makefile target to run it
2009-11-11 22:25:42 +00:00
return n.init_from_cap(cap)
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-17 00:23:48 +00:00
class Deleter(GridTestMixin, unittest.TestCase):
test: multiple by 10 or so all timeouts that Zandr's ARM box just overran
2009-06-10 12:56:39 +00:00
timeout = 3600 # It takes longer than 433 seconds on Zandr's ARM box.
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
def test_retry(self):
# ticket #550, a dirnode.delete which experiences an
# UncoordinatedWriteError will fail with an incorrect "you're
# deleting something which isn't there" NoSuchChildError exception.
# to trigger this, we start by creating a directory with a single
# file in it. Then we create a special dirnode that uses a modified
# MutableFileNode which will raise UncoordinatedWriteError once on
# demand. We then call dirnode.delete, which ought to retry and
# succeed.
self.basedir = self.mktemp()
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-17 00:23:48 +00:00
self.set_up_grid()
c0 = self.g.clients[0]
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c0.create_dirnode()
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
small = upload.Data("Small enough for a LIT", None)
def _created_dir(dn):
self.root = dn
self.root_uri = dn.get_uri()
return dn.add_file(u"file", small)
d.addCallback(_created_dir)
def _do_delete(ignored):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
nm = UCWEingNodeMaker(c0.storage_broker, c0._secret_holder,
c0.get_history(), c0.getServiceNamed("uploader"),
immutable.Downloader: pass StorageBroker to constructor, stop being a Service child of the client, access with client.downloader instead of client.getServiceNamed("downloader"). The single "Downloader" instance is scheduled for demolition anyways, to be replaced by individual filenode.download calls.
2009-08-15 19:25:43 +00:00
c0.downloader,
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
c0.download_cache_dirman,
c0.get_encoding_parameters(),
c0._key_generator)
n = nm.create_from_cap(self.root_uri)
mutable.modify(): after UCWE, publish even if the second invocation of the modifier didn't modify anything. For #551.
2008-12-06 05:49:23 +00:00
assert n._node.please_ucwe_after_next_upload == False
n._node.please_ucwe_after_next_upload = True
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
# This should succeed, not raise an exception
return n.delete(u"file")
d.addCallback(_do_delete)
return d
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class Adder(GridTestMixin, unittest.TestCase, testutil.ShouldFailMixin):
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
def test_overwrite(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
# note: This functionality could be tested without actually creating
# several RSA keys. It would be faster without the GridTestMixin: use
# dn.set_node(nodemaker.create_from_cap(make_chk_file_uri())) instead
# of dn.add_file, and use a special NodeMaker that creates fake
# mutable files.
self.basedir = "dirnode/Adder/test_overwrite"
self.set_up_grid()
c = self.g.clients[0]
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
fileuri = make_chk_file_uri(1234)
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
filenode = c.nodemaker.create_from_cap(fileuri)
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
def _create_directory_tree(root_node):
# Build
# root/file1
# root/file2
# root/dir1
d = root_node.add_file(u'file1', upload.Data("Important Things",
Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'
2009-07-20 03:43:18 +00:00
None))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res:
Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'
2009-07-20 03:43:18 +00:00
root_node.add_file(u'file2', upload.Data("Sekrit Codes", None)))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res:
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
root_node.create_subdirectory(u"dir1"))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res: root_node)
return d
d.addCallback(_create_directory_tree)
def _test_adder(root_node):
d = root_node.set_node(u'file1', filenode)
# We've overwritten file1. Let's try it with a directory
d.addCallback(lambda res:
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
root_node.create_subdirectory(u'dir2'))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res:
root_node.set_node(u'dir2', filenode))
# We try overwriting a file with a child while also specifying
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
# overwrite=False. We should receive an ExistingChildError
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
# when we do this.
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_node",
"child 'file1' already exists",
Trivial whitespace changes.
2010-05-27 19:41:14 +00:00
root_node.set_node, u"file1",
filenode, overwrite=False))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
# If we try with a directory, we should see the same thing
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_node",
"child 'dir1' already exists",
root_node.set_node, u'dir1', filenode,
overwrite=False))
d.addCallback(lambda res:
Trivial whitespace changes.
2010-05-27 19:41:14 +00:00
root_node.set_node(u'file1', filenode,
overwrite="only-files"))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res:
Trivial whitespace changes.
2010-05-27 19:41:14 +00:00
self.shouldFail(ExistingChildError, "set_node",
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
"child 'dir1' already exists",
root_node.set_node, u'dir1', filenode,
Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'
2009-07-20 03:43:18 +00:00
overwrite="only-files"))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
return d
d.addCallback(_test_adder)
return d
Reference in New Issue Copy Permalink