ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-28 16:58:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
3880486f91
tahoe-lafs/src/allmydata/test/test_dirnode.py

1264 lines
62 KiB
Python
Raw Normal View History

test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
import time
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
from zope.interface import implements
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
from twisted.trial import unittest
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
from twisted.internet import defer
move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected.
2008-07-16 20:14:39 +00:00
from allmydata import uri, dirnode
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
from allmydata.client import Client
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
from allmydata.immutable import upload
interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode The proper hierarchy is: IFilesystemNode +IFileNode ++IMutableFileNode ++IImmutableFileNode +IDirectoryNode Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.
2009-11-20 07:52:55 +00:00
from allmydata.interfaces import IImmutableFileNode, IMutableFileNode, \
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
ExistingChildError, NoSuchChildError, NotDeepImmutableError, \
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
IDeepCheckResults, IDeepCheckAndRepairResults, CannotPackUnknownNodeError
mutable: merge renaming with test patches
2008-12-07 15:45:19 +00:00
from allmydata.mutable.filenode import MutableFileNode
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
from allmydata.mutable.common import UncoordinatedWriteError
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
from allmydata.util import hashutil, base32
more #514: pass a Monitor to all checker operations, make mutable-checker honor the cancel flag
2008-10-22 08:38:18 +00:00
from allmydata.monitor import Monitor
refactor web tests, and interfaces.IFileNode
2007-12-05 06:01:37 +00:00
from allmydata.test.common import make_chk_file_uri, make_mutable_file_uri, \
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
ErrorMixin
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-17 00:23:48 +00:00
from allmydata.test.no_network import GridTestMixin
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
from allmydata.unknown import UnknownNode
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
from allmydata.nodemaker import NodeMaker
Add tests for CachingDict, _pack_contents, _unpack_contents
2009-07-04 03:43:28 +00:00
from base64 import b32decode
move testutil into test/common_util.py, since it doesn't count as 'code under test' for our pyflakes numbers
2008-10-29 04:28:31 +00:00
import common_util as testutil
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class Dirnode(GridTestMixin, unittest.TestCase,
test_dirnode: add an explainError call
2008-11-19 23:02:12 +00:00
testutil.ShouldFailMixin, testutil.StallMixin, ErrorMixin):
tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel
2009-06-05 03:14:44 +00:00
timeout = 240 # It takes longer than 120 seconds on Francois's arm box.
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def test_basic(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_basic"
self.set_up_grid()
c = self.g.clients[0]
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _done(res):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.failUnless(isinstance(res, dirnode.DirectoryNode))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
rep = str(res)
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
self.failUnless("RW-MUT" in rep)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_done)
return d
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
def test_initial_children(self):
self.basedir = "dirnode/Dirnode/test_initial_children"
self.set_up_grid()
c = self.g.clients[0]
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
nm = c.nodemaker
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
setup_py_uri = "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861"
one_uri = "URI:LIT:n5xgk" # LIT for "one"
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
kids = {u"one": (nm.create_from_cap(one_uri), {}),
u"two": (nm.create_from_cap(setup_py_uri),
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
{"metakey": "metavalue"}),
}
d = c.create_dirnode(kids)
def _created(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
rep = str(dn)
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
self.failUnless("RW-MUT" in rep)
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
return dn.list()
d.addCallback(_created)
def _check_kids(children):
self.failUnlessEqual(sorted(children.keys()), [u"one", u"two"])
one_node, one_metadata = children[u"one"]
two_node, two_metadata = children[u"two"]
self.failUnlessEqual(one_node.get_size(), 3)
self.failUnlessEqual(two_node.get_size(), 14861)
self.failUnless(isinstance(one_metadata, dict), one_metadata)
self.failUnlessEqual(two_metadata["metakey"], "metavalue")
d.addCallback(_check_kids)
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
d.addCallback(lambda ign: nm.create_new_mutable_directory(kids))
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
d.addCallback(lambda dn: dn.list())
d.addCallback(_check_kids)
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
future_writecap = "x-tahoe-crazy://I_am_from_the_future."
future_readcap = "x-tahoe-crazy-readonly://I_am_from_the_future."
future_node = UnknownNode(future_writecap, future_readcap)
bad_kids1 = {u"one": (future_node, {})}
d.addCallback(lambda ign:
self.shouldFail(AssertionError, "bad_kids1",
"does not accept UnknownNode",
nm.create_new_mutable_directory,
bad_kids1))
bad_kids2 = {u"one": (nm.create_from_cap(one_uri), None)}
d.addCallback(lambda ign:
self.shouldFail(AssertionError, "bad_kids2",
"requires metadata to be a dict",
nm.create_new_mutable_directory,
bad_kids2))
test_dirnode.py: add tests of initial_children= args to client.create_dirnode and nodemaker.create_new_mutable_directory
2009-10-17 19:41:59 +00:00
return d
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
def test_immutable(self):
self.basedir = "dirnode/Dirnode/test_immutable"
self.set_up_grid()
c = self.g.clients[0]
nm = c.nodemaker
setup_py_uri = "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861"
one_uri = "URI:LIT:n5xgk" # LIT for "one"
mut_readcap = "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q"
mut_writecap = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
kids = {u"one": (nm.create_from_cap(one_uri), {}),
u"two": (nm.create_from_cap(setup_py_uri),
{"metakey": "metavalue"}),
}
d = c.create_immutable_dirnode(kids)
def _created(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
self.failIf(dn.is_mutable())
self.failUnless(dn.is_readonly())
rep = str(dn)
self.failUnless("RO-IMM" in rep)
cap = dn.get_cap()
self.failUnlessIn("CHK", cap.to_string())
self.cap = cap
return dn.list()
d.addCallback(_created)
def _check_kids(children):
self.failUnlessEqual(sorted(children.keys()), [u"one", u"two"])
one_node, one_metadata = children[u"one"]
two_node, two_metadata = children[u"two"]
self.failUnlessEqual(one_node.get_size(), 3)
self.failUnlessEqual(two_node.get_size(), 14861)
self.failUnless(isinstance(one_metadata, dict), one_metadata)
self.failUnlessEqual(two_metadata["metakey"], "metavalue")
d.addCallback(_check_kids)
d.addCallback(lambda ign: nm.create_from_cap(self.cap.to_string()))
d.addCallback(lambda dn: dn.list())
d.addCallback(_check_kids)
future_writecap = "x-tahoe-crazy://I_am_from_the_future."
future_readcap = "x-tahoe-crazy-readonly://I_am_from_the_future."
future_node = UnknownNode(future_writecap, future_readcap)
bad_kids1 = {u"one": (future_node, {})}
d.addCallback(lambda ign:
self.shouldFail(AssertionError, "bad_kids1",
"does not accept UnknownNode",
c.create_immutable_dirnode,
bad_kids1))
bad_kids2 = {u"one": (nm.create_from_cap(one_uri), None)}
d.addCallback(lambda ign:
self.shouldFail(AssertionError, "bad_kids2",
"requires metadata to be a dict",
c.create_immutable_dirnode,
bad_kids2))
bad_kids3 = {u"one": (nm.create_from_cap(mut_writecap), {})}
d.addCallback(lambda ign:
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
self.shouldFail(NotDeepImmutableError, "bad_kids3",
"is not immutable",
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
c.create_immutable_dirnode,
bad_kids3))
bad_kids4 = {u"one": (nm.create_from_cap(mut_readcap), {})}
d.addCallback(lambda ign:
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
self.shouldFail(NotDeepImmutableError, "bad_kids4",
"is not immutable",
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
c.create_immutable_dirnode,
bad_kids4))
d.addCallback(lambda ign: c.create_immutable_dirnode({}))
def _created_empty(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
self.failIf(dn.is_mutable())
self.failUnless(dn.is_readonly())
rep = str(dn)
self.failUnless("RO-IMM" in rep)
cap = dn.get_cap()
self.failUnlessIn("LIT", cap.to_string())
self.failUnlessEqual(cap.to_string(), "URI:DIR2-LIT:")
self.cap = cap
return dn.list()
d.addCallback(_created_empty)
d.addCallback(lambda kids: self.failUnlessEqual(kids, {}))
smallkids = {u"o": (nm.create_from_cap(one_uri), {})}
d.addCallback(lambda ign: c.create_immutable_dirnode(smallkids))
def _created_small(dn):
self.failUnless(isinstance(dn, dirnode.DirectoryNode))
self.failIf(dn.is_mutable())
self.failUnless(dn.is_readonly())
rep = str(dn)
self.failUnless("RO-IMM" in rep)
cap = dn.get_cap()
self.failUnlessIn("LIT", cap.to_string())
self.failUnlessEqual(cap.to_string(),
"URI:DIR2-LIT:gi4tumj2n4wdcmz2kvjesosmjfkdu3rvpbtwwlbqhiwdeot3puwcy")
self.cap = cap
return dn.list()
d.addCallback(_created_small)
d.addCallback(lambda kids: self.failUnlessEqual(kids.keys(), [u"o"]))
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
# now test n.create_subdirectory(mutable=False)
d.addCallback(lambda ign: c.create_dirnode())
def _made_parent(n):
d = n.create_subdirectory(u"subdir", kids, mutable=False)
d.addCallback(lambda sd: sd.list())
d.addCallback(_check_kids)
d.addCallback(lambda ign: n.list())
d.addCallback(lambda children:
self.failUnlessEqual(children.keys(), [u"subdir"]))
d.addCallback(lambda ign: n.get(u"subdir"))
d.addCallback(lambda sd: sd.list())
d.addCallback(_check_kids)
d.addCallback(lambda ign: n.get(u"subdir"))
d.addCallback(lambda sd: self.failIf(sd.is_mutable()))
bad_kids = {u"one": (nm.create_from_cap(mut_writecap), {})}
d.addCallback(lambda ign:
self.shouldFail(NotDeepImmutableError, "YZ",
"is not immutable",
n.create_subdirectory,
u"sub2", bad_kids, mutable=False))
return d
d.addCallback(_made_parent)
nodemaker: implement immutable directories (internal interface), for #607 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet
2009-11-12 00:22:33 +00:00
return d
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def test_check(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_check"
self.set_up_grid()
c = self.g.clients[0]
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
more #514: pass a Monitor to all checker operations, make mutable-checker honor the cancel flag
2008-10-22 08:38:18 +00:00
d.addCallback(lambda dn: dn.check(Monitor()))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _done(res):
overhaul checker invocation Removed the Checker service, removed checker results storage (both in-memory and the tiny stub of sqlite-based storage). Added ICheckable, all check/verify is now done by calling the check() method on filenodes and dirnodes (immutable files, literal files, mutable files, and directory instances). Checker results are returned in a Results instance, with an html() method for display. Checker results have been temporarily removed from the wui directory listing until we make some other fixes. Also fixed client.create_node_from_uri() to create LiteralFileNodes properly, since they have different checking behavior. Previously we were creating full FileNodes with LIT uris inside, which were downloadable but not checkable.
2008-07-16 00:23:25 +00:00
self.failUnless(res.is_healthy())
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_done)
return d
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
def _test_deepcheck_create(self):
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
# create a small tree with a loop, and some non-directories
# root/
# root/subdir/
# root/subdir/file1
# root/subdir/link -> root
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
# root/rodir
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
c = self.g.clients[0]
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
def _created_root(rootnode):
self._rootnode = rootnode
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
return rootnode.create_subdirectory(u"subdir")
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
d.addCallback(_created_root)
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
def _created_subdir(subdir):
self._subdir = subdir
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
d = subdir.add_file(u"file1", upload.Data("data"*100, None))
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
d.addCallback(lambda res: subdir.set_node(u"link", self._rootnode))
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d.addCallback(lambda res: c.create_dirnode())
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
d.addCallback(lambda dn:
self._rootnode.set_uri(u"rodir",
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
dn.get_uri(),
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
dn.get_readonly_uri()))
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 21:37:04 +00:00
return d
d.addCallback(_created_subdir)
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
def _done(res):
return self._rootnode
d.addCallback(_done)
return d
def test_deepcheck(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_deepcheck"
self.set_up_grid()
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
d = self._test_deepcheck_create()
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
d.addCallback(lambda rootnode: rootnode.start_deep_check().when_done())
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
def _check_results(r):
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
self.failUnless(IDeepCheckResults.providedBy(r))
c = r.get_counters()
self.failUnlessEqual(c,
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
{"count-objects-checked": 4,
"count-objects-healthy": 4,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-objects-unhealthy": 0,
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-07 05:35:47 +00:00
"count-objects-unrecoverable": 0,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-corrupt-shares": 0,
})
self.failIf(r.get_corrupt_shares())
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
self.failUnlessEqual(len(r.get_all_results()), 4)
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
d.addCallback(_check_results)
return d
def test_deepcheck_and_repair(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_deepcheck_and_repair"
self.set_up_grid()
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
d = self._test_deepcheck_create()
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
d.addCallback(lambda rootnode:
rootnode.start_deep_check_and_repair().when_done())
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
def _check_results(r):
self.failUnless(IDeepCheckAndRepairResults.providedBy(r))
c = r.get_counters()
self.failUnlessEqual(c,
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
{"count-objects-checked": 4,
"count-objects-healthy-pre-repair": 4,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-objects-unhealthy-pre-repair": 0,
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-07 05:35:47 +00:00
"count-objects-unrecoverable-pre-repair": 0,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-corrupt-shares-pre-repair": 0,
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
"count-objects-healthy-post-repair": 4,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-objects-unhealthy-post-repair": 0,
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-07 05:35:47 +00:00
"count-objects-unrecoverable-post-repair": 0,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-corrupt-shares-post-repair": 0,
"count-repairs-attempted": 0,
"count-repairs-successful": 0,
"count-repairs-unsuccessful": 0,
})
self.failIf(r.get_corrupt_shares())
self.failIf(r.get_remaining_corrupt_shares())
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
self.failUnlessEqual(len(r.get_all_results()), 4)
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-17 01:20:57 +00:00
d.addCallback(_check_results)
return d
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
def _mark_file_bad(self, rootnode):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.delete_shares_numbered(rootnode.get_uri(), [0])
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
return rootnode
def test_deepcheck_problems(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_deepcheck_problems"
self.set_up_grid()
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
d = self._test_deepcheck_create()
d.addCallback(lambda rootnode: self._mark_file_bad(rootnode))
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
d.addCallback(lambda rootnode: rootnode.start_deep_check().when_done())
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
def _check_results(r):
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
c = r.get_counters()
self.failUnlessEqual(c,
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 21:53:37 +00:00
{"count-objects-checked": 4,
"count-objects-healthy": 3,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-objects-unhealthy": 1,
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-07 05:35:47 +00:00
"count-objects-unrecoverable": 0,
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 19:44:56 +00:00
"count-corrupt-shares": 0,
})
#self.failUnlessEqual(len(r.get_problems()), 1) # TODO
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-12 04:03:26 +00:00
d.addCallback(_check_results)
return d
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def test_readonly(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_readonly"
self.set_up_grid()
c = self.g.clients[0]
nm = c.nodemaker
filecap = make_chk_file_uri(1234)
filenode = nm.create_from_cap(filecap)
use added secret to protect convergent encryption Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key. If you want traditional convergent encryption behavior, set the added secret to be the empty string. This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names. (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.) This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 16:46:06 +00:00
uploadable = upload.Data("some data", convergence="some convergence string")
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _created(rw_dn):
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d2 = rw_dn.set_uri(u"child", filecap, filecap)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d2.addCallback(lambda res: rw_dn)
return d2
d.addCallback(_created)
def _ready(rw_dn):
ro_uri = rw_dn.get_readonly_uri()
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
ro_dn = c.create_node_from_uri(ro_uri)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.failUnless(ro_dn.is_readonly())
self.failUnless(ro_dn.is_mutable())
self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
ro_dn.set_uri, u"newchild", filecap, filecap)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
ro_dn.set_node, u"newchild", filenode)
test_dirnode: improve coverage of not-mutable-error a bit
2008-05-08 23:53:35 +00:00
self.shouldFail(dirnode.NotMutableError, "set_nodes ro", None,
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
ro_dn.set_nodes, { u"newchild": (filenode, None) })
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
ro_dn.add_file, u"newchild", uploadable)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
ro_dn.delete, u"child")
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
ro_dn.create_subdirectory, u"newchild")
test_dirnode: improve coverage of not-mutable-error a bit
2008-05-08 23:53:35 +00:00
self.shouldFail(dirnode.NotMutableError, "set_metadata_for ro", None,
ro_dn.set_metadata_for, u"child", {})
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
ro_dn.move_child_to, u"child", rw_dn)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
rw_dn.move_child_to, u"child", ro_dn)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
return ro_dn.list()
d.addCallback(_ready)
def _listed(children):
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
self.failUnless(u"child" in children)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_listed)
return d
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
def failUnlessGreaterThan(self, a, b):
test_dirnode.py: improve error messages in case timestamp tests fail
2008-02-12 00:37:51 +00:00
self.failUnless(a > b, "%r should be > %r" % (a, b))
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
test_dirnode.py: add diag output to test-ctime/mtime tests
2008-02-11 21:13:07 +00:00
def failUnlessGreaterOrEqualThan(self, a, b):
test_dirnode.py: improve error messages in case timestamp tests fail
2008-02-12 00:37:51 +00:00
self.failUnless(a >= b, "%r should be >= %r" % (a, b))
test_dirnode.py: add diag output to test-ctime/mtime tests
2008-02-11 21:13:07 +00:00
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def test_create(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.basedir = "dirnode/Dirnode/test_create"
self.set_up_grid()
c = self.g.clients[0]
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.expected_manifest = []
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.expected_verifycaps = set()
self.expected_storage_indexes = set()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _then(n):
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
# /
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.rootnode = n
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.failUnless(n.is_mutable())
u = n.get_uri()
self.failUnless(u)
self.failUnless(u.startswith("URI:DIR2:"), u)
u_ro = n.get_readonly_uri()
self.failUnless(u_ro.startswith("URI:DIR2-RO:"), u_ro)
rename "get_verifier()" to "get_verify_cap()"
2008-12-08 19:44:11 +00:00
u_v = n.get_verify_cap().to_string()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.failUnless(u_v.startswith("URI:DIR2-Verifier:"), u_v)
dirnode: add get_repair_cap()
2009-01-23 04:44:49 +00:00
u_r = n.get_repair_cap().to_string()
self.failUnlessEqual(u_r, u)
web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.
2008-10-07 04:36:18 +00:00
self.expected_manifest.append( ((), u) )
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.expected_verifycaps.add(u_v)
si = n.get_storage_index()
self.expected_storage_indexes.add(base32.b2a(si))
IFilesystemNode: add get_storage_index(), it makes tests easier
2008-08-12 23:14:07 +00:00
expected_si = n._uri._filenode_uri.storage_index
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.failUnlessEqual(si, expected_si)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d = n.list()
d.addCallback(lambda res: self.failUnlessEqual(res, {}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.has_child(u"missing"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: self.failIf(res))
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
fake_file_uri = make_mutable_file_uri()
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
other_file_uri = make_mutable_file_uri()
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
m = c.nodemaker.create_from_cap(fake_file_uri)
rename "get_verifier()" to "get_verify_cap()"
2008-12-08 19:44:11 +00:00
ffu_v = m.get_verify_cap().to_string()
web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.
2008-10-07 04:36:18 +00:00
self.expected_manifest.append( ((u"child",) , m.get_uri()) )
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.expected_verifycaps.add(ffu_v)
self.expected_storage_indexes.add(base32.b2a(m.get_storage_index()))
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"child",
fake_file_uri, fake_file_uri))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_uri-no",
"child 'child' already exists",
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
n.set_uri, u"child",
other_file_uri, other_file_uri,
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
# /
# /child = mutable
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
d.addCallback(lambda res: n.create_subdirectory(u"subdir"))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
# /
# /child = mutable
# /subdir = directory
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _created(subdir):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.failUnless(isinstance(subdir, dirnode.DirectoryNode))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
self.subdir = subdir
rename "get_verifier()" to "get_verify_cap()"
2008-12-08 19:44:11 +00:00
new_v = subdir.get_verify_cap().to_string()
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
assert isinstance(new_v, str)
web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.
2008-10-07 04:36:18 +00:00
self.expected_manifest.append( ((u"subdir",), subdir.get_uri()) )
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.expected_verifycaps.add(new_v)
si = subdir.get_storage_index()
self.expected_storage_indexes.add(base32.b2a(si))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_created)
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "mkdir-no",
"child 'subdir' already exists",
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
n.create_subdirectory, u"subdir",
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
self.failUnlessEqual(sorted(children.keys()),
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
sorted([u"child", u"subdir"])))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
d.addCallback(lambda res: n.start_deep_stats().when_done())
dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too.
2008-05-08 20:21:14 +00:00
def _check_deepstats(stats):
self.failUnless(isinstance(stats, dict))
expected = {"count-immutable-files": 0,
"count-mutable-files": 1,
"count-literal-files": 0,
"count-files": 1,
"count-directories": 2,
"size-immutable-files": 0,
"size-literal-files": 0,
#"size-directories": 616, # varies
#"largest-directory": 616,
"largest-directory-children": 2,
"largest-immutable-file": 0,
}
for k,v in expected.iteritems():
self.failUnlessEqual(stats[k], v,
"stats[%s] was %s, not %s" %
(k, stats[k], v))
test_dirnode: the size of the directories varies, tolerate it as low as 500 bytes
2008-05-08 20:42:42 +00:00
self.failUnless(stats["size-directories"] > 500,
dirnode: refactor deep-stats a bit
2008-05-08 20:33:07 +00:00
stats["size-directories"])
test_dirnode: the size of the directories varies, tolerate it as low as 500 bytes
2008-05-08 20:42:42 +00:00
self.failUnless(stats["largest-directory"] > 500,
dirnode: refactor deep-stats a bit
2008-05-08 20:33:07 +00:00
stats["largest-directory"])
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
self.failUnlessEqual(stats["size-files-histogram"], [])
dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too.
2008-05-08 20:21:14 +00:00
d.addCallback(_check_deepstats)
manifest: include stats in results. webapi is unchanged.
2008-11-19 22:03:47 +00:00
d.addCallback(lambda res: n.build_manifest().when_done())
def _check_manifest(res):
manifest = res["manifest"]
self.failUnlessEqual(sorted(manifest),
sorted(self.expected_manifest))
stats = res["stats"]
_check_deepstats(stats)
dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.
2008-11-24 21:40:46 +00:00
self.failUnlessEqual(self.expected_verifycaps,
res["verifycaps"])
self.failUnlessEqual(self.expected_storage_indexes,
res["storage-index"])
manifest: include stats in results. webapi is unchanged.
2008-11-19 22:03:47 +00:00
d.addCallback(_check_manifest)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _add_subsubdir(res):
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
return self.subdir.create_subdirectory(u"subsubdir")
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_add_subsubdir)
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
# /
# /child = mutable
# /subdir = directory
# /subdir/subsubdir = directory
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_child_at_path(u"subdir/subsubdir"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda subsubdir:
self.failUnless(isinstance(subsubdir,
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
dirnode.DirectoryNode)))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_child_at_path(u""))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: self.failUnlessEqual(res.get_uri(),
n.get_uri()))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"child"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
d.addCallback(lambda res:
dirnode lookup: use distinct NoSuchChildError instead of the generic KeyError when a child can't be found
2008-10-27 20:15:25 +00:00
self.shouldFail(NoSuchChildError, "gcamap-no",
"nope",
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-03 01:08:45 +00:00
n.get_child_and_metadata_at_path,
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
u"subdir/nope"))
d.addCallback(lambda res:
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-03 01:08:45 +00:00
n.get_child_and_metadata_at_path(u""))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
def _check_child_and_metadata1(res):
child, metadata = res
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.failUnless(isinstance(child, dirnode.DirectoryNode))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
# edge-metadata needs at least one path segment
self.failUnlessEqual(sorted(metadata.keys()), [])
d.addCallback(_check_child_and_metadata1)
d.addCallback(lambda res:
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-03 01:08:45 +00:00
n.get_child_and_metadata_at_path(u"child"))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
def _check_child_and_metadata2(res):
child, metadata = res
self.failUnlessEqual(child.get_uri(),
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
fake_file_uri)
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"]))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
d.addCallback(_check_child_and_metadata2)
d.addCallback(lambda res:
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-03 01:08:45 +00:00
n.get_child_and_metadata_at_path(u"subdir/subsubdir"))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
def _check_child_and_metadata3(res):
child, metadata = res
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.failUnless(isinstance(child, dirnode.DirectoryNode))
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"]))
dirnode: add get_child_and_metadata_at_path
2008-10-03 00:52:03 +00:00
d.addCallback(_check_child_and_metadata3)
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# set_uri + metadata
# it should be possible to add a child without any metadata
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"c2",
fake_file_uri, fake_file_uri,
{}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"c2"))
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
d.addCallback(lambda metadata: self.failUnlessEqual(metadata.keys(), ['tahoe']))
# You can't override the link timestamps.
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"c2",
fake_file_uri, fake_file_uri,
{ 'tahoe': {'linkcrtime': "bogus"}}))
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"c2"))
def _has_good_linkcrtime(metadata):
self.failUnless(metadata.has_key('tahoe'))
self.failUnless(metadata['tahoe'].has_key('linkcrtime'))
self.failIfEqual(metadata['tahoe']['linkcrtime'], 'bogus')
d.addCallback(_has_good_linkcrtime)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# if we don't set any defaults, the child should get timestamps
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"c3",
fake_file_uri, fake_file_uri))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"c3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# or we can add specific metadata at set_uri() time, which
# overrides the timestamps
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res: n.set_uri(u"c4",
fake_file_uri, fake_file_uri,
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
{"key": "value"}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"c4"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
trivial: remove trailing whitespace and unused import
2009-04-12 02:17:42 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
(metadata['key'] == "value"), metadata))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"c2"))
d.addCallback(lambda res: n.delete(u"c3"))
d.addCallback(lambda res: n.delete(u"c4"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# set_node + metadata
# it should be possible to add a child without any metadata
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"d2", n, {}))
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d.addCallback(lambda res: c.create_dirnode())
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda n2:
self.shouldFail(ExistingChildError, "set_node-no",
"child 'd2' already exists",
n.set_node, u"d2", n2,
overwrite=False))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"d2"))
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
d.addCallback(lambda metadata: self.failUnlessEqual(metadata.keys(), ['tahoe']))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# if we don't set any defaults, the child should get timestamps
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"d3", n))
d.addCallback(lambda res: n.get_metadata_for(u"d3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# or we can add specific metadata at set_node() time, which
# overrides the timestamps
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"d4", n,
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
{"key": "value"}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"d4"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
trivial: remove trailing whitespace and unused import
2009-04-12 02:17:42 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
(metadata['key'] == "value"), metadata))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"d2"))
d.addCallback(lambda res: n.delete(u"d3"))
d.addCallback(lambda res: n.delete(u"d4"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
wapi: add POST /uri/$DIRECTORY?t=set_children Unfinished bits: doc in webapi.txt, test handling of badly formed JSON, return reasonable HTTP response, examination of the effect of this patch on code coverage -- but I'm committing it anyway because MikeB can use it and I'm being called to dinner...
2008-03-01 01:40:27 +00:00
# metadata through set_children()
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
d.addCallback(lambda res:
dirnode.set_children: take a dict, not a list
2009-10-13 00:24:40 +00:00
n.set_children({
u"e1": (fake_file_uri, fake_file_uri),
u"e2": (fake_file_uri, fake_file_uri, {}),
u"e3": (fake_file_uri, fake_file_uri,
{"key": "value"}),
}))
dirnode.set_children: change return value: fire with self instead of None
2009-10-13 01:50:26 +00:00
d.addCallback(lambda n2: self.failUnlessIdentical(n2, n))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_children-no",
"child 'e1' already exists",
n.set_children,
dirnode.set_children: take a dict, not a list
2009-10-13 00:24:40 +00:00
{ u"e1": (other_file_uri,
other_file_uri),
u"new": (other_file_uri,
other_file_uri),
},
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
# and 'new' should not have been created
d.addCallback(lambda res: n.list())
d.addCallback(lambda children: self.failIf(u"new" in children))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"e1"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"e2"))
trivial: remove trailing whitespace and unused import
2009-04-12 02:17:42 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()), set(['tahoe'])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"e3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
trivial: remove trailing whitespace and unused import
2009-04-12 02:17:42 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"]))
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
and (metadata['key'] == "value"), metadata))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"e1"))
d.addCallback(lambda res: n.delete(u"e2"))
d.addCallback(lambda res: n.delete(u"e3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
# metadata through set_nodes()
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
d.addCallback(lambda res:
n.set_nodes({ u"f1": (n, None),
u"f2": (n, {}),
u"f3": (n, {"key": "value"}),
}))
dirnode.set_nodes: change return value: fire with self instead of None
2009-10-13 01:45:46 +00:00
d.addCallback(lambda n2: self.failUnlessIdentical(n2, n))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_nodes-no",
"child 'f1' already exists",
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
n.set_nodes, { u"f1": (n, None),
u"new": (n, None), },
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
# and 'new' should not have been created
d.addCallback(lambda res: n.list())
d.addCallback(lambda children: self.failIf(u"new" in children))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"f1"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"f2"))
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
d.addCallback(
lambda metadata: self.failUnlessEqual(set(metadata.keys()), set(['tahoe'])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"f3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
trivial: remove trailing whitespace and unused import
2009-04-12 02:17:42 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
(metadata['key'] == "value"), metadata))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"f1"))
d.addCallback(lambda res: n.delete(u"f2"))
d.addCallback(lambda res: n.delete(u"f3"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
d.addCallback(lambda res:
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
n.set_metadata_for(u"child",
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
{"tags": ["web2.0-compatible"]}))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda n1: n1.get_metadata_for(u"child"))
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
d.addCallback(lambda metadata:
self.failUnlessEqual(metadata,
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
{"tags": ["web2.0-compatible"]}))
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
def _start(res):
self._start_timestamp = time.time()
d.addCallback(_start)
test_dirnode.py: simplejson-1.7.1 incorrectly rounds floats to two decimal places. Don't let this bug flunk the timestamp test.
2008-02-12 00:35:17 +00:00
# simplejson-1.7.1 (as shipped on Ubuntu 'gutsy') rounds all
# floats to hundredeths (it uses str(num) instead of repr(num)).
# simplejson-1.7.3 does not have this bug. To prevent this bug
# from causing the test to fail, stall for more than a few
# hundrededths of a second.
d.addCallback(self.stall, 0.1)
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.add_file(u"timestamps",
use added secret to protect convergent encryption Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key. If you want traditional convergent encryption behavior, set the added secret to be the empty string. This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names. (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.) This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 16:46:06 +00:00
upload.Data("stamp me", convergence="some convergence string")))
test_dirnode.py: simplejson-1.7.1 incorrectly rounds floats to two decimal places. Don't let this bug flunk the timestamp test.
2008-02-12 00:35:17 +00:00
d.addCallback(self.stall, 0.1)
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
def _stop(res):
self._stop_timestamp = time.time()
d.addCallback(_stop)
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"timestamps"))
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
def _check_timestamp1(metadata):
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
self.failUnless("ctime" in metadata)
self.failUnless("mtime" in metadata)
test_dirnode.py: add diag output to test-ctime/mtime tests
2008-02-11 21:13:07 +00:00
self.failUnlessGreaterOrEqualThan(metadata["ctime"],
self._start_timestamp)
self.failUnlessGreaterOrEqualThan(self._stop_timestamp,
metadata["ctime"])
self.failUnlessGreaterOrEqualThan(metadata["mtime"],
self._start_timestamp)
self.failUnlessGreaterOrEqualThan(self._stop_timestamp,
metadata["mtime"])
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
# Our current timestamp rules say that replacing an existing
# child should preserve the 'ctime' but update the mtime
self._old_ctime = metadata["ctime"]
self._old_mtime = metadata["mtime"]
d.addCallback(_check_timestamp1)
d.addCallback(self.stall, 2.0) # accomodate low-res timestamps
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"timestamps", n))
d.addCallback(lambda res: n.get_metadata_for(u"timestamps"))
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
def _check_timestamp2(metadata):
self.failUnlessEqual(metadata["ctime"], self._old_ctime,
"%s != %s" % (metadata["ctime"],
self._old_ctime))
self.failUnlessGreaterThan(metadata["mtime"], self._old_mtime)
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
return n.delete(u"timestamps")
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
d.addCallback(_check_timestamp2)
# also make sure we can add/update timestamps on a
# previously-existing child that didn't have any, since there are
# a lot of 0.7.0-generated edges around out there
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.set_node(u"no_timestamps", n, {}))
d.addCallback(lambda res: n.set_node(u"no_timestamps", n))
d.addCallback(lambda res: n.get_metadata_for(u"no_timestamps"))
test_dirnode.py: assert that we update mtime and preserve ctime
2008-02-11 22:12:55 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"])))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"no_timestamps"))
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.
2008-02-09 01:43:47 +00:00
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"subdir"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda old_child:
self.failUnlessEqual(old_child.get_uri(),
self.subdir.get_uri()))
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
self.failUnlessEqual(sorted(children.keys()),
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
sorted([u"child"])))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable1 = upload.Data("some data", convergence="converge")
d.addCallback(lambda res: n.add_file(u"newfile", uploadable1))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda newnode:
interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode The proper hierarchy is: IFilesystemNode +IFileNode ++IMutableFileNode ++IImmutableFileNode +IDirectoryNode Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.
2009-11-20 07:52:55 +00:00
self.failUnless(IImmutableFileNode.providedBy(newnode)))
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable2 = upload.Data("some data", convergence="stuff")
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "add_file-no",
"child 'newfile' already exists",
n.add_file, u"newfile",
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable2,
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
overwrite=False))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
self.failUnlessEqual(sorted(children.keys()),
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
sorted([u"child", u"newfile"])))
d.addCallback(lambda res: n.get_metadata_for(u"newfile"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
self.failUnlessEqual(set(metadata.keys()),
set(["tahoe", "ctime", "mtime"])))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable3 = upload.Data("some data", convergence="converge")
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.add_file(u"newfile-metadata",
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
uploadable3,
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
{"key": "value"}))
d.addCallback(lambda newnode:
interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode The proper hierarchy is: IFilesystemNode +IFileNode ++IMutableFileNode ++IImmutableFileNode +IDirectoryNode Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.
2009-11-20 07:52:55 +00:00
self.failUnless(IImmutableFileNode.providedBy(newnode)))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.get_metadata_for(u"newfile-metadata"))
dirnode.py: add metadata= to add_file(), add tests
2008-02-11 21:53:28 +00:00
d.addCallback(lambda metadata:
trivial: remove trailing whitespace and unused import
2009-04-12 02:17:42 +00:00
self.failUnless((set(metadata.keys()) == set(["key", "tahoe"])) and
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
2009-04-11 22:52:05 +00:00
(metadata['key'] == "value"), metadata))
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
d.addCallback(lambda res: n.delete(u"newfile-metadata"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
d.addCallback(lambda res: n.create_subdirectory(u"subdir2"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
def _created2(subdir2):
self.subdir2 = subdir2
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
# put something in the way, to make sure it gets overwritten
return subdir2.add_file(u"child", upload.Data("overwrite me",
"converge"))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(_created2)
d.addCallback(lambda res:
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
n.move_child_to(u"child", self.subdir2))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
self.failUnlessEqual(sorted(children.keys()),
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
sorted([u"newfile", u"subdir2"])))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
d.addCallback(lambda res: self.subdir2.list())
d.addCallback(lambda children:
self.failUnlessEqual(sorted(children.keys()),
unicode handling: declare dirnodes to contain unicode child names, update webish to match
2008-02-14 22:45:56 +00:00
sorted([u"child"])))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res: self.subdir2.get(u"child"))
d.addCallback(lambda child:
self.failUnlessEqual(child.get_uri(),
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
fake_file_uri))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
# move it back, using new_child_name=
d.addCallback(lambda res:
self.subdir2.move_child_to(u"child", n, u"newchild"))
d.addCallback(lambda res: n.list())
d.addCallback(lambda children:
self.failUnlessEqual(sorted(children.keys()),
sorted([u"newchild", u"newfile",
u"subdir2"])))
d.addCallback(lambda res: self.subdir2.list())
d.addCallback(lambda children:
self.failUnlessEqual(sorted(children.keys()), []))
# now make sure that we honor overwrite=False
d.addCallback(lambda res:
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
self.subdir2.set_uri(u"newchild",
other_file_uri, other_file_uri))
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 23:09:47 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "move_child_to-no",
"child 'newchild' already exists",
n.move_child_to, u"newchild",
self.subdir2,
overwrite=False))
d.addCallback(lambda res: self.subdir2.get(u"newchild"))
d.addCallback(lambda child:
self.failUnlessEqual(child.get_uri(),
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
other_file_uri))
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
return d
d.addCallback(_then)
test_dirnode: add an explainError call
2008-11-19 23:02:12 +00:00
d.addErrback(self.explain_error)
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 21:32:04 +00:00
return d
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
def test_create_subdirectory(self):
self.basedir = "dirnode/Dirnode/test_create_subdirectory"
self.set_up_grid()
c = self.g.clients[0]
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
nm = c.nodemaker
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
d = c.create_dirnode()
def _then(n):
# /
self.rootnode = n
fake_file_uri = make_mutable_file_uri()
other_file_uri = make_mutable_file_uri()
md = {"metakey": "metavalue"}
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
kids = {u"kid1": (nm.create_from_cap(fake_file_uri), {}),
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match
2009-10-17 19:28:29 +00:00
u"kid2": (nm.create_from_cap(other_file_uri), md),
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
}
d = n.create_subdirectory(u"subdir", kids)
def _check(sub):
d = n.get_child_at_path(u"subdir")
d.addCallback(lambda sub2: self.failUnlessEqual(sub2.get_uri(),
sub.get_uri()))
d.addCallback(lambda ign: sub.list())
return d
d.addCallback(_check)
def _check_kids(kids2):
self.failUnlessEqual(sorted(kids.keys()), sorted(kids2.keys()))
self.failUnlessEqual(kids2[u"kid2"][1]["metakey"], "metavalue")
d.addCallback(_check_kids)
return d
d.addCallback(_then)
return d
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
class MinimalFakeMutableFile:
def get_writekey(self):
return "writekey"
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class Packing(unittest.TestCase):
# This is a base32-encoded representation of the directory tree
# root/file1
# root/file2
# root/file3
# as represented after being fed to _pack_contents.
# We have it here so we can decode it, feed it to
# _unpack_contents, and verify that _unpack_contents
# works correctly.
known_tree = "GM4TOORVHJTGS3DFGEWDSNJ2KVJESOSDJBFTU33MPB2GS3LZNVYG6N3GGI3WU5TIORTXC3DOMJ2G4NB2MVWXUZDONBVTE5LNGRZWK2LYN55GY23XGNYXQMTOMZUWU5TENN4DG23ZG5UTO2L2NQ2DO6LFMRWDMZJWGRQTUMZ2GEYDUMJQFQYTIMZ22XZKZORX5XS7CAQCSK3URR6QOHISHRCMGER5LRFSZRNAS5ZSALCS6TWFQAE754IVOIKJVK73WZPP3VUUEDTX3WHTBBZ5YX3CEKHCPG3ZWQLYA4QM6LDRCF7TJQYWLIZHKGN5ROA3AUZPXESBNLQQ6JTC2DBJU2D47IZJTLR3PKZ4RVF57XLPWY7FX7SZV3T6IJ3ORFW37FXUPGOE3ROPFNUX5DCGMAQJ3PGGULBRGM3TU6ZCMN2GS3LFEI5CAMJSGQ3DMNRTHA4TOLRUGI3TKNRWGEWCAITUMFUG6ZJCHIQHWITMNFXGW3LPORUW2ZJCHIQDCMRUGY3DMMZYHE3S4NBSG42TMNRRFQQCE3DJNZVWG4TUNFWWKIR2EAYTENBWGY3DGOBZG4XDIMRXGU3DMML5FQQCE3LUNFWWKIR2EAYTENBWGY3DGOBZG4XDIMRXGU3DMML5FQWDGOJRHI2TUZTJNRSTELBZGQ5FKUSJHJBUQSZ2MFYGKZ3SOBSWQ43IO52WO23CNAZWU3DUGVSWSNTIOE5DK33POVTW4ZLNMNWDK6DHPA2GS2THNF2W25DEN5VGY2LQNFRGG5DKNNRHO5TZPFTWI6LNMRYGQ2LCGJTHM4J2GM5DCMB2GQWDCNBSHKVVQBGRYMACKJ27CVQ6O6B4QPR72RFVTGOZUI76XUSWAX73JRV5PYRHMIFYZIA25MXDPGUGML6M2NMRSG4YD4W4K37ZDYSXHMJ3IUVT4F64YTQQVBJFFFOUC7J7LAB2VFCL5UKKGMR2D3F4EPOYC7UYWQZNR5KXHBSNXLCNBX2SNF22DCXJIHSMEKWEWOG5XCJEVVZ7UW5IB6I64XXQSJ34B5CAYZGZIIMR6LBRGMZTU6ZCMN2GS3LFEI5CAMJSGQ3DMNRTHA4TOLRUGMYDEMJYFQQCE5DBNBXWKIR2EB5SE3DJNZVW233UNFWWKIR2EAYTENBWGY3DGOBZG4XDIMZQGIYTQLBAEJWGS3TLMNZHI2LNMURDUIBRGI2DMNRWGM4DSNZOGQZTAMRRHB6SYIBCNV2GS3LFEI5CAMJSGQ3DMNRTHA4TOLRUGMYDEMJYPUWCYMZZGU5DKOTGNFWGKMZMHE2DUVKSJE5EGSCLHJRW25DDPBYTO2DXPB3GM6DBNYZTI6LJMV3DM2LWNB4TU4LWMNSWW3LKORXWK5DEMN3TI23NNE3WEM3SORRGY5THPA3TKNBUMNZG453BOF2GSZLXMVWWI3DJOFZW623RHIZTUMJQHI2SYMJUGI5BOSHWDPG3WKPAVXCF3XMKA7QVIWPRMWJHDTQHD27AHDCPJWDQENQ5H5ZZILTXQNIXXCIW4LKQABU2GCFRG5FHQN7CHD7HF4EKNRZFIV2ZYQIBM7IQU7F4RGB3XCX3FREPBKQ7UCICHVWPCYFGA6OLH3J45LXQ6GWWICJ3PGWJNLZ7PCRNLAPNYUGU6BENS7OXMBEOOFRIZV3PF2FFWZ5WHDPKXERYP7GNHKRMGEZTOOT3EJRXI2LNMURDUIBRGI2DMNRWGM4DSNZOGQZTGNRSGY4SYIBCORQWQ33FEI5CA6ZCNRUW423NN52GS3LFEI5CAMJSGQ3DMNRTHA4TOLRUGMZTMMRWHEWCAITMNFXGWY3SORUW2ZJCHIQDCMRUGY3DMMZYHE3S4NBTGM3DENRZPUWCAITNORUW2ZJCHIQDCMRUGY3DMMZYHE3S4NBTGM3DENRZPUWCY==="
Add tests for CachingDict, _pack_contents, _unpack_contents
2009-07-04 03:43:28 +00:00
def test_unpack_and_pack_behavior(self):
known_tree = b32decode(self.known_tree)
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
nodemaker = NodeMaker(None, None, None,
None, None, None,
{"k": 3, "n": 10}, None)
writecap = "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q"
filenode = nodemaker.create_from_cap(writecap)
node = dirnode.DirectoryNode(filenode, nodemaker, None)
children = node._unpack_contents(known_tree)
self._check_children(children)
packed_children = node._pack_contents(children)
children = node._unpack_contents(packed_children)
self._check_children(children)
def _check_children(self, children):
# Are all the expected child nodes there?
self.failUnless(children.has_key(u'file1'))
self.failUnless(children.has_key(u'file2'))
self.failUnless(children.has_key(u'file3'))
# Are the metadata for child 3 right?
file3_rocap = "URI:CHK:cmtcxq7hwxvfxan34yiev6ivhy:qvcekmjtoetdcw4kmi7b3rtblvgx7544crnwaqtiewemdliqsokq:3:10:5"
file3_rwcap = "URI:CHK:cmtcxq7hwxvfxan34yiev6ivhy:qvcekmjtoetdcw4kmi7b3rtblvgx7544crnwaqtiewemdliqsokq:3:10:5"
file3_metadata = {'ctime': 1246663897.4336269, 'tahoe': {'linkmotime': 1246663897.4336269, 'linkcrtime': 1246663897.4336269}, 'mtime': 1246663897.4336269}
self.failUnlessEqual(file3_metadata, children[u'file3'][1])
self.failUnlessEqual(file3_rocap,
children[u'file3'][0].get_readonly_uri())
self.failUnlessEqual(file3_rwcap,
children[u'file3'][0].get_uri())
# Are the metadata for child 2 right?
file2_rocap = "URI:CHK:apegrpehshwugkbh3jlt5ei6hq:5oougnemcl5xgx4ijgiumtdojlipibctjkbwvyygdymdphib2fvq:3:10:4"
file2_rwcap = "URI:CHK:apegrpehshwugkbh3jlt5ei6hq:5oougnemcl5xgx4ijgiumtdojlipibctjkbwvyygdymdphib2fvq:3:10:4"
file2_metadata = {'ctime': 1246663897.430218, 'tahoe': {'linkmotime': 1246663897.430218, 'linkcrtime': 1246663897.430218}, 'mtime': 1246663897.430218}
self.failUnlessEqual(file2_metadata, children[u'file2'][1])
self.failUnlessEqual(file2_rocap,
children[u'file2'][0].get_readonly_uri())
self.failUnlessEqual(file2_rwcap,
children[u'file2'][0].get_uri())
# Are the metadata for child 1 right?
file1_rocap = "URI:CHK:olxtimympo7f27jvhtgqlnbtn4:emzdnhk2um4seixozlkw3qx2nfijvdkx3ky7i7izl47yedl6e64a:3:10:10"
file1_rwcap = "URI:CHK:olxtimympo7f27jvhtgqlnbtn4:emzdnhk2um4seixozlkw3qx2nfijvdkx3ky7i7izl47yedl6e64a:3:10:10"
file1_metadata = {'ctime': 1246663897.4275661, 'tahoe': {'linkmotime': 1246663897.4275661, 'linkcrtime': 1246663897.4275661}, 'mtime': 1246663897.4275661}
self.failUnlessEqual(file1_metadata, children[u'file1'][1])
self.failUnlessEqual(file1_rocap,
children[u'file1'][0].get_readonly_uri())
self.failUnlessEqual(file1_rwcap,
children[u'file1'][0].get_uri())
Add tests for CachingDict, _pack_contents, _unpack_contents
2009-07-04 03:43:28 +00:00
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
def _make_kids(self, nm, which):
caps = {"imm": "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861",
"lit": "URI:LIT:n5xgk", # LIT for "one"
"write": "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq",
"read": "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q",
"dirwrite": "URI:DIR2:n6x24zd3seu725yluj75q5boaa:mm6yoqjhl6ueh7iereldqxue4nene4wl7rqfjfybqrehdqmqskvq",
"dirread": "URI:DIR2-RO:b7sr5qsifnicca7cbk3rhrhbvq:mm6yoqjhl6ueh7iereldqxue4nene4wl7rqfjfybqrehdqmqskvq",
}
kids = {}
for name in which:
kids[unicode(name)] = (nm.create_from_cap(caps[name]), {})
return kids
def test_deep_immutable(self):
nm = NodeMaker(None, None, None, None, None, None, {"k": 3, "n": 10},
None)
fn = MinimalFakeMutableFile()
kids = self._make_kids(nm, ["imm", "lit", "write", "read",
"dirwrite", "dirread"])
packed = dirnode.pack_children(fn, kids, deep_immutable=False)
self.failUnlessIn("lit", packed)
kids = self._make_kids(nm, ["imm", "lit"])
packed = dirnode.pack_children(fn, kids, deep_immutable=True)
self.failUnlessIn("lit", packed)
kids = self._make_kids(nm, ["imm", "lit", "write"])
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
self.failUnlessRaises(dirnode.MustBeDeepImmutable,
dirnode.pack_children,
fn, kids, deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
# read-only is not enough: all children must be immutable
kids = self._make_kids(nm, ["imm", "lit", "read"])
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
self.failUnlessRaises(dirnode.MustBeDeepImmutable,
dirnode.pack_children,
fn, kids, deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
kids = self._make_kids(nm, ["imm", "lit", "dirwrite"])
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
self.failUnlessRaises(dirnode.MustBeDeepImmutable,
dirnode.pack_children,
fn, kids, deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
kids = self._make_kids(nm, ["imm", "lit", "dirread"])
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
2010-01-14 22:15:29 +00:00
self.failUnlessRaises(dirnode.MustBeDeepImmutable,
dirnode.pack_children,
fn, kids, deep_immutable=True)
dirnode.pack_children(): add deep_immutable= argument This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 16:28:09 +00:00
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
class FakeMutableFile:
Add t=mkdir-immutable to the webapi. Closes #607. * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-18 07:09:00 +00:00
implements(IMutableFileNode)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
counter = 0
def __init__(self, initial_contents=""):
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
self.data = self._get_initial_contents(initial_contents)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
counter = FakeMutableFile.counter
FakeMutableFile.counter += 1
writekey = hashutil.ssk_writekey_hash(str(counter))
fingerprint = hashutil.ssk_pubkey_fingerprint_hash(str(counter))
self.uri = uri.WriteableSSKFileURI(writekey, fingerprint)
nodemaker.create_new_mutable_directory: pack_children() in initial_contents= instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-17 20:31:46 +00:00
def _get_initial_contents(self, contents):
if isinstance(contents, str):
return contents
if contents is None:
return ""
assert callable(contents), "%s should be callable, not %s" % \
(contents, type(contents))
return contents(self)
stop using IURI()/etc as an adapter
2009-11-11 22:45:42 +00:00
def get_cap(self):
return self.uri
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def get_uri(self):
return self.uri.to_string()
def download_best_version(self):
return defer.succeed(self.data)
def get_writekey(self):
return "writekey"
def is_readonly(self):
return False
def is_mutable(self):
return True
def modify(self, modifier):
self.data = modifier(self.data, None, True)
return defer.succeed(None)
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class FakeNodeMaker(NodeMaker):
def create_mutable_file(self, contents="", keysize=None):
return defer.succeed(FakeMutableFile(contents))
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
class FakeClient2(Client):
def __init__(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
self.nodemaker = FakeNodeMaker(None, None, None,
None, None, None,
{"k":3,"n":10}, None)
def create_node_from_uri(self, rwcap, rocap):
return self.nodemaker.create_from_cap(rwcap, rocap)
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
class Dirnode2(unittest.TestCase, testutil.ShouldFailMixin):
def setUp(self):
test_dirnode.py: reduce scope of a Client instance, suggested by Kevan.
2010-01-15 06:27:13 +00:00
client = FakeClient2()
self.nodemaker = client.nodemaker
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
def test_from_future(self):
# create a dirnode that contains unknown URI types, and make sure we
# tolerate them properly. Since dirnodes aren't allowed to add
# unknown node types, we have to be tricky.
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
d = self.nodemaker.create_new_mutable_directory()
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
future_writecap = "x-tahoe-crazy://I_am_from_the_future."
future_readcap = "x-tahoe-crazy-readonly://I_am_from_the_future."
future_node = UnknownNode(future_writecap, future_readcap)
def _then(n):
self._node = n
return n.set_node(u"future", future_node)
d.addCallback(_then)
# we should be prohibited from adding an unknown URI to a directory,
# since we don't know how to diminish the cap to a readcap (for the
# dirnode's rocap slot), and we don't want to accidentally grant
# write access to a holder of the dirnode's readcap.
d.addCallback(lambda ign:
self.shouldFail(CannotPackUnknownNodeError,
"copy unknown",
"cannot pack unknown node as child add",
dirnode.set_uri/set_children: change signature to take writecap+readcap instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 23:51:26 +00:00
self._node.set_uri, u"add",
future_writecap, future_readcap))
Tolerate unknown URI types in directory structures. Part of #683. The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).
2009-07-03 01:07:49 +00:00
d.addCallback(lambda ign: self._node.list())
def _check(children):
self.failUnlessEqual(len(children), 1)
(fn, metadata) = children[u"future"]
self.failUnless(isinstance(fn, UnknownNode), fn)
self.failUnlessEqual(fn.get_uri(), future_writecap)
self.failUnlessEqual(fn.get_readonly_uri(), future_readcap)
# but we *should* be allowed to copy this node, because the
# UnknownNode contains all the information that was in the
# original directory (readcap and writecap), so we're preserving
# everything.
return self._node.set_node(u"copy", fn)
d.addCallback(_check)
d.addCallback(lambda ign: self._node.list())
def _check2(children):
self.failUnlessEqual(len(children), 2)
(fn, metadata) = children[u"copy"]
self.failUnless(isinstance(fn, UnknownNode), fn)
self.failUnlessEqual(fn.get_uri(), future_writecap)
self.failUnlessEqual(fn.get_readonly_uri(), future_readcap)
return d
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
class DeepStats(unittest.TestCase):
tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel
2009-06-05 03:14:44 +00:00
timeout = 240 # It takes longer than 120 seconds on Francois's arm box.
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
def test_stats(self):
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
ds = dirnode.DeepStats(None)
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
ds.add("count-files")
ds.add("size-immutable-files", 123)
ds.histogram("size-files-histogram", 123)
ds.max("largest-directory", 444)
s = ds.get_results()
self.failUnlessEqual(s["count-files"], 1)
self.failUnlessEqual(s["size-immutable-files"], 123)
self.failUnlessEqual(s["largest-directory"], 444)
self.failUnlessEqual(s["count-literal-files"], 0)
ds.add("count-files")
ds.add("size-immutable-files", 321)
ds.histogram("size-files-histogram", 321)
ds.max("largest-directory", 2)
s = ds.get_results()
self.failUnlessEqual(s["count-files"], 2)
self.failUnlessEqual(s["size-immutable-files"], 444)
self.failUnlessEqual(s["largest-directory"], 444)
self.failUnlessEqual(s["count-literal-files"], 0)
self.failUnlessEqual(s["size-files-histogram"],
[ (101, 316, 1), (317, 1000, 1) ])
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.
2008-10-22 00:03:07 +00:00
ds = dirnode.DeepStats(None)
deep-stats: add file-size histogram
2008-05-08 23:19:42 +00:00
for i in range(1, 1100):
ds.histogram("size-files-histogram", i)
ds.histogram("size-files-histogram", 4*1000*1000*1000*1000) # 4TB
s = ds.get_results()
self.failUnlessEqual(s["size-files-histogram"],
[ (1, 3, 3),
(4, 10, 7),
(11, 31, 21),
(32, 100, 69),
(101, 316, 216),
(317, 1000, 684),
(1001, 3162, 99),
(3162277660169L, 10000000000000L, 1),
])
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
class UCWEingMutableFileNode(MutableFileNode):
please_ucwe_after_next_upload = False
def _upload(self, new_contents, servermap):
d = MutableFileNode._upload(self, new_contents, servermap)
def _ucwe(res):
mutable.modify(): after UCWE, publish even if the second invocation of the modifier didn't modify anything. For #551.
2008-12-06 05:49:23 +00:00
if self.please_ucwe_after_next_upload:
self.please_ucwe_after_next_upload = False
raise UncoordinatedWriteError()
return res
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
d.addCallback(_ucwe)
return d
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class UCWEingNodeMaker(NodeMaker):
def _create_mutable(self, cap):
n = UCWEingMutableFileNode(self.storage_broker, self.secret_holder,
self.default_encoding_parameters,
self.history)
clean up uri-vs-cap terminology, emphasize cap instances instead of URI strings * "cap" means a python instance which encapsulates a filecap/dircap (uri.py) * "uri" means a string with a "URI:" prefix * FileNode instances are created with (and retain) a cap instance, and generate uri strings on demand * .get_cap/get_readcap/get_verifycap/get_repaircap return cap instances * .get_uri/get_readonly_uri return uri strings * add filenode.download_to_filename() for control.py, should find a better way * use MutableFileNode.init_from_cap, not .init_from_uri * directory URI instances: use get_filenode_cap, not get_filenode_uri * update/cleanup bench_dirnode.py to match, add Makefile target to run it
2009-11-11 22:25:42 +00:00
return n.init_from_cap(cap)
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-17 00:23:48 +00:00
class Deleter(GridTestMixin, unittest.TestCase):
test: multiple by 10 or so all timeouts that Zandr's ARM box just overran
2009-06-10 12:56:39 +00:00
timeout = 3600 # It takes longer than 433 seconds on Zandr's ARM box.
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
def test_retry(self):
# ticket #550, a dirnode.delete which experiences an
# UncoordinatedWriteError will fail with an incorrect "you're
# deleting something which isn't there" NoSuchChildError exception.
# to trigger this, we start by creating a directory with a single
# file in it. Then we create a special dirnode that uses a modified
# MutableFileNode which will raise UncoordinatedWriteError once on
# demand. We then call dirnode.delete, which ought to retry and
# succeed.
self.basedir = self.mktemp()
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-17 00:23:48 +00:00
self.set_up_grid()
c0 = self.g.clients[0]
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c0.create_dirnode()
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
small = upload.Data("Small enough for a LIT", None)
def _created_dir(dn):
self.root = dn
self.root_uri = dn.get_uri()
return dn.add_file(u"file", small)
d.addCallback(_created_dir)
def _do_delete(ignored):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
nm = UCWEingNodeMaker(c0.storage_broker, c0._secret_holder,
c0.get_history(), c0.getServiceNamed("uploader"),
immutable.Downloader: pass StorageBroker to constructor, stop being a Service child of the client, access with client.downloader instead of client.getServiceNamed("downloader"). The single "Downloader" instance is scheduled for demolition anyways, to be replaced by individual filenode.download calls.
2009-08-15 19:25:43 +00:00
c0.downloader,
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
c0.download_cache_dirman,
c0.get_encoding_parameters(),
c0._key_generator)
n = nm.create_from_cap(self.root_uri)
mutable.modify(): after UCWE, publish even if the second invocation of the modifier didn't modify anything. For #551.
2008-12-06 05:49:23 +00:00
assert n._node.please_ucwe_after_next_upload == False
n._node.please_ucwe_after_next_upload = True
dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.
2008-12-06 05:08:37 +00:00
# This should succeed, not raise an exception
return n.delete(u"file")
d.addCallback(_do_delete)
return d
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
class Adder(GridTestMixin, unittest.TestCase, testutil.ShouldFailMixin):
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
def test_overwrite(self):
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
# note: This functionality could be tested without actually creating
# several RSA keys. It would be faster without the GridTestMixin: use
# dn.set_node(nodemaker.create_from_cap(make_chk_file_uri())) instead
# of dn.add_file, and use a special NodeMaker that creates fake
# mutable files.
self.basedir = "dirnode/Adder/test_overwrite"
self.set_up_grid()
c = self.g.clients[0]
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
fileuri = make_chk_file_uri(1234)
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
filenode = c.nodemaker.create_from_cap(fileuri)
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation of adding initial_children= argument. Includes stubbed-out initial_children= support.
2009-10-12 22:45:06 +00:00
d = c.create_dirnode()
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
def _create_directory_tree(root_node):
# Build
# root/file1
# root/file2
# root/dir1
d = root_node.add_file(u'file1', upload.Data("Important Things",
Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'
2009-07-20 03:43:18 +00:00
None))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res:
Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'
2009-07-20 03:43:18 +00:00
root_node.add_file(u'file2', upload.Data("Sekrit Codes", None)))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res:
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
root_node.create_subdirectory(u"dir1"))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res: root_node)
return d
d.addCallback(_create_directory_tree)
def _test_adder(root_node):
d = root_node.set_node(u'file1', filenode)
# We've overwritten file1. Let's try it with a directory
d.addCallback(lambda res:
replace dirnode.create_empty_directory() with create_subdirectory(), which takes an initial_children= argument
2009-10-13 02:15:20 +00:00
root_node.create_subdirectory(u'dir2'))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res:
root_node.set_node(u'dir2', filenode))
# We try overwriting a file with a child while also specifying
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo
2009-08-15 11:02:56 +00:00
# overwrite=False. We should receive an ExistingChildError
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
# when we do this.
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_node",
"child 'file1' already exists",
root_node.set_node, u"file1",
filenode, overwrite=False))
# If we try with a directory, we should see the same thing
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_node",
"child 'dir1' already exists",
root_node.set_node, u'dir1', filenode,
overwrite=False))
d.addCallback(lambda res:
root_node.set_node(u'file1', filenode,
Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'
2009-07-20 03:43:18 +00:00
overwrite="only-files"))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
d.addCallback(lambda res:
self.shouldFail(ExistingChildError, "set_node",
"child 'dir1' already exists",
root_node.set_node, u'dir1', filenode,
Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'
2009-07-20 03:43:18 +00:00
overwrite="only-files"))
Add unit tests for the Adder in dirnode.py
2009-07-18 19:50:49 +00:00
return d
d.addCallback(_test_adder)
return d
Reference in New Issue Copy Permalink