Brian Warner
6db747b1f0
figleaf_htmlizer: rewrite with twisted.python.usage, remove logging: should behave the same as before
2009-02-11 19:16:43 -07:00
Brian Warner
f3ed579e74
figleaf_htmlizer: expand tabs, fix to 4-space indents. No functional changes.
2009-02-11 19:05:42 -07:00
Zooko O'Whielacronx
9247dc70e5
trivial: whitespace
2009-01-18 10:54:58 -07:00
Zooko O'Whielacronx
e449052a17
util: dictutil: add DictOfSets.union(key, values) and DictOfSets.update(otherdictofsets)
2009-01-12 10:55:39 -07:00
Zooko O'Whielacronx
c01cfc0035
util: log: allow empty msgs (because downloader is using the "format" alternative with no "msg" argument)
2009-01-07 11:54:11 -07:00
Zooko O'Whielacronx
2122ba78b3
util: deferredutil: undo my recent patch to use our own implementation of gatherResults
...
It seems to cause lots of failures on some builders.
2009-01-07 11:00:05 -07:00
Zooko O'Whielacronx
79535e4f5b
util: deferredutil: implement our own gatherResults instead of using Twisted's
...
Because we want to maintain backwards compatibility to Twisted 2.4.0.
2009-01-07 10:32:07 -07:00
Zooko O'Whielacronx
78306cf7fa
trivial: M-x whitespace-cleanup
2009-01-07 10:25:28 -07:00
Zooko O'Whielacronx
8e92dd1231
trivial: whitespace cleanup
2009-01-06 11:20:58 -07:00
Zooko O'Whielacronx
d4a5f9131b
util: base32: require str-not-unicode inputs -- effectively rolls back [3306] and [3307]
2009-01-06 10:41:22 -07:00
Zooko O'Whielacronx
5d5e89d96d
util: add gatherResults which is a deferred-list-like thing that doesn't wrap failures in a FirstError
2009-01-04 10:52:02 -07:00
Zooko O'Whielacronx
b8904e023c
trivial: remove unused import -- thanks, pyflakes
2008-12-31 15:25:56 -07:00
Zooko O'Whielacronx
c00d20361f
rrefutil: generically wrap any errback from callRemote() in a ServerFailure instance
...
This facilitates client code to easily catch ServerFailures without also catching exceptions arising from client-side code.
See also:
http://foolscap.lothar.com/trac/ticket/105 # make it easy to distinguish server-side failures/exceptions from client-side
2008-12-31 14:28:30 -07:00
Zooko O'Whielacronx
ce5effbedf
util/base32: the identity trans table needn't have any contents -- we are using string.translate solely to delete known chars
2008-12-22 17:48:08 -07:00
Zooko O'Whielacronx
3a26a4bd58
util/base32: allow unicode inputs to a2b() or could_be_base32_encoded(), and encode them with utf-8 before processing them
2008-12-22 17:47:13 -07:00
Zooko O'Whielacronx
dde2376c4e
util/base32: loosen the precondition forbidding unicode and requiring str -- now it requires either unicode or str
...
Hopefully this will make it so that tests pass with François Deppierraz's patch to fix the tahoe cli's handling of unicode argument.
2008-12-22 16:22:37 -07:00
Zooko O'Whielacronx
f1fbd4feae
dirnode: don't check MAC on entries in dirnodes
...
In an ancient version of directories, we needed a MAC on each entry. In modern times, the entire dirnode comes with a digital signature, so the MAC on each entry is redundant.
With this patch, we no longer check those MACs when reading directories, but we still produce them so that older readers will accept directories that we write.
2008-12-21 17:35:18 -07:00
Zooko O'Whielacronx
2b2c2a566b
util: logging: refactor some common logging behavior into mixins
2008-12-16 17:38:07 -07:00
Zooko O'Whielacronx
5fb44a3e79
pyutil: assertutil: copy in simplified assertutil from pyutil
2008-12-16 17:37:45 -07:00
Zooko O'Whielacronx
e29e4c04d9
pyutil: assertutil: simplify handling of exception during formatting of precondition message, and reduce dependency to just the Python Standard Library's logging module
2008-12-10 07:10:57 -07:00
Brian Warner
39a41f1d85
util/abbreviate: add abbreviated-size parser
2008-12-01 17:24:12 -07:00
Brian Warner
bf06492a90
#538 : fetch version and attach to the rref. Make IntroducerClient demand v1 support.
2008-11-21 20:07:27 -07:00
Brian Warner
7391f63fc1
util/abbreviate: little utility to abbreviate seconds and bytes
2008-11-18 20:11:42 -07:00
Brian Warner
fc7cd23bd1
limiter.py: fix stack blowout by inserting an eventual-send between _done and maybe_start_task. This was causing failures during a 'tahoe manifest' of a large set of directories
2008-11-14 21:11:44 -07:00
Zooko O'Whielacronx
9c35ca76a4
util: copy in pyutil.fileutil.ReopenableNamedTemporaryFile
2008-11-04 17:47:15 -07:00
Zooko O'Whielacronx
cef24792e7
util: copy in nummedobj from pyutil
2008-11-04 13:55:50 -07:00
Zooko O'Whielacronx
fb85a6280f
util: copy in dictutil from pyutil
2008-11-04 13:53:27 -07:00
Brian Warner
c205a54965
util/cachedir.py: add a cache-directory manager class, which expires+deletes unused files after a while
2008-10-30 13:01:20 -07:00
Brian Warner
b73c380cdb
move testutil into test/common_util.py, since it doesn't count as 'code under test' for our pyflakes numbers
2008-10-28 21:28:31 -07:00
Brian Warner
294e3fb682
util: move PollMixin to a separate file (pollmixin.py), so testutil can be moved into test/
2008-10-28 21:15:48 -07:00
Brian Warner
2d4e895096
iputil.py: avoid a DNS lookup at startup (which may timeout tests when run on a partially-offline host) by using 198.41.0.4 instead of A.ROOT-SERVERS.NET
2008-10-28 13:36:46 -07:00
Zooko O'Whielacronx
57d9f6beb6
repairer: fix flaw in testutil.flip_one_bit() that Brian pointed out
2008-10-16 12:48:48 -07:00
Zooko O'Whielacronx
8a6d1e5da6
repairer: test all different kinds of corruption that can happen to share files on disk
2008-10-14 16:09:20 -07:00
Brian Warner
b0a00e0f33
util/time_format.py: accept space separator, add unit tests
2008-10-13 15:52:58 -07:00
Brian Warner
93725a6935
util/limiter: add a repr
2008-10-07 13:19:45 -07:00
Brian Warner
1e3500ae5f
testutil.PollMixin: set default timeout (to 100s), emit a more helpful error when the timeout is hit
2008-09-29 22:23:09 -07:00
Brian Warner
98c8e25709
netstring: add required_trailer= argument
2008-09-26 09:57:54 -07:00
Brian Warner
e8cf581e3f
move netstring() and split_netstring() into a separate util.netstring module
2008-09-25 21:38:24 -07:00
Brian Warner
0a59991ce6
various: use util.log.err instead of twisted.log.err, so we get both Incidents and trial-test-flunking
2008-09-20 10:35:45 -07:00
Brian Warner
8854d0c1b5
util.log: send log.err to Twisted too, so that Trial tests are flunked
2008-09-20 10:34:27 -07:00
Brian Warner
bab799297f
testutil.PollMixin: use a custom exception (and convert it) to avoid the ugly 'stash' cycle
2008-09-02 20:32:51 -07:00
Zooko O'Whielacronx
9d14341c7d
test: add testutil.flip_one_bit which flips a randomly chosen bit of the input string
2008-07-28 16:42:17 -07:00
Brian Warner
dba0ad3d28
test_mutable.py: add more tests of post-mapupdate corruption, to support #474 testing
2008-06-24 11:08:10 -07:00
Brian Warner
fe44b8fb4b
util.dictutil: move DictOfSets out to a separate utility module
2008-05-21 09:43:49 -07:00
Brian Warner
1121df6664
testutil.shouldFail: mention the 'which' string in substring failures too
2008-05-16 16:08:38 -07:00
Brian Warner
d76cf6f748
limiter: fix bug that ignores the limit= argument
2008-05-07 18:35:53 -07:00
Brian Warner
c7e441309d
add a basic concurrency limiter utility
2008-05-07 16:53:30 -07:00
ben
c85869fc08
support freebsd 6
2008-04-28 00:41:40 -07:00
Brian Warner
81ab1ec785
testutil: factor stall() out into a common location
2008-04-22 16:47:15 -07:00
Brian Warner
b6b3b6b1bd
testutil: minor reformatting
2008-04-17 19:56:45 -07:00
Zooko O'Whielacronx
bdda08df8c
copy pyutil.time_format into src/allmydata/util
...
time_format() provides a good format for logging timestamps
2007-08-14 09:33:49 -07:00
Zooko O'Whielacronx
fc3bd0c987
use added secret to protect convergent encryption
...
Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key. If you want traditional convergent encryption behavior, set the added secret to be the empty string.
This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names. (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.)
This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 09:46:06 -07:00
Zooko O'Whielacronx
b2f5ac9a0a
Merge patch which switches to SHA-256d with patch that adds punctuation and capitalization to the comment about the hash value.
2008-02-15 13:16:43 -07:00
Brian Warner
72874390fe
BIG COMPATIBILITY BREAK: update hash tags, switch to SHA-256d everywhere
2008-02-14 19:58:01 -07:00
Zooko O'Whielacronx
fc0d637523
docs: update install and usage docs, improve cli "usage" output, make new example directories, add unit test that fails code which prints out sentences that don't end with punctuation marks
2008-02-15 13:11:02 -07:00
robk-tahoe
ca41693162
idlib: make failures much clearer when encountering unicode
...
while investigating fuse related stuff, I spent quite a while staring at
very cryptic explosions I got from idlib. it turns out that unicode
objects and str objects have .translate() methods with differing signatures.
to save anyone else the headache, this makes it very clear if you accidentally
try to pass a unicode object in to a2b() etc.
2008-02-14 17:23:07 -07:00
Brian Warner
09bfa3bea2
remove bencode, yay
2008-02-14 18:06:41 -07:00
Zooko O'Whielacronx
7c6de95bc6
switch from base62 to base32 for storage indices, switch from z-base-32 to rfc 3548 base-32 for everything, separate out base32 encoding from idlib
2008-02-14 19:27:47 -07:00
Zooko O'Whielacronx
e89edf8803
fix a few unused imports and suchlike, discovered by pyflakes
2008-02-13 07:38:08 -07:00
Zooko O'Whielacronx
3f8df27063
use base62 encoding for storage indexes, on disk and in verifier caps, and in logging and diagnostic tools
...
base62 encoding fits more information into alphanumeric chars while avoiding the troublesome non-alphanumeric chars of base64 encoding. In particular, this allows us to work around the ext3 "32,000 entries in a directory" limit while retaining the convenient property that the intermediate directory names are leading prefixes of the storage index file names.
2008-02-12 20:48:37 -07:00
Brian Warner
da9610e90a
change encryption-key hash to include encoding parameters. This is a minor compatibility break: CHK files encoded (with convergence) before and after this will have different keys and ciphertexts. Also switched to SHA-256d for both the data-to-key hash and the key-to-storageindex hash
2008-02-06 19:50:47 -07:00
Brian Warner
9f9a458249
hashutil: add tagged_hash_256d and tagged_hasher_256d
2008-02-06 19:36:43 -07:00
Brian Warner
3a5ba35215
PollMixin: add timeout= argument, rewrite to avoid tail-recursion problems
2008-02-04 20:35:07 -07:00
Zooko O'Whielacronx
1d1628e525
rename storage_index_chk_hash() to storage_index_hash() and add TODO about how our use of it now includes keys that are not CHKs
2008-02-01 12:27:37 -07:00
Brian Warner
a1b155725f
log.py: update log.err() to take advantage of current foolscap's log.err
2008-01-30 18:47:23 -07:00
robk-tahoe
5085c35002
cleanup mac and windows build code
...
this moves some of the code common to both windows and mac builds into the
allmydata module hierarchy, and cleans up the windows and mac build directories
to import the code from there.
2008-01-23 21:06:41 -07:00
Brian Warner
f664dcb489
testutil.py: hush the new (more strict) pyflakes
2008-01-14 18:27:55 -07:00
Brian Warner
c24a1bbaaa
iputil.py: hush the new (more strict) pyflakes
2008-01-14 18:27:43 -07:00
robk-tahoe
9ed5533760
added tweaked sibpath implementation
...
use of twisted.python.util.sibpath to find files relative to modules doesn't
work when those modules are bundled into a library by py2exe. this provides
an alternative implementation (in allmydata.util.sibpath) which checks for
the existence of the file, and if it is not found, attempts to find it relative
to sys.executable instead.
2008-01-10 15:23:41 -07:00
Brian Warner
d9d1697996
util.log: add levels like UNUSUAL
2008-01-07 17:32:45 -07:00
Zooko O'Whielacronx
46db7cf1dc
add regexes to idlib to match certain kinds of base32-encoded values
2008-01-03 16:44:46 -07:00
Zooko O'Whielacronx
c066903566
fix idlib.could_be_base32_encoded_l() to accept *only* valid strings
2008-01-03 11:43:11 -07:00
Zooko O'Whielacronx
1ac11301a7
setup: finish switching from Tahoe's versions of autoversioning tools to pyutil's versions
2007-12-21 14:42:38 -07:00
Zooko O'Whielacronx
08a64c3a2b
rename "secret" to "lease_secret" and change its size from 16 to 32 bytes
2007-12-17 18:34:11 -07:00
Zooko O'Whielacronx
8c65bdcf9d
put all private state in $BASEDIR/private
...
fixes #219
The only part of #219 that this doesn't include is the part about
logpublisher, which has been moved out of tahoe into foolscap.
2007-12-17 16:39:54 -07:00
Brian Warner
e52e14cc66
remove tests of logging functionality that's been subsumed by foolscap logging
2007-12-12 20:23:53 -07:00
Zooko O'Whielacronx
1ac09840a4
a few documentation and naming convention updates
...
Notable: the argument to make REPORTER has been renamed to TRIALARGS.
2007-12-12 19:34:08 -07:00
Zooko O'Whielacronx
192477e4a2
move increase_rlimits() into iputil and make it a no-op on Windows
2007-12-07 08:03:43 -07:00
Brian Warner
0f5ef5184d
test_dirnode.py: obtain full coverage of dirnode.py
2007-12-04 14:32:04 -07:00
Brian Warner
0bf5a762a9
use AES from pycryptopp instead of pycrypto, also truncate the keys slightly differently
2007-12-03 17:27:46 -07:00
Brian Warner
cc1612aee5
hierarchical logging: add numbered messages and parent= args
2007-11-19 18:23:18 -07:00
Zooko O'Whielacronx
59d6c3c822
decentralized directories: integration and testing
...
* use new decentralized directories everywhere instead of old centralized directories
* provide UI to them through the web server
* provide UI to them through the CLI
* update unit tests to simulate decentralized mutable directories in order to test other components that rely on them
* remove the notion of a "vdrive server" and a client thereof
* remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example)
* add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start. The default is 1.
* add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!)
* fix a few bugs where we used to equate "mutable" with "not read-only". Nowadays all directories are mutable, but some might be read-only (to you).
* fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge
* sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.
2007-12-03 14:52:42 -07:00
Zooko O'Whielacronx
3c7f96400d
hashutil.py: switch from pycrypto to pycryptopp SHA256
2007-11-09 14:40:13 -07:00
Brian Warner
e7ad7f8791
mutable: grab encprivkey when necessary during publish, fix test_mutable
2007-11-08 02:46:27 -07:00
Zooko O'Whielacronx
f60dc4adfb
UNDO: hashutil: replace pycrypto's SHA256 with pycryptopp's SHA256
2007-11-07 18:02:39 -07:00
Zooko O'Whielacronx
11412da436
hashutil: replace pycrypto's SHA256 with pycryptopp's SHA256
2007-11-07 18:02:39 -07:00
Brian Warner
5739126d11
mutable: storage_index is always 16 bytes
2007-11-06 18:54:34 -07:00
Brian Warner
c4f7412f1c
stabilize on 20-byte nodeids everywhere, printed with foolscap's base32
2007-11-06 18:49:59 -07:00
Brian Warner
59632c6812
mutable: use proper enable/renew/cancel secrets
2007-11-05 21:51:08 -07:00
Brian Warner
78c45c82d1
mutable: implement filenode share-packing, still pretty rough
2007-11-02 20:51:39 -07:00
Brian Warner
57373c9889
bencode.py: reencode as UTF-8
2007-11-01 15:42:57 -07:00
Brian Warner
4423bc7510
trailing-whitespace eradication, no functional changes
2007-11-01 15:34:35 -07:00
Brian Warner
c63b43bdbc
trailing-whitespace eradication, no functional changes
2007-11-01 15:34:31 -07:00
Brian Warner
f714c5915a
trailing-whitespace eradication, no functional changes
2007-11-01 15:34:28 -07:00
Brian Warner
f425ee3600
trailing-whitespace eradication, no functional changes
2007-11-01 15:34:21 -07:00
Brian Warner
6849647856
trailing-whitespace eradication, no functional changes
2007-11-01 15:34:18 -07:00
Brian Warner
ef546da72b
trailing-whitespace eradication, no functional changes
2007-11-01 15:29:31 -07:00
Brian Warner
8d000aa1ec
trailing-whitespace eradication, no functional changes
2007-11-01 15:29:28 -07:00
Brian Warner
8390d83322
trailing-whitespace eradication, no functional changes
2007-11-01 15:29:17 -07:00
Brian Warner
fb59627faa
trailing-whitespace eradication, no functional changes
2007-11-01 15:25:20 -07:00
