Brian Warner
80cf789817
download: remove unused import
2007-04-17 21:11:20 -07:00
Brian Warner
b76aa1ce17
download: oops, NotEnoughHashesError comes from hashtree, not hashutil
2007-04-17 20:37:51 -07:00
Brian Warner
b84d6ed07f
encode: fix multi-segment uploads: lambdas inside for loops require special attention to make sure you are capturing the *value* of the loop variable and not just the slot it lives in
2007-04-17 20:29:08 -07:00
Brian Warner
a4d7607a9e
test_system: minor comment
2007-04-17 20:28:19 -07:00
Brian Warner
e7ec4ff4e5
factor out the tagged hash function used for subshares/blocks
2007-04-17 20:27:56 -07:00
Brian Warner
76e28b3484
comment out some verbose log messages, add commented-out new ones
2007-04-17 20:25:52 -07:00
Brian Warner
008e418523
storageserver: assert that blocks are written in-order, clean up tests a bit
2007-04-17 20:21:05 -07:00
Zooko O'Whielacronx
0ae8f98ac9
GNUmakefile: rm -rf ./src/pyfec/build on clean
2007-04-18 09:49:47 -07:00
Zooko O'Whielacronx
f4b00f9737
GNUmakefile: don't stop clean if clean-pyfec fails
2007-04-18 09:48:24 -07:00
Zooko O'Whielacronx
acde096405
pyfec: rename pyfec to zfec
...
It turns out that "pyfec" turns off people who aren't Python hackers.
"allmyfec" is too long for a low-level core utility library.
"fec" is too generic (it is already used by Luigi Rizzo's library which this
library is based on, for one thing).
I'm open to other naming suggestions, especially before we widely announce this
library, which I expect will happen within a few days.
2007-04-18 09:19:00 -07:00
Zooko O'Whielacronx
c2dfcb1f01
don't clobber existing storefile every put block
2007-04-17 20:14:44 -07:00
Zooko O'Whielacronx
723f4078a3
pyutil: fileutil.open_or_create()
2007-04-17 20:14:26 -07:00
Brian Warner
e040b85f5d
test_storage: add (failing) test of the BucketWriter/BucketReader implementation
2007-04-17 20:03:44 -07:00
Brian Warner
dbbbc96cbf
storageserver: the last segment is not always going to be the same size as the rest, so don't assert such a thing
2007-04-17 13:41:45 -07:00
Brian Warner
2e15c9aed2
system_test: exercise multiple segments
2007-04-17 13:40:47 -07:00
Brian Warner
c3268ca394
download.py: don't truncate tail segments that are the same size as all the others
2007-04-17 13:39:35 -07:00
Brian Warner
a0dc26ee11
test_encode.Roundtrip: cover more combinations of data size relative to segment size and number of block hash tree leaves
2007-04-17 12:57:55 -07:00
Brian Warner
96812507a0
test_encode.Encode: cover more combinations of data size relative to segment size and number of block hash tree leaves
2007-04-17 12:29:56 -07:00
Brian Warner
a05b713076
test_encode: test filesizes which are an exact multiple of the segment size. This test fails right now.
2007-04-16 19:55:03 -07:00
Brian Warner
ff8cb4d32e
encode: make MAX_SEGMENT_SIZE controllable, to support tests which force the use of multiple segments. Also, remove not-very-useful upload-side debug messages
2007-04-16 19:29:57 -07:00
Brian Warner
b9624502c9
download: more test coverage
2007-04-16 17:21:37 -07:00
Brian Warner
6bdabd2cea
download: remove some leftover (and not very useful) debug logging
2007-04-16 17:17:57 -07:00
Brian Warner
2f5fb51848
download: validate handling of missing sharehashes too
2007-04-16 17:15:44 -07:00
Brian Warner
a5a30d6bee
iputil.py: remove unused import
2007-04-16 17:08:00 -07:00
Brian Warner
42179e5ae2
download: verify that bad blocks or hashes are caught by the download process
2007-04-16 16:30:21 -07:00
Zooko O'Whielacronx
5b3e923093
storageserver: ignore files in verifierdir whose filenames aren't of the right form for shares
2007-04-18 07:41:56 -07:00
Zooko O'Whielacronx
f4de079ba0
pyutil: iputil: fix netbsd, irix, sunos
2007-04-18 07:40:26 -07:00
Brian Warner
281a0d376c
test_system: bump up timeout again, the new extra download forms take more time on poor overloaded slave1
2007-04-16 15:53:25 -07:00
Brian Warner
88a7fdcaab
test_iputil: improve error message
2007-04-16 15:05:25 -07:00
Brian Warner
2fef5dac1f
download: log more information when hashtree checks fail
2007-04-16 13:08:19 -07:00
Brian Warner
7dabb68a51
download: improve test coverage on our IDownloadTarget classes, make FileHandle return the filehandle when its done so that it is easier to close
2007-04-16 13:07:36 -07:00
Brian Warner
1a6da72861
don't include test code itself in the test-coverage numbers
2007-04-16 12:32:45 -07:00
Brian Warner
2d832efa64
fec: add test for mathutil
2007-04-16 12:01:58 -07:00
Brian Warner
8e9d090bdb
fec: remove unused mathutil.linear_fit_slope
2007-04-16 11:51:36 -07:00
Brian Warner
8aa2694595
fec.util.mathutil.permute: fix docstring
2007-04-16 11:51:04 -07:00
Brian Warner
652e8a735f
interfaces: use explicit TupleOf and ChoiceOf constraints, since the upcoming version of Foolscap changes the meaning of bare tuples (from ChoiceOf to TupleOf)
2007-04-13 19:04:38 -07:00
Brian Warner
03fbee6ade
encode.py: remove an unused import
2007-04-12 20:09:32 -07:00
Brian Warner
0ecb26ec44
test_hashtree.py: get full coverage for hashtree.py
2007-04-12 19:58:13 -07:00
Brian Warner
30133a7cdf
hash trees: further cleanup, to make sure we're validating the right thing
...
hashtree.py: improve the methods available for finding out which hash nodes
are needed. Change set_hashes() to require that every hash provided can
be validated up to the root.
download.py: validate from the top down, including the URI-derived roothash
in the share hash tree, and stashing the thus-validated share hash for use
in the block hash tree.
2007-04-12 19:41:48 -07:00
Brian Warner
d351cd7674
download: always validate the blockhash, and don't let the bucket trick us into not validating hashes
2007-04-12 15:18:46 -07:00
Brian Warner
2e314ad47f
hashtree.py: reindent from 2-spaces to 4-spaces. No functional changes.
2007-04-12 14:24:11 -07:00
Brian Warner
d8215e0c6f
rename chunk.py to hashtree.py
2007-04-12 13:13:25 -07:00
Brian Warner
8f58b30db9
verify hash chains on incoming blocks
...
Implement enough of chunk.IncompleteHashTree to be usable.
Rearrange download: all block/hash requests now go through
a ValidatedBucket instance, which is responsible for retrieving
and verifying hashes before providing validated data. Download
was changed to use ValidatedBuckets everywhere instead of
unwrapped RIBucketReader references.
2007-04-12 13:07:40 -07:00
Zooko O'Whielacronx
234951041f
oops -- the previous commit of iputil wasn't the right version
...
Too bad synchronizing pyutil and allmydata.util includes a manual step.
2007-04-16 15:12:01 -07:00
Zooko O'Whielacronx
ac4d23d336
port iputil to Windows (and Irix, and NetBSD, and Solaris 2, ...)
2007-04-16 14:59:13 -07:00
Zooko O'Whielacronx
0f54468f66
pyfec: bump version number to 1.0.0a1-2-STABLE
2007-04-15 12:08:12 -07:00
Zooko O'Whielacronx
278f9ad5c9
pyfec: add mention of sha256sum to README
2007-04-15 12:07:35 -07:00
Zooko O'Whielacronx
b4e25737ff
pyfec: bump the performance measurement bragging up higher in the README
2007-04-14 18:03:54 -07:00
Zooko O'Whielacronx
625f230954
pyfec: bump version number to 1.0.0a1-1-STABLE
2007-04-14 17:50:40 -07:00
Zooko O'Whielacronx
2d9f728e27
pyfec: add -f option to fec, add more user-friendly handling of filesystem errors and user errors
2007-04-14 17:48:32 -07:00
