Commit Graph

23 Commits

Author SHA1 Message Date
Brian Warner
5cbdc240e2 encode: add plaintext/crypttext merkle trees to the shares, and the thingA block. Still needs tests and download-side verification. 2007-06-06 19:40:20 -07:00
Brian Warner
6bb9debc16 encode: tolerate lost peers, as long as we still get enough shares out. Closes #17. 2007-06-06 10:32:40 -07:00
Brian Warner
3dfd26970b move validation data to thingA, URI has storage_index plus thingA hash
This (compatibility-breaking) change moves much of the validation data and
encoding parameters out of the URI and into the so-called "thingA" block
(which will get a better name as soon as we find one we're comfortable with).
The URI retains the "storage_index" (a generalized term for the role that
we're currently using the verifierid for, the unique index for each file
that gets used by storage servers to decide which shares to return), the
decryption key, the needed_shares/total_shares counts (since they affect
peer selection), and the hash of the thingA block.

This shortens the URI and lets us add more kinds of validation data without
growing the URI (like plaintext merkle trees, to enable strong incremental
plaintext validation), at the cost of maybe 150 bytes of alacrity. Each
storage server holds an identical copy of the thingA block.

This is an incompatible change: new messages have been added to the storage
server interface, and the URI format has changed drastically.
2007-06-01 18:48:01 -07:00
Brian Warner
4b2298937b use real encryption, generate/store/verify verifierid and fileid 2007-04-25 17:53:10 -07:00
Brian Warner
49e992b8b6 make test_encode less CPU-intense by using 4-out-of-10 encoding instead of 25-out-of-100 2007-04-19 10:56:15 -07:00
Brian Warner
a0dc26ee11 test_encode.Roundtrip: cover more combinations of data size relative to segment size and number of block hash tree leaves 2007-04-17 12:57:55 -07:00
Brian Warner
96812507a0 test_encode.Encode: cover more combinations of data size relative to segment size and number of block hash tree leaves 2007-04-17 12:29:56 -07:00
Brian Warner
a05b713076 test_encode: test filesizes which are an exact multiple of the segment size. This test fails right now. 2007-04-16 19:55:03 -07:00
Brian Warner
ff8cb4d32e encode: make MAX_SEGMENT_SIZE controllable, to support tests which force the use of multiple segments. Also, remove not-very-useful upload-side debug messages 2007-04-16 19:29:57 -07:00
Brian Warner
b9624502c9 download: more test coverage 2007-04-16 17:21:37 -07:00
Brian Warner
2f5fb51848 download: validate handling of missing sharehashes too 2007-04-16 17:15:44 -07:00
Brian Warner
42179e5ae2 download: verify that bad blocks or hashes are caught by the download process 2007-04-16 16:30:21 -07:00
Brian Warner
8f58b30db9 verify hash chains on incoming blocks
Implement enough of chunk.IncompleteHashTree to be usable.
Rearrange download: all block/hash requests now go through
a ValidatedBucket instance, which is responsible for retrieving
and verifying hashes before providing validated data. Download
was changed to use ValidatedBuckets everywhere instead of
unwrapped RIBucketReader references.
2007-04-12 13:07:40 -07:00
Brian Warner
8d2def5b04 encode: clean up some weirdness that was there to make unit tests easier to write 2007-04-05 22:36:18 -07:00
Brian Warner
919ca3e902 rename encode_new.py to encode.py, now that there isn't an old one anymore 2007-04-05 21:17:42 -07:00
Brian Warner
7cd9ef3bbf finish making the new encoder/decoder/upload/download work 2007-03-30 16:50:50 -07:00
Brian Warner
234b2f354e add new test for doing an encode/decode round trip, and make it almost work 2007-03-30 13:20:01 -07:00
Brian Warner
9a2e0cf28e switch upload to use encode_new, fix a few things (but not nearly all of them) 2007-03-30 11:53:03 -07:00
Brian Warner
7d7ce7a059 test_encode: make sure encode_new can produce the data it is supposed to 2007-03-30 11:32:57 -07:00
Zooko O'Whielacronx
4b4f5bbcba change #!/usr/bin/python to #!/usr/bin/env python
Note that using "whatever version of python the name 'python' maps to in the current shell environment" is more error-prone that specifying which python you mean, such as by executing "/usr/bin/python setup.py" instead of executing "./setup.py".  When you build tahoe (by running "make") it will make a copy of bin/allmydata-tahoe in instdir/bin/allmydata-tahoe with the shebang line rewritten to execute the specific version of python that was used when building instead of to execute "/usr/bin/env python".

However, it seems better that the default for lazy people be "whatever 'python' means currently" instead of "whatever 'python' meant to the manufacturer of your operating system".
2007-03-29 14:01:28 -07:00
Brian Warner
eb3b9b16af encode: start to fix a few problems, still a lot of work left to go 2006-12-14 04:31:17 -07:00
Brian Warner
3e5e2ee9fe make initial simple encode_new test pass 2006-12-14 04:17:01 -07:00
Brian Warner
3490378551 move all packages into src/, fix allmydata.Crypto build. Now you must perform a 'setup.py build' before using anything, and you must add the build directory (build/lib.linux-i686-2.4) to your PYTHONPATH before doing anything 2006-12-14 03:39:50 -07:00