robk-tahoe
ef0f523fe0
stats gathering: added counters to upload,download,mutablewatcher
...
counting number of operations, and for immutable files, bytes transferred
2008-04-09 18:08:59 -07:00
Brian Warner
1869d7c29b
upload.py: remove spurious trailing quotes that confuse syntax-highlighting
2008-03-24 15:51:19 -07:00
Zooko O'Whielacronx
fc3bd0c987
use added secret to protect convergent encryption
...
Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key. If you want traditional convergent encryption behavior, set the added secret to be the empty string.
This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names. (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.)
This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 09:46:06 -07:00
Brian Warner
567979ba85
lower max_segsize for immutable files to 128KiB, to make download-progress smoother and improve alacrity. Closes #252 . Note that mutable files continue to use 1MiB segments.
2008-03-07 19:24:51 -07:00
Brian Warner
3a8bd1b772
webish: add more share information to upload status, including assisted uploads
2008-03-05 19:51:51 -07:00
Brian Warner
ca1a1762e2
web: status: add 'started' timestamps to all operations
2008-03-04 18:50:44 -07:00
Brian Warner
c8e24f0904
webish: make upload timings visible on the recent uploads/downloads status page
2008-03-03 14:48:52 -07:00
Brian Warner
1a7651ce82
retain 10 most recent upload/download status objects, show them in /status . Prep for showing individual status objects
2008-02-29 22:19:03 -07:00
Brian Warner
93d3960d4d
upload: fix up some log messages
2008-02-29 20:00:45 -07:00
Brian Warner
301dd3d489
webish status: distinguish active uploads/downloads from recent ones
2008-02-26 15:35:28 -07:00
Zooko O'Whielacronx
7c6de95bc6
switch from base62 to base32 for storage indices, switch from z-base-32 to rfc 3548 base-32 for everything, separate out base32 encoding from idlib
2008-02-14 19:27:47 -07:00
Zooko O'Whielacronx
3f8df27063
use base62 encoding for storage indexes, on disk and in verifier caps, and in logging and diagnostic tools
...
base62 encoding fits more information into alphanumeric chars while avoiding the troublesome non-alphanumeric chars of base64 encoding. In particular, this allows us to work around the ext3 "32,000 entries in a directory" limit while retaining the convenient property that the intermediate directory names are leading prefixes of the storage index file names.
2008-02-12 20:48:37 -07:00
Brian Warner
e6af3b845c
make current upload/download status objects available from the client
2008-02-12 15:39:45 -07:00
Brian Warner
d0ce8694c1
add upload-status objects, to track upload progress
2008-02-12 15:36:05 -07:00
Brian Warner
873dee53e9
test_system: remove the hackish debug_interrupt= attribute magic used to exercise interrupted-upload resumption, instead just make the Uploadable bounce the helper halfway through the upload
2008-02-07 20:15:37 -07:00
Brian Warner
3bd79917b2
test_system.py: remove that ugly debug_stash_RemoteencryptedUploadable hack, now that UploadResults give us a better approach
2008-02-07 17:27:30 -07:00
Brian Warner
c7b10f4498
upload: if we lose the helper, go back to doing direct-to-server uploads instead of causing all subsequent uploads to fail
2008-02-07 17:26:59 -07:00
Brian Warner
b2d31f0a8a
upload.EncryptAnUploadable: use 50KiB read chunks (instead of 50KB), to match the chunksize requested by the upload helper
2008-02-07 17:10:13 -07:00
Brian Warner
da9610e90a
change encryption-key hash to include encoding parameters. This is a minor compatibility break: CHK files encoded (with convergence) before and after this will have different keys and ciphertexts. Also switched to SHA-256d for both the data-to-key hash and the key-to-storageindex hash
2008-02-06 19:50:47 -07:00
Brian Warner
81c5ceae16
upload: rework passing of default encoding parameters: move more responsibility into BaseUploadable
2008-02-06 18:39:03 -07:00
Brian Warner
75e4332462
offloaded: oops, need more tricks to make the unit tests pass
2008-02-06 17:51:11 -07:00
Brian Warner
a2cace9cfb
helper: return full uri-extension data to the client, so it can compare encoding parameters
2008-02-06 17:30:58 -07:00
Brian Warner
652d50d7e0
UploadResults: add more helper timing stats (ciphertext fetch times)
2008-02-06 02:12:35 -07:00
Brian Warner
124fb5ecdf
add upload-results timing info for helper uploads. This changes the Helper protocol, and introduces a compatibility break
2008-02-06 01:52:25 -07:00
Brian Warner
93d45abb02
add upload timings and rates to the POST /uri?t=upload results page
2008-02-06 00:41:51 -07:00
Brian Warner
66f33ee504
upload: return an UploadResults instance (with .uri) instead of just a URI
2008-02-05 21:01:38 -07:00
Brian Warner
d98fde952c
introducer: remove remaining bits of 'push-to-myself' flags. The uploading/downloading node is no longer special.
2008-02-05 14:16:01 -07:00
Brian Warner
daecca6589
big introducer refactoring: separate publish+subscribe. Addresses #271 .
2008-02-05 13:05:13 -07:00
Zooko O'Whielacronx
1d1628e525
rename storage_index_chk_hash() to storage_index_hash() and add TODO about how our use of it now includes keys that are not CHKs
2008-02-01 12:27:37 -07:00
Brian Warner
909d848071
remove upload.upload_(data,filename,filehandle) convenience functions
2008-01-30 19:03:19 -07:00
Brian Warner
44ff4c6605
upload: Data should use convergence by default
2008-01-30 19:02:56 -07:00
Zooko O'Whielacronx
ca971559e6
make content-hash-key encryption a parameter of uploading
...
fixes #293
2008-01-30 12:24:50 -07:00
Brian Warner
c2c922b5e9
upload: add log message when AssistedUploader is done
2008-01-29 17:38:12 -07:00
Brian Warner
c6c5a37d3f
upload: fix 'skipping_ahead' log message
2008-01-28 19:13:36 -07:00
Brian Warner
8063aa8b86
WriteBucketProxy: improve __repr__
2008-01-28 18:53:51 -07:00
Brian Warner
aa3b509323
upload.py: the 'skipping encryption' message was emitted exactly backwards
2008-01-28 18:38:38 -07:00
Brian Warner
8485ff53fc
display the Helper FURL and our connection status on the welcome page. Closes #285 .
2008-01-28 13:56:22 -07:00
Brian Warner
09a99ce822
offloaded upload: avoid tail-recursion problem that would break large files
2008-01-24 21:51:34 -07:00
Brian Warner
46fe024612
offloaded uploader: don't use a huge amount of memory when skipping over previously-uploaded data
2008-01-24 17:25:33 -07:00
Brian Warner
e9307d3fda
offloaded: close the local filehandle after encoding is done, otherwise windows fails
2008-01-17 01:52:33 -07:00
Brian Warner
6b08c28f5d
upload.py: make it easier to have an IUploadable that overrides encoding parameters: just set an attribute instead of subclassing
2008-01-17 01:17:42 -07:00
Brian Warner
812383a369
offloaded: upload.py: handle forward skips, to allow resumed uploads to send less than all the data. We still read all the data (to hash it, 'paranoid mode'), but we don't send it over the wire
2008-01-17 01:16:56 -07:00
Brian Warner
c597e67c2b
offloaded: improve logging across the board
2008-01-17 01:11:35 -07:00
Brian Warner
51321944f0
megapatch: overhaul encoding_parameters handling: now it comes from the Uploadable, or the Client. Removed options= too. Also move helper towards resumability.
2008-01-16 03:03:35 -07:00
Brian Warner
a6ca98ac53
upload: add Encoder.abort(), to abandon the upload in progress. Add some debug hooks to enable unit tests.
2008-01-14 21:22:55 -07:00
Brian Warner
60090fb9f2
upload: improve logging
2008-01-14 21:19:20 -07:00
Brian Warner
e65967da49
upload: pass options through to the encoder
2008-01-14 21:17:32 -07:00
Brian Warner
7ac2b94aba
remove wait_for_numpeers and the when_enough_peers call in mutable.Publish
2008-01-14 14:55:59 -07:00
Brian Warner
964edadf44
offloaded: add a system test, make it pass. files are now being uploaded through the helper.
2008-01-11 05:42:55 -07:00
Brian Warner
6ac01fde4c
offloaded: more test coverage on client side, change interfaces a bit
2008-01-11 04:53:37 -07:00
