Commit Graph

110 Commits

Author SHA1 Message Date
Brian Warner
f6b2072af1 check-speed: test SSK upload/download speed too. SDMF imposes a limit on the file sizes, no 10MB or 100MB test 2007-12-14 02:05:31 -07:00
Brian Warner
0dc84963f1 the wait_for_numpeers= argument to client.upload() is optional: make both the code and the Interface reflect this 2007-12-06 18:36:58 -07:00
Brian Warner
f190382d5e refactor web tests, and interfaces.IFileNode 2007-12-04 23:01:37 -07:00
Brian Warner
0f5ef5184d test_dirnode.py: obtain full coverage of dirnode.py 2007-12-04 14:32:04 -07:00
Zooko O'Whielacronx
59d6c3c822 decentralized directories: integration and testing
* use new decentralized directories everywhere instead of old centralized directories
 * provide UI to them through the web server
 * provide UI to them through the CLI
 * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them
 * remove the notion of a "vdrive server" and a client thereof
 * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example)
 * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start.  The default is 1.
 * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!)
 * fix a few bugs where we used to equate "mutable" with "not read-only".  Nowadays all directories are mutable, but some might be read-only (to you).
 * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge
 * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.
2007-12-03 14:52:42 -07:00
Zooko O'Whielacronx
ae727a550a IMutableFileNode is a subtype of IFileNode
I'm not 100% sure that this is correct, but it looks reasonable, it passes unit
tests (although note that unit tests are currently not covering the new mutable
files very well), and it makes the "view JSON" link on a directory work instead
of raising an assertion error.
2007-11-10 16:37:18 -07:00
Brian Warner
be94960680 mutable: test roundtrip, make it work 2007-11-07 14:19:01 -07:00
Brian Warner
7e43c7b5f9 mutable: fix use of storage API 2007-11-06 18:53:34 -07:00
Brian Warner
c4f7412f1c stabilize on 20-byte nodeids everywhere, printed with foolscap's base32 2007-11-06 18:49:59 -07:00
Brian Warner
e08b091d9f storage: rewrite slot API, now use testv_and_readv_and_writev or readv 2007-11-05 20:17:14 -07:00
Brian Warner
8f21424449 storage: add readv_slots: get data from all shares 2007-11-05 00:37:01 -07:00
Brian Warner
516ce89a1e trailing-whitespace eradication, no functional changes 2007-11-01 15:33:47 -07:00
Brian Warner
f4946bc22e trailing-whitespace eradication, no functional changes 2007-11-01 15:22:57 -07:00
Brian Warner
1d8a4cdfe7 mutable: first pass at dirnodes, filenodes, new URIs. Some test coverage.
The URI typenames need revision, and only a few dirnode methods are
implemented. Filenodes are non-functional, but URI/key-management is in
place. There are a lot of classes with names like "NewDirectoryNode" that
will need to be rename once we decide what (if any) backwards compatibility
want to retain.
2007-11-01 15:15:29 -07:00
Brian Warner
fb3eddafdb move NotMutableError from dirnode.py into interfaces.py 2007-11-01 15:03:07 -07:00
Brian Warner
68d3d62002 mutable slots: finish up basic coding on server-side containers, add some tests. Remove all caching from MutableShareFile. 2007-10-31 00:10:40 -07:00
Brian Warner
b24c2925e8 checkpointing mutable-file work. Storage layer is 80% in place. 2007-10-30 19:47:36 -07:00
Brian Warner
046bda2b47 webish: add checker results and a 'Check' button to the web interface 2007-10-23 17:23:57 -07:00
Brian Warner
afca99c485 interfaces: clarify IChecker.checker_results_for a bit 2007-10-22 18:10:46 -07:00
Brian Warner
57f994fb02 checker: remember checker results, but only in ram for now 2007-10-22 17:46:24 -07:00
Brian Warner
9da1d70676 add a simple checker, for both files and directories 2007-10-15 16:16:39 -07:00
Zooko O'Whielacronx
426721f3f2 update a few documents, comments, and defaults to mention 3-of-10 instead of 25-of-100 2007-10-15 19:53:59 -07:00
Brian Warner
c301b41f50 control: add measure_peer_response_time(), to estimate RTT for the mesh 2007-09-26 12:21:15 -07:00
Brian Warner
d58d794757 check_speed: do both upload and download tests 2007-09-20 18:52:44 -07:00
Brian Warner
2632c0ad8c check_speed: upload multiple files, measure Ax+B 2007-09-20 16:55:33 -07:00
Brian Warner
b9d5a4ead4 check_speed.py: run two 1MB uploads and measure the time it takes 2007-09-19 18:40:18 -07:00
Zooko O'Whielacronx
09ff5af99c loosen constraint on share data size 2007-09-11 11:29:18 -07:00
Brian Warner
76be4a582c webish: implement 'PUT /uri?t=mkdir' (to create anonymous dirnodes) 2007-09-05 17:23:06 -07:00
wilcoxjg
33bae16ed2 created DirnodeURI schema 2007-08-28 23:28:26 -07:00
Brian Warner
2a63fe8b01 deletion phase3: add a sqlite database to track renew/cancel-lease secrets, implement renew/cancel_lease (but nobody calls them yet). Also, move the shares from BASEDIR/storage/* down to BASEDIR/storage/shares/* 2007-08-27 23:41:40 -07:00
Brian Warner
739ae1ccde deletion phase1: send renew/cancel-lease secrets, but my_secret is fake, and the StorageServer discards them 2007-08-27 17:28:51 -07:00
Brian Warner
fe06b3be8b dirnode: change the defined behavior of RIVirtualDriveServer.set to allow replace-in-place without raising an exception 2007-08-16 17:03:19 -07:00
Brian Warner
42dcc3088e IDirectoryNode: add has_child() method 2007-08-15 13:22:01 -07:00
Brian Warner
c3293f4f37 interfaces: move signatures into docstrings, to reduce lines of code and improve code-coverage numbers 2007-07-24 19:43:21 -07:00
Brian Warner
e6e9ddc588 refactor upload/encode, to split encrypt and encode responsibilities 2007-07-23 19:31:53 -07:00
Brian Warner
9c5ab89afe truncate storage index to 128 bits, since it's derived from a 128 bit AES key 2007-07-22 19:48:44 -07:00
Brian Warner
1d9a58977f uri: implement URI-processing classes, IFileURI/IDirnodeURI, use internally 2007-07-21 15:40:36 -07:00
Brian Warner
9af506900b upload: refactor to enable streaming upload. not all tests pass yet 2007-07-19 18:21:44 -07:00
Brian Warner
20c980d02b reduce MAX_SEGMENT_SIZE from 2MB to 1MB, to compensate for the large blocks that 3-of-10 produces 2007-07-16 13:48:34 -07:00
Brian Warner
5c08be170e interfaces: increase ShareSize now that our default k is smaller (hence blocks are bigger) 2007-07-13 19:29:31 -07:00
Brian Warner
1f8e407d9c more #85 work, system test still fails 2007-07-13 15:09:01 -07:00
Brian Warner
cd8648d39b storage: use one file per share instead of 7 (#85). work-in-progress, tests still fail 2007-07-13 14:04:49 -07:00
Brian Warner
b0c65ed232 note that setting k=1 is equivalent to replication 2007-07-12 16:22:12 -07:00
Brian Warner
5399395c27 allow the introducer to set default encoding parameters. Closes #84.
By writing something like "25 75 100" into a file named 'encoding_parameters'
in the central Introducer's base directory, all clients which use that
introducer will be advised to use 25-out-of-100 encoding for files (i.e.
100 shares will be produced, 25 are required to reconstruct, and the upload
process will be happy if it can find homes for at least 75 shares). The
default values are "3 7 10". For small meshes, the defaults are probably
good, but for larger ones it may be appropriate to increase the number of
shares.
2007-07-12 15:33:30 -07:00
Brian Warner
b62252183d interfaces.py: increase RIVirtualDriveServer.list constraint from 100 entries to 1000, for now 2007-07-12 13:23:52 -07:00
Brian Warner
dce1dc2730 storage: wrap buckets in a local proxy
This will make it easier to change RIBucketWriter in the future to reduce the wire
protocol to just open/write(offset,data)/close, and do all the structuring on the
client end. The ultimate goal is to store each bucket in a single file, to reduce
the considerable filesystem-quantization/inode overhead on the storage servers.
2007-07-08 23:27:46 -07:00
Brian Warner
a34040df99 interfaces: remove spurious 'self' from interface declarations 2007-07-06 19:39:47 -07:00
Brian Warner
9e42dda6a4 add IDirectoryNode.get_child_at_path 2007-07-06 19:38:37 -07:00
Brian Warner
c4a8db3eb2 webish: provide a valid Content-Length header on downloads 2007-07-03 15:09:00 -07:00
Brian Warner
622acc690a webish: improve reporting of web download errors that occur early enough
If the error occurs before any data has been sent, we can give a sensible
error message (code 500, stack trace, etc). This will cover most of the error
cases. The ones that aren't covered are when we run out of good peers after
successfully decoding the first segment, either because they go away or
because their shares are corrupt.
2007-07-03 13:47:37 -07:00