ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2025-01-04 04:04:10 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
3,717 Commits 185 Branches 90 Tags 89 MiB
cc3c1ae8df
Commit Graph

1064 Commits

Author SHA1 Message Date
Zooko O'Whielacronx
872e4fc84d doc: sundry amendments to docs and in-line code comments 2008-12-28 16:59:54 -07:00
Zooko O'Whielacronx
9f117dbe8f cli: mark unicode filenames as unsupported -- see #534 for details 2008-12-24 13:28:02 -07:00
Zooko O'Whielacronx
25d962c67d test: extend timeout on the hotline file that prevents the client from stopping itself 
The 20-second timeout was apparently tripped on my Powerbook G4 "draco".
 2008-12-21 21:06:29 -07:00
Zooko O'Whielacronx
f1fbd4feae dirnode: don't check MAC on entries in dirnodes 
In an ancient version of directories, we needed a MAC on each entry.  In modern times, the entire dirnode comes with a digital signature, so the MAC on each entry is redundant.
With this patch, we no longer check those MACs when reading directories, but we still produce them so that older readers will accept directories that we write.
 2008-12-21 17:35:18 -07:00
Zooko O'Whielacronx
8b7ce325d7 immutable, checker, and tests: improve docstrings, assertions, tests 
No functional changes, but remove unused code, improve or fix docstrings, etc.
 2008-12-21 15:07:52 -07:00
Zooko O'Whielacronx
471e1f1b9b try to tidy up uri-as-string vs. uri-as-object 
I get confused about whether a given argument or return value is a uri-as-string or uri-as-object.  This patch adds a lot of assertions that it is one or the other, and also changes CheckerResults to take objects not strings.
In the future, I hope that we generally use Python objects except when importing into or exporting from the Python interpreter e.g. over the wire, the UI, or a stored file.
 2008-12-19 08:39:24 -07:00
Zooko O'Whielacronx
7b285ebcb1 immutable: remove the last bits of code (only test code or unused code) which did something with plaintext hashes or plaintext hash trees 2008-12-19 08:18:07 -07:00
Zooko O'Whielacronx
82ee44ed5b debug: pass empty optional arguments to ReadBucketProxy 
because those arguments are about to become non-optional (for other code than test/debug code)
 2008-12-16 17:51:45 -07:00
Brian Warner
278c47b9bd mutable publish: if we are surprised by shares that match what we would have written anyways, don't be surprised. This should fix one of the two #546 problems, in which we re-use a server and forget that we already sent them a share. 2008-12-09 22:44:49 -07:00
Zooko O'Whielacronx
c456ff8591 rename "get_verifier()" to "get_verify_cap()" 2008-12-08 12:44:11 -07:00
Zooko O'Whielacronx
16b20794dd mutable: merge renaming with test patches 2008-12-07 08:45:19 -07:00
Zooko O'Whielacronx
b58875fe43 mutable: rename mutable/node.py to mutable/filenode.py and mutable/repair.py to mutable/repairer.py 
To be more consistent with the immutable layout that I am working on.
 2008-12-07 08:20:08 -07:00
francois
db7ad6da12 filenode.py: Fix partial HTTP Range header handling according to RFC2616 
Tahoe webapi was failing on HTTP request containing a partial Range header.
This change allows movies players like mplayer to seek in movie files stored in
tahoe.

Associated tests for GET and HEAD methods are also included
 2008-11-18 07:41:35 -07:00
Brian Warner
ffb5985146 mutable.modify(): after UCWE, publish even if the second invocation of the modifier didn't modify anything. For #551. 2008-12-05 22:49:23 -07:00
Brian Warner
7a0afb59a4 dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550. 2008-12-05 22:08:37 -07:00
Brian Warner
fb9af2c7a0 MutableFileNode.modify: pass first_time= and servermap= to the modifier callback 2008-12-05 22:07:10 -07:00
Zooko O'Whielacronx
60bbc46a53 minor: fix unused imports -- thanks, pyflakes 2008-12-05 13:07:23 -07:00
Zooko O'Whielacronx
b315619d6b download: refactor handling of URI Extension Block and crypttext hash tree, simplify things 
Refactor into a class the logic of asking each server in turn until one of them gives an answer 
that validates.  It is called ValidatedThingObtainer.

Refactor the downloading and verification of the URI Extension Block into a class named 
ValidatedExtendedURIProxy.

The new logic of validating UEBs is minimalist: it doesn't require the UEB to contain any 
unncessary information, but of course it still accepts such information for backwards 
compatibility (so that this new download code is able to download files uploaded with old, and 
for that matter with current, upload code).

The new logic of validating UEBs follows the practice of doing all validation up front.  This 
practice advises one to isolate the validation of incoming data into one place, so that all of 
the rest of the code can assume only valid data.

If any redundant information is present in the UEB+URI, the new code cross-checks and asserts 
that it is all fully consistent.  This closes some issues where the uploader could have 
uploaded inconsistent redundant data, which would probably have caused the old downloader to 
simply reject that download after getting a Python exception, but perhaps could have caused 
greater harm to the old downloader.

I removed the notion of selecting an erasure codec from codec.py based on the string that was 
passed in the UEB.  Currently "crs" is the only such string that works, so 
"_assert(codec_name == 'crs')" is simpler and more explicit.  This is also in keeping with the 
"validate up front" strategy -- now if someone sets a different string than "crs" in their UEB, 
the downloader will reject the download in the "validate this UEB" function instead of in a 
separate "select the codec instance" function.

I removed the code to check plaintext hashes and plaintext Merkle Trees.  Uploaders do not 
produce this information any more (since it potentially exposes confidential information about 
the file), and the unit tests for it were disabled.  The downloader before this patch would 
check that plaintext hash or plaintext merkle tree if they were present, but not complain if 
they were absent.  The new downloader in this patch complains if they are present and doesn't 
check them.  (We might in the future re-introduce such hashes over the plaintext, but encrypt 
the hashes which are stored in the UEB to preserve confidentiality.  This would be a double-
check on the correctness of our own source code -- the current Merkle Tree over the ciphertext 
is already sufficient to guarantee the integrity of the download unless there is a bug in our 
Merkle Tree or AES implementation.) 

This patch increases the lines-of-code count by 8 (from 17,770 to 17,778), and reduces the 
uncovered-by-tests lines-of-code count by 24 (from 1408 to 1384).  Those numbers would be more 
meaningful if we omitted src/allmydata/util/ from the test-coverage statistics.
 2008-12-05 08:17:54 -07:00
Brian Warner
a0b5b5ab2b test_web: add get_permuted_peers, to unbreak recent checker_results change 2008-12-05 02:12:10 -07:00
Brian Warner
e942ab141b test_system.py: assert less about the stats we get, since shares (and thus allocate() calls) are distributed randomly 2008-12-04 17:27:04 -07:00
Brian Warner
7c4edac582 stats: don't return booleans: it violates the schema. Add a test. 2008-12-04 15:01:24 -07:00
Brian Warner
7cfc74bcc9 test_system.py: don't ask the stats-gatherer to poll: it tolerates failures, so it isn't really giving us enough test coverage. Removing the call will make it more clear that we need to improve the tests later 2008-12-04 15:00:53 -07:00
Brian Warner
a2ce315074 CLI: check for pre-existing aliases in 'tahoe create-alias' and 'tahoe add-alias' 2008-12-02 20:20:22 -07:00
Brian Warner
cdbb72d5ff test_cli: pass rc out of do_cli() too 2008-12-02 20:08:28 -07:00
Brian Warner
6958b7fa90 test_storage.py: more windows-vs-readonly-storage fixes 2008-12-02 19:41:02 -07:00
Brian Warner
d3783475b4 test_download.py: remove extra base32 import 2008-12-02 18:31:26 -07:00
Brian Warner
b997e6ca7c test_download: test both mutable and immutable pre-generated shares 2008-12-02 18:30:07 -07:00
Brian Warner
2d70f7fc0b test_download.py: added 'known-answer-tests', to make sure current code can download a file that was created by earlier code 2008-12-02 18:22:08 -07:00
Brian Warner
71c62f8fcb #542 'tahoe create-key-generator': fix the .tac file this creates to be compatible with modern code, add a test 2008-12-01 17:47:21 -07:00
Brian Warner
cfba882b30 storage: replace sizelimit with reserved_space, make the stats 'disk_avail' number incorporate this reservation 2008-12-01 17:24:21 -07:00
Brian Warner
39a41f1d85 util/abbreviate: add abbreviated-size parser 2008-12-01 17:24:12 -07:00
Zooko O'Whielacronx
fe6abac87b wui/wapi: change the default port number from 8123 to 3456 to avoid conflict with TorButton 
See ticket #536 for details.
 2008-11-25 17:57:37 -07:00
Brian Warner
bc53c24003 dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed. 2008-11-24 14:40:46 -07:00
Brian Warner
0fab511be5 upload: don't use servers which can't support the share size we need. This ought to avoid #439 problems. Some day we'll have a storage server which advertises support for a larger share size. No tests yet. 2008-11-21 20:28:12 -07:00
Brian Warner
bf06492a90 #538: fetch version and attach to the rref. Make IntroducerClient demand v1 support. 2008-11-21 20:07:27 -07:00
Brian Warner
0eb6b324a4 #538: add remote_get_version() to four main Referenceable objects: Introducer Service, Storage Server, Helper, CHK Upload Helper. Remove unused storage-server get_versions(). 2008-11-21 17:43:52 -07:00
Brian Warner
4ada26101b test_dirnode: add an explainError call 2008-11-19 16:02:12 -07:00
Brian Warner
b84c2c6541 manifest: add storage-index strings to the json results 2008-11-19 16:00:27 -07:00
Brian Warner
815e0673e6 manifest: include stats in results. webapi is unchanged. 2008-11-19 15:03:47 -07:00
Brian Warner
6f6556da22 mutable: respect the new tahoe.cfg 'shares.needed' and 'shares.total' settings 2008-11-19 14:05:01 -07:00
Brian Warner
82fe7ba360 oops, update tests to match 'tahoe stats' change 2008-11-18 20:32:59 -07:00
Brian Warner
7391f63fc1 util/abbreviate: little utility to abbreviate seconds and bytes 2008-11-18 20:11:42 -07:00
Brian Warner
5c3e153d0e cli: tahoe stats/manifest: change --verbose to --raw, since I want -v for --verify for check/deep-check/repair 2008-11-18 18:36:08 -07:00
Brian Warner
d657d22c0c test_system: make 'where' strings more helpful, to track down test failures better 2008-11-18 18:29:50 -07:00
Brian Warner
7932fadb5e webapi: add 'summary' string to checker results JSON 2008-11-18 18:28:26 -07:00
Brian Warner
994d97c644 webapi: introducer stats: add 'announcement_distinct_hosts' to the t=json form, to show how many distinct hosts are providing e.g. storage services 2008-11-18 15:30:15 -07:00
Brian Warner
8473a96ada #330: convert stats-gatherer into a .tac file service, add 'tahoe create-stats-gatherer' 2008-11-18 01:46:20 -07:00
Brian Warner
cae54ab118 tahoe.cfg: add controls for k and N (and shares-of-happiness) 2008-11-18 00:29:44 -07:00
Brian Warner
ead0e4d6ca cli: add tests for 'tahoe stats --verbose' 2008-11-17 22:11:14 -07:00
Brian Warner
d6a67cd566 dirnode manifest/stats: process more than one LIT file per tree; we were accidentally ignoring all but the first 2008-11-14 22:50:49 -07:00
francois
1a3a464534 test_cli.py: Ensure that we can read our uploaded files back 2008-11-14 07:44:58 -07:00
francois
32b5bc6330 test_cli.py: use str objects instead of unicode ones 
This will hopefully fix failing tests with LC_ALL=C
 2008-11-14 07:41:37 -07:00
Brian Warner
5a60086dbc CLI: add 'tahoe stats', to run start-deep-stats and print the results 2008-11-13 19:43:50 -07:00
Brian Warner
9c9994300e test_system.py: fix new 'tahoe manifest' tests to not break on windows, by providing --node-directory instead of --node-url 2008-11-13 15:27:48 -07:00
francois
5c0c5bfc81 test for bug #534, unicode filenames 
This test assure that uploading a file whose name contains unicode character
doesn't prevent further uploads in the same directory.
 2008-11-13 05:19:51 -07:00
Brian Warner
0d93d6244e CLI: add 'tahoe manifest', which takes a directory and returns a list of things you can reach from it 2008-11-12 20:17:25 -07:00
Brian Warner
9976bd439a tahoe.cfg: add tub.location, to override the location hints we include in our FURL. This replaces advertised_ip_addresses, which doesn't remain useful enough to retain it. Helps with #517 (Tor). 2008-11-12 18:44:58 -07:00
Zooko O'Whielacronx
14444dbcc0 tests: fix comment 1970-01-05 04:10:55 -07:00
Zooko O'Whielacronx
3a0dc8e4ec tests: add tahoe_cp to the list of scripts that we don't actually have tests for yet 1970-01-05 04:00:58 -07:00
Brian Warner
dfa2408157 checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests 2008-11-06 22:35:47 -07:00
Zooko O'Whielacronx
eae5403e5a shebang: replace "/usr/bin/python" with "/usr/bin/env python" 2008-11-04 18:03:06 -07:00
Brian Warner
6fa41e738b immutable: tolerate filenode.read() with a size= that's too big, rather than hanging 2008-11-04 15:29:19 -07:00
Brian Warner
a2e195df3e debug catalog-shares: tolerate even more errors on bad files/directories 2008-10-30 14:54:47 -07:00
Brian Warner
ba019bfd3a #527: expire the cached files that are used to support Range: headers, every hour, when the file is unused and older than an hour 2008-10-30 13:39:09 -07:00
Brian Warner
c205a54965 util/cachedir.py: add a cache-directory manager class, which expires+deletes unused files after a while 2008-10-30 13:01:20 -07:00
Brian Warner
1419db6ac7 test_cli: try to fix windows again 2008-10-30 12:32:04 -07:00
Brian Warner
d97c34d622 debug/test_cli: fix error handling for catalog-shares, to make the test stop failing on windows 2008-10-30 12:06:51 -07:00
Brian Warner
b1db6d9ff2 web: add 'Repair' button to checker results when they indicate unhealthyness. Also add the object's uri to the CheckerResults instance. 2008-10-29 18:09:17 -07:00
Brian Warner
d3f2df00be webapi: serve the /static URL tree from /public_html (configurable) 2008-10-29 15:34:31 -07:00
Brian Warner
a9101112a4 catalog-shares command: tolerate errors, log them to stderr, handle v2-immutable shares 2008-10-29 15:10:10 -07:00
Brian Warner
186b64b633 test_web.py: one more line of test coverage 2008-10-28 22:00:15 -07:00
Brian Warner
0350994133 test_web: improve test coverage of PUT DIRURL t=uri replace=false 2008-10-28 21:57:44 -07:00
Brian Warner
a5aebf0cbd web: test (and fix) PUT DIRURL t=uri, which replaces a directory in-place with some other cap 2008-10-28 21:54:46 -07:00
Brian Warner
248033d78c test_client.py: improve test coverage a bit 2008-10-28 21:43:35 -07:00
Brian Warner
b73c380cdb move testutil into test/common_util.py, since it doesn't count as 'code under test' for our pyflakes numbers 2008-10-28 21:28:31 -07:00
Brian Warner
294e3fb682 util: move PollMixin to a separate file (pollmixin.py), so testutil can be moved into test/ 2008-10-28 21:15:48 -07:00
Brian Warner
4c2235fcef web/filenode: oops, fix test failures, not everything has a storage index 2008-10-28 18:17:20 -07:00
Brian Warner
b1ca238176 #527: respond to GETs with early ranges quickly, without waiting for the whole file to download. Fixes the alacrity problems with the earlier code. Still needs cache expiration. 2008-10-28 17:56:18 -07:00
Brian Warner
37e3d8e47c #527: support HTTP 'Range:' requests, using a cachefile. Adds filenode.read(consumer, offset, size) method. Still needs: cache expiration, reduced alacrity. 2008-10-28 13:41:04 -07:00
Brian Warner
914655c52b interfaces.py: promote immutable.encode.NotEnoughSharesError.. it isn't just for immutable files any more 2008-10-27 13:34:49 -07:00
Brian Warner
fca158e83a dirnode lookup: use distinct NoSuchChildError instead of the generic KeyError when a child can't be found 2008-10-27 13:15:25 -07:00
Brian Warner
9f21f7cf65 mutable: call remove_advise_corrupt_share when we see share corruption in mapupdate/download/check, tolerate servers that do not implement it 2008-10-24 13:21:28 -07:00
Brian Warner
db37c14ab7 storage: add remote_advise_corrupt_share, for clients to tell storage servers about share corruption that they've discovered. The server logs the report. 2008-10-24 11:52:48 -07:00
Brian Warner
6dbef907ac mutable/servermap.py: fix needs_merge(), it was incorrectly claiming that mixed shares with distinct seqnums needed a merge, causing repair(force=False) to fail 2008-10-23 21:00:24 -07:00
Brian Warner
0383ec5a49 test_web.test_POST_DIRURL_deepcheck: confirm that /operations/HANDLE/ works with or without the slash 2008-10-23 19:17:59 -07:00
Brian Warner
cadb045288 test_web: more test coverage 2008-10-23 17:11:18 -07:00
Brian Warner
a1cfac89f9 webapi: fix t=rename from==to, it used to delete the file 2008-10-23 16:32:36 -07:00
Brian Warner
d4b4cd8ab8 test_system: update test to match web checker results 2008-10-23 16:32:02 -07:00
Brian Warner
c455d52453 deep-check: add webapi links to detailed per-file/dir results 2008-10-23 16:00:31 -07:00
Brian Warner
76b7b9d835 webapi: make the /operations/ 't=status' qualifier optional, remove it from examples 2008-10-23 15:56:58 -07:00
Brian Warner
2ee6356c89 test_dirnode.py: oops, missed a Monitor(), unbreak tests 2008-10-22 01:50:54 -07:00
Brian Warner
977c6ac510 more #514: pass a Monitor to all checker operations, make mutable-checker honor the cancel flag 2008-10-22 01:38:18 -07:00
Brian Warner
8178b10ef1 dirnode.py: check for cancel during deep-traverse operations, and don't initiate any new ones if we've been cancelled. Gets us closer to #514. 2008-10-22 00:55:52 -07:00
Brian Warner
34ab4e3de3 more #514 log-webop status/cancel: add handle-expiration, test coverage 2008-10-21 22:13:54 -07:00
Brian Warner
0f7c1fd6fd #514: improve test coverage 2008-10-21 17:52:56 -07:00
Brian Warner
ad3d9207a9 Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514. 2008-10-21 17:03:07 -07:00
Zooko O'Whielacronx
8a6d1e5da6 repairer: test all different kinds of corruption that can happen to share files on disk 2008-10-14 16:09:20 -07:00
Brian Warner
b0a00e0f33 util/time_format.py: accept space separator, add unit tests 2008-10-13 15:52:58 -07:00
Brian Warner
46657b797f test_storage: use different filenames, poor stupid windows 2008-10-09 19:11:39 -07:00
Brian Warner
7031a69bee storage: introduce v2 immutable shares, with 8-byte offsets fields, to remove two of the three size limitations in #346. This code handles v2 shares but does not generate them. We'll make a release with this v2-tolerance, wait a while, then make a second release that actually generates v2 shares, to avoid compatibility problems. 2008-10-09 18:13:27 -07:00
Brian Warner
288d55825c storage: split WriteBucketProxy and ReadBucketProxy out into immutable/layout.py . No behavioral changes. 2008-10-09 17:08:00 -07:00
Brian Warner
2edfe12b88 test_web: deep-size is more variable than I thought, so assert less 2008-10-06 22:11:47 -07:00
Brian Warner
3ffaded809 web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json. 2008-10-06 21:36:18 -07:00
Brian Warner
8e6d122ecf web: rewrite t=deep-size in terms of deep-stats, update test to match inclusion of directory sizes 2008-10-06 21:35:39 -07:00
Brian Warner
d90a3ed7f8 test_system: add test coverage for immutable download.ConsumerAdapter, remove debug messages 2008-10-06 15:50:37 -07:00
Brian Warner
934dc6aa77 test_cli.py: remove unused imports 2008-10-06 17:42:04 -07:00
Brian Warner
dbc2337639 CLI: remove 'tahoe admin generate-keypair', since the pycryptopp ecdsa API is about to change incompatibly. We'll undo this once pycryptopp is updated 2008-10-06 17:23:20 -07:00
Brian Warner
09341a969a dirnode: fix my remarkably-consistent 'metdadata' typo 2008-10-02 18:08:45 -07:00
Brian Warner
d0bdf9a611 dirnode: add get_child_and_metadata_at_path 2008-10-02 17:52:03 -07:00
Brian Warner
9c505e49c2 stop using 'as' as an identifier: as with 'with', 'as' has become a reserved word in python 2.6 2008-10-02 17:27:49 -07:00
Brian Warner
935d185696 tolerate simplejson-2.0.0 and newer, which frequently return bytestrings instead of unicode objects. Closes #523 2008-09-30 15:21:06 -07:00
Brian Warner
78d58875a8 test_node: improve coverage of advertised_ip_addresses a bit 2008-09-29 23:08:16 -07:00
Zooko O'Whielacronx
2e8b89e9e4 repair: fix test to map from storage index to directory structure properly (thanks, cygwin buildbot, for being so kloodgey that you won't accept random binary filenames and thus making me notice this bug) 2008-09-26 15:49:13 -07:00
Zooko O'Whielacronx
cd0af638f3 repairer: assert that the test code isn't accidentally allowing the repairer code which is being tested to do impossible things 2008-09-26 15:23:53 -07:00
Zooko O'Whielacronx
ebee8ee9f4 repairer: enhance the repairer tests 
Make sure the file can actually be downloaded afterward, that it used one of the 
deleted and then repaired shares to do so, and that it repairs from multiple 
deletions at once (without using more than a reasonable amount of calls to 
storage server allocate).
 2008-09-26 10:47:19 -07:00
Brian Warner
98c8e25709 netstring: add required_trailer= argument 2008-09-26 09:57:54 -07:00
Brian Warner
27a30b90d7 test_netstring.py: move netstring tests to a separate file 2008-09-26 09:55:26 -07:00
Zooko O'Whielacronx
39fa9018ef repairer: remove a test that doesn't apply to the repair-from-corruption case 2008-09-25 15:09:54 -07:00
Zooko O'Whielacronx
73f92ea571 repairer: add a test that repairer fixes corrupted shares (in addition to the test that it fixes deleted shares) 2008-09-25 15:07:12 -07:00
Zooko O'Whielacronx
a3b3d2b7bc repairer: fix swapped docstrings; thanks Brian 2008-09-25 11:24:36 -07:00
Zooko O'Whielacronx
30111811e3 trivial: remove unused imports; thanks, pyflakes 2008-09-25 11:04:22 -07:00
Zooko O'Whielacronx
a363994c4c trivial: remove unused imports -- thanks, pyflakes 2008-09-25 10:34:53 -07:00
Zooko O'Whielacronx
1e8d37cc2d repairer: add basic test of repairer, move tests of immutable checker/repairer from test_system to test_immutable_checker, remove obsolete test helper code from test_filenode 
Hm...  "Checker" ought to be renamed to "CheckerRepairer" or "Repairer" at some point...
 2008-09-25 10:16:53 -07:00
Zooko O'Whielacronx
39f305e44f setup: remove a few minimal unit tests from test_filenode which have been obviated by much better tests in test_mutable and test_system 2008-09-25 09:15:44 -07:00
robk-tahoe
ed9873c2a9 CLI: rework webopen, and moreover its tests w.r.t. path handling 
in the recent reconciliation of webopen patches, I wound up adjusting
webopen to 'pass through' the state of the trailing slash on the given
argument to the resultant url passed to the browser.  this change 
removes the requirement that arguments must be directories, and allows
webopen to be used with files.  it also broke the tests that assumed
that webopen would always normalise the url to have a trailing slash.

in fixing the tests, I realised that, IMHO, there's something deeply
awry with the way tahoe handles paths; specifically in the combination
of '/' being the name of the root path within an alias, but a leading
slash on paths, e.g. 'alias:/path', is catagorically incorrect. i.e.
 'tahoe:' == 'tahoe:/' == '/' 
but 'tahoe:/foo' is an invalid path, and must be 'tahoe:foo'

I wound up making the internals of webopen simply spot a 'path' of
'/' and smash it to '', which 'fixes' webopen to match the behaviour
of tahoe's path handling elsewhere, but that special case sort of
points to the weirdness.

(fwiw, I personally found the fact that the leading / in a path was
disallowed to be weird - I'm just used to seeing paths qualified by
the leading / I guess - so in a debate about normalising path handling
I'd vote to include the /)
 2008-09-24 09:45:23 -07:00
Brian Warner
54fce5c066 webapi: survive slashes in filenames better: make t=info and t=delete to work, and let t=rename fix the problem 2008-09-24 13:35:05 -07:00
Brian Warner
dd9171eb72 node.py: add BASEDIR/keepalive_timeout and BASEDIR/disconnect_timeout, to set/enable the foolscap timers, for #521 2008-09-24 10:51:12 -07:00
Brian Warner
0799e09910 test_cli: disable generate-keypair test on OS-X, pycryptopp still has a bug 2008-09-19 12:38:55 -07:00
Brian Warner
6607fdc586 CLI: add 'tahoe admin generate-keypair' command 2008-09-18 17:11:33 -07:00
Brian Warner
99d5a8d8b9 web: add 'more info' pages for files and directories, move URI/checker-buttons/deep-size/etc off to them 2008-09-17 22:00:41 -07:00
Brian Warner
f570ad7ba5 disallow deep-check on non-directories, simplifies the code a bit 2008-09-10 13:44:58 -07:00
Brian Warner
4bb88fd2ee dirnode: refactor recursive-traversal methods, add stats to deep_check() method results and t=deep-check webapi 2008-09-10 01:45:04 -07:00
Brian Warner
f6eeb3161f dirnode: cleanup, make get_verifier() always return a URI instance, not a string 2008-09-10 01:37:55 -07:00
Brian Warner
28a9f8f076 test_system: check t=deep-stats too 2008-09-09 23:54:57 -07:00
Brian Warner
12ff8a6eae test_system: add deep-check-JSON tests, fix a bug 2008-09-09 23:14:16 -07:00
Brian Warner
ce7fcbde36 test_system: oops, re-enable some tests that got bypassed 2008-09-09 23:02:45 -07:00
Brian Warner
ba336aed3e test_system: add deep-stats test 2008-09-09 22:56:34 -07:00
Brian Warner
80d8f3e862 hush pyflakes 2008-09-09 19:50:17 -07:00
Brian Warner
1d2d6a35a6 checker results: add output=JSON to webapi, add tests, clean up APIs 
to make the internal ones use binary strings (nodeid, storage index) and
the web/JSON ones use base32-encoded strings. The immutable verifier is
still incomplete (it returns imaginary healty results).
 2008-09-09 19:45:17 -07:00
Brian Warner
04513e3ac5 immutable verifier: provide some dummy results so deep-check works, make the tests ignore these results until we finish it off 2008-09-09 18:08:27 -07:00
Brian Warner
7fb3308498 mutable checker: even more tests. Everything in ICheckerResults should be covered now, except for immutable-verify which is incomplete 2008-09-09 17:57:06 -07:00
Brian Warner
84a5778507 checker results: more tests, update interface docs 2008-09-09 17:30:10 -07:00
Brian Warner
f895e39d48 checker results: more tests, more results. immutable verifier tests are disabled until they emit more complete results 2008-09-09 17:15:46 -07:00
Brian Warner
90b934eb71 checker: add tests, add stub for immutable check_and_repair 2008-09-09 16:34:49 -07:00
Brian Warner
918b0543b9 test_system: make log() tolerate the format= form 2008-09-07 20:03:36 -07:00
Brian Warner
3408d552cd checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays 2008-09-07 12:44:56 -07:00
Brian Warner
e2c484a761 introducer: add get_nickname_for_peerid 2008-09-05 22:07:00 -07:00
Brian Warner
1668401c16 mutable: make mutable-repair work for non-verifier runs, add tests 2008-08-26 16:34:54 -07:00
Brian Warner
bcbb51fe4a test_web: workaround broken HEAD behavior in twisted-2.5.0 and earlier 2008-08-12 19:45:20 -07:00
Brian Warner
6af4619928 test_web: oops, actually use HEAD (instead of GET) in the HEAD test 2008-08-12 19:04:51 -07:00
Brian Warner
d43baa2ad7 mutable: add get_size_of_best_version to the interface, to simplify the web HEAD code, and tests 2008-08-12 19:02:52 -07:00
Brian Warner
014c9b5969 CLI: add 'tahoe debug corrupt-share', and use it for deep-verify tests, and fix non-deep web checker API to pass verify=true into node 2008-08-12 17:05:01 -07:00
Brian Warner
c80e352951 IFilesystemNode: add get_storage_index(), it makes tests easier 2008-08-12 16:14:07 -07:00
Brian Warner
5c0bfd7681 test_system: rename Checker to ImmutableChecker, to make room for a mutable one 2008-08-12 15:59:32 -07:00
Brian Warner
67e7333955 'tahoe debug dump-share': add --offsets, to show section offsets 2008-08-12 14:46:56 -07:00
Brian Warner
022fcf3db8 test_cli: oops, fix tests after recent stdout/stderr cleanup 2008-08-12 14:46:34 -07:00
Brian Warner
bb33e3e4c2 CLI: move all debug commands (dump-share, dump-cap, find-shares, catalog-shares) into a 'debug' subcommand, and improve --help output 2008-08-12 13:37:32 -07:00
Brian Warner
d106e411af checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted 2008-08-11 21:03:26 -07:00
Brian Warner
376df2eb5a CLI: make 'tahoe webopen' command accept aliases like 'tahoe ls' 2008-08-11 18:20:23 -07:00
Brian Warner
c7c57bd85c mutable: more repair tests, one with force=True to check out merging 2008-08-06 12:06:07 -07:00
Brian Warner
a68c9364b8 test/common: add ShouldFailMixin 2008-08-06 12:05:52 -07:00
Brian Warner
572115a405 test_mutable: add comment about minimal-bandwidth repairer, comma lack of 2008-08-06 10:38:50 -07:00
Brian Warner
cdbfccd23e test_mutable: factor out common setup code 2008-08-06 10:38:04 -07:00
Brian Warner
dd6ec73efa mutable: start adding Repair tests, fix a simple bug 2008-08-05 23:12:39 -07:00
Brian Warner
3ad0168fbf test_system: factor out find_shares/replace_shares to a common class, so they can be used by other tests 2008-08-05 18:49:58 -07:00
Brian Warner
45d12baba2 test_cli: more coverage for 'tahoe put' modifying a mutable file in-place, by filename, closes #441 2008-08-04 13:26:43 -07:00
Brian Warner
4fb03c2510 check_grid.py: update to match new CLI: 'put - TARGET' instead of 'put TARGET' 2008-08-01 19:48:56 -07:00
Brian Warner
b9f4006b96 test_cli: remove windows-worrying newlines from test data 2008-08-01 19:47:34 -07:00
Brian Warner
35afe726c1 test_cli.py: factor out CLITestMixin 2008-08-01 19:29:38 -07:00
Brian Warner
f7d2fcc233 CLI: change one-arg forms of 'tahoe put' to make an unlinked file, fix replace-mutable #441 2008-08-01 19:27:29 -07:00
Brian Warner
2eec38279d CLI: add create-alias command, to merge mkdir and add-alias into a single (secure-from-argv-snooping) step 2008-08-01 19:10:41 -07:00
Brian Warner
dc57ab4d36 test_cli: add system-based tests for PUT, including a mutable put that fails/todo (#441) 2008-08-01 15:10:09 -07:00
Brian Warner
071cbec865 tests: simplify CLI tests that use stdin, now that runner supports it 2008-08-01 15:05:14 -07:00
Brian Warner
f181a0458a CLI: simplify argument-passing, use options= for everthing, including stdout 2008-08-01 11:46:24 -07:00
Zooko O'Whielacronx
6d3ec390b0 tests: add test that verifier notices any (randomly chosen) bit flipped in the verifiable part of any (randomly chosen) share 
The currently verifier doesn't (usually) pass this randomized test, hence the TODO.
 2008-07-30 17:20:15 -07:00
Zooko O'Whielacronx
86160c3fc9 tests: test that checker doesn't cause reads on the storage servers 
It would still pass the test if it noticed a corrupted share.  (It won't
notice, of course.)  But it is required to do its work without causing storage
servers to read blocks from the filesystem.
 2008-07-30 16:54:20 -07:00
Zooko O'Whielacronx
dbd1c2397e tests: use the handy dandy TestCase.mktemp() function from trial to give unique and nicely named directories for each testcase 2008-07-30 15:49:20 -07:00
Zooko O'Whielacronx
6aabaa0797 tests: don't use SignalMixin 
It seems like we no longer need it, and it screws up something internal in
trial which causes trial's TestCase.mktemp() method to exhibit wrong behavior
(always using a certain test method name instead of using the current test
method name), and I wish to use TestCase.mktemp().

Of course, it is possible that the buildbot is about to tell me that we do
still require SignalMixin on some of our platforms...
 2008-07-30 15:35:36 -07:00
Zooko O'Whielacronx
56282613c5 tests: add test_system.Checker which tests basic checking (without verification) functionality 2008-07-28 16:43:17 -07:00
Zooko O'Whielacronx
fe522e31a4 tests: make it so that you can use common.py's SystemTestMixin.set_up_nodes() more than once with the same introducer 2008-07-28 16:40:29 -07:00
Brian Warner
9e22353773 test/common.py: use pre-computed Tub certificates for the system-test mixin, to speed such tests up by maybe 15%. The goal is to encourage more full-grid tests. 2008-07-28 12:44:21 -07:00
Brian Warner
914783acdf web: add /status/?t=json, with active upload/download ops. Addresses #493. 2008-07-25 17:41:10 -07:00
Brian Warner
48c64b7a29 test_system.py: factor SystemTestMixin out of SystemTest 2008-07-25 15:33:49 -07:00
Brian Warner
10bbc740e9 test_system.py: modify system-test setup code in preparation for merge with common.SystemTestMixin 2008-07-25 15:29:31 -07:00
Brian Warner
766cdd9c52 test_system.py: move SystemTestMixin out into common.py, where further improvements will occur 2008-07-25 15:17:58 -07:00
Brian Warner
e317fcc08c test_system.py: create SystemTestMixin, with less cruft, for faster system-like tests 2008-07-25 15:13:00 -07:00
Brian Warner
afda2a43e4 storage: remove update_write_enabler method, it won't serve the desired purpose, and I have a better scheme in mind. See #489 for details 2008-07-21 17:28:28 -07:00
Zooko O'Whielacronx
9461887e0a immutable file download: make the ciphertext hash tree mandatory 
This fixes #491 (URIs do not refer to unique files in Allmydata Tahoe).
Fortunately all of the versions of Tahoe currently in use are already producing
this ciphertext hash tree when uploading, so there is no
backwards-compatibility problem with having the downloader require it to be
present.
 2008-07-21 09:31:02 -07:00
Brian Warner
edec917060 test_web: test that save=true filename=unicode doesn't crash 2008-07-18 18:58:57 -07:00
Brian Warner
879fefe5f3 first pass at a mutable repairer. not tested at all yet, but of course all existing tests pass 2008-07-17 21:09:23 -07:00
Brian Warner
923c9c242a oops, fix import/pyflakes problems 2008-07-17 17:06:20 -07:00
Brian Warner
67db0a4967 deep-check: add webapi, add 'DEEP-CHECK' button to wui, add tests, rearrange checker API a bit 2008-07-17 16:47:09 -07:00
Brian Warner
69156aeb28 dirnode deep-check: add tests of cycles, fix failures 2008-07-17 14:37:04 -07:00
Brian Warner
9289433ba3 first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests 2008-07-16 18:20:57 -07:00
Brian Warner
3e9322bcb6 checker: re-enable checker web results (although they just say 'Healthy' right now) 2008-07-16 15:42:56 -07:00
Brian Warner
7394607141 move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected. 2008-07-16 13:14:39 -07:00
Brian Warner
94e619c1f6 overhaul checker invocation 
Removed the Checker service, removed checker results storage (both in-memory
and the tiny stub of sqlite-based storage). Added ICheckable, all
check/verify is now done by calling the check() method on filenodes and
dirnodes (immutable files, literal files, mutable files, and directory
instances).

Checker results are returned in a Results instance, with an html() method for
display. Checker results have been temporarily removed from the wui directory
listing until we make some other fixes.

Also fixed client.create_node_from_uri() to create LiteralFileNodes properly,
since they have different checking behavior. Previously we were creating full
FileNodes with LIT uris inside, which were downloadable but not checkable.
 2008-07-15 17:23:25 -07:00
Brian Warner
fd465b4aaf download: fix stopProducing failure ('self._paused_at not defined'), add tests 2008-07-14 15:25:21 -07:00
Brian Warner
c46f5bc634 storage: rename the latency key names so they sort properly 2008-07-11 21:51:02 -07:00
Brian Warner
60725ed065 storage: add add_lease/update_write_enabler to remote API, revamp lease handling 2008-07-09 18:06:55 -07:00