ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2025-01-05 20:54:09 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
4,161 Commits 185 Branches 90 Tags 89 MiB
7143707178
Commit Graph

289 Commits

Author SHA1 Message Date
Brian Warner
5c3e153d0e cli: tahoe stats/manifest: change --verbose to --raw, since I want -v for --verify for check/deep-check/repair 2008-11-18 18:36:08 -07:00
Brian Warner
d657d22c0c test_system: make 'where' strings more helpful, to track down test failures better 2008-11-18 18:29:50 -07:00
Brian Warner
7932fadb5e webapi: add 'summary' string to checker results JSON 2008-11-18 18:28:26 -07:00
Brian Warner
994d97c644 webapi: introducer stats: add 'announcement_distinct_hosts' to the t=json form, to show how many distinct hosts are providing e.g. storage services 2008-11-18 15:30:15 -07:00
Brian Warner
ead0e4d6ca cli: add tests for 'tahoe stats --verbose' 2008-11-17 22:11:14 -07:00
Brian Warner
d6a67cd566 dirnode manifest/stats: process more than one LIT file per tree; we were accidentally ignoring all but the first 2008-11-14 22:50:49 -07:00
Brian Warner
5a60086dbc CLI: add 'tahoe stats', to run start-deep-stats and print the results 2008-11-13 19:43:50 -07:00
Brian Warner
9c9994300e test_system.py: fix new 'tahoe manifest' tests to not break on windows, by providing --node-directory instead of --node-url 2008-11-13 15:27:48 -07:00
Brian Warner
0d93d6244e CLI: add 'tahoe manifest', which takes a directory and returns a list of things you can reach from it 2008-11-12 20:17:25 -07:00
Brian Warner
dfa2408157 checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests 2008-11-06 22:35:47 -07:00
Brian Warner
6fa41e738b immutable: tolerate filenode.read() with a size= that's too big, rather than hanging 2008-11-04 15:29:19 -07:00
Brian Warner
b1ca238176 #527: respond to GETs with early ranges quickly, without waiting for the whole file to download. Fixes the alacrity problems with the earlier code. Still needs cache expiration. 2008-10-28 17:56:18 -07:00
Brian Warner
37e3d8e47c #527: support HTTP 'Range:' requests, using a cachefile. Adds filenode.read(consumer, offset, size) method. Still needs: cache expiration, reduced alacrity. 2008-10-28 13:41:04 -07:00
Brian Warner
914655c52b interfaces.py: promote immutable.encode.NotEnoughSharesError.. it isn't just for immutable files any more 2008-10-27 13:34:49 -07:00
Brian Warner
fca158e83a dirnode lookup: use distinct NoSuchChildError instead of the generic KeyError when a child can't be found 2008-10-27 13:15:25 -07:00
Brian Warner
d4b4cd8ab8 test_system: update test to match web checker results 2008-10-23 16:32:02 -07:00
Brian Warner
977c6ac510 more #514: pass a Monitor to all checker operations, make mutable-checker honor the cancel flag 2008-10-22 01:38:18 -07:00
Brian Warner
8178b10ef1 dirnode.py: check for cancel during deep-traverse operations, and don't initiate any new ones if we've been cancelled. Gets us closer to #514. 2008-10-22 00:55:52 -07:00
Brian Warner
ad3d9207a9 Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514. 2008-10-21 17:03:07 -07:00
Brian Warner
3ffaded809 web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json. 2008-10-06 21:36:18 -07:00
Brian Warner
d90a3ed7f8 test_system: add test coverage for immutable download.ConsumerAdapter, remove debug messages 2008-10-06 15:50:37 -07:00
Zooko O'Whielacronx
a363994c4c trivial: remove unused imports -- thanks, pyflakes 2008-09-25 10:34:53 -07:00
Zooko O'Whielacronx
1e8d37cc2d repairer: add basic test of repairer, move tests of immutable checker/repairer from test_system to test_immutable_checker, remove obsolete test helper code from test_filenode 
Hm...  "Checker" ought to be renamed to "CheckerRepairer" or "Repairer" at some point...
 2008-09-25 10:16:53 -07:00
Brian Warner
99d5a8d8b9 web: add 'more info' pages for files and directories, move URI/checker-buttons/deep-size/etc off to them 2008-09-17 22:00:41 -07:00
Brian Warner
f570ad7ba5 disallow deep-check on non-directories, simplifies the code a bit 2008-09-10 13:44:58 -07:00
Brian Warner
4bb88fd2ee dirnode: refactor recursive-traversal methods, add stats to deep_check() method results and t=deep-check webapi 2008-09-10 01:45:04 -07:00
Brian Warner
28a9f8f076 test_system: check t=deep-stats too 2008-09-09 23:54:57 -07:00
Brian Warner
12ff8a6eae test_system: add deep-check-JSON tests, fix a bug 2008-09-09 23:14:16 -07:00
Brian Warner
ce7fcbde36 test_system: oops, re-enable some tests that got bypassed 2008-09-09 23:02:45 -07:00
Brian Warner
ba336aed3e test_system: add deep-stats test 2008-09-09 22:56:34 -07:00
Brian Warner
80d8f3e862 hush pyflakes 2008-09-09 19:50:17 -07:00
Brian Warner
1d2d6a35a6 checker results: add output=JSON to webapi, add tests, clean up APIs 
to make the internal ones use binary strings (nodeid, storage index) and
the web/JSON ones use base32-encoded strings. The immutable verifier is
still incomplete (it returns imaginary healty results).
 2008-09-09 19:45:17 -07:00
Brian Warner
04513e3ac5 immutable verifier: provide some dummy results so deep-check works, make the tests ignore these results until we finish it off 2008-09-09 18:08:27 -07:00
Brian Warner
7fb3308498 mutable checker: even more tests. Everything in ICheckerResults should be covered now, except for immutable-verify which is incomplete 2008-09-09 17:57:06 -07:00
Brian Warner
84a5778507 checker results: more tests, update interface docs 2008-09-09 17:30:10 -07:00
Brian Warner
f895e39d48 checker results: more tests, more results. immutable verifier tests are disabled until they emit more complete results 2008-09-09 17:15:46 -07:00
Brian Warner
90b934eb71 checker: add tests, add stub for immutable check_and_repair 2008-09-09 16:34:49 -07:00
Brian Warner
918b0543b9 test_system: make log() tolerate the format= form 2008-09-07 20:03:36 -07:00
Brian Warner
3408d552cd checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays 2008-09-07 12:44:56 -07:00
Brian Warner
1668401c16 mutable: make mutable-repair work for non-verifier runs, add tests 2008-08-26 16:34:54 -07:00
Brian Warner
014c9b5969 CLI: add 'tahoe debug corrupt-share', and use it for deep-verify tests, and fix non-deep web checker API to pass verify=true into node 2008-08-12 17:05:01 -07:00
Brian Warner
5c0bfd7681 test_system: rename Checker to ImmutableChecker, to make room for a mutable one 2008-08-12 15:59:32 -07:00
Brian Warner
67e7333955 'tahoe debug dump-share': add --offsets, to show section offsets 2008-08-12 14:46:56 -07:00
Brian Warner
bb33e3e4c2 CLI: move all debug commands (dump-share, dump-cap, find-shares, catalog-shares) into a 'debug' subcommand, and improve --help output 2008-08-12 13:37:32 -07:00
Brian Warner
3ad0168fbf test_system: factor out find_shares/replace_shares to a common class, so they can be used by other tests 2008-08-05 18:49:58 -07:00
Brian Warner
f7d2fcc233 CLI: change one-arg forms of 'tahoe put' to make an unlinked file, fix replace-mutable #441 2008-08-01 19:27:29 -07:00
Brian Warner
071cbec865 tests: simplify CLI tests that use stdin, now that runner supports it 2008-08-01 15:05:14 -07:00
Brian Warner
f181a0458a CLI: simplify argument-passing, use options= for everthing, including stdout 2008-08-01 11:46:24 -07:00
Zooko O'Whielacronx
6d3ec390b0 tests: add test that verifier notices any (randomly chosen) bit flipped in the verifiable part of any (randomly chosen) share 
The currently verifier doesn't (usually) pass this randomized test, hence the TODO.
 2008-07-30 17:20:15 -07:00
Zooko O'Whielacronx
86160c3fc9 tests: test that checker doesn't cause reads on the storage servers 
It would still pass the test if it noticed a corrupted share.  (It won't
notice, of course.)  But it is required to do its work without causing storage
servers to read blocks from the filesystem.
 2008-07-30 16:54:20 -07:00
Zooko O'Whielacronx
dbd1c2397e tests: use the handy dandy TestCase.mktemp() function from trial to give unique and nicely named directories for each testcase 2008-07-30 15:49:20 -07:00
Zooko O'Whielacronx
56282613c5 tests: add test_system.Checker which tests basic checking (without verification) functionality 2008-07-28 16:43:17 -07:00
Brian Warner
48c64b7a29 test_system.py: factor SystemTestMixin out of SystemTest 2008-07-25 15:33:49 -07:00
Brian Warner
10bbc740e9 test_system.py: modify system-test setup code in preparation for merge with common.SystemTestMixin 2008-07-25 15:29:31 -07:00
Brian Warner
766cdd9c52 test_system.py: move SystemTestMixin out into common.py, where further improvements will occur 2008-07-25 15:17:58 -07:00
Brian Warner
e317fcc08c test_system.py: create SystemTestMixin, with less cruft, for faster system-like tests 2008-07-25 15:13:00 -07:00
Brian Warner
7394607141 move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected. 2008-07-16 13:14:39 -07:00
Brian Warner
94e619c1f6 overhaul checker invocation 
Removed the Checker service, removed checker results storage (both in-memory
and the tiny stub of sqlite-based storage). Added ICheckable, all
check/verify is now done by calling the check() method on filenodes and
dirnodes (immutable files, literal files, mutable files, and directory
instances).

Checker results are returned in a Results instance, with an html() method for
display. Checker results have been temporarily removed from the wui directory
listing until we make some other fixes.

Also fixed client.create_node_from_uri() to create LiteralFileNodes properly,
since they have different checking behavior. Previously we were creating full
FileNodes with LIT uris inside, which were downloadable but not checkable.
 2008-07-15 17:23:25 -07:00
Brian Warner
9b6f23ac05 'tahoe dump-share': show verify-cap too 2008-07-07 14:11:02 -07:00
Brian Warner
28f4652b96 break introducer up into separate modules in the new allmydata.introducer package 2008-06-18 12:24:16 -07:00
Brian Warner
b6c2f80170 test_system: even more 'cp -r' coverage 2008-05-21 18:40:49 -07:00
Brian Warner
4824d058cc test_system.py: improve 'cp -r' coverage: exercise copy from tahoe to local disk 2008-05-21 18:36:25 -07:00
Brian Warner
0f8f213baf CLI: remove the '-r' shortcut for --dir-cap, to make it available for cp -r 2008-05-21 17:55:32 -07:00
Brian Warner
457248f023 tahoe_cp: rewrite, make --recursive work 2008-05-21 17:35:21 -07:00
Brian Warner
15fbe05092 tahoe_ls: improve error message when the target is missing 2008-05-21 17:34:52 -07:00
Brian Warner
7afd869854 cli: initial implementation of 'cp -r', probably doesn't work yet 2008-05-21 11:49:22 -07:00
Brian Warner
0df663b7ae CLI: implement the easy part of cp (no -r, only two arguments) 2008-05-20 16:56:03 -07:00
Brian Warner
1d8e452184 CLI: add 'list-aliases', factor out get_aliases 2008-05-20 14:36:04 -07:00
Brian Warner
1236bc3408 CLI: add 'ln', just like move but without the delete 2008-05-20 13:30:31 -07:00
Brian Warner
f09f92c371 test_system.py: remove dead code 2008-05-20 13:20:48 -07:00
Brian Warner
a8ed3208ba CLI ls: add --readonly-uri to display readonly URIs for all children 2008-05-20 12:49:11 -07:00
Brian Warner
e889463f0c CLI: add put --mutable, enhance ls to show mutable vs immutable as rw/r- 2008-05-20 12:36:55 -07:00
Brian Warner
88457fbc32 test_web/test_system: improve test coverage 2008-05-19 23:28:52 -07:00
Brian Warner
506ce2dfd0 test_system: fix windows test failure, open files with binary mode 2008-05-19 20:24:12 -07:00
Brian Warner
8e92dfcb50 overhaul CLI: not quite complete but it works a lot better than it used to. The new scheme uses 'tahoe add-alias' and rsync/scp-style 'alias:foo/bar.txt' arguments 2008-05-19 19:28:50 -07:00
Brian Warner
b1d2922ea3 test_system: match webapi change, making new files now returns 201 Created, not 200 OK 2008-05-19 14:09:31 -07:00
Brian Warner
26187bfc81 use a weakref cache in the client to manage singleton filenodes/dirnodes, to avoid autocollision. Should close #391. 2008-05-08 18:02:55 -07:00
Brian Warner
fabdc28c06 deep-stats: add file-size histogram 2008-05-08 16:19:42 -07:00
Brian Warner
acdd98432d hush pyflakes warning about code that got moved in the recent StallMixin refactoring 2008-04-22 17:14:26 -07:00
Brian Warner
81ab1ec785 testutil: factor stall() out into a common location 2008-04-22 16:47:15 -07:00
Brian Warner
a379690b04 mutable: replace MutableFileNode API, update tests. Changed all callers to use overwrite(), but that will change soon 2008-04-17 17:51:38 -07:00
Brian Warner
e1838ba217 mutable WIP: clean up status handling, shrink the code a lot, improve test coverage 2008-04-17 13:02:22 -07:00
Brian Warner
a1670497a8 mutable WIP: add servermap update status pages 2008-04-16 19:05:41 -07:00
Brian Warner
1b4b4cbd4a mutable WIP: rename NotEnoughPeersError to NotEnoughSharesError 2008-04-15 16:08:32 -07:00
Brian Warner
cf84bae850 mutable WIP: merge conflicts in test_system.py 2008-04-15 15:57:28 -07:00
Brian Warner
d4230d1781 mutable WIP: split mutable.py into separate files. All tests pass. 2008-04-11 14:31:16 -07:00
Brian Warner
0515b1099c mutable WIP: all tests pass, but publish/retrieve status is still stubbed out 2008-04-10 19:26:35 -07:00
Brian Warner
6de2a0794f web: don't break status page when there is no helper running 2008-04-14 23:42:20 -07:00
Brian Warner
86578bc3cd webish: fix 'not running helper' status indicator on welcome page, add tests 2008-04-14 16:28:11 -07:00
Brian Warner
5b8320442a stats: add /statistics web page to show them, add tests 2008-04-14 14:17:08 -07:00
Brian Warner
500934b72f helper stats: fix the /helper_status page, the recent conflict merging missed some uses. Added tests, updated the munin plugins to match 2008-04-14 13:18:53 -07:00
robk-tahoe
1ae2c39862 key_generator: service related cleanups, incorporation into system test 
this cleans up KeyGenerator to be a service (a subservice of the
KeyGeneratorService as instantiated by the key-generator.tac app)
this means that the timer which replenishes the keypool will be
shutdown cleanly when the service is stopped.

adds checks on the key_generator service and client into the system
test 'test_mutable' such that one of the nodes (clients[3]) uses
the key_generator service, and checks that mutable file creation
in that node, via a variety of means, are all consuming keys from
the key_generator.
 2008-04-03 15:57:07 -07:00
Brian Warner
4531d1e953 introweb.py: add ?t=json, to provide machine-readable subscriber counts 2008-03-25 12:56:12 -07:00
Brian Warner
1e097766c9 disable plaintext hashes in shares, but leave a switch to turn it back on 2008-03-24 13:39:51 -07:00
Brian Warner
7b21054c33 UNDO: upload: stop putting plaintext and ciphertext hashes in shares. 
This removes the guess-partial-information attack vector, and reduces
the amount of overhead that we consume with each file. It also introduces
a forwards-compability break: older versions of the code (before the
previous download-time "make hashes optional" patch) will be unable
to read files uploaded by this version, as they will complain about the
missing hashes. This patch is experimental, and is being pushed into
trunk to obtain test coverage. We may undo it before releasing 1.0.
 2008-03-23 15:35:54 -07:00
Zooko O'Whielacronx
fc3bd0c987 use added secret to protect convergent encryption 
Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key.  If you want traditional convergent encryption behavior, set the added secret to be the empty string.

This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names.  (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.)

This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
 2008-03-24 09:46:06 -07:00
Brian Warner
7996131a0a upload: stop putting plaintext and ciphertext hashes in shares. 
This removes the guess-partial-information attack vector, and reduces
the amount of overhead that we consume with each file. It also introduces
a forwards-compability break: older versions of the code (before the
previous download-time "make hashes optional" patch) will be unable
to read files uploaded by this version, as they will complain about the
missing hashes. This patch is experimental, and is being pushed into
trunk to obtain test coverage. We may undo it before releasing 1.0.
 2008-03-23 15:35:54 -07:00
Brian Warner
2ef70ab814 mutable.py: split replace() into update() and overwrite(). Addresses #328. 2008-03-12 18:00:43 -07:00
Brian Warner
89be2e1bea introweb: combine announcement and subscriber information to show version+nickname for each client 2008-03-11 19:21:29 -07:00
Brian Warner
810ba68343 add a webserver for the Introducer, showing service announcements and subscriber lists 2008-03-11 17:36:25 -07:00