ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-28 08:48:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,396 Commits 185 Branches 90 Tags 89 MiB
ee72a5adfd
Commit Graph

669 Commits

Author SHA1 Message Date
Brian Warner
1340c484c6 download.py: use producer/consumer to reduce memory usage, closes #129. 
If the DownloadTarget is also an IConsumer, give it control of the brakes
by offering ourselves to target.registerProducer(). When they tell us to
pause, set a flag, which is checked between segment downloads and decodes.
webish.py: make WebDownloadTarget an IConsumer and pass control along to
the http.Request, which already knows how to be an IConsumer.
This reduces the memory footprint of stalled HTTP GETs to a bare minimum,
and thus closes #129.
 2007-09-19 00:34:47 -07:00
Brian Warner
9c9a793540 upload.py: fix signature of NonConvergentUploadMixin.get_encryption_key 2007-09-18 23:38:11 -07:00
Brian Warner
ee9b7fd678 check_memory: oops, silly bug make 'upload' push to ourselves, raising the usage 2007-09-19 04:14:48 -07:00
Brian Warner
08cc32bb77 check_memory: add download-GET-slow, to simulate memory usage of a node feeding downloaded data via HTTP GET to a slow client 2007-09-18 20:35:27 -07:00
Brian Warner
a5fab9935b test_runner: better diagnostics in case test_client fails 2007-09-18 20:03:18 -07:00
Brian Warner
2886c5aeae check_memory: don't accept shares for download/download-GET test, since that hits bug #97 2007-09-18 19:40:29 -07:00
Brian Warner
f6acf5a0c3 check_memory: add download, download-GET 2007-09-18 18:56:05 -07:00
Brian Warner
38946439f3 test_runner.py: fix race conditions in start/stop node, should run on cygwin now 2007-09-18 15:17:26 -07:00
Brian Warner
645927ca73 test_runner.py: skip the start/stop test when sys.platform is win32/cygwin. 
The previous twisted.python.runtime.platformType approach didn't catch cygwin
for some reason.
 2007-09-17 21:56:08 -07:00
Brian Warner
f72ef4d8f9 test_runner.py: add tests for startstop_node.py 2007-09-17 02:25:31 -07:00
Brian Warner
cf69accfb6 startstop_node.py: use SIGINT instead of SIGTERM, to allow the node a chance to clean up 2007-09-17 02:24:21 -07:00
Brian Warner
b7c18d6505 webish: improve test coverage 2007-09-17 01:53:46 -07:00
Brian Warner
6cd11037c6 debug: remove the 'dump-share-leases' command, it is subsumed by 'dump-share' 2007-09-17 01:44:39 -07:00
Brian Warner
79bd7d422d provisioning.py: get full test coverage 2007-09-17 01:38:54 -07:00
Brian Warner
311ed144f8 uri.py: improve test coverage a bit 2007-09-17 01:09:47 -07:00
Brian Warner
8451b485a4 storage: fill alreadygot= with all known shares for the given storageindex, not just the ones they asked about 2007-09-17 00:48:40 -07:00
Brian Warner
808f851589 upload: make peer-selection a bit more uniform. Closes #132. 2007-09-16 17:08:34 -07:00
Brian Warner
24e6ccddce peer-selection: if we must loop, send a minimal number of queries (by asking for more than one share per peer on the second pass) 2007-09-16 01:53:00 -07:00
Brian Warner
8a251d8670 upload: remove Tahoe3 peer-selection algorithm 2007-09-16 01:26:11 -07:00
Brian Warner
baa16087cd upload: switch to Tahoe2, add test for uniform share allocation 2007-09-16 01:25:03 -07:00
Brian Warner
979d12cd42 upload.py: implement Tahoe2 peer-selection algorithm 2007-09-16 01:24:07 -07:00
Brian Warner
f24d7450a7 vdrive.py: oops, forgot an import 2007-09-15 15:21:57 -07:00
Brian Warner
d37d74fa1d vdrive: log an error if we weren't able to use the vdrive the way we wanted to 2007-09-15 15:17:29 -07:00
Brian Warner
d628d5f503 storage: remove the leftover incoming/XYZ/ directory when we're done with it 2007-09-15 14:34:04 -07:00
Brian Warner
e9d32f14b9 check_memory.py: preserve client.log and stats.out in _test_memory/ 
Put the nodes in _test_memory/test/, which is clobbered on each test. Also
kill the client with SIGINT instead of SIGKILL. Also don't daemonize the 
client, since we're going to kill it at the end of the test anyways: this
cleans up shutdown a bit.
 2007-09-15 20:53:06 -07:00
Brian Warner
d5ed60e4ad webish: upload: when the localdir= doesn't exist, say so in the HTTP response 2007-09-15 12:49:07 -07:00
Brian Warner
c7986389e5 check_memory.py: have all clients write their logs to _test_memory/client.log instead of a separate file per client 2007-09-15 12:34:05 -07:00
Brian Warner
e7f61a3a6c check_memory: oops, only bail if the process ended abnormally 2007-09-15 11:47:29 -07:00
Brian Warner
32f25f33e3 check-memory: if the child process fails to start, shut down instead of hanging forever 2007-09-14 20:16:57 -07:00
Brian Warner
df49fa55e0 test_client.py: remove unused 'stat' import 2007-09-13 20:17:42 -07:00
Zooko O'Whielacronx
8fd98624a3 don't try to use bindann 
It causes a mysterious misbehavior in Python import which causes the previous patch to fail (the patch to not run trial tests if dependencies can't be imported)
 2007-09-13 19:14:46 -07:00
Zooko O'Whielacronx
f443f8adbc remove test in test_client -- we can't easily assert that files aren't readable by others, on Windows 2007-09-13 20:12:26 -07:00
Brian Warner
e1e037e9b5 storage: always record lease expiration times as integers 2007-09-11 14:53:31 -07:00
nejucomo
96f55be844 Add some passing unit tests for testutil.PollMixin. 2007-09-07 16:15:41 -07:00
Brian Warner
89a25f1987 provisioning: include the cost of replacing failed hard drives 2007-09-10 15:46:45 -07:00
Brian Warner
0497d62939 provisioning: add cost information 2007-09-10 15:29:33 -07:00
Zooko O'Whielacronx
09ff5af99c loosen constraint on share data size 2007-09-11 11:29:18 -07:00
Brian Warner
3eaa888a0e provisioning: oops, fix test_provisioning too 2007-09-07 15:01:17 -07:00
Brian Warner
0ae7d31d33 provisioning.py: update test with new encoding-parameters format 2007-09-07 14:45:43 -07:00
Brian Warner
2a48283e53 provisioning: require at least one drive per server 2007-09-06 23:08:21 -07:00
Brian Warner
5561d3afe4 provisioning: explain the 'survival' number better 2007-09-06 23:01:07 -07:00
Brian Warner
421277cafa provisioning: more repair/survivability data 2007-09-06 22:54:53 -07:00
Brian Warner
69d680d820 provisioning: add some drive failure and repair rate info 2007-09-06 18:47:41 -07:00
Brian Warner
849dbb4950 provisioning.py: add file/server availability numbers 2007-09-05 18:16:21 -07:00
Brian Warner
76be4a582c webish: implement 'PUT /uri?t=mkdir' (to create anonymous dirnodes) 2007-09-05 17:23:06 -07:00
Brian Warner
db3f11648c webish: implement 'PUT /uri' (to create anonymous files) 2007-09-05 17:12:27 -07:00
Brian Warner
77394da447 client.py: only import webish.py if 'webport' is set, to save 3MB of footprint 2007-09-04 16:33:06 -07:00
Brian Warner
277e720f7c storage: add version number to share data. Closes #90. 2007-09-04 09:00:24 -07:00
Brian Warner
fb65aadd82 storage: don't add a duplicate lease, renew the old one instead 2007-09-02 21:39:47 -07:00
Brian Warner
89c7f27572 storage: remove get_or_add_owner, since I don't know what we need yet 2007-09-02 15:03:40 -07:00
Brian Warner
94233b8813 storage: remove unused delete_bucket() method, lease-cancellation covers it 2007-09-02 15:00:29 -07:00
Brian Warner
85f3107b12 storage: handle simultanous uploads: add a lease for the pre-empted client 2007-09-02 14:57:49 -07:00
Brian Warner
dbcabc1142 debug: 'dump-uri-extension' command becomes 'dump-share', add 'dump-share-leases'. Both display leases. 2007-09-02 14:48:20 -07:00
Brian Warner
0fe1205789 storage: replace sqlite with in-share lease records 2007-09-02 14:47:15 -07:00
Brian Warner
c7288286ae test_uri.py: remove tiny whitespace 2007-08-28 23:40:03 -07:00
wilcoxjg
33bae16ed2 created DirnodeURI schema 2007-08-28 23:28:26 -07:00
Brian Warner
a605fe5cad storage: use sqlite from either python2.5's stdlib or the pysqlite2 package 2007-08-28 23:28:52 -07:00
Brian Warner
2a63fe8b01 deletion phase3: add a sqlite database to track renew/cancel-lease secrets, implement renew/cancel_lease (but nobody calls them yet). Also, move the shares from BASEDIR/storage/* down to BASEDIR/storage/shares/* 2007-08-27 23:41:40 -07:00
Brian Warner
850bc9da02 deletion phase2b: create renew/cancel secrets for real. 2007-08-27 19:30:26 -07:00
Brian Warner
1428079a84 node: change get_or_create_config to strip whitespace and accept a filemode= argument 2007-08-27 19:23:50 -07:00
Brian Warner
89ceb49309 node.py: change get_or_create_config() to accept a function 2007-08-27 19:07:12 -07:00
Brian Warner
56afda11d1 deletion phase2a: improve creation of renew/cancel secrets. Still fake though. 2007-08-27 19:00:18 -07:00
Brian Warner
77a7232867 node.py: refactor config-file getting and setting 2007-08-27 18:58:39 -07:00
Brian Warner
739ae1ccde deletion phase1: send renew/cancel-lease secrets, but my_secret is fake, and the StorageServer discards them 2007-08-27 17:28:51 -07:00
Brian Warner
ed525f7478 test_web: improve provisioning.py test coverage a bit by using a live web hit 2007-08-27 17:26:39 -07:00
Brian Warner
5946985c07 add a provisioning utility page which shows necessary storage space and transfer rates for grids of various sizes 2007-08-26 23:44:24 -07:00
Brian Warner
b620d0955b debug.py: add share-overhead size info to dump-uri-extension 2007-08-26 23:42:39 -07:00
Brian Warner
0a39430730 webish: strip leading/tailing whitespace from user-provided filenames, to avoid confusion 2007-08-25 12:05:24 -07:00
Brian Warner
44db5fd8a2 webish: strip leading/tailing whitespace from user-provided URIs 2007-08-25 12:05:06 -07:00
Zooko O'Whielacronx
427bf2cdc9 change name of the query "allow_local_access()?" to "local_access_is_allowed()" 
So as to avoid confusing it with the setter method  
"allow_local_access(allowed=True)".
 2007-08-22 10:32:00 -07:00
Brian Warner
ac358fc32f tahoe_put.py: hush pyflakes by removing unused 'sys' import 2007-08-17 16:29:50 -07:00
tahoe
f49194cb5e trying to introduce old style humanreadablied logs hopefully without breaking the existing ones 2007-08-11 14:52:37 -07:00
Brian Warner
664a0e6af4 crypto: fix compiler warnings in the .c files 2007-08-15 17:48:52 -07:00
Brian Warner
6c24848797 copy RSA from PyCrypto into the allmydata/ tree, we'll use it eventually 2007-08-15 17:36:39 -07:00
Brian Warner
6ac237b845 hush pyflakes warning in Crypto.Util.number 2007-08-14 14:17:12 -07:00
Brian Warner
b7703df6a5 remove PyCrypto, copy AES/SHA256/Util.number into the allmydata/ tree 2007-08-14 13:57:41 -07:00
Brian Warner
3e1c417f81 webish: modify JSON to match zooko's proposed API changes in #118 2007-08-23 13:00:39 -07:00
Zooko O'Whielacronx
0c22044f7e command-line: remove the --vdrive option (it is now hardcoded to "global") 2007-08-23 13:27:00 -07:00
Brian Warner
f3353ee5e4 web: replace FILE links with /uri -based ones, to prevent an XSS attack against the secret vdrive URI contained in the current URL 2007-08-22 17:35:01 -07:00
Brian Warner
930618fba2 web: oops, forgot to add start.html 2007-08-22 15:08:00 -07:00
Brian Warner
4de5767c98 web: remove /vdrive/private, replace with a start.html file that points at the /uri/PRIVATE_URI, to prevent XSRF attacks 2007-08-22 14:54:34 -07:00
Zooko O'Whielacronx
8717905e35 refactor reading of configuration files in client.py 
This makes it so that an optional file which is unreadable or is rm'ed 
at the wrong moment will be ignored instead of raising an exception.   
It also bums out a couple of unnecessary lines of code (the explicit  
".close()" call).
 2007-08-22 10:29:57 -07:00
Brian Warner
d5262ec7e1 client.py: add a newline to "myself.furl" contents 2007-08-17 23:28:22 -07:00
Zooko O'Whielacronx
9996836d23 command-line: fix ticket #111 by requiring input to be a local file and sending Content-Length header 2007-08-17 14:59:49 -07:00
Zooko O'Whielacronx
594912e334 command-line: fix a few bugs in the "execute this python file" way to execute rm 2007-08-17 14:17:31 -07:00
Brian Warner
3e2d1e28eb tahoe_put-web2ish.py: clean up imports to be pyflakes-compliant, add newline to output 2007-08-16 17:47:52 -07:00
Brian Warner
2014662b47 cli.py: fix typo in synopsis 2007-08-16 17:47:24 -07:00
Brian Warner
75056b6222 test_system: weaken the assertion about the 410 error message when downloading a bogus URI, since some systems emit error messages differently 2007-08-16 17:05:26 -07:00
Brian Warner
6b5e05cc67 test_system.py: verify that we can replace files in place 2007-08-16 17:03:50 -07:00
Brian Warner
fe06b3be8b dirnode: change the defined behavior of RIVirtualDriveServer.set to allow replace-in-place without raising an exception 2007-08-16 17:03:19 -07:00
Brian Warner
be0ab3b1f1 test_system.py: many (failing) web tests were accidentally bypassed, fix those. Add some PUT tests. 2007-08-16 16:49:40 -07:00
Zooko O'Whielacronx
a12ef1ae82 command-line: add "rm", and tidy-up variable names, and make it so "allmydata-tahoe spam" prints a usage message instead of returning silently 2007-08-17 13:23:16 -07:00
Zooko O'Whielacronx
e554962ac3 command-line: remove some redundant options checking 2007-08-17 13:06:43 -07:00
Zooko O'Whielacronx
bffd0c97f5 command-line: fix all three commands and all two ways to invoke them to require node-url and give a useful usage string if node-url is absent or of the wrong form 2007-08-17 12:54:47 -07:00
Zooko O'Whielacronx
e4f07a975f tahoe_put.py: don't treat "^HTTP 200 OK" appearing somewhere in the body as if it appeared in the header 2007-08-17 12:06:41 -07:00
Zooko O'Whielacronx
7e8c6eb50b tahoe_put.py: require node-url to be provided 2007-08-17 12:06:11 -07:00
Zooko O'Whielacronx
922663a7e2 cmdline: change "--server" to "--node-url" and make it have no default value 2007-08-16 16:53:27 -07:00
Zooko O'Whielacronx
1564987e6c put now exits after doing its work, and it prints a terse message if 200 or 201, and a full dump of response elsewise 2007-08-16 16:30:39 -07:00
Zooko O'Whielacronx
6ed3a23c62 incomplete version of tahoe-put.py 
It doesn't exit properly afterward, and it might not do the best things with non-success responses from the server.
(See tahoe-put-web2ish.py for an example of better response handling.)
 2007-08-16 16:01:01 -07:00
Zooko O'Whielacronx
9ad02dbe80 fix version class to preferred format and correct parsing 2007-08-16 15:38:01 -07:00