Brian Warner
e1f604f7cd
change our default HTTP port to 8123
2007-10-11 13:17:23 -07:00
Brian Warner
dc573554c8
rename bin/allmydata-tahoe to bin/tahoe. Closes #155 .
2007-10-11 03:38:24 -07:00
Brian Warner
ee72a5adfd
rename client.tac to tahoe-client.tac, so that 'ps ax|grep tahoe' works. Closes #156 .
2007-10-11 02:48:06 -07:00
Brian Warner
34a25b9b28
node.py: don't append 'Z' to the timestamp, since it's really localtime. We need deeper changes to make it be UTC
2007-10-11 02:24:17 -07:00
Brian Warner
6ed8664c7f
check_memory.py: don't use self.keepalive_file until it's been initialized
2007-10-11 02:20:20 -07:00
Brian Warner
a76898244b
create_node.py: allow config['webport'] to be missing, for check_memory
2007-10-11 02:19:59 -07:00
Brian Warner
1b406f3c32
node.py: set logging timestamp to '2007-10-11 02:11:14.000Z', per ticket #171 . No milliseconds yet, though
2007-10-11 02:13:05 -07:00
Brian Warner
0f000784ce
create_node: use a webport by default, on localhost:8011
2007-10-11 02:01:23 -07:00
Brian Warner
8ce93b5e27
test_cli.py: hush pyflakes with a dummy usage, until we get some real CLI tests
2007-10-11 01:55:29 -07:00
Brian Warner
7fd1964c6f
runner: make most commands use ~/.tahoe by default (create-client, start/stop/restart, all CLI tools, but *not* create-introducer
2007-10-11 01:54:23 -07:00
Brian Warner
a29ab33379
webish: write node.url, for the benefit of CLI tools
2007-10-11 01:38:04 -07:00
Brian Warner
a470947cc3
cli: use urllib.escape on all URIs
2007-10-11 01:34:44 -07:00
Brian Warner
5b28411b04
cli: add --node-directory and --root-uri to all commands
2007-10-11 00:30:36 -07:00
Brian Warner
ec8781e587
'tahoe restart': re-enable --force option, which was accidentally lost
2007-10-10 20:10:03 -07:00
Zooko O'Whielacronx
021b5d4f2a
more information SkipTest for XMLRPC
2007-10-04 11:07:46 -07:00
Brian Warner
8fa19fa273
check_speed: average multiple pings when measuring RTT
2007-09-26 18:16:15 -07:00
Brian Warner
51b59b1219
dump-share: clarify the label on the size of the original file
2007-09-26 15:00:59 -07:00
Brian Warner
3fcee101ee
check_speed: measure RTT, report per-file times as a multiple of RTT
2007-09-26 13:07:33 -07:00
Brian Warner
c301b41f50
control: add measure_peer_response_time(), to estimate RTT for the mesh
2007-09-26 12:21:15 -07:00
Brian Warner
c10da1e1ef
client.py: add the missing remote_get_nodeid() method claimed in interfaces.py
2007-09-26 12:20:48 -07:00
Brian Warner
0596b9fe19
test_system.py minor typo
2007-09-26 12:07:37 -07:00
Brian Warner
0cfed96796
test_system.py: add coverage for allmydata.control
2007-09-26 12:06:55 -07:00
Brian Warner
aafccf7008
test_client.py: add TODO to test the hotline-timeout file
2007-09-25 21:03:54 -07:00
Brian Warner
c14349f695
check_speed.py: minor comment
2007-09-25 20:07:03 -07:00
Brian Warner
e05311f69a
client.py: increase hotline timeout, the check_memory test is failing
2007-09-25 19:22:33 -07:00
Brian Warner
1691524044
check_speed.py: use more small-file tests to improve accuracy of per-file time
2007-09-25 18:57:36 -07:00
Brian Warner
50415cf9c4
check_memory: more keepalive-file debug messages
2007-09-25 18:47:48 -07:00
Brian Warner
8d7368f748
check_memory: put framework log in a stable place, add debug code to figure
...
out why the keepalive-file isn't doing its job
2007-09-25 18:26:54 -07:00
Brian Warner
9b09fd3bf1
test_system.py: do one upload, then test debug scripts, then do other uploads
2007-09-24 18:12:37 -07:00
Zooko O'Whielacronx
e3d782da52
cli: use port 8123 as the example port number
2007-09-24 13:17:27 -07:00
Zooko O'Whielacronx
93efd2902e
startstop_node.py: use SIGINT instead of SIGTERM to stop the node ( fixes #143 )
2007-09-23 07:06:54 -07:00
Zooko O'Whielacronx
a18f7aa6d5
fix test_vdrive ( fixes #144 )
...
It turns out that we actually have *two* files in our storage servers at the
time that test_vdrive asserts things about the shares. I suppose that
test_vdrive happens to pass on all other operating systems because the
filesystem happens to return the right share as the first one in a
"listdir()". The fix in this patch is slightly kludgey -- allow either share
to pass -- but good enough.
2007-09-22 15:26:27 -07:00
Brian Warner
ff07ce6234
check_speed: add optional 100MB test, if the 10MB test finished fast enough
2007-09-22 00:04:46 -07:00
Brian Warner
325c7a7bfa
node.py: chmod the foolscap private key (node.pem) to 0600, since it's secret
2007-09-21 16:52:55 -07:00
Zooko O'Whielacronx
594a104c71
startstop_node.py: fix bug with options passed to restart and enable restart --profile
2007-09-21 15:48:15 -07:00
Zooko O'Whielacronx
3e4616f4ad
cli: add --profile flag
2007-09-21 15:03:47 -07:00
Zooko O'Whielacronx
20079e3019
startstop_node.py: don't pass config to do_start() and do_stop()
2007-09-21 15:03:15 -07:00
Zooko O'Whielacronx
28b2c19f1e
startstop_node.py: find twistd.py in sys.prefix/scripts if there isn't a twistd executable on the path
2007-09-21 14:45:20 -07:00
Brian Warner
1c29065f1c
check_memory: touch the hotline file more often, to avoid accidental timeout
2007-09-20 19:35:31 -07:00
Brian Warner
d58d794757
check_speed: do both upload and download tests
2007-09-20 18:52:44 -07:00
Brian Warner
2632c0ad8c
check_speed: upload multiple files, measure Ax+B
2007-09-20 16:55:33 -07:00
Zooko O'Whielacronx
f0b427485b
startstop_node.py: remove debug printout which breaks unit tests
2007-09-20 17:02:36 -07:00
Zooko O'Whielacronx
c2e310825d
startstop_node.py: improve the hack of launching twistd
...
Thanks to Brian for helping me figure out the cleaner way to do this: take the
first result from which("twistd"), and if it has the extension ".bat" or
".exe" then execute it, else execute python and give it as the first argument.
2007-09-20 12:37:50 -07:00
Brian Warner
3774ce59ea
check_memory: fix race condition for startup of in-process server nodes
2007-09-20 15:33:58 -07:00
Brian Warner
77fae0b338
check_memory.py: record initial memory usage (before any connections are made)
2007-09-20 12:36:27 -07:00
Brian Warner
b9d5a4ead4
check_speed.py: run two 1MB uploads and measure the time it takes
2007-09-19 18:40:18 -07:00
Brian Warner
f6be35e122
started framework for an automated speed-checking tool. Doesn't do much yet.
2007-09-19 18:27:47 -07:00
Brian Warner
2b93164a03
test_runner: try harder to work on slow buildslaves and cygwin
2007-09-19 13:56:00 -07:00
Brian Warner
bede39a156
check-memory: add 'receive' mode, for #97 (consumption during share receive
2007-09-19 12:59:32 -07:00
Brian Warner
0cbf2cca08
introducer.py: add test coverage of _disconnected()
2007-09-19 11:50:13 -07:00
Brian Warner
e2f19825d9
startstop_node.py: improve test coverage a little bit
2007-09-19 01:50:27 -07:00
Brian Warner
d31d7ff941
test_system.py: do a large-file test (1.5MB) to trigger pauseProducing
2007-09-19 01:43:44 -07:00
Brian Warner
1340c484c6
download.py: use producer/consumer to reduce memory usage, closes #129 .
...
If the DownloadTarget is also an IConsumer, give it control of the brakes
by offering ourselves to target.registerProducer(). When they tell us to
pause, set a flag, which is checked between segment downloads and decodes.
webish.py: make WebDownloadTarget an IConsumer and pass control along to
the http.Request, which already knows how to be an IConsumer.
This reduces the memory footprint of stalled HTTP GETs to a bare minimum,
and thus closes #129 .
2007-09-19 00:34:47 -07:00
Brian Warner
9c9a793540
upload.py: fix signature of NonConvergentUploadMixin.get_encryption_key
2007-09-18 23:38:11 -07:00
Brian Warner
ee9b7fd678
check_memory: oops, silly bug make 'upload' push to ourselves, raising the usage
2007-09-19 04:14:48 -07:00
Brian Warner
08cc32bb77
check_memory: add download-GET-slow, to simulate memory usage of a node feeding downloaded data via HTTP GET to a slow client
2007-09-18 20:35:27 -07:00
Brian Warner
a5fab9935b
test_runner: better diagnostics in case test_client fails
2007-09-18 20:03:18 -07:00
Brian Warner
2886c5aeae
check_memory: don't accept shares for download/download-GET test, since that hits bug #97
2007-09-18 19:40:29 -07:00
Brian Warner
f6acf5a0c3
check_memory: add download, download-GET
2007-09-18 18:56:05 -07:00
Brian Warner
38946439f3
test_runner.py: fix race conditions in start/stop node, should run on cygwin now
2007-09-18 15:17:26 -07:00
Brian Warner
645927ca73
test_runner.py: skip the start/stop test when sys.platform is win32/cygwin.
...
The previous twisted.python.runtime.platformType approach didn't catch cygwin
for some reason.
2007-09-17 21:56:08 -07:00
Brian Warner
f72ef4d8f9
test_runner.py: add tests for startstop_node.py
2007-09-17 02:25:31 -07:00
Brian Warner
cf69accfb6
startstop_node.py: use SIGINT instead of SIGTERM, to allow the node a chance to clean up
2007-09-17 02:24:21 -07:00
Brian Warner
b7c18d6505
webish: improve test coverage
2007-09-17 01:53:46 -07:00
Brian Warner
6cd11037c6
debug: remove the 'dump-share-leases' command, it is subsumed by 'dump-share'
2007-09-17 01:44:39 -07:00
Brian Warner
79bd7d422d
provisioning.py: get full test coverage
2007-09-17 01:38:54 -07:00
Brian Warner
311ed144f8
uri.py: improve test coverage a bit
2007-09-17 01:09:47 -07:00
Brian Warner
8451b485a4
storage: fill alreadygot= with all known shares for the given storageindex, not just the ones they asked about
2007-09-17 00:48:40 -07:00
Brian Warner
808f851589
upload: make peer-selection a bit more uniform. Closes #132 .
2007-09-16 17:08:34 -07:00
Brian Warner
24e6ccddce
peer-selection: if we must loop, send a minimal number of queries (by asking for more than one share per peer on the second pass)
2007-09-16 01:53:00 -07:00
Brian Warner
8a251d8670
upload: remove Tahoe3 peer-selection algorithm
2007-09-16 01:26:11 -07:00
Brian Warner
baa16087cd
upload: switch to Tahoe2, add test for uniform share allocation
2007-09-16 01:25:03 -07:00
Brian Warner
979d12cd42
upload.py: implement Tahoe2 peer-selection algorithm
2007-09-16 01:24:07 -07:00
Brian Warner
f24d7450a7
vdrive.py: oops, forgot an import
2007-09-15 15:21:57 -07:00
Brian Warner
d37d74fa1d
vdrive: log an error if we weren't able to use the vdrive the way we wanted to
2007-09-15 15:17:29 -07:00
Brian Warner
d628d5f503
storage: remove the leftover incoming/XYZ/ directory when we're done with it
2007-09-15 14:34:04 -07:00
Brian Warner
e9d32f14b9
check_memory.py: preserve client.log and stats.out in _test_memory/
...
Put the nodes in _test_memory/test/, which is clobbered on each test. Also
kill the client with SIGINT instead of SIGKILL. Also don't daemonize the
client, since we're going to kill it at the end of the test anyways: this
cleans up shutdown a bit.
2007-09-15 20:53:06 -07:00
Brian Warner
d5ed60e4ad
webish: upload: when the localdir= doesn't exist, say so in the HTTP response
2007-09-15 12:49:07 -07:00
Brian Warner
c7986389e5
check_memory.py: have all clients write their logs to _test_memory/client.log instead of a separate file per client
2007-09-15 12:34:05 -07:00
Brian Warner
e7f61a3a6c
check_memory: oops, only bail if the process ended abnormally
2007-09-15 11:47:29 -07:00
Brian Warner
32f25f33e3
check-memory: if the child process fails to start, shut down instead of hanging forever
2007-09-14 20:16:57 -07:00
Brian Warner
df49fa55e0
test_client.py: remove unused 'stat' import
2007-09-13 20:17:42 -07:00
Zooko O'Whielacronx
8fd98624a3
don't try to use bindann
...
It causes a mysterious misbehavior in Python import which causes the previous patch to fail (the patch to not run trial tests if dependencies can't be imported)
2007-09-13 19:14:46 -07:00
Zooko O'Whielacronx
f443f8adbc
remove test in test_client -- we can't easily assert that files aren't readable by others, on Windows
2007-09-13 20:12:26 -07:00
Brian Warner
e1e037e9b5
storage: always record lease expiration times as integers
2007-09-11 14:53:31 -07:00
nejucomo
96f55be844
Add some passing unit tests for testutil.PollMixin.
2007-09-07 16:15:41 -07:00
Brian Warner
89a25f1987
provisioning: include the cost of replacing failed hard drives
2007-09-10 15:46:45 -07:00
Brian Warner
0497d62939
provisioning: add cost information
2007-09-10 15:29:33 -07:00
Zooko O'Whielacronx
09ff5af99c
loosen constraint on share data size
2007-09-11 11:29:18 -07:00
Brian Warner
3eaa888a0e
provisioning: oops, fix test_provisioning too
2007-09-07 15:01:17 -07:00
Brian Warner
0ae7d31d33
provisioning.py: update test with new encoding-parameters format
2007-09-07 14:45:43 -07:00
Brian Warner
2a48283e53
provisioning: require at least one drive per server
2007-09-06 23:08:21 -07:00
Brian Warner
5561d3afe4
provisioning: explain the 'survival' number better
2007-09-06 23:01:07 -07:00
Brian Warner
421277cafa
provisioning: more repair/survivability data
2007-09-06 22:54:53 -07:00
Brian Warner
69d680d820
provisioning: add some drive failure and repair rate info
2007-09-06 18:47:41 -07:00
Brian Warner
849dbb4950
provisioning.py: add file/server availability numbers
2007-09-05 18:16:21 -07:00
Brian Warner
76be4a582c
webish: implement 'PUT /uri?t=mkdir' (to create anonymous dirnodes)
2007-09-05 17:23:06 -07:00
Brian Warner
db3f11648c
webish: implement 'PUT /uri' (to create anonymous files)
2007-09-05 17:12:27 -07:00
Brian Warner
77394da447
client.py: only import webish.py if 'webport' is set, to save 3MB of footprint
2007-09-04 16:33:06 -07:00
Brian Warner
277e720f7c
storage: add version number to share data. Closes #90 .
2007-09-04 09:00:24 -07:00
Brian Warner
fb65aadd82
storage: don't add a duplicate lease, renew the old one instead
2007-09-02 21:39:47 -07:00
Brian Warner
89c7f27572
storage: remove get_or_add_owner, since I don't know what we need yet
2007-09-02 15:03:40 -07:00
Brian Warner
94233b8813
storage: remove unused delete_bucket() method, lease-cancellation covers it
2007-09-02 15:00:29 -07:00
Brian Warner
85f3107b12
storage: handle simultanous uploads: add a lease for the pre-empted client
2007-09-02 14:57:49 -07:00
Brian Warner
dbcabc1142
debug: 'dump-uri-extension' command becomes 'dump-share', add 'dump-share-leases'. Both display leases.
2007-09-02 14:48:20 -07:00
Brian Warner
0fe1205789
storage: replace sqlite with in-share lease records
2007-09-02 14:47:15 -07:00
Brian Warner
c7288286ae
test_uri.py: remove tiny whitespace
2007-08-28 23:40:03 -07:00
wilcoxjg
33bae16ed2
created DirnodeURI schema
2007-08-28 23:28:26 -07:00
Brian Warner
a605fe5cad
storage: use sqlite from either python2.5's stdlib or the pysqlite2 package
2007-08-28 23:28:52 -07:00
Brian Warner
2a63fe8b01
deletion phase3: add a sqlite database to track renew/cancel-lease secrets, implement renew/cancel_lease (but nobody calls them yet). Also, move the shares from BASEDIR/storage/* down to BASEDIR/storage/shares/*
2007-08-27 23:41:40 -07:00
Brian Warner
850bc9da02
deletion phase2b: create renew/cancel secrets for real.
2007-08-27 19:30:26 -07:00
Brian Warner
1428079a84
node: change get_or_create_config to strip whitespace and accept a filemode= argument
2007-08-27 19:23:50 -07:00
Brian Warner
89ceb49309
node.py: change get_or_create_config() to accept a function
2007-08-27 19:07:12 -07:00
Brian Warner
56afda11d1
deletion phase2a: improve creation of renew/cancel secrets. Still fake though.
2007-08-27 19:00:18 -07:00
Brian Warner
77a7232867
node.py: refactor config-file getting and setting
2007-08-27 18:58:39 -07:00
Brian Warner
739ae1ccde
deletion phase1: send renew/cancel-lease secrets, but my_secret is fake, and the StorageServer discards them
2007-08-27 17:28:51 -07:00
Brian Warner
ed525f7478
test_web: improve provisioning.py test coverage a bit by using a live web hit
2007-08-27 17:26:39 -07:00
Brian Warner
5946985c07
add a provisioning utility page which shows necessary storage space and transfer rates for grids of various sizes
2007-08-26 23:44:24 -07:00
Brian Warner
b620d0955b
debug.py: add share-overhead size info to dump-uri-extension
2007-08-26 23:42:39 -07:00
Brian Warner
0a39430730
webish: strip leading/tailing whitespace from user-provided filenames, to avoid confusion
2007-08-25 12:05:24 -07:00
Brian Warner
44db5fd8a2
webish: strip leading/tailing whitespace from user-provided URIs
2007-08-25 12:05:06 -07:00
Zooko O'Whielacronx
427bf2cdc9
change name of the query "allow_local_access()?" to "local_access_is_allowed()"
...
So as to avoid confusing it with the setter method
"allow_local_access(allowed=True)".
2007-08-22 10:32:00 -07:00
Brian Warner
ac358fc32f
tahoe_put.py: hush pyflakes by removing unused 'sys' import
2007-08-17 16:29:50 -07:00
tahoe
f49194cb5e
trying to introduce old style humanreadablied logs hopefully without breaking the existing ones
2007-08-11 14:52:37 -07:00
Brian Warner
664a0e6af4
crypto: fix compiler warnings in the .c files
2007-08-15 17:48:52 -07:00
Brian Warner
6c24848797
copy RSA from PyCrypto into the allmydata/ tree, we'll use it eventually
2007-08-15 17:36:39 -07:00
Brian Warner
6ac237b845
hush pyflakes warning in Crypto.Util.number
2007-08-14 14:17:12 -07:00
Brian Warner
b7703df6a5
remove PyCrypto, copy AES/SHA256/Util.number into the allmydata/ tree
2007-08-14 13:57:41 -07:00
Brian Warner
3e1c417f81
webish: modify JSON to match zooko's proposed API changes in #118
2007-08-23 13:00:39 -07:00
Zooko O'Whielacronx
0c22044f7e
command-line: remove the --vdrive option (it is now hardcoded to "global")
2007-08-23 13:27:00 -07:00
Brian Warner
f3353ee5e4
web: replace FILE links with /uri -based ones, to prevent an XSS attack against the secret vdrive URI contained in the current URL
2007-08-22 17:35:01 -07:00
Brian Warner
930618fba2
web: oops, forgot to add start.html
2007-08-22 15:08:00 -07:00
Brian Warner
4de5767c98
web: remove /vdrive/private, replace with a start.html file that points at the /uri/PRIVATE_URI, to prevent XSRF attacks
2007-08-22 14:54:34 -07:00
Zooko O'Whielacronx
8717905e35
refactor reading of configuration files in client.py
...
This makes it so that an optional file which is unreadable or is rm'ed
at the wrong moment will be ignored instead of raising an exception.
It also bums out a couple of unnecessary lines of code (the explicit
".close()" call).
2007-08-22 10:29:57 -07:00
Brian Warner
d5262ec7e1
client.py: add a newline to "myself.furl" contents
2007-08-17 23:28:22 -07:00
Zooko O'Whielacronx
9996836d23
command-line: fix ticket #111 by requiring input to be a local file and sending Content-Length header
2007-08-17 14:59:49 -07:00
Zooko O'Whielacronx
594912e334
command-line: fix a few bugs in the "execute this python file" way to execute rm
2007-08-17 14:17:31 -07:00
Brian Warner
3e2d1e28eb
tahoe_put-web2ish.py: clean up imports to be pyflakes-compliant, add newline to output
2007-08-16 17:47:52 -07:00
Brian Warner
2014662b47
cli.py: fix typo in synopsis
2007-08-16 17:47:24 -07:00
Brian Warner
75056b6222
test_system: weaken the assertion about the 410 error message when downloading a bogus URI, since some systems emit error messages differently
2007-08-16 17:05:26 -07:00
Brian Warner
6b5e05cc67
test_system.py: verify that we can replace files in place
2007-08-16 17:03:50 -07:00
Brian Warner
fe06b3be8b
dirnode: change the defined behavior of RIVirtualDriveServer.set to allow replace-in-place without raising an exception
2007-08-16 17:03:19 -07:00
Brian Warner
be0ab3b1f1
test_system.py: many (failing) web tests were accidentally bypassed, fix those. Add some PUT tests.
2007-08-16 16:49:40 -07:00
Zooko O'Whielacronx
a12ef1ae82
command-line: add "rm", and tidy-up variable names, and make it so "allmydata-tahoe spam" prints a usage message instead of returning silently
2007-08-17 13:23:16 -07:00
Zooko O'Whielacronx
e554962ac3
command-line: remove some redundant options checking
2007-08-17 13:06:43 -07:00
Zooko O'Whielacronx
bffd0c97f5
command-line: fix all three commands and all two ways to invoke them to require node-url and give a useful usage string if node-url is absent or of the wrong form
2007-08-17 12:54:47 -07:00
Zooko O'Whielacronx
e4f07a975f
tahoe_put.py: don't treat "^HTTP 200 OK" appearing somewhere in the body as if it appeared in the header
2007-08-17 12:06:41 -07:00
Zooko O'Whielacronx
7e8c6eb50b
tahoe_put.py: require node-url to be provided
2007-08-17 12:06:11 -07:00
Zooko O'Whielacronx
922663a7e2
cmdline: change "--server" to "--node-url" and make it have no default value
2007-08-16 16:53:27 -07:00
Zooko O'Whielacronx
1564987e6c
put now exits after doing its work, and it prints a terse message if 200 or 201, and a full dump of response elsewise
2007-08-16 16:30:39 -07:00
Zooko O'Whielacronx
6ed3a23c62
incomplete version of tahoe-put.py
...
It doesn't exit properly afterward, and it might not do the best things with non-success responses from the server.
(See tahoe-put-web2ish.py for an example of better response handling.)
2007-08-16 16:01:01 -07:00
Zooko O'Whielacronx
9ad02dbe80
fix version class to preferred format and correct parsing
2007-08-16 15:38:01 -07:00
Zooko O'Whielacronx
5a1d4aa8b7
version_class.py: if you don't have pkg_resources for comparing version numbers, use distutils.version.LooseVersion
2007-08-16 16:16:41 -07:00
Brian Warner
ceef80bee6
scripts: rearrange Options, make --version behave the same for all commands
2007-08-16 12:50:19 -07:00
Zooko O'Whielacronx
4ed7f0ac2a
fix small bug in unit tests which caused spurious failures on Windows
2007-08-16 14:14:41 -07:00
Zooko O'Whielacronx
a431811011
in-line doc edit: s/pyutil/allmydata.util/
2007-08-16 14:10:48 -07:00
Zooko O'Whielacronx
094b687d6e
import version class and make-version script from pyutil -- fixes win32 build, improves error handling, and eliminates unused features
2007-08-16 14:09:30 -07:00
Zooko O'Whielacronx
6c4fb6fd93
a first crack at the "put" command-line
...
There are actually two versions in this patch, one of which requires twisted.web2 and the other of which uses the Python standard library's socket module. The socketish one doesn't know when the web server is done so it hangs after doing its thing. (Oh -- maybe I should add an HTTP header asking the web server to close the connection when finished.) The web2ish one works better in that respect. Neither of them handle error responses from the server very well yet.
After lunch I intend to finish the socketish one.
To try one, mv src/allmydata/scripts/tahoe_put-{socketish,web2ish}.py src/allmydata/scripts/tahoe_put.py .
If you want to try the web2ish one, and you can't find a web2 package to install, you can get one from:
http://allmydata.org/~zooko/repos/twistedweb2snarf/
2007-08-16 12:15:38 -07:00
Brian Warner
ee6a6c79fa
webish.py: allow users to delete (but not create) empty-named files. Closes #94 .
2007-08-15 23:14:05 -07:00
Brian Warner
c1da0c11bc
webish: implement replace= for POST commands
2007-08-15 15:21:38 -07:00
Brian Warner
1752c9e29e
webish: implement replace= for PUT commands
2007-08-15 13:22:23 -07:00
Brian Warner
42dcc3088e
IDirectoryNode: add has_child() method
2007-08-15 13:22:01 -07:00
Brian Warner
244471dcc4
check_memory.py: Disable the 100MB test for now: our buildslave can't currently handle it because the testnet/framework processes uses something like 600M of RSS.
2007-08-15 12:55:11 -07:00
Brian Warner
a0c16f1a36
client.py: make a note in the logs when the auto-shutdown feature is in use
2007-08-14 02:12:30 -07:00
Brian Warner
179d5462c0
webish: look for when_done= in POST fields as well as queryargs. Closes #101 .
...
We need to look in the fields because that's how we build the mkdir/upload
forms. Without this, uploading or creating directories would leave us on a
page that had just a URI, instead of something actually useful to a human.
2007-08-13 17:45:02 -07:00
Brian Warner
6c6c62037d
fix pyflakes warnings from recent b32decode change
2007-08-12 16:33:51 -07:00
Brian Warner
0332e94656
test_system.py: add coverage for get_permuted_peers()
2007-08-12 16:29:34 -07:00
Brian Warner
1ac83637a6
Client.get_permuted_peers: use self.nodeid now that it's fixed
2007-08-12 16:24:51 -07:00
tahoe
53349c571f
shorten ids
2007-08-12 11:53:18 -07:00
Zooko O'Whielacronx
42f8e57416
don't over-encode the nodeid many times with ascii-encoding
2007-08-12 10:29:38 -07:00
Brian Warner
2f5a27316f
webish: localfile=/localdir= are now disabled by default, a special switch is required to enable them
2007-08-10 18:21:22 -07:00
Brian Warner
9281a0aedc
webish: reduce POST memory footprint by overriding http.Request
...
The original twisted.web.http.Request class has a requestReceived method
that parses the form body (in the request's .content filehandle) using
the stdlib cgi.parse_multipart() function. parse_multipart() consumes a
lot of memory when handling large file uploads, because it holds the
arguments entirely in RAM. Nevow's subclass of Request uses cgi.FieldStorage
instead, which is much more memory-efficient.
This patch uses a local subclass of Request and a modified copy of the
requestReceived method. It disables the cgi.parse_multipart parsing and
instead relies upon Nevow's use of FieldStorage. Our code must look for
form elements (which come from the body of the POST request) in req.fields,
rather than assuming that they will be copied into req.args (which now
only contains the query arguments that appear in the URL).
As a result, memory usage is no longer inflated by the size of the file
being uploaded in a POST upload request. Note that cgi.FieldStorage uses
temporary files (tempfile.TemporaryFile) to hold the data.
This closes #29 .
2007-08-10 17:25:33 -07:00
Brian Warner
b6ff38e4c7
webish.POSTHandler: fix typo that meant we didn't look for 'name' in req.fields
2007-08-10 17:25:28 -07:00
Brian Warner
998802fd6d
#96 : add flag to enable pushing data to ourselves, defaulting to False
2007-08-09 18:30:24 -07:00
Brian Warner
8b14ad1673
encode.py: log a percentage complete as well as the raw byte counts
2007-08-09 18:28:45 -07:00
Brian Warner
08ee80176a
Encoder.__repr__: mention the file being encoded
2007-08-09 18:26:56 -07:00
Brian Warner
684966d103
encode.py: add a reactor turn barrier between segments, to allow Deferreds to retire and free their arguments, all in the name of reducing memory footprint
2007-08-09 18:26:17 -07:00
Zooko O'Whielacronx
531109e803
fix test_web refactoring so that the WebMixin class isn't a TestCase
...
Thanks, Brian.
2007-08-10 08:40:02 -07:00
wilcoxjg
e13783cd6a
If do_stop attempts to stop a nonexistent process the OSError exception is caught (by errno) and handled tersely.
2007-08-09 08:57:45 -07:00
Zooko O'Whielacronx
39f218ab90
refactor test_web so that other tests can use the part of test_web that sets up a simple filesystem
2007-08-09 13:08:42 -07:00
Brian Warner
9ac5c14037
introducer: don't log.err() an initial connection failure, since that flunks tests. Use self.log() instead. Also improve test_client.py to always trigger this case, before it was finishing to quickly to always hit the problem.
2007-08-09 12:53:44 -07:00
Brian Warner
84d41adfcc
check_memory.py: include a single 100MB test
2007-08-09 11:30:33 -07:00
Zooko O'Whielacronx
9b08691f51
fix iputil so that it doesn't launch dozens of processes when you give it a full path and so that it tries executables in preference order and stops as soon as one gives out a dotted-quad string
2007-08-09 10:56:47 -07:00
Zooko O'Whielacronx
524c9f1fc2
startstop_node now works if the logs directory already exists
...
Thanks to Arno and Arch for pointing this out.
2007-08-08 21:43:48 -07:00
Brian Warner
57df32aac5
now that the buildslave is moved to a new machine, enable the 50MB test
2007-08-09 01:32:52 -07:00
Zooko O'Whielacronx
42d3afffa2
refactor test_uri and add a test of the empty file
2007-07-26 10:48:30 -07:00
Brian Warner
2ba415bd3e
move nodelogs from BASEDIR/twistd.log* to BASEDIR/logs/twistd.log*, closes #61
2007-07-24 20:44:33 -07:00
Brian Warner
8de93635af
test_web.py: oops, fix syntax error caused by removing comments
2007-07-25 03:21:30 -07:00
Brian Warner
6aff425fd2
test_web.py: remove spurious '# YES' lines, now that all the tests pass
2007-07-24 20:16:21 -07:00
Brian Warner
f664d4d80d
vdrive: eventually create public/private dirnodes event if the vdrive server isn't available at start up
2007-07-24 19:44:25 -07:00
Brian Warner
c3293f4f37
interfaces: move signatures into docstrings, to reduce lines of code and improve code-coverage numbers
2007-07-24 19:43:21 -07:00
Brian Warner
e3eba2d6c5
remove unused imports: hush pyflakes warnings
2007-07-24 18:13:58 -07:00
Zooko O'Whielacronx
32c90c6468
switch from rfc 3548 base-32 to z-base-32 except for tubids/nodeids
2007-07-24 13:46:06 -07:00
Brian Warner
e6e9ddc588
refactor upload/encode, to split encrypt and encode responsibilities
2007-07-23 19:31:53 -07:00
Brian Warner
9c5ab89afe
truncate storage index to 128 bits, since it's derived from a 128 bit AES key
2007-07-22 19:48:44 -07:00
Brian Warner
81a9904455
CHK: remove the storage index from the URI, deriving it from the key instead
2007-07-21 18:23:15 -07:00
Brian Warner
2bc3c163b6
uri.py: get keys and index from the URI instance
2007-07-21 17:45:00 -07:00
Brian Warner
1d9a58977f
uri: implement URI-processing classes, IFileURI/IDirnodeURI, use internally
2007-07-21 15:40:36 -07:00
Brian Warner
32fcf0b405
dirnode.build_manifest(): tolerate cycles in the directory graph
2007-07-21 15:40:13 -07:00
Brian Warner
e3a57fca98
upload: finish refactoring, all unit tests pass now
2007-07-19 22:53:29 -07:00
Brian Warner
9af506900b
upload: refactor to enable streaming upload. not all tests pass yet
2007-07-19 18:21:44 -07:00
Brian Warner
a7f04616bf
webish: improve test coverage further
2007-07-17 12:28:29 -07:00
Brian Warner
3197e1d39b
webish: improve test coverage of WebDownloadTarget
2007-07-17 12:16:45 -07:00
Brian Warner
d927d371b7
check_memory.py: test POST-based uploads as well as from-disk ones
2007-07-16 20:17:51 -07:00
Brian Warner
4db205b8b3
vdrive.py: log a note when we successfully retrieve the global root directory
2007-07-16 20:17:06 -07:00
Brian Warner
0826b1bd34
introducer.py: minor rearrangement of methods
2007-07-16 19:47:42 -07:00
Brian Warner
0269a6229b
check_memory.py: fix benign-but-noisy vdrive.furl handling bug
2007-07-16 19:34:52 -07:00
Brian Warner
0fd5dcacf6
check_memory: update it, write stats to a file, disable 50MB test for now
2007-07-16 18:08:55 -07:00
Brian Warner
1aa22b9abd
client.py: add a 'debug_no_storage' option to throw out all share data
2007-07-16 18:07:03 -07:00
Brian Warner
e91288d9ed
client.py: write control.furl in the right place
2007-07-16 18:06:27 -07:00
robk-org
920fed7f2a
added unit test to webish's rename function
...
added unit tests to test various permutations of the rename function, and
some sanity checks on the rename-form function.
also added a guard to prevent '/' in from_/to_name in rename calls.
2007-07-16 17:05:01 -07:00
Brian Warner
9be3da03f6
runner.py: make 'allmydata-tahoe --version' emit version numbers of everything
2007-07-16 14:58:36 -07:00
Brian Warner
20c980d02b
reduce MAX_SEGMENT_SIZE from 2MB to 1MB, to compensate for the large blocks that 3-of-10 produces
2007-07-16 13:48:34 -07:00
Brian Warner
56dcb814a8
webish: test error cases more thoroughly by looking inside the response text
2007-07-16 12:01:19 -07:00
Brian Warner
3f294d5597
webish: oops, handle POST without localfile= too
2007-07-16 12:00:54 -07:00
Brian Warner
f207f4a199
webish.py: disallow slashes in POSTed filenames. Closes #75 .
2007-07-16 11:53:12 -07:00
Brian Warner
7bc031a81d
webish.Directory: add a 'text/plain' link for files, to download as plaintext.
...
This is useful to view (e.g.) .py files that would otherwise be served as
some weird text/x-python that browsers will just want to save to disk.
2007-07-13 23:45:35 -07:00
Brian Warner
225fbfee57
storage.py: add a test for the next_power_of_k fix I made a few hours ago, basically do an upload with a non-power-of-two number of segments
2007-07-13 22:24:06 -07:00
Brian Warner
6e1ad64c9d
webish.py: test that _get_or_create_directories fix I made a few hours ago
2007-07-13 22:19:16 -07:00
Brian Warner
a290205ade
webish: handle PUTs to direct children of the root
2007-07-13 19:31:52 -07:00
Brian Warner
8a39ee9034
storage.py: turn some assertions into preconditions
2007-07-13 19:30:48 -07:00
Brian Warner
9fc687cdfc
storage.py: handle num_segments != power-of-two without an assertion
2007-07-13 19:30:21 -07:00
Brian Warner
5c08be170e
interfaces: increase ShareSize now that our default k is smaller (hence blocks are bigger)
2007-07-13 19:29:31 -07:00
Brian Warner
c6f52e379a
rename storageserver.py to just storage.py, since it has both server and client sides now
2007-07-13 17:25:45 -07:00
Brian Warner
0468448b85
welcome.xhtml: add a link to allmydata.org
2007-07-13 17:15:44 -07:00
Brian Warner
d206843625
encode.py: minor typo
2007-07-13 17:00:06 -07:00
Brian Warner
4d868e6649
fix dump-uri-extension
2007-07-13 16:58:08 -07:00
Brian Warner
35117d77a0
storageserver.ReadBucketProxy: break out _parse_offsets, for debug tools
2007-07-13 16:52:17 -07:00
Brian Warner
7589a8ee82
storage: we must truncate short segments. Now most tests pass (except uri_extension)
2007-07-13 16:38:25 -07:00
Brian Warner
1f8e407d9c
more #85 work, system test still fails
2007-07-13 15:09:01 -07:00
Brian Warner
cd8648d39b
storage: use one file per share instead of 7 ( #85 ). work-in-progress, tests still fail
2007-07-13 14:04:49 -07:00
Brian Warner
6ff94541a8
control.py: make get_memory_usage() callable from elsewhere
2007-07-08 19:07:54 -07:00
robk-org
d65d02fa58
add a 'rename' button to the webish dir view
...
alongside the 'del' button is now presented a 'rename' button, which takes
the user to a new page, the 't=rename-form' page, which asks ther user for
the new name of the child and ultimately submits a POST request to the dir
for 't=rename' to perform the actual rename i.e. an attach followed by a
delete of children.
2007-07-12 16:53:54 -07:00
robk-org
15c3055176
fix bug in arg handling around 'delete' button in directory view
...
the code composing the form providing the 'delete' button in a dir
view was broken in that it tried to put some of the arguments into
the url query, rather than as the form's post args. worse, nevow
was kind enough to escape the querystring making it invalid.
2007-07-12 16:46:54 -07:00
robk-org
cacc9ac7b1
include the filename on URI-link urls for files
...
this is likely to induce browsers to do more useful things with the result
than they would when given neither content-type nor filename. (i.e. they
can guess that a .jpg file is an image, even with a bogus content type)
2007-07-11 15:26:47 -07:00
Brian Warner
798b714bef
webish.DirnodeWalkerMixin: comment out the code that demonstrates what we
...
would do if we could just use generators, since we don't use it.
2007-07-12 23:28:09 -07:00
Brian Warner
b0c65ed232
note that setting k=1 is equivalent to replication
2007-07-12 16:22:12 -07:00
Brian Warner
ba7e14a870
fix several methods to handle LIT URIs correctly, rather than assuming that all filenodes are CHK URIs
2007-07-12 16:17:49 -07:00
Brian Warner
53cf757be3
make it possible to download LIT uris. oops.
2007-07-12 16:16:59 -07:00
Brian Warner
5399395c27
allow the introducer to set default encoding parameters. Closes #84 .
...
By writing something like "25 75 100" into a file named 'encoding_parameters'
in the central Introducer's base directory, all clients which use that
introducer will be advised to use 25-out-of-100 encoding for files (i.e.
100 shares will be produced, 25 are required to reconstruct, and the upload
process will be happy if it can find homes for at least 75 shares). The
default values are "3 7 10". For small meshes, the defaults are probably
good, but for larger ones it may be appropriate to increase the number of
shares.
2007-07-12 15:33:30 -07:00
Brian Warner
def63d193e
implement URI:LIT, which stores small (<55B) files inside the URI itself. Fixes #81 .
2007-07-12 13:22:36 -07:00
Brian Warner
b62252183d
interfaces.py: increase RIVirtualDriveServer.list constraint from 100 entries to 1000, for now
2007-07-12 13:23:52 -07:00
Brian Warner
9238c61224
cli: implement 'get'
2007-07-11 10:26:19 -07:00
Zooko O'Whielacronx
32166c360a
tahoe_ls: remove a couple of vestigial or debugging bits
2007-07-10 19:46:40 -07:00
Brian Warner
b419b853ea
cli: implement 'allmydata-tahoe ls'
2007-07-10 19:37:37 -07:00
Brian Warner
d8a878da86
startstop_node.py: refactor find_twistd() out so it is only run when you need to start a node
2007-07-10 19:13:55 -07:00
Brian Warner
f22801aa33
runner.py: further refactoring
2007-07-10 19:05:18 -07:00
Brian Warner
4a16d20d85
runner.py: start to refactor into separate files
2007-07-10 18:41:52 -07:00
Brian Warner
944583a599
tahoe-ls.py: hush a pyflakes warning
2007-07-10 16:26:05 -07:00
Brian Warner
ea3c3e6780
import simplejson directly, and remove our local copies of the component .py files from util/json*
2007-07-10 15:49:32 -07:00
