tahoe
53349c571f
shorten ids
2007-08-12 11:53:18 -07:00
Zooko O'Whielacronx
42f8e57416
don't over-encode the nodeid many times with ascii-encoding
2007-08-12 10:29:38 -07:00
Brian Warner
2f5a27316f
webish: localfile=/localdir= are now disabled by default, a special switch is required to enable them
2007-08-10 18:21:22 -07:00
Brian Warner
9281a0aedc
webish: reduce POST memory footprint by overriding http.Request
...
The original twisted.web.http.Request class has a requestReceived method
that parses the form body (in the request's .content filehandle) using
the stdlib cgi.parse_multipart() function. parse_multipart() consumes a
lot of memory when handling large file uploads, because it holds the
arguments entirely in RAM. Nevow's subclass of Request uses cgi.FieldStorage
instead, which is much more memory-efficient.
This patch uses a local subclass of Request and a modified copy of the
requestReceived method. It disables the cgi.parse_multipart parsing and
instead relies upon Nevow's use of FieldStorage. Our code must look for
form elements (which come from the body of the POST request) in req.fields,
rather than assuming that they will be copied into req.args (which now
only contains the query arguments that appear in the URL).
As a result, memory usage is no longer inflated by the size of the file
being uploaded in a POST upload request. Note that cgi.FieldStorage uses
temporary files (tempfile.TemporaryFile) to hold the data.
This closes #29 .
2007-08-10 17:25:33 -07:00
Brian Warner
b6ff38e4c7
webish.POSTHandler: fix typo that meant we didn't look for 'name' in req.fields
2007-08-10 17:25:28 -07:00
Brian Warner
998802fd6d
#96 : add flag to enable pushing data to ourselves, defaulting to False
2007-08-09 18:30:24 -07:00
Brian Warner
8b14ad1673
encode.py: log a percentage complete as well as the raw byte counts
2007-08-09 18:28:45 -07:00
Brian Warner
08ee80176a
Encoder.__repr__: mention the file being encoded
2007-08-09 18:26:56 -07:00
Brian Warner
684966d103
encode.py: add a reactor turn barrier between segments, to allow Deferreds to retire and free their arguments, all in the name of reducing memory footprint
2007-08-09 18:26:17 -07:00
Zooko O'Whielacronx
531109e803
fix test_web refactoring so that the WebMixin class isn't a TestCase
...
Thanks, Brian.
2007-08-10 08:40:02 -07:00
wilcoxjg
e13783cd6a
If do_stop attempts to stop a nonexistent process the OSError exception is caught (by errno) and handled tersely.
2007-08-09 08:57:45 -07:00
Zooko O'Whielacronx
39f218ab90
refactor test_web so that other tests can use the part of test_web that sets up a simple filesystem
2007-08-09 13:08:42 -07:00
Brian Warner
9ac5c14037
introducer: don't log.err() an initial connection failure, since that flunks tests. Use self.log() instead. Also improve test_client.py to always trigger this case, before it was finishing to quickly to always hit the problem.
2007-08-09 12:53:44 -07:00
Brian Warner
84d41adfcc
check_memory.py: include a single 100MB test
2007-08-09 11:30:33 -07:00
Zooko O'Whielacronx
9b08691f51
fix iputil so that it doesn't launch dozens of processes when you give it a full path and so that it tries executables in preference order and stops as soon as one gives out a dotted-quad string
2007-08-09 10:56:47 -07:00
Zooko O'Whielacronx
524c9f1fc2
startstop_node now works if the logs directory already exists
...
Thanks to Arno and Arch for pointing this out.
2007-08-08 21:43:48 -07:00
Brian Warner
57df32aac5
now that the buildslave is moved to a new machine, enable the 50MB test
2007-08-09 01:32:52 -07:00
Zooko O'Whielacronx
42d3afffa2
refactor test_uri and add a test of the empty file
2007-07-26 10:48:30 -07:00
Brian Warner
2ba415bd3e
move nodelogs from BASEDIR/twistd.log* to BASEDIR/logs/twistd.log*, closes #61
2007-07-24 20:44:33 -07:00
Brian Warner
8de93635af
test_web.py: oops, fix syntax error caused by removing comments
2007-07-25 03:21:30 -07:00
Brian Warner
6aff425fd2
test_web.py: remove spurious '# YES' lines, now that all the tests pass
2007-07-24 20:16:21 -07:00
Brian Warner
f664d4d80d
vdrive: eventually create public/private dirnodes event if the vdrive server isn't available at start up
2007-07-24 19:44:25 -07:00
Brian Warner
c3293f4f37
interfaces: move signatures into docstrings, to reduce lines of code and improve code-coverage numbers
2007-07-24 19:43:21 -07:00
Brian Warner
e3eba2d6c5
remove unused imports: hush pyflakes warnings
2007-07-24 18:13:58 -07:00
Zooko O'Whielacronx
32c90c6468
switch from rfc 3548 base-32 to z-base-32 except for tubids/nodeids
2007-07-24 13:46:06 -07:00
Brian Warner
e6e9ddc588
refactor upload/encode, to split encrypt and encode responsibilities
2007-07-23 19:31:53 -07:00
Brian Warner
9c5ab89afe
truncate storage index to 128 bits, since it's derived from a 128 bit AES key
2007-07-22 19:48:44 -07:00
Brian Warner
81a9904455
CHK: remove the storage index from the URI, deriving it from the key instead
2007-07-21 18:23:15 -07:00
Brian Warner
2bc3c163b6
uri.py: get keys and index from the URI instance
2007-07-21 17:45:00 -07:00
Brian Warner
1d9a58977f
uri: implement URI-processing classes, IFileURI/IDirnodeURI, use internally
2007-07-21 15:40:36 -07:00
Brian Warner
32fcf0b405
dirnode.build_manifest(): tolerate cycles in the directory graph
2007-07-21 15:40:13 -07:00
Brian Warner
e3a57fca98
upload: finish refactoring, all unit tests pass now
2007-07-19 22:53:29 -07:00
Brian Warner
9af506900b
upload: refactor to enable streaming upload. not all tests pass yet
2007-07-19 18:21:44 -07:00
Brian Warner
a7f04616bf
webish: improve test coverage further
2007-07-17 12:28:29 -07:00
Brian Warner
3197e1d39b
webish: improve test coverage of WebDownloadTarget
2007-07-17 12:16:45 -07:00
Brian Warner
d927d371b7
check_memory.py: test POST-based uploads as well as from-disk ones
2007-07-16 20:17:51 -07:00
Brian Warner
4db205b8b3
vdrive.py: log a note when we successfully retrieve the global root directory
2007-07-16 20:17:06 -07:00
Brian Warner
0826b1bd34
introducer.py: minor rearrangement of methods
2007-07-16 19:47:42 -07:00
Brian Warner
0269a6229b
check_memory.py: fix benign-but-noisy vdrive.furl handling bug
2007-07-16 19:34:52 -07:00
Brian Warner
0fd5dcacf6
check_memory: update it, write stats to a file, disable 50MB test for now
2007-07-16 18:08:55 -07:00
Brian Warner
1aa22b9abd
client.py: add a 'debug_no_storage' option to throw out all share data
2007-07-16 18:07:03 -07:00
Brian Warner
e91288d9ed
client.py: write control.furl in the right place
2007-07-16 18:06:27 -07:00
robk-org
920fed7f2a
added unit test to webish's rename function
...
added unit tests to test various permutations of the rename function, and
some sanity checks on the rename-form function.
also added a guard to prevent '/' in from_/to_name in rename calls.
2007-07-16 17:05:01 -07:00
Brian Warner
9be3da03f6
runner.py: make 'allmydata-tahoe --version' emit version numbers of everything
2007-07-16 14:58:36 -07:00
Brian Warner
20c980d02b
reduce MAX_SEGMENT_SIZE from 2MB to 1MB, to compensate for the large blocks that 3-of-10 produces
2007-07-16 13:48:34 -07:00
Brian Warner
56dcb814a8
webish: test error cases more thoroughly by looking inside the response text
2007-07-16 12:01:19 -07:00
Brian Warner
3f294d5597
webish: oops, handle POST without localfile= too
2007-07-16 12:00:54 -07:00
Brian Warner
f207f4a199
webish.py: disallow slashes in POSTed filenames. Closes #75 .
2007-07-16 11:53:12 -07:00
Brian Warner
7bc031a81d
webish.Directory: add a 'text/plain' link for files, to download as plaintext.
...
This is useful to view (e.g.) .py files that would otherwise be served as
some weird text/x-python that browsers will just want to save to disk.
2007-07-13 23:45:35 -07:00
Brian Warner
225fbfee57
storage.py: add a test for the next_power_of_k fix I made a few hours ago, basically do an upload with a non-power-of-two number of segments
2007-07-13 22:24:06 -07:00
