Zooko O'Whielacronx
42f8e57416
don't over-encode the nodeid many times with ascii-encoding
2007-08-12 10:29:38 -07:00
Zooko O'Whielacronx
3bc708529f
foolscap: change using-foolscap.xhtml to say "public key" instead of "private key"
...
In the first instance, I hope that the tubid is generated from the hash of the public key and not the hash of the private key!
In the latter instances, the phrase "public key" as in "public key certificate" is more common.
2007-08-11 09:45:03 -07:00
Brian Warner
2f5a27316f
webish: localfile=/localdir= are now disabled by default, a special switch is required to enable them
2007-08-10 18:21:22 -07:00
Brian Warner
9281a0aedc
webish: reduce POST memory footprint by overriding http.Request
...
The original twisted.web.http.Request class has a requestReceived method
that parses the form body (in the request's .content filehandle) using
the stdlib cgi.parse_multipart() function. parse_multipart() consumes a
lot of memory when handling large file uploads, because it holds the
arguments entirely in RAM. Nevow's subclass of Request uses cgi.FieldStorage
instead, which is much more memory-efficient.
This patch uses a local subclass of Request and a modified copy of the
requestReceived method. It disables the cgi.parse_multipart parsing and
instead relies upon Nevow's use of FieldStorage. Our code must look for
form elements (which come from the body of the POST request) in req.fields,
rather than assuming that they will be copied into req.args (which now
only contains the query arguments that appear in the URL).
As a result, memory usage is no longer inflated by the size of the file
being uploaded in a POST upload request. Note that cgi.FieldStorage uses
temporary files (tempfile.TemporaryFile) to hold the data.
This closes #29 .
2007-08-10 17:25:33 -07:00
Brian Warner
b6ff38e4c7
webish.POSTHandler: fix typo that meant we didn't look for 'name' in req.fields
2007-08-10 17:25:28 -07:00
Brian Warner
998802fd6d
#96 : add flag to enable pushing data to ourselves, defaulting to False
2007-08-09 18:30:24 -07:00
Brian Warner
8b14ad1673
encode.py: log a percentage complete as well as the raw byte counts
2007-08-09 18:28:45 -07:00
Brian Warner
08ee80176a
Encoder.__repr__: mention the file being encoded
2007-08-09 18:26:56 -07:00
Brian Warner
684966d103
encode.py: add a reactor turn barrier between segments, to allow Deferreds to retire and free their arguments, all in the name of reducing memory footprint
2007-08-09 18:26:17 -07:00
Zooko O'Whielacronx
531109e803
fix test_web refactoring so that the WebMixin class isn't a TestCase
...
Thanks, Brian.
2007-08-10 08:40:02 -07:00
wilcoxjg
e13783cd6a
If do_stop attempts to stop a nonexistent process the OSError exception is caught (by errno) and handled tersely.
2007-08-09 08:57:45 -07:00
Zooko O'Whielacronx
39f218ab90
refactor test_web so that other tests can use the part of test_web that sets up a simple filesystem
2007-08-09 13:08:42 -07:00
Brian Warner
9ac5c14037
introducer: don't log.err() an initial connection failure, since that flunks tests. Use self.log() instead. Also improve test_client.py to always trigger this case, before it was finishing to quickly to always hit the problem.
2007-08-09 12:53:44 -07:00
Brian Warner
84d41adfcc
check_memory.py: include a single 100MB test
2007-08-09 11:30:33 -07:00
Zooko O'Whielacronx
6d905265a1
zfec: required version of setuptools is 0.6c6 on cygwin, 0.6a9 on other platforms
2007-08-09 10:29:03 -07:00
Zooko O'Whielacronx
5bbbdd1e91
zfec: default setuptools version is now 0.6c6
2007-08-09 10:28:38 -07:00
Zooko O'Whielacronx
d1c40a7ab5
zfec: add benchmark utility
2007-08-09 10:19:13 -07:00
Zooko O'Whielacronx
279561b61b
zfec: add to README.txt how to run the unit tests
2007-07-20 15:51:24 -07:00
Zooko O'Whielacronx
31a1e03194
zfec: fix link to zfec-dev mailing list
2007-07-20 13:57:47 -07:00
Zooko O'Whielacronx
a1fbfc9e53
zfec: fix README.txt to explain how to use setuptools with GNU stow better
...
Also remove requirement of setuptools >= v0.6c3. I *think* that using setuptools in the modern way relieves this requirement, but I haven't tested it on dapper yet.
2007-07-20 13:57:01 -07:00
Zooko O'Whielacronx
df3fc9b5ed
zfec: clarify licensing requirements (transitive grace period)
2007-07-20 13:55:46 -07:00
Zooko O'Whielacronx
8522689641
zfec: shebang usr bin env python in ez_setup.py
2007-06-04 04:45:17 -07:00
Zooko O'Whielacronx
5643ef81a1
zfec: update doc strings in fileutil
2007-06-04 04:44:44 -07:00
Zooko O'Whielacronx
9b08691f51
fix iputil so that it doesn't launch dozens of processes when you give it a full path and so that it tries executables in preference order and stops as soon as one gives out a dotted-quad string
2007-08-09 10:56:47 -07:00
Zooko O'Whielacronx
524c9f1fc2
startstop_node now works if the logs directory already exists
...
Thanks to Arno and Arch for pointing this out.
2007-08-08 21:43:48 -07:00
Brian Warner
57df32aac5
now that the buildslave is moved to a new machine, enable the 50MB test
2007-08-09 01:32:52 -07:00
warner-tahoe
1b863ed433
foolscap: fix .deb-breaking bug (foolscap.lothar.com: #8 )
2007-08-07 19:00:08 -07:00
warner-tahoe
afe006d700
update foolscap to foolscap-0.1.5, the latest release
2007-08-07 18:55:47 -07:00
Zooko O'Whielacronx
c367dfed5c
shebang /usr/bin/env python in ez_setup.py
...
This makes it so that "./ez_setup.py" works on more platforms.
2007-08-08 14:37:40 -07:00
Zooko O'Whielacronx
13c18a06d9
fix missing import -- this bug was introduced by the recent patch to relax setuptools version requirements
2007-08-07 14:25:44 -07:00
Zooko O'Whielacronx
9a8abab722
require setuptools >= 0.6c6 on cygwin, and >= 0.6a9 on other platforms
...
Earlier I tried 0.6a9 (which comes in .deb format on Dapper) and something
didn't work, so I raised it to 0.6c3. However, I didn't make a note of what
failed, and so now I'm hoping that what failed was cygwin-specific. Anyway,
everyone look out for setuptools compatibility problems on the your favorite
platform (and I'll check the buildslaves), and we'll raise the required version
as little as possible and only on the problematic platform.
2007-08-07 14:15:54 -07:00
tahoe
c4e5510813
setuptools upgrade to 0.6c6
2007-07-31 16:18:11 -07:00
Zooko O'Whielacronx
42d3afffa2
refactor test_uri and add a test of the empty file
2007-07-26 10:48:30 -07:00
Brian Warner
2ba415bd3e
move nodelogs from BASEDIR/twistd.log* to BASEDIR/logs/twistd.log*, closes #61
2007-07-24 20:44:33 -07:00
Brian Warner
8de93635af
test_web.py: oops, fix syntax error caused by removing comments
2007-07-25 03:21:30 -07:00
Brian Warner
6aff425fd2
test_web.py: remove spurious '# YES' lines, now that all the tests pass
2007-07-24 20:16:21 -07:00
Brian Warner
f664d4d80d
vdrive: eventually create public/private dirnodes event if the vdrive server isn't available at start up
2007-07-24 19:44:25 -07:00
Brian Warner
c3293f4f37
interfaces: move signatures into docstrings, to reduce lines of code and improve code-coverage numbers
2007-07-24 19:43:21 -07:00
Brian Warner
e3eba2d6c5
remove unused imports: hush pyflakes warnings
2007-07-24 18:13:58 -07:00
Zooko O'Whielacronx
32c90c6468
switch from rfc 3548 base-32 to z-base-32 except for tubids/nodeids
2007-07-24 13:46:06 -07:00
Brian Warner
e6e9ddc588
refactor upload/encode, to split encrypt and encode responsibilities
2007-07-23 19:31:53 -07:00
Brian Warner
9c5ab89afe
truncate storage index to 128 bits, since it's derived from a 128 bit AES key
2007-07-22 19:48:44 -07:00
Brian Warner
81a9904455
CHK: remove the storage index from the URI, deriving it from the key instead
2007-07-21 18:23:15 -07:00
Brian Warner
2bc3c163b6
uri.py: get keys and index from the URI instance
2007-07-21 17:45:00 -07:00
Brian Warner
1d9a58977f
uri: implement URI-processing classes, IFileURI/IDirnodeURI, use internally
2007-07-21 15:40:36 -07:00
Brian Warner
32fcf0b405
dirnode.build_manifest(): tolerate cycles in the directory graph
2007-07-21 15:40:13 -07:00
Brian Warner
e3a57fca98
upload: finish refactoring, all unit tests pass now
2007-07-19 22:53:29 -07:00
Brian Warner
9af506900b
upload: refactor to enable streaming upload. not all tests pass yet
2007-07-19 18:21:44 -07:00
Brian Warner
a7f04616bf
webish: improve test coverage further
2007-07-17 12:28:29 -07:00
Brian Warner
3197e1d39b
webish: improve test coverage of WebDownloadTarget
2007-07-17 12:16:45 -07:00
