Commit Graph

252 Commits

Author SHA1 Message Date
Brian Warner
c6c30ac5d4 webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation 2009-02-20 12:15:54 -07:00
Zooko O'Whielacronx
7d91dc5efe rollback the feature of making "ambient upload authority" configurable
This reverses some, but not all, of the changes that were committed in the following set of patches.

rolling back:

Sun Jan 18 09:54:30 MST 2009  toby.murray
  * add 'web.ambient_upload_authority' as a paramater to tahoe.cfg

    M ./src/allmydata/client.py -1 +3
    M ./src/allmydata/test/common.py -7 +9
    A ./src/allmydata/test/test_ambient_upload_authority.py
    M ./src/allmydata/web/root.py +12
    M ./src/allmydata/webish.py -1 +4
Sun Jan 18 09:56:08 MST 2009  zooko@zooko.com
  * trivial: whitespace
  I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.

    M ./src/allmydata/test/test_ambient_upload_authority.py -9 +8
    M ./src/allmydata/web/root.py -2 +1
    M ./src/allmydata/webish.py -2 +1
Mon Jan 19 14:16:19 MST 2009  zooko@zooko.com
  * trivial: remove unused import noticed by pyflakes

    M ./src/allmydata/test/test_ambient_upload_authority.py -1
Mon Jan 19 21:38:35 MST 2009  toby.murray
  * doc: describe web.ambient_upload_authority

    M ./docs/configuration.txt +14
    M ./docs/frontends/webapi.txt +11
Mon Jan 19 21:38:57 MST 2009  zooko@zooko.com
  * doc: add Toby Murray to the CREDITS

    M ./CREDITS +4
2009-01-20 20:47:35 -07:00
Zooko O'Whielacronx
ce7c448bc0 trivial: whitespace
I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.
2009-01-18 10:56:08 -07:00
toby.murray
66f83c7356 add 'web.ambient_upload_authority' as a paramater to tahoe.cfg 2009-01-18 10:54:30 -07:00
Brian Warner
d3f2df00be webapi: serve the /static URL tree from /public_html (configurable) 2008-10-29 15:34:31 -07:00
Brian Warner
34ab4e3de3 more #514 log-webop status/cancel: add handle-expiration, test coverage 2008-10-21 22:13:54 -07:00
Brian Warner
ad3d9207a9 Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514. 2008-10-21 17:03:07 -07:00
Brian Warner
e5252fd5b2 web/deep-check: show the webapi runtime at the bottom of the page 2008-08-12 20:34:26 -07:00
Brian Warner
242b98f750 webish: censor URLs that start with /file and /named too 2008-05-19 14:34:13 -07:00
Brian Warner
f9cd30d9bc webish: complete rewrite, break into smaller pieces, auto-create directories, improve error handling 2008-05-19 12:57:04 -07:00
Brian Warner
304abfee32 webish: add /file links, change directory page to use them. This fixes filenames in wget. Closes #221. 2008-05-14 14:32:21 -07:00
Brian Warner
6c00a70dbc dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too. 2008-05-08 13:21:14 -07:00
Brian Warner
f153dafaa0 web: return a proper error upon POST with a bad t= value 2008-04-15 11:11:29 -07:00
Brian Warner
86578bc3cd webish: fix 'not running helper' status indicator on welcome page, add tests 2008-04-14 16:28:11 -07:00
Brian Warner
5b8320442a stats: add /statistics web page to show them, add tests 2008-04-14 14:17:08 -07:00
Brian Warner
2c96a32633 helper: add more stats to webapi, at /helper_status 2008-03-27 16:46:08 -07:00
Brian Warner
36f5c025a6 helper: add stats for the gatherer, show some on the webish welcome page 2008-03-27 15:55:32 -07:00
Brian Warner
9b3a32d0b3 add GET /uri/URI/?t=deep-size, to compute the total size of immutable files reachable from a given directory 2008-03-27 11:33:42 -07:00
Zooko O'Whielacronx
fc3bd0c987 use added secret to protect convergent encryption
Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key.  If you want traditional convergent encryption behavior, set the added secret to be the empty string.

This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names.  (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.)

This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 09:46:06 -07:00
robk-tahoe
08fd16ef78 add a mkdir-p POST handler
this adds a t=mkdir-p call to directories (accessed by their uri as
/uri/<URI>?t=mkdir=p&path=/some/path) which returns the uri for a 
directory at a specified path before the given uri, regardless of
whether the directory exists or whether intermediate directories
need to be created to satisfy the request.

this is used by the migration code in MV to optimise the work of
path traversal which was other wise done on every file PUT
2008-03-17 18:13:01 -07:00
Zooko O'Whielacronx
7bc06872ac refactor one of the dispatch routines in webish.py
The behavior is intended to be unchanged by this refactoring.  Unit tests show no change in behavior.
2008-03-20 12:11:09 -07:00
Brian Warner
2ef70ab814 mutable.py: split replace() into update() and overwrite(). Addresses #328. 2008-03-12 18:00:43 -07:00
Brian Warner
0c40513e75 webish: put a trailing slash in the node.url file, since our unit tests use it, and some versions of twisted or nevow don't automatically add one 2008-03-11 19:01:18 -07:00
Brian Warner
810ba68343 add a webserver for the Introducer, showing service announcements and subscriber lists 2008-03-11 17:36:25 -07:00
Zooko O'Whielacronx
a8a85da326 make the nodeurl start with '127.0.0.1' instead of with 'localhost'
This is because there exist in the wild computers that are misconfigured so that 'localhost' doesn't resolve to 127.0.0.1.  On those computers, using 'localhost' for the nodeurl is a security problem, because the user commonly sends valuable caps to the nodeurl.
2008-03-11 06:18:03 -07:00
Brian Warner
8815b30b9b webish: show storage sizelimit, abbreviate current usage 2008-03-06 21:16:38 -07:00
Brian Warner
3a331f6822 webish: split out 'unlinked' operations 2008-03-05 15:12:42 -07:00
Brian Warner
4fa622f9b1 webish: this file is too big, start breaking it into pieces, beginning with status 2008-03-05 14:59:56 -07:00
Brian Warner
ec23da28a6 webish: add more mutable-retrieve timing status 2008-03-04 21:04:36 -07:00
Brian Warner
ca1a1762e2 web: status: add 'started' timestamps to all operations 2008-03-04 18:50:44 -07:00
Brian Warner
68fbd89e66 webish: add primitive publish/retrieve status pages 2008-03-04 01:07:44 -07:00
Brian Warner
18eb00d136 webish: download-results: add per-server response times 2008-03-03 20:53:45 -07:00
Brian Warner
886ef22335 webish: download-results: add server_problems 2008-03-03 20:30:35 -07:00
Brian Warner
def910c391 webish download results: add servermap, decrypt time 2008-03-03 20:09:32 -07:00
Brian Warner
2b49605c51 webish: add 'download results', with some basic timing information 2008-03-03 19:19:21 -07:00
Brian Warner
c8e24f0904 webish: make upload timings visible on the recent uploads/downloads status page 2008-03-03 14:48:52 -07:00
Brian Warner
436baa1b19 webish: add per-file upload/download status pages 2008-02-29 23:03:00 -07:00
Brian Warner
1a7651ce82 retain 10 most recent upload/download status objects, show them in /status . Prep for showing individual status objects 2008-02-29 22:19:03 -07:00
Zooko O'Whielacronx
30c9b21d6b wui/wapi/webish: HTML form checkboxes send the value "on", so let's interpret that as boolean true 2008-02-29 20:29:42 -07:00
Zooko O'Whielacronx
99f006c584 wapi: add POST /uri/$DIRECTORY?t=set_children
Unfinished bits: doc in webapi.txt, test handling of badly formed JSON, return reasonable HTTP response, examination of the effect of this patch on code coverage -- but I'm committing it anyway because MikeB can use it and I'm being called to dinner...
2008-02-29 18:40:27 -07:00
Zooko O'Whielacronx
ef46e16aa9 test: refactor webist.POSTHandler() to have a method for each "?t=" command
Input validation and decoding is still done in the body of POSTHandler.renderHTTP().
2008-02-29 13:11:18 -07:00
Brian Warner
301dd3d489 webish status: distinguish active uploads/downloads from recent ones 2008-02-26 15:35:28 -07:00
Brian Warner
2f6fd7d000 webish.py: remove some dead code 2008-02-15 04:02:38 -07:00
Brian Warner
7927495cbe unicode handling: declare dirnodes to contain unicode child names, update webish to match 2008-02-14 15:45:56 -07:00
Zooko O'Whielacronx
7c6de95bc6 switch from base62 to base32 for storage indices, switch from z-base-32 to rfc 3548 base-32 for everything, separate out base32 encoding from idlib 2008-02-14 19:27:47 -07:00
Brian Warner
124b2160b9 webish.py: fix for #237: when listing large directories, insert a turn break once every 100 children, to work around non-optimized tail recursion Deferreds 2008-02-12 21:28:52 -07:00
Brian Warner
55d30790bc webish: flogify the remaining log messages 2008-02-12 19:32:33 -07:00
Brian Warner
94348b7182 webish: censor all caps before logging the HTTP request, to preserve user privacy 2008-02-12 19:31:23 -07:00
Brian Warner
b17b7695a4 webish: add /status page to display current uploads/downloads 2008-02-12 15:40:05 -07:00
Brian Warner
a70fba4ef1 webish: add edge metadata to t=json output, including timestamps 2008-02-11 19:14:10 -07:00
Brian Warner
3d72d00826 webish: display timestamps 2008-02-11 15:13:18 -07:00
Brian Warner
590b020a44 webish: more upload stats: total encode-and-push rate, already-in-grid existence check time 2008-02-06 02:39:01 -07:00
Brian Warner
6e0d3059a3 webish: add when_done= to POST /uri?t=upload . I did not add a 'recent uploads' section to the welcome page, but I think the new upload-results page provides the desired data 2008-02-06 02:38:16 -07:00
Brian Warner
652d50d7e0 UploadResults: add more helper timing stats (ciphertext fetch times) 2008-02-06 02:12:35 -07:00
Brian Warner
124fb5ecdf add upload-results timing info for helper uploads. This changes the Helper protocol, and introduces a compatibility break 2008-02-06 01:52:25 -07:00
Brian Warner
93d45abb02 add upload timings and rates to the POST /uri?t=upload results page 2008-02-06 00:41:51 -07:00
Brian Warner
1e4504463c webish: make POST /uri?t=upload deposit you on an 'Upload Results' page 2008-02-05 23:01:37 -07:00
Brian Warner
604d6d54ef webish.py: refactor /uri handlers, one rend.Page class per operation 2008-02-05 22:38:20 -07:00
Brian Warner
23c2c8d660 webish: add PUT /uri?mutable=true 2008-02-05 22:18:02 -07:00
Brian Warner
e5dc9a8486 webish: add POST /uri?t=upload&mutable=true 2008-02-05 22:10:22 -07:00
Brian Warner
6afe50e2aa webish: add upload/view-uri forms (not associated with any particular directory) to the welcome page. Document POST /uri?t=upload . 2008-02-05 21:44:40 -07:00
Brian Warner
66f33ee504 upload: return an UploadResults instance (with .uri) instead of just a URI 2008-02-05 21:01:38 -07:00
Brian Warner
2f09b03bcc webish: add storage-consumed estimate on welcome page 2008-02-05 20:29:39 -07:00
Brian Warner
4b4eba867b webish: display tahoe import path on the welcome page, to help figure out where the code is coming from 2008-02-05 20:08:49 -07:00
Brian Warner
eb9023ab0e webish: condense display of nickname a little bit 2008-02-05 18:29:28 -07:00
Brian Warner
96e6c78c7a webish: show nickname too 2008-02-05 18:26:05 -07:00
Brian Warner
d146ef7e09 webish: add extra introducer data (version, timestamps) to Welcome page 2008-02-05 17:32:27 -07:00
Brian Warner
583cc34d2f webish: remove 'URI-link' from directory page, now that we only use URI-links 2008-01-29 18:11:09 -07:00
Brian Warner
d9d0920177 webish: link to directory URIs rather than a child path. Addresses #103. 2008-01-29 18:04:32 -07:00
Brian Warner
4c5518faef webish: upload+localdir=missing should give an error 2008-01-28 14:48:06 -07:00
Brian Warner
3b41c939f8 webish: oops, unit tests don't have an Uploader, don't rely upon it for helper-status display 2008-01-28 14:03:29 -07:00
Brian Warner
8485ff53fc display the Helper FURL and our connection status on the welcome page. Closes #285. 2008-01-28 13:56:22 -07:00
robk-tahoe
6ef4606534 tweak webish to use resource_filename to find css and html files
using sibpath to find web template files relative to source code is functional
when running from source environments, but not especially flexible when running
from bundled built environments.  the more 'orthodox' mechanism, pkg_resources,
in theory at least, knows how to find resource files in various environments.

this makes the 'web' directory in allmydata into an actual allmydata.web module
(since pkg_resources looks for files relative to a named module, and that module
must be importable) and uses pkg_resources.resource_filename to find the files
therein.
2008-01-22 17:44:58 -07:00
robk-tahoe
9ed5533760 added tweaked sibpath implementation
use of twisted.python.util.sibpath to find files relative to modules doesn't
work when those modules are bundled into a library by py2exe.  this provides
an alternative implementation (in allmydata.util.sibpath) which checks for
the existence of the file, and if it is not found, attempts to find it relative
to sys.executable instead.
2008-01-10 15:23:41 -07:00
robk-tahoe
3a7c688270 fix webish unit tests by making node.url file optional 2008-01-08 12:36:14 -07:00
robk-tahoe
08c3ee73a2 reinstate creation of node.url files upon startup
a recent purge of the start.html code also took away the logic that wrote
'node.url' into the node root.  this is required for the tahoe cli tool to 
find the node.  this puts back a limited fraction of that code, so that the
node writes out a node.url file upon startup.
2008-01-07 18:04:56 -07:00
Zooko O'Whielacronx
5995d11a3c remove automatic private dir
* rename my_private_dir.cap to root_dir.cap
 * move it into the private subdir
 * change the cmdline argument "--root-uri=[private]" to "--dir-uri=[root]"
2008-01-03 17:02:05 -07:00
Brian Warner
6b81ebfd8e refactor webish.py slightly, improve the test coverage a bit. One test is marked SKIP pending improvements in webish.py 2007-12-25 03:48:57 -07:00
Brian Warner
0e804da8d9 webish: use get_args() everywhere, put .fields=None in MyRequest 2007-12-25 00:07:04 -07:00
Brian Warner
4ba1af37a6 test_web: more coverage of URIPOSTHandler error cases 2007-12-24 22:49:35 -07:00
Brian Warner
47761bde93 webish: factor out queryargs-or-formargs extraction, make /uri?t=mkdir work and redirect to a valid url 2007-12-24 17:26:52 -07:00
Zooko O'Whielacronx
a85a4d71f7 webish: add button to make directories and unit test of it
Unfortunately although it passes the unit tests, it doesn't work, because the unit tests and the implementation use the "encode params into URL" technique but the button uses the "encode params into request body" technique.
2007-12-20 15:31:12 -07:00
Zooko O'Whielacronx
a4bc623fa5 make the wapi/wui [*] support creation of a new directory without there already being an existing directory to link the new one into
[*] WebAPI/WebUI
2007-12-20 12:58:17 -07:00
Brian Warner
50bc0d2fb3 the new pyflakes is stricter, complaining about function definitions that overshadow earlier definitions or imports. Fix some of its complaints. 2007-12-18 18:47:28 -07:00
Zooko O'Whielacronx
36e8e4e561 do not put a link from the welcoXme page to the start.html if there is not a private directory 2007-12-17 18:35:59 -07:00
Zooko O'Whielacronx
8c65bdcf9d put all private state in $BASEDIR/private
fixes #219

The only part of #219 that this doesn't include is the part about 
logpublisher, which has been moved out of tahoe into foolscap.
2007-12-17 16:39:54 -07:00
Zooko O'Whielacronx
d3f23d4052 we no longer need to replace "/" with "!" in urls 2007-12-14 18:25:50 -07:00
Zooko O'Whielacronx
ae90016845 fix the overwrite button and add unit test for it 2007-12-14 17:52:05 -07:00
Zooko O'Whielacronx
caa69b007e refactor the feature of getting versions of packages, include the version numbers of more of the packages that we use 2007-12-12 19:37:37 -07:00
Brian Warner
74820cc3de webish: append save=true to a GET URL to save-to-disk. Closes #222. 2007-12-11 18:04:44 -07:00
Zooko O'Whielacronx
8ad2e85fdf add POST /uri?t=upload and tests thereof
Hm...  I refactored processing of segments in a way that I marked as "XXX HELP
I AM YUCKY", and then I ran out of time for rerefactoring it before I
committed.  At least all the tests pass.
2007-12-06 17:17:02 -07:00
Brian Warner
51af1fa2e3 webish: fix overwrite form display 2007-12-05 00:15:13 -07:00
Brian Warner
53e865cf49 test_web.py: add coverage for directory listings that include mutable files 2007-12-04 23:57:40 -07:00
Brian Warner
785f21b9bf test_web.py: add coverage for POST t=check 2007-12-04 23:49:38 -07:00
Brian Warner
56e02b274b webish: add POST t=mutable, make it replace files in-place, add t=overwrite 2007-12-04 23:42:54 -07:00
Zooko O'Whielacronx
3605354a95 fix several bugs and warnings -- thanks, pyflakes 2007-12-03 15:42:35 -07:00
Zooko O'Whielacronx
59d6c3c822 decentralized directories: integration and testing
* use new decentralized directories everywhere instead of old centralized directories
 * provide UI to them through the web server
 * provide UI to them through the CLI
 * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them
 * remove the notion of a "vdrive server" and a client thereof
 * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example)
 * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start.  The default is 1.
 * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!)
 * fix a few bugs where we used to equate "mutable" with "not read-only".  Nowadays all directories are mutable, but some might be read-only (to you).
 * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge
 * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.
2007-12-03 14:52:42 -07:00
Zooko O'Whielacronx
35aea346d3 don't provide the "overwrite" button if the file is readonly to you 2007-11-10 16:07:05 -07:00
Zooko O'Whielacronx
85b972a4c3 WUI: hook up an "overwrite" button on mutable files 2007-11-10 10:53:45 -07:00
Brian Warner
1f22768dc7 webish: add preliminary mutable file support: upload, download, listings, JSON, URI, RO-URI. No replace yet. 2007-11-09 03:54:27 -07:00