Commit Graph

1077 Commits

Author SHA1 Message Date
Brian Warner
8f1212edac encode.py: don't allow a shareholder which dies in start() to kill the whole upload 2008-01-28 12:14:48 -07:00
Brian Warner
a36ed9b5b7 encode.py: don't record BAD log event unless there is actually a problem 2008-01-28 11:59:10 -07:00
robk-tahoe
0207dc85b9 confwiz: use get_config call to backend
this will write an arbitrary number of config files, instead of being restricted
to just the introducer.furl, based on the response of the php backend.  
the get_config is passed username/password
2008-01-25 19:01:32 -07:00
Brian Warner
09a99ce822 offloaded upload: avoid tail-recursion problem that would break large files 2008-01-24 21:51:34 -07:00
Brian Warner
081d27a65d encode.py: improve error message when segment lengths come out wrong 2008-01-24 21:51:09 -07:00
Brian Warner
46fe024612 offloaded uploader: don't use a huge amount of memory when skipping over previously-uploaded data 2008-01-24 17:25:33 -07:00
robk-tahoe
2f2de9df2e Reworked mac gui to not lock up upon launch
Previously, once the node itself was launched, the UI event loop was no longer
running.  This meant that the app would sit around seemingly 'wedged' and being
reported as 'Not Responding' by the os.

This chnages that by actually implementing a wxPython gui which is left running
while the reactor, and the node within it, is launched in another thread.
Beyond 'quit' -> reactor.stop, there are no interactions between the threads.

The ui provides 'open web root' and 'open account page' actions, both in the
file menu, and in the (right click) dock icon menu.


Something weird in the handling of wxpython's per-frame menubar stuff seems to
mean that the menu bar only displays the file menu and about etc (i.e. the items
from the wx menubar) if the focus changes from and back to the app while the 
frame the menubar belongs to is displayed.  Hence a splash frame comes up at
startup to provide an opportunity.

It also seems that, in the case that the file menu is not available, that one
can induce it to reappear by choosing 'about' from the dock menu, and then 
closing the about window.
2008-01-24 20:00:28 -07:00
robk-tahoe
5085c35002 cleanup mac and windows build code
this moves some of the code common to both windows and mac builds into the
allmydata module hierarchy, and cleans up the windows and mac build directories
to import the code from there.
2008-01-23 21:06:41 -07:00
Brian Warner
a0945de0e0 encode.py: log the contents of the uri_extension block 2008-01-23 18:08:04 -07:00
Brian Warner
1ff21d1d64 test_upload.py: implement remote_abort on our fake BucketWriter 2008-01-23 18:07:34 -07:00
robk-tahoe
6ef4606534 tweak webish to use resource_filename to find css and html files
using sibpath to find web template files relative to source code is functional
when running from source environments, but not especially flexible when running
from bundled built environments.  the more 'orthodox' mechanism, pkg_resources,
in theory at least, knows how to find resource files in various environments.

this makes the 'web' directory in allmydata into an actual allmydata.web module
(since pkg_resources looks for files relative to a named module, and that module
must be importable) and uses pkg_resources.resource_filename to find the files
therein.
2008-01-22 17:44:58 -07:00
Zooko O'Whielacronx
348eecd615 setup: require specific versions of dependencies, both at run-time (if pkg_resources is available) and at build-time, and make there be only once place where we specify those versions
Using pkg_resources.require() like this also apparently allows people to install multiple different versions of packages on their system and tahoe (if pkg_resources is available to it) will import the version of the package that it requires.  I haven't tested this feature.
2008-01-22 17:24:33 -07:00
Zooko O'Whielacronx
eaed7a0690 setup: use setuptools (if it is present) at run-time to give a specific error message on startup if a too-old version of a dependency is installed 2008-01-22 17:42:54 -07:00
robk-tahoe
d1de1f180a offloaded: reinstate fix for windows tests
in a discussion the other day, brian had asked me to try removing this fix, since
it leads to double-closing the reader.  since on my windows box, the test failures
I'd experienced were related to the ConnectionLost exception problem, and this
close didn't see to make a difference to test results, I agreed.

turns out that the buildbot's environment does fail without this fix, even with
the exception fix, as I'd kind of expected.

it makes sense, because the reader (specifically the file handle) must be closed
before it can be unlinked. at any rate, I'm reinstating this, in order to fix the
windows build
2008-01-21 15:25:15 -07:00
robk-tahoe
7b990cc9af UNDO: offloaded: close reader before removing its file
unlinking a file before closing it is not portable. it works on unix, but fails
since an open file holds a lock on windows.

this closes the reader before trying to unlink the encoding file within the 
CHKUploadHelper.
2008-01-17 17:36:28 -07:00
Zooko O'Whielacronx
84289b2446 setup: update some docs, metadata, and docstrings 2008-01-22 10:22:51 -07:00
robk-tahoe
7c34658438 offloaded: fix failure in unit test on windows
in trying to test my fix for the failure of the offloaded unit test on windows
(by closing the reader before unlinking the encoding file - which, perhaps
disturbingly doesn't actually make a difference in my windows environment)
I was unable too because the unit test failed every time with a connection lost
error.

after much more time than I'd like to admit it took, I eventually managed to
track that down to a part of the unit test which is supposed to be be dropping
a connection.   it looks like the exceptions that get thrown on unix, or at
least all the specific environments brian tested in, for that dropped 
connection are different from what is thrown on my box (which is running py2.4
and twisted 2.4.0, for reference)  adding ConnectionLost to the list of
expected exceptions makes the test pass.

though curiously still my test logs a NotEnoughWritersError error, and I'm not
currently able to fathom why that exception isn't leading to any overall 
failure of the unit test itself.

for general interest, a large part of the time spent trying to track this down
was lost to the state of logging.  I added a whole bunch of logging to try
and track down where the tests were failing, but then spent a bunch of time
searching in vain for that log output.  as far as I can tell at this point
the unit tests are themselves logging to foolscap's log module, but that isn't
being directed anywhere, so all the test's logging is being black holed.
2008-01-17 20:57:29 -07:00
robk-tahoe
46154beace offloaded: close reader before removing its file
unlinking a file before closing it is not portable. it works on unix, but fails
since an open file holds a lock on windows.

this closes the reader before trying to unlink the encoding file within the 
CHKUploadHelper.
2008-01-17 17:36:28 -07:00
Brian Warner
e9307d3fda offloaded: close the local filehandle after encoding is done, otherwise windows fails 2008-01-17 01:52:33 -07:00
Brian Warner
fd0dc3013c offloaded: update unit tests: assert that interrupt/resume works, and that the helper deletes tempfiles 2008-01-17 01:18:10 -07:00
Brian Warner
6b08c28f5d upload.py: make it easier to have an IUploadable that overrides encoding parameters: just set an attribute instead of subclassing 2008-01-17 01:17:42 -07:00
Brian Warner
812383a369 offloaded: upload.py: handle forward skips, to allow resumed uploads to send less than all the data. We still read all the data (to hash it, 'paranoid mode'), but we don't send it over the wire 2008-01-17 01:16:56 -07:00
Brian Warner
0e3b218e7d offloaded.py: delete encoding tempfile when upload is complete 2008-01-17 01:15:54 -07:00
Brian Warner
41cbd1e3a7 offloaded.py: when resuming, append new data to incoming file, rather than overwrite it. 2008-01-17 01:15:32 -07:00
Brian Warner
8386b0226b offloaded.py: remove dead/redundant code 2008-01-17 01:14:45 -07:00
Brian Warner
c597e67c2b offloaded: improve logging across the board 2008-01-17 01:11:35 -07:00
Brian Warner
51321944f0 megapatch: overhaul encoding_parameters handling: now it comes from the Uploadable, or the Client. Removed options= too. Also move helper towards resumability. 2008-01-16 03:03:35 -07:00
Brian Warner
547375a488 node.py: when calling os.abort(), announce it to stdout as well as the log 2008-01-16 03:01:32 -07:00
Brian Warner
4cdb252d5a cli scripts: remove the for-educational-purposes standalone clauses. Closes #261. 2008-01-16 00:08:51 -07:00
Brian Warner
f0430ccc48 offloaded: cleanup to handle multiple simultaneous uploaders gracefully 2008-01-14 22:20:03 -07:00
Brian Warner
7bb9307871 encode: actually define the UploadAborted exception 2008-01-14 21:27:02 -07:00
Brian Warner
821521cc3e test_storage: fix pyflakes warnings 2008-01-14 21:26:48 -07:00
Brian Warner
9f0ee4b758 test_system: fix pyflakes warnings 2008-01-14 21:26:28 -07:00
Brian Warner
168a8c3b73 offloaded: improve logging, pass through options, get ready for testing interrupted uploads. test_system: add (disabled) interrupted-upload test 2008-01-14 21:24:26 -07:00
Brian Warner
a6ca98ac53 upload: add Encoder.abort(), to abandon the upload in progress. Add some debug hooks to enable unit tests. 2008-01-14 21:22:55 -07:00
Brian Warner
60090fb9f2 upload: improve logging 2008-01-14 21:19:20 -07:00
Brian Warner
e65967da49 upload: pass options through to the encoder 2008-01-14 21:17:32 -07:00
Brian Warner
cb76ed36f9 logging: enable flogging in more places, replace Node.log with flogging 2008-01-14 21:16:58 -07:00
Brian Warner
f664dcb489 testutil.py: hush the new (more strict) pyflakes 2008-01-14 18:27:55 -07:00
Brian Warner
c24a1bbaaa iputil.py: hush the new (more strict) pyflakes 2008-01-14 18:27:43 -07:00
robk-tahoe
d87a80dca6 added is_uri() function to allmydata.uri 2008-01-10 20:43:42 -07:00
Brian Warner
7ac2b94aba remove wait_for_numpeers and the when_enough_peers call in mutable.Publish 2008-01-14 14:55:59 -07:00
Brian Warner
222718f13c test_GET_DIRURL_large: reduce from 400 to 200 children: the test fails to
fail anyways, and 200 ought to be enough to trigger the problem, so 400 is
overkill, and just wastes CPU.
2008-01-14 22:31:41 -07:00
Brian Warner
37b7f136ce 'tahoe dump-cap': accept http:// -prefixed URLs too 2008-01-14 14:12:27 -07:00
Brian Warner
6ca0efeef6 add 'tahoe dump-cap' command, to show storage index, lease secrets, etc 2008-01-14 13:43:25 -07:00
Brian Warner
76ee9cccfe storage: improve logging a bit 2008-01-14 11:58:58 -07:00
Zooko O'Whielacronx
676ded6a7e tests: put back skipped and todo tests
closes #258 -- "put back skipped and todo tests"
2008-01-14 21:02:41 -07:00
Brian Warner
964edadf44 offloaded: add a system test, make it pass. files are now being uploaded through the helper. 2008-01-11 05:42:55 -07:00
Brian Warner
08f9c3f2ab offloaded.py: hush pyflakes 2008-01-11 05:05:14 -07:00
Brian Warner
6ac01fde4c offloaded: more test coverage on client side, change interfaces a bit 2008-01-11 04:53:37 -07:00