Commit Graph

2564 Commits

Author SHA1 Message Date
david-sarah
6e114e44b5 SFTP: avoid race condition where .write could be called on an OverwriteableFileConsumer after it had been closed. 2010-05-23 16:38:30 -07:00
david-sarah
c6b414c815 SFTP: log tracebacks for RAISEd exceptions. 2010-05-23 15:15:35 -07:00
david-sarah
725daee788 SFTP: more logging to investigate behaviour of getAttrs(path). 2010-05-23 13:42:36 -07:00
david-sarah
75357fd2d5 SFTP: fix pyflakes warnings; drop 'noisy' versions of eventually_callback and eventually_errback; robustify conversion of exception messages to UTF-8. 2010-05-23 07:09:05 -07:00
david-sarah
38964fb35a SFTP: fixes and test cases for renaming of open files. 2010-05-22 20:25:49 -07:00
david-sarah
42ecd77f4e SFTP: Increase test_sftp timeout to cater for francois' ARM buildslave. 2010-05-22 12:16:39 -07:00
david-sarah
5974773969 SFTP: Fix error in support for getAttrs on an open file, to index open files by directory entry rather than path. Extend that support to renaming open files. Also, implement the extposix-rename@openssh.org extension, and some other minor refactoring. 2010-05-21 20:58:36 -07:00
david-sarah
5545a678bf SFTP tests: fix test_openDirectory_and_attrs that was failing in timezones west of UTC. 2010-05-20 11:10:27 -07:00
david-sarah
ce3872d10e SFTP: allow getAttrs to succeed on a file that has been opened for creation but not yet uploaded or linked (part of #1050). 2010-05-19 20:56:13 -07:00
david-sarah
7f9e351bce SFTP: improve logging so that results of requests are (usually) logged. 2010-05-19 17:36:52 -07:00
david-sarah
a62733ed08 SFTP: add tests for more combinations of open flags. 2010-05-18 22:39:33 -07:00
david-sarah
f0945526ce SFTP: allow FXF_WRITE | FXF_TRUNC (#1050). 2010-05-18 21:32:40 -07:00
david-sarah
bde27cab57 SFTP: remove another case where we were logging data. 2010-05-18 18:27:13 -07:00
david-sarah
d8b096a054 SFTP: avoid logging all data passed to callbacks. 2010-05-18 17:06:51 -07:00
david-sarah
819eaa74c1 SFTP: fixes related to reporting of permissions (needed for sshfs). 2010-05-17 22:45:21 -07:00
david-sarah
8d118e28f1 SFTP: change error code returned for ExistingChildError to FX_FAILURE (fixes gvfs with some picky programs such as gedit). 2010-05-17 17:42:05 -07:00
david-sarah
57699fd1eb SFTP: fixed bugs that caused hangs during write (#1037). 2010-05-16 21:42:28 -07:00
david-sarah
5f9c10901b SFTP: work around a probable bug in twisted.conch.ssh.session:loseConnection(). Also some minor error handling cleanups. 2010-05-16 18:26:06 -07:00
david-sarah
6a2b0229f0 SFTP: Support statvfs extensions, avoid logging actual data, and decline shell sessions politely. 2010-05-16 08:43:47 -07:00
david-sarah
31994aaf74 SFTP: fix error in SFTPUserHandler arguments introduced by execCommand patch. 2010-05-15 18:40:45 -07:00
david-sarah
8028bf01bc SFTP: implement execCommand to interoperate with clients that issue a 'df -P -k /' command. Also eliminate use of Zope adaptation. 2010-05-15 18:27:54 -07:00
david-sarah
6ba6c97a7a sftpd.py: 'log.OPERATIONAL' should be just 'OPERATIONAL'. 2010-05-15 08:55:33 -07:00
david-sarah
5ab4a376b7 Attempt to fix #1040 by making SFTPUser implement ISession. 2010-05-14 17:57:19 -07:00
david-sarah
7409d1e019 Eliminate Windows newlines from sftpd.py. 2010-05-14 17:56:56 -07:00
david-sarah
14d0721246 Update SFTP implementation and tests: fix #1038 and switch to foolscap logging; also some code reorganization. 2010-05-13 21:31:13 -07:00
david-sarah
5fdd63d0ad Tests for new SFTP implementation 2010-05-11 23:05:52 -07:00
david-sarah
5248c808ac New SFTP implementation: mutable files, read/write support, streaming download, Unicode filenames, and more 2010-05-11 22:54:07 -07:00
david-sarah
9214dbda50 Add must_exist, must_be_directory, and must_be_file arguments to DirectoryNode.delete. This will be used to fixes a minor condition in the SFTP frontend. 2010-05-27 12:45:29 -07:00
david-sarah
4556702044 Fix test failures in test_web caused by changes to web page titles in #1062. Also, change a 'target' field to '_blank' instead of 'blank' in welcome.xhtml. 2010-06-03 16:21:05 -07:00
david-sarah
4f58355a0e dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present. 2010-06-01 18:46:44 -07:00
david-sarah
53f7d2c7fe dirnode.py: Fix #1034 (MetadataSetter does not enforce restriction on setting 'tahoe' subkeys), and expose the metadata updater for use by SFTP. Also, support diminishing a child cap to read-only if 'no-write' is set in the metadata. 2010-05-31 21:54:28 -07:00
david-sarah
027e7701bd Change doc comments in interfaces.py to take into account unknown nodes. 2010-05-28 10:19:22 -07:00
david-sarah
6d91b1e00e Trivial whitespace changes. 2010-05-27 12:41:14 -07:00
david-sarah
aac94a8d89 Suppress 'integer argument expected, got float' DeprecationWarning everywhere 2010-05-23 15:11:57 -07:00
david-sarah
e706f043f0 Change shouldFail to avoid Unicode errors when converting Failure to str 2010-05-11 23:07:54 -07:00
david-sarah
b8641f8f9b allmydata.org -> tahoe-lafs.org in __init__.py 2010-06-02 23:35:30 -07:00
david-sarah
90e9fe28af Resolve conflict in patch to change imports to absolute. 2010-06-02 22:46:08 -07:00
freestorm77
59a77f82d7 title_rename_xhtml.dpatch.txt
- Renamed xhtml Title from "Allmydata - Tahoe" to "Tahoe-LAFS"
- Renamed Tahoe to Tahoe-LAFS in page content
- Changed Tahoe-LAFS home page link to http://tahoe-lafs.org (added target="blank")
- Deleted commented css script in info.xhtml
2010-05-29 10:25:42 -07:00
Zooko O'Whielacronx
9e2da05837 tests: refactor test_web.py to have less duplication of literal caps-from-the-future
This is a prelude to a patch which will add tests of caps from the future which have non-ascii chars in them.
2010-05-18 22:51:46 -07:00
david-sarah
a80f19a084 Correct harmless indentation errors found by pylint 2010-02-25 23:21:51 -07:00
david-sarah
e76092e16c Change relative imports to absolute 2010-02-26 01:14:33 -07:00
david-sarah
e880fc2ce4 Suppress deprecation warning for twisted.web.error.NoResource when using Twisted >= 9.0.0. 2010-05-16 13:56:25 -07:00
Zooko O'Whielacronx
d4f6d4092c test: show stdout and stderr in case of non-zero exit code from "tahoe" command 2010-05-24 00:33:48 -07:00
Zooko O'Whielacronx
294c0a6279 tests: fix pyflakes warnings in bench_dirnode.py 2010-05-21 13:25:11 -07:00
Zooko O'Whielacronx
35efdf04ca tests: don't let bench_dirnode.py do stuff and have side-effects at import time (unless __name__ == '__main__') 2010-05-21 05:20:52 -07:00
Zooko O'Whielacronx
a48979516e tests: increase timeout to give François's ARM buildslave a chance to complete the tests 2010-05-20 06:45:26 -07:00
Kevan Carstensen
d0706d27ea Hush pyflakes warnings 2010-05-15 11:43:44 -07:00
Kevan Carstensen
9179dd099a Improve code coverage of the Tahoe2PeerSelector tests. 2010-05-14 20:29:13 -07:00
Kevan Carstensen
55a2849f38 Remove a comment that no longer makes sense. 2010-05-14 13:35:16 -07:00
Kevan Carstensen
e225f573b9 Fix up the behavior of #778, per reviewers' comments
- Make some important utility functions clearer and more thoroughly 
    documented.
  - Assert in upload.servers_of_happiness that the buckets attributes
    of PeerTrackers passed to it are mutually disjoint.
  - Get rid of some silly non-Pythonisms that I didn't see when I first
    wrote these patches.
  - Make sure that should_add_server returns true when queried about a 
    shnum that it doesn't know about yet.
  - Change Tahoe2PeerSelector.preexisting_shares to map a shareid to a set
    of peerids, alter dependencies to deal with that.
  - Remove upload.should_add_servers, because it is no longer necessary
  - Move upload.shares_of_happiness and upload.shares_by_server to a utility
    file.
  - Change some points in Tahoe2PeerSelector.
  - Compute servers_of_happiness using a bipartite matching algorithm that 
    we know is optimal instead of an ad-hoc greedy algorithm that isn't.
  - Change servers_of_happiness to just take a sharemap as an argument,
    change its callers to merge existing_shares and used_peers before 
    calling it.
  - Change an error message in the encoder to be more appropriate for 
    servers of happiness.
  - Clarify the wording of an error message in immutable/upload.py
  - Refactor a happiness failure message to happinessutil.py, and make
    immutable/upload.py and immutable/encode.py use it.
  - Move the word "only" as far to the right as possible in failure 
    messages.
  - Use a better definition of progress during peer selection.
  - Do read-only peer share detection queries in parallel, not sequentially.
  - Clean up logging semantics; print the query statistics whenever an
    upload is unsuccessful, not just in one case.
2010-05-13 17:49:17 -07:00
Kevan Carstensen
9bc71d3da0 Alter the error message when an upload fails, per some comments in #778.
When I first implemented #778, I just altered the error messages to refer to
servers where they referred to shares. The resulting error messages weren't
very good. These are a bit better.
2009-12-30 15:03:44 -07:00
Kevan Carstensen
8bcc771e26 Change "UploadHappinessError" to "UploadUnhappinessError" 2009-12-04 22:30:37 -07:00
Kevan Carstensen
68fb556e93 Alter the error message returned when peer selection fails
The Tahoe2PeerSelector returned either NoSharesError or NotEnoughSharesError
for a variety of error conditions that weren't informatively described by them.
This patch creates a new error, UploadHappinessError, replaces uses of 
NoSharesError and NotEnoughSharesError with it, and alters the error message
raised with the errors to be more in line with the new servers_of_happiness
behavior. See ticket #834 for more information.
2009-11-22 18:24:05 -07:00
Kevan Carstensen
320582be5a Eliminate overcounting iof servers_of_happiness in Tahoe2PeerSelector; also reorganize some things. 2009-11-17 19:45:42 -07:00
Kevan Carstensen
4e29060847 Change stray "shares_of_happiness" to "servers_of_happiness" 2009-11-16 15:24:59 -07:00
Kevan Carstensen
a816de3f23 Alter Tahoe2PeerSelector to make sure that it recognizes existing shares on readonly servers, fixing an issue in #778 2009-11-16 13:28:05 -07:00
Kevan Carstensen
8c71df53f9 Alter 'immutable/encode.py' and 'immutable/upload.py' to use servers_of_happiness instead of shares_of_happiness. 2009-11-04 05:12:22 -07:00
Kevan Carstensen
b2d8a7cec2 Alter the signature of set_shareholders in IEncoder to add a 'servermap' parameter, which gives IEncoders enough information to perform a sane check for servers_of_happiness. 2009-11-03 21:32:41 -07:00
Kevan Carstensen
89a8da9dd2 Alter CiphertextDownloader to work with servers_of_happiness 2009-09-23 21:19:32 -07:00
Kevan Carstensen
5683112a02 Revisions of the #778 tests, per reviewers' comments
- Fix comments and confusing naming.
- Add tests for the new error messages suggested by David-Sarah
  and Zooko.
- Alter existing tests for new error messages.
- Make sure that the tests continue to work with the trunk.
- Add a test for a mutual disjointedness assertion that I added to
  upload.servers_of_happiness.
- Fix the comments to correctly reflect read-onlyness
- Add a test for an edge case in should_add_server
- Add an assertion to make sure that share redistribution works as it 
  should
- Alter tests to work with revised servers_of_happiness semantics
- Remove tests for should_add_server, since that function no longer exists.
- Alter tests to know about merge_peers, and to use it before calling 
  servers_of_happiness.
- Add tests for merge_peers.
- Add Zooko's puzzles to the tests.
- Edit encoding tests to expect the new kind of failure message.
- Edit tests to expect error messages with the word "only" moved as far
  to the right as possible.
- Extended and cleaned up some helper functions.
- Changed some tests to call more appropriate helper functions.
- Added a test for the failing redistribution algorithm
- Added a test for the progress message
- Added a test for the upper bound on readonly peer share discovery.
2010-05-13 18:25:42 -07:00
Kevan Carstensen
d360ee0d2e Alter various unit tests to work with the new happy behavior 2010-01-07 12:13:25 -07:00
Kevan Carstensen
953c9555ca Replace "UploadHappinessError" with "UploadUnhappinessError" in tests. 2009-12-04 22:34:53 -07:00
Kevan Carstensen
c0f3dbb919 Add tests for the behavior described in #834. 2009-11-22 19:20:08 -07:00
Kevan Carstensen
c3b11dedea Re-work 'test_upload.py' to be more readable; add more tests for #778 2009-11-16 13:23:34 -07:00
Kevan Carstensen
9590690450 Test Tahoe2PeerSelector to make sure that it recognizeses existing shares on readonly servers 2009-11-08 18:37:35 -07:00
Kevan Carstensen
324a72e4a3 Add more tests for comment:53 in ticket #778 2009-11-04 05:28:49 -07:00
Kevan Carstensen
697f1cfae8 Add a test for upload.shares_by_server 2009-11-04 05:13:24 -07:00
Kevan Carstensen
0d8e70bfe5 Minor tweak to an existing test -- make the first server read-write, instead of read-only 2009-11-03 21:42:32 -07:00
Kevan Carstensen
5ad29d4e8b Alter tests to use the new form of set_shareholders 2009-11-03 21:36:02 -07:00
"Kevan Carstensen"
ee9690b357 Refactor some behavior into a mixin, and add tests for the behavior described in #778 2009-10-30 02:19:08 -07:00
Kevan Carstensen
362f204075 Alter NoNetworkGrid to allow the creation of readonly servers for testing purposes. 2009-10-17 18:30:13 -07:00
"Kevan Carstensen"
5fe125ed74 Alter wording in 'interfaces.py' to be correct wrt #778 2009-12-04 21:40:05 -07:00
Zooko O'Whielacronx
6037df55b3 web: add a simple test that you can load directory.xhtml 2010-05-09 23:37:29 -07:00
Francois Deppierraz
dd95e8e71b CLI: Support for https url in option --node-url
This patch modifies the regular expression used for verifying of '--node-url'
parameter.  Support for accessing a Tahoe gateway over HTTPS was already
present, thanks to Python's urllib.
2010-04-30 11:56:09 -07:00
Brian Warner
e2031aff0f backupdb.did_create_directory: use REPLACE INTO, not INSERT INTO + ignore error
This handles the case where we upload a new tahoe directory for a
previously-processed local directory, possibly creating a new dircap (if the
metadata had changed). Now we replace the old dirhash->dircap record. The
previous behavior left the old record in place (with the old dircap and
timestamps), so we'd never stop creating new directories and never converge
on a null backup.
2010-04-27 22:08:03 -07:00
Brian Warner
f6f8256b35 "tahoe webopen": add --info flag, to get ?t=info
Also fix some trailing whitespace.
2010-04-24 16:30:03 -07:00
Zooko O'Whielacronx
80ae6f2706 setup: fix error in licensing declaration from Shawn Willden, as noted by the FSF compliance division 2010-03-09 08:37:36 -08:00
Zooko O'Whielacronx
34a50b857a docs: a few small edits to try to guide newcomers through the docs
These edits were suggested by my watching over Jake Appelbaum's shoulder as he completely ignored/skipped/missed install.html and also as he decided that debian.txt wouldn't help him with basic installation. Then I threw in a few docs edits that have been sitting around in my sandbox asking to be committed for months.
2010-03-03 15:19:02 -08:00
david-sarah
40edf8f419 Change code that gives a base32 SI or an empty string to be more straightforward. (#948) 2010-02-26 22:55:51 -08:00
david-sarah
a2ed17f2a0 Additional test for DIR2-LIT directories in test_web.py, fixed version (#948) 2010-02-24 20:18:24 -08:00
david-sarah
d7b50a3b86 Additional fixes for DIR2-LIT More Info page and deep-check/manifest operations (#948) 2010-02-24 00:02:20 -08:00
david-sarah
e6aee33bb7 directories: add DIR2-LIT directories to test_deepcheck.py (#948) 2010-02-23 23:54:33 -08:00
david-sarah
187d837c1d dirnode: add tests of literal dirnodes (current and fix for #948) 2010-02-23 20:33:45 -08:00
david-sarah
fec9185f2a Additional fix for abbrev_si, with test 2010-02-21 19:36:52 -08:00
david-sarah
d29ec184a6 Additions to test_web.py for #948 2010-02-21 18:53:52 -08:00
david-sarah
973f0afdd3 Change direct accesses to an_uri.storage_index to calls to .get_storage_index() (fixes #948) 2010-02-21 18:45:04 -08:00
david-sarah
6a7feea455 Tweak to 'tahoe ls --help' output (#837) 2010-02-23 19:02:31 -08:00
david-sarah
9741b9655f Test behaviour of 'tahoe ls' for unknown objects (#837) 2010-02-23 18:59:13 -08:00
david-sarah
03134eedb5 Improve behaviour of 'tahoe ls' for unknown objects, addressing kevan's comments 2010-02-19 22:13:13 -08:00
Brian Warner
1de4d2c594 test_util.py: improve coverage of util.time_format 2010-02-23 23:46:37 -05:00
Kevan Carstensen
81ad52d6eb Change OphandleTable to use a deterministic clock, so we can test it
To test the changes for #577, we need a deterministic way to simulate
the passage of long periods of time. twisted.internet.task.Clock seems,
from my Googling, to be the way to go for this functionality. I changed
a few things so that OphandleTable would use twisted.internet.task.Clock
when testing:

  * WebishServer.__init___ now takes an optional 'clock' parameter,
  * which it passes to the root.Root instance it creates.
  * root.Root.__init__ now takes an optional 'clock' parameter, which it
    passes to the OphandleTable.__init__ method.
  * OphandleTable.__init__ now takes an optional 'clock' parameter. If
    it is provided, and it isn't None, its callLater method will be used
    to schedule ophandle expirations (as opposed to using
    reactor.callLater, which is what OphandleTable does normally).
  * The WebMixin object in test_web.py now sets a self.clock parameter,
    which is a twisted.internet.task.Clock that it feeds to the 
    WebishServer it creates. 

Tests using the WebMixin can control the passage of time in
OphandleTable by accessing self.clock.
2010-02-20 13:07:13 -08:00
Kevan Carstensen
bcdc78bd44 Add tests for the ophandle expiration behavior in #577 2010-02-20 17:04:55 -08:00
Kevan Carstensen
e9b59a4949 Increase ophandle expiration times, per #577 2010-02-20 17:05:12 -08:00
david-sarah
85a50feeaa More cleanups to test_cli using new utilities for reading and writing files. 2010-02-05 17:38:55 -08:00
david-sarah
c984a09fe7 Fix race conditions and missing callback in allmydata.test.test_cli.Cp.test_copy_using_filecap, add utilities for one-liner reading and writing of files, and fix cases in test_cli where files were not being closed after writing. 2010-02-05 17:37:27 -08:00
Kevan Carstensen
c00a62a2e7 Add tests for #939 2010-02-11 22:21:37 -08:00
Kevan Carstensen
63c8c65747 Alter CLI utilities to handle nonexistent aliases better 2010-02-10 18:43:18 -08:00
Brian Warner
4040b1000b web/storage.py: display total-seen on the last-complete-cycle line. For #940. 2010-02-07 16:20:10 -08:00
Brian Warner
880f824103 code coverage: replace figleaf with coverage.py, should work on py2.6 now.
It still lacks the right HTML report (the builtin report is very pretty, but
lacks the "lines uncovered" numbers that I want), and the half-finished
delta-from-last-run measurements.
2010-02-03 08:54:21 -08:00
Zooko O'Whielacronx
3e4342ecb3 immutable: downloader accepts notifications of buckets even if those notifications arrive after he has begun downloading shares.
This can be useful if one of the ones that he has already begun downloading fails. See #287 for discussion. This fixes part of #287 which part was a regression caused by #928, namely this fixes fail-over in case a share is corrupted (or the server returns an error or disconnects). This does not fix the related issue mentioned in #287 if a server hangs and doesn't reply to requests for blocks.
2010-01-31 22:16:10 -08:00