Commit Graph

34 Commits

Author SHA1 Message Date
Ross Patterson
95f2d53f92 chore(refs #3455) Address feedback, BBB comments
https://github.com/tahoe-lafs/tahoe-lafs/pull/845#issuecomment-704469561
2020-10-07 12:28:41 -07:00
Ross Patterson
362c121426 feat(py3): Convert unicode-only modules to str
Modules that reference `unicode` but do *not* reference `str` can safely be converted to
use `str` in a way that's closest to the way it should be done under Python 3 but that
is still Python 2 compatible [per
`python-future`](https://python-future.org/compatible_idioms.html?highlight=unicode#unicode).

This change results in 4 additional tests passing under Python 3 that weren't before,
one previous test error is now a failure, and more coverage in a few modules.  Here's
the diff of the output from running all tests under Python 3 before these changes and
after.  I've elided the irrelevant changes (time stamps, object ids, etc.):

```diff
--- .tox/make-test-py3-all-old.log	2020-09-27 20:56:55.761691130 -0700
+++ .tox/make-test-py3-all-new.log	2020-09-27 20:58:16.242075678 -0700
@@ -1,6 +1,6 @@
...
@@ -4218,7 +4218,7 @@
 [ERROR]
 (#.### secs)
 allmydata.test.mutable.test_version.Version.test_download_version ... Traceback (most recent call last):
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/mutable/test_version.py", line 274, in test_download_version
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/mutable/test_version.py", line 279, in test_download_version
     d = self.publish_multiple()
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/mutable/util.py", line 372, in publish_multiple
     self._nodemaker = make_nodemaker(self._storage)
@@ -4438,40 +4438,26 @@
 allmydata.test.test_abbreviate.Abbreviate.test_time ... [OK]
 (#.### secs)
 allmydata.test.test_auth.AccountFileCheckerKeyTests.test_authenticated ... Traceback (most recent call last):
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_auth.py", line 42, in setUp
-    abspath = abspath_expanduser_unicode(unicode(self.account_file.path))
-builtins.NameError: name 'unicode' is not defined
+Failure: twisted.cred.error.UnauthorizedLogin:
 [ERROR]
 (#.### secs)
 allmydata.test.test_auth.AccountFileCheckerKeyTests.test_missing_signature ... Traceback (most recent call last):
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_auth.py", line 42, in setUp
-    abspath = abspath_expanduser_unicode(unicode(self.account_file.path))
-builtins.NameError: name 'unicode' is not defined
-[ERROR]
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/trial/_asynctest.py", line 75, in _eb
+    raise self.failureException(output)
+twisted.trial.unittest.FailTest:
+Expected: (<class 'twisted.conch.error.ValidPublicKey'>,)
+Got:
+[Failure instance: Traceback (failure with no frames): <class 'twisted.cred.error.UnauthorizedLogin'>:
+]
+[FAILURE]
 (#.### secs)
-allmydata.test.test_auth.AccountFileCheckerKeyTests.test_password_auth_user ... Traceback (most recent call last):
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_auth.py", line 42, in setUp
-    abspath = abspath_expanduser_unicode(unicode(self.account_file.path))
-builtins.NameError: name 'unicode' is not defined
-[ERROR]
+allmydata.test.test_auth.AccountFileCheckerKeyTests.test_password_auth_user ... [OK]
 (#.### secs)
-allmydata.test.test_auth.AccountFileCheckerKeyTests.test_unknown_user ... Traceback (most recent call last):
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_auth.py", line 42, in setUp
-    abspath = abspath_expanduser_unicode(unicode(self.account_file.path))
-builtins.NameError: name 'unicode' is not defined
-[ERROR]
+allmydata.test.test_auth.AccountFileCheckerKeyTests.test_unknown_user ... [OK]
 (#.### secs)
-allmydata.test.test_auth.AccountFileCheckerKeyTests.test_unrecognized_key ... Traceback (most recent call last):
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_auth.py", line 42, in setUp
-    abspath = abspath_expanduser_unicode(unicode(self.account_file.path))
-builtins.NameError: name 'unicode' is not defined
-[ERROR]
+allmydata.test.test_auth.AccountFileCheckerKeyTests.test_unrecognized_key ... [OK]
 (#.### secs)
-allmydata.test.test_auth.AccountFileCheckerKeyTests.test_wrong_signature ... Traceback (most recent call last):
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_auth.py", line 42, in setUp
-    abspath = abspath_expanduser_unicode(unicode(self.account_file.path))
-builtins.NameError: name 'unicode' is not defined
-[ERROR]
+allmydata.test.test_auth.AccountFileCheckerKeyTests.test_wrong_signature ... [OK]
 (#.### secs)
 allmydata.test.test_backupdb.BackupDB.test_basic ... [OK]
 (#.### secs)
@@ -4615,7 +4601,7 @@
 src/allmydata/crypto/util.py                            12      2      4      2    75%   13, 32, 12->13, 30->32
 src/allmydata/deep_stats.py                             83     63     26      0    18%   27-52, 56-58, 62-82, 86-91, 94, 97, 103-114, 117-121, 125-131, 135
 src/allmydata/dirnode.py                               525    420    178      0    15%   70-103, 112-116, 119-135, 140-143, 146-160, 165-173, 176-177, 180-205, 208-217, 223-229, 248-286, 293-299, 302, 310, 315, 318-324, 327-332, 336-340, 344-346, 355-406, 410, 413, 416, 419, 422, 425, 428, 431-433, 436, 439, 442, 445, 448-450, 453, 457, 459, 464, 469-472, 475-478, 481-484, 489-492, 498-501, 504-507, 510-518, 530-532, 539-555, 558-566, 570-589, 600-610, 613-620, 628-641, 646-652, 657-678, 693-714, 752-761, 765-770, 774-812, 819-820, 825, 828, 831, 836-839, 842-849, 852-853, 862-877, 880-881, 884-891, 894, 897-899
-src/allmydata/frontends/auth.py                        100     71     28      0    26%   21-22, 30-48, 51, 54-56, 59-70, 80-87, 100-110, 117-118, 121, 124-142, 147-150, 156-159
+src/allmydata/frontends/auth.py                        100     52     28      4    47%   21-22, 38, 41-44, 51, 54-56, 65-70, 80-87, 106-108, 117-118, 121, 124-142, 147-150, 156-159, 37->38, 40->41, 59->65, 101->106
 src/allmydata/frontends/ftpd.py                        255    254     84      0     1%   4-337
 src/allmydata/frontends/sftpd.py                      1211   1208    488      0     1%   4-2014
 src/allmydata/hashtree.py                              174    135     72      1    16%   59, 75-78, 106-108, 114-117, 123-126, 132-136, 142-149, 152-162, 165-169, 172, 175, 180, 183, 186, 218-232, 259-262, 295-306, 320-323, 326-331, 384-484, 58->59
@@ -4653,7 +4639,7 @@
 src/allmydata/scripts/admin.py                          51     31      2      0    38%   9-14, 17-21, 25, 28, 31-37, 40-46, 56-57, 59, 61-66, 74-78
 src/allmydata/scripts/backupdb.py                      146     91     14      1    36%   84-91, 94-96, 99, 103, 106, 111-114, 117-119, 122, 125, 128, 176-221, 231-242, 245-263, 266-272, 308-324, 327-333, 336-341, 306->308
 src/allmydata/scripts/cli.py                           259    124     46      6    46%   25-49, 69-72, 79-81, 103, 142-146, 175, 221-222, 258, 265-266, 284-285, 330-331, 338-341, 346-355, 361-362, 366-373, 388, 405, 417, 432, 449, 479-481, 484-486, 489-491, 494-496, 499-501, 504-515, 518-520, 523-525, 528-530, 533, 536-538, 541-543, 546-548, 551-553, 556-558, 561-563, 566-568, 571-573, 576-577, 60->exit, 61->exit, 174->175, 180->exit, 181->exit, 219->221
-src/allmydata/scripts/common.py                        153     74     60      4    48%   64, 82, 88, 100, 114-126, 130-152, 159-163, 168-169, 172, 177, 191-236, 240-241, 47->49, 63->64, 79->82, 87->88
+src/allmydata/scripts/common.py                        154     74     60      4    49%   69, 87, 93, 105, 119-131, 135-157, 164-168, 173-174, 177, 182, 196-241, 245-246, 52->54, 68->69, 84->87, 92->93
 src/allmydata/scripts/common_http.py                    77     58     20      0    20%   15-30, 34-36, 38, 42-83, 87, 90, 94-96, 101
 src/allmydata/scripts/create_node.py                   302    185    114      8    30%   24, 61-96, 99-111, 114-128, 136-139, 169-174, 191-194, 205-208, 224-229, 235, 242, 256-278, 289-292, 295-298, 329, 339, 347-380, 385-445, 448-450, 455-477, 223->224, 234->235, 241->242, 252->256, 288->289, 294->295, 328->329, 338->339
 src/allmydata/scripts/debug.py                         719    638    202      0     9%   14, 31-32, 35-49, 52-60, 63-142, 146-154, 157-164, 168-217, 220-304, 307-401, 407, 417, 437-465, 468-485, 488-602, 606, 609-611, 637-648, 653-656, 659, 683-689, 692-810, 813-842, 845-848, 851-865, 869, 888, 891-940, 946, 949-950, 957, 960-961, 967-972, 984-985, 999-1000, 1003-1004, 1020-1021, 1025-1031, 1046-1050
@@ -4661,10 +4647,10 @@
 src/allmydata/scripts/run_common.py                    135     18     24      6    85%   37, 41-46, 59-60, 149, 158, 192-193, 216-220, 226-227, 55->62, 135->exit, 135->exit, 148->149, 191->192, 225->226
 src/allmydata/scripts/runner.py                        138     53     42     11    56%   84-85, 91, 97-99, 104, 114, 123-132, 140, 146, 149-160, 174-181, 186, 189-190, 204-232, 248, 255, 31->36, 103->104, 113->114, 139->140, 145->146, 147->149, 185->186, 188->189, 202->204, 247->248, 254->255
 src/allmydata/scripts/slow_operation.py                 69     56     22      0    14%   15-44, 47-52, 55-61, 64-83
-src/allmydata/scripts/stats_gatherer.py                 41     25     10      0    31%   20-25, 62-86
+src/allmydata/scripts/stats_gatherer.py                 42     25     10      0    33%   25-30, 67-91
 src/allmydata/scripts/tahoe_add_alias.py               106     91     30      0    11%   20-32, 35-59, 63-98, 102-111, 115-144
 src/allmydata/scripts/tahoe_backup.py                  331    267     85      0    15%   20-35, 38-51, 54-58, 71-73, 76-152, 155-157, 160-161, 164-174, 178-209, 212-242, 246-274, 278-279, 287-311, 322-331, 336, 339, 342-351, 356, 359, 362-367, 372-374, 379, 384, 389, 398, 417-425, 428, 431-461, 469-480, 483-486, 500-504, 511-512, 525, 538-542, 545-549, 552-555, 558-561, 564, 571, 578, 586-594
-src/allmydata/scripts/tahoe_check.py                   263    235    121      0     7%   15, 20-100, 103-112, 120-129, 132-167, 170-173, 179-192, 195-256, 259-270, 277-323, 327-336, 339
+src/allmydata/scripts/tahoe_check.py                   264    235    121      0     8%   20, 25-105, 108-117, 125-134, 137-172, 175-178, 184-197, 200-261, 264-275, 282-328, 332-341, 344
 src/allmydata/scripts/tahoe_cp.py                      602    503    226      0    12%   22, 26, 30-31, 34-37, 40-41, 44-47, 50-53, 56-60, 63-70, 75-77, 80, 83, 86, 90-91, 94, 98-99, 102, 106-111, 114, 117-134, 138-142, 145-159, 162-172, 175-177, 180, 185-189, 192, 195-197, 200-203, 206, 210-214, 218-223, 230-233, 236, 239-253, 256-263, 266-297, 303, 307-309, 316, 320-323, 326-333, 336-350, 354-358, 361-397, 403-413, 416-433, 436-437, 440-454, 465-496, 504-580, 583, 589-630, 636-689, 693-698, 701-703, 706-719, 723-762, 765-775, 778-806, 810-818, 821-838, 842, 845-857, 862-863, 867
 src/allmydata/scripts/tahoe_get.py                      37     32     12      0    10%   9-45
 src/allmydata/scripts/tahoe_invite.py                   59     41      8      0    27%   27-31, 36-71, 76-101
@@ -4679,7 +4665,7 @@
 src/allmydata/scripts/tahoe_stop.py                     60     47     10      0    19%   16, 24-84
 src/allmydata/scripts/tahoe_unlink.py                   28     23      6      0    15%   12-40
 src/allmydata/scripts/tahoe_webopen.py                  27     24     12      0     8%   7-31
-src/allmydata/stats.py                                 242    156     54      3    33%   28-34, 37-40, 43-47, 50-64, 67-72, 101, 104-110, 113-125, 144-146, 154-155, 160-163, 169-174, 178-187, 191, 200-207, 210, 213-219, 222-228, 232-234, 237, 241, 246-250, 253, 256-257, 263-278, 281-285, 288-293, 299-325, 100->101, 143->144, 153->154
+src/allmydata/stats.py                                 242    156     54      3    33%   29-35, 38-41, 44-48, 51-65, 68-73, 102, 105-111, 114-126, 145-147, 155-156, 161-164, 170-175, 179-188, 192, 201-208, 211, 214-220, 223-229, 233-235, 238, 242, 247-251, 254, 257-258, 264-279, 282-286, 289-294, 300-326, 101->102, 144->145, 154->155
 src/allmydata/storage/common.py                         24      2      4      2    86%   11, 28, 10->11, 36->39
 src/allmydata/storage/crawler.py                       222    125     64      6    37%   16, 90, 111-113, 148-178, 192-193, 231, 244, 251, 275-312, 315-363, 377-384, 393, 416, 428, 445, 453, 488-492, 495-508, 13->16, 89->90, 96->99, 228->231, 248->251, 268->271
 src/allmydata/storage/expirer.py                       240    183     81      2    21%   9, 74-79, 119, 122, 125-167, 171-233, 236-253, 256-261, 264-266, 269-274, 280-284, 288-322, 388-435, 7->9, 71->74
@@ -4748,7 +4734,7 @@
 src/allmydata/windows/fixups.py                        133    133     54      0     0%   1-237
 src/allmydata/windows/registry.py                       42     42     12      0     0%   1-77
 ------------------------------------------------------------------------------------------------
-TOTAL                                                27427  20411   8234    294    22%
+TOTAL                                                27430  20392   8234    298    22%

 18 files skipped due to complete coverage.
 + '[' '!' -z 1 ']'
```

Trac: refs #3448, https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3448
2020-09-30 08:47:02 -07:00
Itamar Turner-Trauring
e8743a607f Fix failing tests. 2020-09-16 14:37:16 -04:00
Jean-Paul Calderone
628c7e7c5f remove the custom timeouts 2019-05-08 18:39:26 -06:00
heartsucker
a087a5be72
removed tuple unpacking in function definitons for python3 compatibility 2019-04-18 13:37:18 +02:00
heartsucker
0750dbac38
updated all python files to use pep-3110 exception syntax for python3 compatibility 2019-04-04 11:57:58 +02:00
Brian Warner
2ec3791411 test_deepcheck: replace t.w.client.getPage with treq 2017-07-12 17:10:46 -07:00
meejah
031a88c25f simplejson -> json 2017-02-23 15:21:36 -07:00
Brian Warner
7193bff48b tests: use shared run_cli()/do_cli()
A couple of test classes which defined their own flavors were changed to
use the common one.
2016-09-09 15:41:00 -07:00
Daira Hopwood
0ef5939477 Remove 'needs-rebalancing' and add 'count-happiness' to checker reports; repair tests. fixes #1784, #2105
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-03-20 16:13:57 +00:00
Brian Warner
5874a7d3a7 bin/tahoe: clean up global-vs-subcommand arguments like --node-directory
The new rules for "bin/tahoe ARG1.. SUBCOMMAND ARG2.." arg:

* --node-directory is only accepted in ARG1, not ARG2
* create-*/start/stop/restart accept --basedir in ARG2, or an explicit
  basedir argument
* only one of --node-directory/--basedir/explicit-basedir is accepted
* --quiet/--version is only accepted in ARG1, not ARG2

Closes #166
2013-04-09 19:11:32 +01:00
Brian Warner
da9ac55294 CheckResults.get_servers_responding() now returns IServers
Remove temporary get_new_servers_responding().
2012-06-02 11:39:12 -07:00
Brian Warner
957a5315aa CheckResults.get_sharemap() now returns IServers
Remove temporary get_new_sharemap().
2012-06-02 11:39:11 -07:00
Brian Warner
8daacbcf69 CheckResults: replace get_data() with as_dict(), use getters in web status 2012-06-02 11:39:10 -07:00
Brian Warner
4867dca3f0 use the new CheckResult getters almost everywhere
The remaining get_data() calls are either in
web.check_results.json_check_results(), or functioning as repr()s in
various unit test failure cases.
2012-06-02 11:39:10 -07:00
Kevan Carstensen
47eab03c6c test: fix assorted tests broken by MDMF changes 2011-08-06 17:44:59 -07:00
Brian Warner
ffd296fc5a Refactor StorageFarmBroker handling of servers
Pass around IServer instance instead of (peerid, rref) tuple. Replace
"descriptor" with "server". Other replacements:

 get_all_servers -> get_connected_servers/get_known_servers
 get_servers_for_index -> get_servers_for_psi (now returns IServers)

This change still needs to be pushed further down: lots of code is now
getting the IServer and then distributing (peerid, rref) internally.
Instead, it ought to distribute the IServer internally and delay
extracting a serverid or rref until the last moment.

no_network.py was updated to retain parallelism.
2011-02-20 17:58:04 -08:00
Zooko O'Whielacronx
1dbfcf753d immutable: extend the tests to check that the shares that got uploaded really do make a sufficiently Happy distribution
This patch also renames some instances of "find_shares()" to "find_all_shares()" and other instances to "find_uri_shares()" as appropriate -- the conflation between those names confused me at first when writing these tests.
2010-07-18 21:50:47 -07:00
david-sarah
e6aee33bb7 directories: add DIR2-LIT directories to test_deepcheck.py (#948) 2010-02-23 23:54:33 -08:00
Brian Warner
731d15e56f hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900. 2010-01-14 14:15:29 -08:00
Brian Warner
cf65cc2ae3 replace dirnode.create_empty_directory() with create_subdirectory(), which
takes an initial_children= argument
2009-10-12 19:15:20 -07:00
Brian Warner
304aadd4f7 dirnode.set_children: take a dict, not a list 2009-10-12 17:24:40 -07:00
Brian Warner
e2ffc3dc03 dirnode.set_uri/set_children: change signature to take writecap+readcap
instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 16:51:26 -07:00
Brian Warner
3ee740628a replace Client.create_empty_dirnode() with create_dirnode(), in anticipation
of adding initial_children= argument.

Includes stubbed-out initial_children= support.
2009-10-12 15:45:06 -07:00
Brian Warner
0d5dc51617 Overhaul IFilesystemNode handling, to simplify tests and use POLA internally.
* stop using IURI as an adapter
* pass cap strings around instead of URI instances
* move filenode/dirnode creation duties from Client to new NodeMaker class
* move other Client duties to KeyGenerator, SecretHolder, History classes
* stop passing Client reference to dirnode/filenode constructors
  - pass less-powerful references instead, like StorageBroker or Uploader
* always create DirectoryNodes by wrapping a filenode (mutable for now)
* remove some specialized mock classes from unit tests

Detailed list of changes (done one at a time, then merged together)

always pass a string to create_node_from_uri(), not an IURI instance
always pass a string to IFilesystemNode constructors, not an IURI instance
stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri()
client.py: move SecretHolder code out to a separate class
test_web.py: hush pyflakes
client.py: move NodeMaker functionality out into a separate object
LiteralFileNode: stop storing a Client reference
immutable Checker: remove Client reference, it only needs a SecretHolder
immutable Upload: remove Client reference, leave SecretHolder and StorageBroker
immutable Repairer: replace Client reference with StorageBroker and SecretHolder
immutable FileNode: remove Client reference
mutable.Publish: stop passing Client
mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference
MutableChecker: reference StorageBroker and History directly, not through Client
mutable.FileNode: removed unused indirection to checker classes
mutable.FileNode: remove Client reference
client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker
move create_mutable_file() into NodeMaker
test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests.
test_mutable.py: clean up basedir names
client.py: move create_empty_dirnode() into NodeMaker
dirnode.py: get rid of DirectoryNode.create
remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match
stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker
remove Client from NodeMaker
move helper status into History, pass History to web.Status instead of Client
test_mutable.py: fix minor typo
2009-08-15 04:28:46 -07:00
Zooko O'Whielacronx
6a93caf7a3 tests: bump up a timeout that expired on Zandr's box 2009-06-13 12:57:03 -07:00
Zooko O'Whielacronx
69a3b23b34 test: multiple by 10 or so all timeouts that Zandr's ARM box just overran 2009-06-10 05:56:39 -07:00
Zooko O'Whielacronx
63c8f0c424 tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel 2009-06-04 20:14:44 -07:00
Brian Warner
2b525a42d3 test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly 2009-03-15 17:51:34 -07:00
Brian Warner
6e57576f2e dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237. 2009-03-13 16:31:35 -07:00
Brian Warner
fd4ceb6a87 webapi: modify streaming deep-manifest/deep-checker to emit an ERROR: line if they encounter an unrecoverable+untraversable directory. For #590. 2009-02-24 23:13:35 -07:00
Brian Warner
51e3baeefc test_deepcheck: switch deep-check tests to use no-network too. This cuts the runtime down by about 50% 2009-02-24 21:04:57 -07:00
Brian Warner
6cf4096156 test_deepcheck: convert MutableChecker to no-network GridTest 2009-02-24 20:00:10 -07:00
Brian Warner
b398a69f14 test_system: split off checker tests to test_deepcheck.py, this file is too big 2009-02-18 15:42:34 -07:00