262 Commits

Author SHA1 Message Date
Ross Patterson
a89715ebe8 feat(py3): feat(py3): Fix use of deprecated type
Python 3 did away with the unbound method type entirely, they're just functions under
Python 3, and IIUC the unbound type is just an alias for the method type in Python 2.
As such, this approach should preserve the behavior under Python 2 and should work under
Python 3.

With this change, the diff in test output shows one test error converted to a failure,
increases coverage in all the modules that have a coverage change, and reveals the next
porting bug:

```
...
  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
builtins.TypeError: can't concat str to bytes
```

There are no regressions I see under Python 2, so this seems like the right fix to this
particular issue AFAICT.

```diff
--- ../../.tox/make-test-py3-all-old.log	2020-10-04 15:59:59.355692613 -0700
+++ ../../.tox/make-test-py3-all-new.log	2020-10-04 16:59:27.870208496 -0700
@@ -1206,7 +1206,7 @@
     test_location2 ...                                                  [ERROR]
     test_location_auto_and_explicit ...                                 [ERROR]
     test_location_not_set ...                                           [ERROR]
-    test_logdir_is_str ...                                              [ERROR]
+    test_logdir_is_str ...                                               [FAIL]
     test_private_config ...                                             [ERROR]
     test_private_config_missing ...                                        [OK]
     test_private_config_unreadable ...                                  [ERROR]
@@ -2254,7 +2254,7 @@
     yield create_introducer(basedir)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/trial/_synctest.py", line 355, in __exit__
     self._expectedName, reason.getTraceback()),
-twisted.trial.unittest.FailTest: builtins.AttributeError raised instead of ValueError:
+twisted.trial.unittest.FailTest: builtins.TypeError raised instead of ValueError:
  Traceback (most recent call last):
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
     _inlineCallbacks(None, g, status)
@@ -2271,11 +2271,9 @@
     tor_provider,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/introducer/server.py", line 87, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+builtins.TypeError: can't concat str to bytes

 allmydata.test.test_node.IntroducerNotListening.test_port_none_introducer
@@ -2307,6 +2305,30 @@
 ===============================================================================
 [FAIL]
 Traceback (most recent call last):
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_node.py", line 340, in test_logdir_is_str
+    yield client.create_client(basedir)
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
+    result = g.send(result)
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/client.py", line 298, in create_client_from_config
+    storage_broker,
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/client.py", line 676, in __init__
+    node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
+    self.setup_logging()
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 842, in setup_logging
+    foolscap.logging.log.setLogDir(incident_dir.encode(get_filesystem_encoding()))
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_node.py", line 336, in call_setLogDir
+    self.failUnless(isinstance(logdir, str), logdir)
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/trial/_synctest.py", line 397, in assertTrue
+    super(_Assertions, self).assertTrue(condition, msg)
+  File "/usr/lib/python3.6/unittest/case.py", line 682, in assertTrue
+    raise self.failureException(msg)
+twisted.trial.unittest.FailTest: False is not true : b'/home/rpatterson/src/work/sfu/tahoe-lafs/_trial_temp/test_node/test_logdir_is_str/logs/incidents'
+
+allmydata.test.test_node.TestCase.test_logdir_is_str
+===============================================================================
+[FAIL]
+Traceback (most recent call last):
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
     result = g.send(result)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_runner.py", line 192, in test_eliot_destination
@@ -5933,11 +5955,9 @@
     storage_broker,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/client.py", line 676, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+builtins.TypeError: can't concat str to bytes

 allmydata.test.test_client.Basic.test_web_apiauthtoken
 ===============================================================================
@@ -7130,8 +7150,12 @@
 ===============================================================================
 [ERROR]
 Traceback (most recent call last):
-Failure: testtools.testresult.real._StringException: Empty attachments:
-  twisted-log
+Failure: testtools.testresult.real._StringException: twisted-log: {{{
+2020-10-04 23:57:37.636Z [-] Foolscap logging initialized
+2020-10-04 23:57:37.636Z [-] Note to developers: twistd.log does not receive very much.
+2020-10-04 23:57:37.636Z [-] Use 'flogtool tail -c NODEDIR/private/logport.furl' instead
+2020-10-04 23:57:37.636Z [-] and read docs/logging.rst
+}}}

 Traceback (most recent call last):
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
@@ -7144,19 +7168,21 @@
     tor_provider,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/introducer/server.py", line 87, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+TypeError: can't concat str to bytes

 allmydata.test.test_introducer.Node.test_create
 ===============================================================================
 [ERROR]
 Traceback (most recent call last):
-Failure: testtools.testresult.real._StringException: Empty attachments:
-  twisted-log
+Failure: testtools.testresult.real._StringException: twisted-log: {{{
+2020-10-04 23:57:38.284Z [-] Foolscap logging initialized
+2020-10-04 23:57:38.284Z [-] Note to developers: twistd.log does not receive very much.
+2020-10-04 23:57:38.284Z [-] Use 'flogtool tail -c NODEDIR/private/logport.furl' instead
+2020-10-04 23:57:38.284Z [-] and read docs/logging.rst
+}}}

 Traceback (most recent call last):
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
@@ -7169,11 +7195,9 @@
     tor_provider,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/introducer/server.py", line 87, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+TypeError: can't concat str to bytes

 allmydata.test.test_introducer.Node.test_furl
@@ -7540,24 +7564,6 @@
 ===============================================================================
 [ERROR]
 Traceback (most recent call last):
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_node.py", line 340, in test_logdir_is_str
-    yield client.create_client(basedir)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/.tox/py36-coverage/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
-    result = g.send(result)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/client.py", line 298, in create_client_from_config
-    storage_broker,
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/client.py", line 676, in __init__
-    node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
-
-allmydata.test.test_node.TestCase.test_logdir_is_str
-===============================================================================
-[ERROR]
-Traceback (most recent call last):
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/test/test_node.py", line 259, in test_private_config
     config = config_from_string(basedir, "", "")
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 209, in config_from_string
@@ -8446,11 +8452,9 @@
     tor_provider,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/introducer/server.py", line 87, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+builtins.TypeError: can't concat str to bytes

 allmydata.test.test_system.Connections.test_rref
 allmydata.test.test_system.SystemTest.test_filesystem
@@ -8565,11 +8569,9 @@
     tor_provider,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/introducer/server.py", line 87, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+builtins.TypeError: can't concat str to bytes

 allmydata.test.web.test_introducer.IntroducerWeb.test_basic_information
 ===============================================================================
@@ -8587,11 +8589,9 @@
     tor_provider,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/introducer/server.py", line 87, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+builtins.TypeError: can't concat str to bytes

 allmydata.test.web.test_introducer.IntroducerWeb.test_json_front_page
 ===============================================================================
@@ -8609,11 +8609,9 @@
     tor_provider,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/introducer/server.py", line 87, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+builtins.TypeError: can't concat str to bytes

 allmydata.test.web.test_introducer.IntroducerWeb.test_tahoe_css
 ===============================================================================
@@ -8631,11 +8629,9 @@
     tor_provider,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/introducer/server.py", line 87, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+builtins.TypeError: can't concat str to bytes

 allmydata.test.web.test_introducer.IntroducerWeb.test_welcome
 ===============================================================================
@@ -8721,7 +8717,7 @@
 -------------------------------------------------------------------------------
 Ran 1300 tests in ###.###s

-FAILED (skips=42, expectedFailures=1, failures=34, errors=532, successes=707)
+FAILED (skips=42, expectedFailures=1, failures=35, errors=531, successes=707)

 Unknown error
 Traceback (most recent call last):
@@ -8740,11 +8736,9 @@
     storage_broker,
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/client.py", line 676, in __init__
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
-    self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
-    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
-builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 740, in __init__
+    self.config.write_config_file("my_nodeid", b32encode(self.nodeid).lower() + "\n")
+builtins.TypeError: can't concat str to bytes
 + test_exit=1
 + ./.tox/py36-coverage/bin/coverage combine
 + ./.tox/py36-coverage/bin/coverage html
@@ -8775,11 +8769,11 @@
 src/allmydata/immutable/downloader/__init__.py           7      1      2      1    78%   12, 11->12
 src/allmydata/immutable/downloader/common.py            14      1      2      1    88%   12, 11->12
 src/allmydata/immutable/downloader/fetcher.py          147      1     56      1    99%   12, 11->12
-src/allmydata/immutable/downloader/finder.py           143     11     40      5    91%   11, 88-89, 108, 115, 170-173, 231-232, 10->11, 87->88, 107->108, 113->115, 165->exit
+src/allmydata/immutable/downloader/finder.py           143      9     40      5    92%   11, 88-89, 108, 115, 170-173, 10->11, 87->88, 107->108, 113->115, 165->exit
 src/allmydata/immutable/downloader/node.py             282      9     66      9    94%   11, 170-172, 224-235, 271, 10->11, 43->exit, 60->68, 165->169, 169->170, 256->exit, 270->271, 517->exit, 537->exit
 src/allmydata/immutable/downloader/segmentation.py     118      1     22      1    99%   11, 10->11
-src/allmydata/immutable/downloader/share.py            453     23    154     11    94%   11, 210, 239-250, 289-290, 354, 399-400, 430-437, 518, 722-725, 775, 862, 10->11, 209->210, 222->239, 288->289, 352->354, 397->399, 515->518, 660->663, 710->722, 762->exit, 774->775
-src/allmydata/immutable/downloader/status.py           154     12     36      1    93%   11, 65-67, 223, 226, 230, 232, 234, 274, 285, 287, 10->11
+src/allmydata/immutable/downloader/share.py            453     15    154      8    96%   11, 210, 248-250, 354, 430-437, 518, 722-725, 862, 10->11, 209->210, 239->248, 352->354, 515->518, 660->663, 710->722, 762->exit
+src/allmydata/immutable/downloader/status.py           154      9     36      1    95%   11, 223, 226, 230, 232, 234, 274, 285, 287, 10->11
 src/allmydata/immutable/encode.py                      421     13    124     10    95%   114, 197-200, 278-280, 405, 412, 462, 509, 562, 687, 102->104, 113->114, 195->197, 404->405, 411->412, 461->462, 499->509, 505->511, 561->562, 685->687
 src/allmydata/immutable/filenode.py                    196     48     30      5    72%   77-78, 83, 85, 88, 94-101, 104-124, 127-172, 254, 258, 315, 40->42, 224->227, 227->229, 251->254, 257->258
 src/allmydata/immutable/happiness_upload.py            214      1    132      3    99%   15, 13->15, 213->211, 280->279
@@ -8802,7 +8796,7 @@
 src/allmydata/mutable/repairer.py                       57     37     18      0    29%   13, 15, 17, 19, 29-34, 65-71, 74-126, 129-131
 src/allmydata/mutable/retrieve.py                      489    123    120     33    71%   46, 48, 50, 52, 56, 58, 60, 62, 64, 89-90, 133, 186-193, 204-208, 211-212, 224-226, 231, 240, 251, 312, 318, 344-354, 377, 385-386, 399-400, 425-434, 490, 501, 515-516, 529-540, 564-578, 591-592, 629-630, 653-654, 674-675, 681-682, 698, 712-729, 758-760, 765, 772-774, 790-792, 871, 883, 909-910, 919-941, 965-966, 981-994, 999-1005, 129->133, 167->169, 169->171, 201->204, 223->224, 230->231, 237->239, 239->240, 243->247, 249->251, 309->312, 317->318, 376->377, 381->385, 391->394, 396->399, 424->425, 489->490, 499->501, 514->515, 590->591, 628->629, 652->653, 673->674, 677->687, 680->681, 687->694, 694->698, 755->764, 764->765, 868->871, 880->883, 964->965
 src/allmydata/mutable/servermap.py                     612    240    186     26    56%   45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 74, 130-139, 142, 148, 159-161, 175, 177, 183, 186-199, 206, 213, 217-220, 231, 234-238, 243-252, 255-259, 315, 328-350, 358-363, 370-372, 379, 429, 433, 443-447, 495, 498, 506-508, 514-516, 569-570, 603-611, 623-638, 718-721, 732-741, 792, 796, 803-804, 850-851, 872-874, 911-915, 929-945, 961-975, 982-999, 1003-1013, 1043-1045, 1050-1052, 1060-1064, 1069-1070, 1093-1100, 1106-1186, 1214-1215, 1232-1233, 313->315, 427->429, 432->433, 439->443, 459->461, 493->495, 497->498, 504->506, 509->514, 566->569, 597->603, 687->exit, 702->exit, 710->exit, 717->718, 727->732, 759->exit, 791->792, 795->796, 869->872, 1039->1043, 1047->1050, 1059->1060, 1066->1069, 1092->1093, 1213->1214
-src/allmydata/node.py                                  388     82    146     33    76%   120, 132, 190, 241, 243-245, 278, 284, 294-295, 303-306, 315, 320, 339, 341, 361, 393-396, 422, 449, 453, 490, 493, 500, 511-512, 548, 566, 574, 581, 583, 590-591, 601, 612, 629-633, 679, 681, 738-741, 747, 756, 764, 792-805, 808-809, 814-815, 827-846, 189->190, 240->241, 242->243, 277->278, 314->315, 319->320, 338->339, 340->341, 360->361, 391->393, 421->422, 448->449, 451->453, 489->490, 492->493, 499->500, 510->511, 547->548, 565->566, 573->574, 580->581, 582->583, 589->590, 600->601, 611->612, 622->629, 673->679, 680->681, 737->738, 746->747, 763->764, 821->830, 823->821
+src/allmydata/node.py                                  388     66    146     33    80%   120, 132, 190, 241, 243-245, 278, 284, 294-295, 303-306, 315, 320, 339, 341, 361, 393-396, 422, 449, 453, 490, 493, 500, 511-512, 548, 566, 574, 581, 583, 590-591, 601, 612, 629-633, 679, 681, 741, 747, 756, 764, 792-805, 808-809, 814-815, 832, 837, 189->190, 240->241, 242->243, 277->278, 314->315, 319->320, 338->339, 340->341, 360->361, 391->393, 421->422, 448->449, 451->453, 489->490, 492->493, 499->500, 510->511, 547->548, 565->566, 573->574, 580->581, 582->583, 589->590, 600->601, 611->612, 622->629, 673->679, 680->681, 746->747, 763->764, 823->821, 831->832, 835->837
 src/allmydata/nodemaker.py                              97     23     38     10    70%   49, 61, 66, 70, 81, 94, 107-115, 130-138, 141-150, 57->61, 65->66, 69->70, 79->81, 86->95, 90->94, 104->107, 124->exit, 129->130, 129->133
 src/allmydata/scripts/admin.py                          51     20      2      1    60%   9-14, 25, 28, 31-37, 40-46, 57, 59, 61-66, 56->57
 src/allmydata/scripts/backupdb.py                      146     91     14      1    36%   84-91, 94-96, 99, 103, 106, 111-114, 117-119, 122, 125, 128, 176-221, 231-242, 245-263, 266-272, 308-324, 327-333, 336-341, 306->308
@@ -8837,7 +8831,7 @@
 src/allmydata/storage/common.py                         24      1      4      2    89%   11, 10->11, 36->39
 src/allmydata/storage/crawler.py                       222      1     64      3    99%   16, 13->16, 96->99, 496->505
 src/allmydata/storage/expirer.py                       240      1     81      2    99%   9, 7->9, 250->exitp
-src/allmydata/storage/immutable.py                     198      2     48      6    97%   12, 101, 11->12, 100->101, 142->140, 155->160, 187->197, 273->exit
+src/allmydata/storage/immutable.py                     198      1     48      5    98%   12, 11->12, 142->140, 155->160, 187->197, 273->exit
 src/allmydata/storage/lease.py                          35      1      4      1    95%   12, 11->12
 src/allmydata/storage/mutable.py                       289      8     90      6    96%   12, 162, 252, 289, 362-367, 11->12, 160->162, 247->252, 307->311, 354->362, 448->exit
 src/allmydata/storage/server.py                        371      9    120      9    96%   13, 92, 222, 243, 329, 349, 375, 422-423, 10->13, 91->92, 221->222, 241->243, 289->300, 317->329, 325->305, 346->349, 374->375
@@ -8899,7 +8893,7 @@
 src/allmydata/windows/fixups.py                        133    133     54      0     0%   1-237
 src/allmydata/windows/registry.py                       42     42     12      0     0%   1-77
 ------------------------------------------------------------------------------------------------
-TOTAL                                                27477  11798   8244    605    54%
+TOTAL                                                27477  11768   8244    601    54%

 18 files skipped due to complete coverage.
 + '[' '!' -z 1 ']'
```
2020-10-04 20:56:14 -07:00
Ross Patterson
baa36157b6 feat(py3): Update Python internal data model refs
As of Python 2.7, some of [the `func_*` and `im_*` attributes also can be accessed
through their corresponding `__*__` attributes for Python 3
compatibility](https://docs.python.org/2/reference/datamodel.html?highlight=__self__#the-standard-type-hierarchy).
I searched for all such occurrences and this is all that needed changing AFAICT.

Converts 9 test errors to errors with new exceptions and improves Python 3 test coverage
a smidge:

```diff
--- ../../.tox/make-test-py3-all-old.log	2020-10-04 15:16:34.054975263 -0700
+++ ../../.tox/make-test-py3-all-new.log	2020-10-04 15:59:59.355692613 -0700
@@ -2273,9 +2273,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.test_node.IntroducerNotListening.test_port_none_introducer
@@ -5935,9 +5935,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.test_client.Basic.test_web_apiauthtoken
 ===============================================================================
@@ -7146,9 +7146,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.test_introducer.Node.test_create
@@ -7171,9 +7171,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.test_introducer.Node.test_furl
@@ -7550,9 +7550,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.test_node.TestCase.test_logdir_is_str
 ===============================================================================
@@ -8448,9 +8448,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.test_system.Connections.test_rref
 allmydata.test.test_system.SystemTest.test_filesystem
@@ -8567,9 +8567,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.web.test_introducer.IntroducerWeb.test_basic_information
 ===============================================================================
@@ -8589,9 +8589,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.web.test_introducer.IntroducerWeb.test_json_front_page
 ===============================================================================
@@ -8611,9 +8611,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.web.test_introducer.IntroducerWeb.test_tahoe_css
 ===============================================================================
@@ -8633,9 +8633,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'

 allmydata.test.web.test_introducer.IntroducerWeb.test_welcome
 ===============================================================================
@@ -8742,9 +8742,9 @@
     node.Node.__init__(self, config, main_tub, control_tub, i2p_provider, tor_provider)
   File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 734, in __init__
     self.setup_logging()
-  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 824, in setup_logging
-    ob = o.im_self
-builtins.AttributeError: 'function' object has no attribute 'im_self'
+  File "/home/rpatterson/src/work/sfu/tahoe-lafs/src/allmydata/node.py", line 826, in setup_logging
+    newmeth = types.UnboundMethodType(formatTimeTahoeStyle, ob, ob.__class__)
+builtins.AttributeError: module 'types' has no attribute 'UnboundMethodType'
 + test_exit=1
 + ./.tox/py36-coverage/bin/coverage combine
 + ./.tox/py36-coverage/bin/coverage html
@@ -8802,7 +8802,7 @@
 src/allmydata/mutable/repairer.py                       57     37     18      0    29%   13, 15, 17, 19, 29-34, 65-71, 74-126, 129-131
 src/allmydata/mutable/retrieve.py                      489    123    120     33    71%   46, 48, 50, 52, 56, 58, 60, 62, 64, 89-90, 133, 186-193, 204-208, 211-212, 224-226, 231, 240, 251, 312, 318, 344-354, 377, 385-386, 399-400, 425-434, 490, 501, 515-516, 529-540, 564-578, 591-592, 629-630, 653-654, 674-675, 681-682, 698, 712-729, 758-760, 765, 772-774, 790-792, 871, 883, 909-910, 919-941, 965-966, 981-994, 999-1005, 129->133, 167->169, 169->171, 201->204, 223->224, 230->231, 237->239, 239->240, 243->247, 249->251, 309->312, 317->318, 376->377, 381->385, 391->394, 396->399, 424->425, 489->490, 499->501, 514->515, 590->591, 628->629, 652->653, 673->674, 677->687, 680->681, 687->694, 694->698, 755->764, 764->765, 868->871, 880->883, 964->965
 src/allmydata/mutable/servermap.py                     612    240    186     26    56%   45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 74, 130-139, 142, 148, 159-161, 175, 177, 183, 186-199, 206, 213, 217-220, 231, 234-238, 243-252, 255-259, 315, 328-350, 358-363, 370-372, 379, 429, 433, 443-447, 495, 498, 506-508, 514-516, 569-570, 603-611, 623-638, 718-721, 732-741, 792, 796, 803-804, 850-851, 872-874, 911-915, 929-945, 961-975, 982-999, 1003-1013, 1043-1045, 1050-1052, 1060-1064, 1069-1070, 1093-1100, 1106-1186, 1214-1215, 1232-1233, 313->315, 427->429, 432->433, 439->443, 459->461, 493->495, 497->498, 504->506, 509->514, 566->569, 597->603, 687->exit, 702->exit, 710->exit, 717->718, 727->732, 759->exit, 791->792, 795->796, 869->872, 1039->1043, 1047->1050, 1059->1060, 1066->1069, 1092->1093, 1213->1214
-src/allmydata/node.py                                  388     84    146     33    75%   120, 132, 190, 241, 243-245, 278, 284, 294-295, 303-306, 315, 320, 339, 341, 361, 393-396, 422, 449, 453, 490, 493, 500, 511-512, 548, 566, 574, 581, 583, 590-591, 601, 612, 629-633, 679, 681, 738-741, 747, 756, 764, 792-805, 808-809, 814-815, 825-846, 189->190, 240->241, 242->243, 277->278, 314->315, 319->320, 338->339, 340->341, 360->361, 391->393, 421->422, 448->449, 451->453, 489->490, 492->493, 499->500, 510->511, 547->548, 565->566, 573->574, 580->581, 582->583, 589->590, 600->601, 611->612, 622->629, 673->679, 680->681, 737->738, 746->747, 763->764, 821->830, 823->821
+src/allmydata/node.py                                  388     82    146     33    76%   120, 132, 190, 241, 243-245, 278, 284, 294-295, 303-306, 315, 320, 339, 341, 361, 393-396, 422, 449, 453, 490, 493, 500, 511-512, 548, 566, 574, 581, 583, 590-591, 601, 612, 629-633, 679, 681, 738-741, 747, 756, 764, 792-805, 808-809, 814-815, 827-846, 189->190, 240->241, 242->243, 277->278, 314->315, 319->320, 338->339, 340->341, 360->361, 391->393, 421->422, 448->449, 451->453, 489->490, 492->493, 499->500, 510->511, 547->548, 565->566, 573->574, 580->581, 582->583, 589->590, 600->601, 611->612, 622->629, 673->679, 680->681, 737->738, 746->747, 763->764, 821->830, 823->821
 src/allmydata/nodemaker.py                              97     23     38     10    70%   49, 61, 66, 70, 81, 94, 107-115, 130-138, 141-150, 57->61, 65->66, 69->70, 79->81, 86->95, 90->94, 104->107, 124->exit, 129->130, 129->133
 src/allmydata/scripts/admin.py                          51     20      2      1    60%   9-14, 25, 28, 31-37, 40-46, 57, 59, 61-66, 56->57
 src/allmydata/scripts/backupdb.py                      146     91     14      1    36%   84-91, 94-96, 99, 103, 106, 111-114, 117-119, 122, 125, 128, 176-221, 231-242, 245-263, 266-272, 308-324, 327-333, 336-341, 306->308
@@ -8899,7 +8899,7 @@
 src/allmydata/windows/fixups.py                        133    133     54      0     0%   1-237
 src/allmydata/windows/registry.py                       42     42     12      0     0%   1-77
 ------------------------------------------------------------------------------------------------
-TOTAL                                                27477  11800   8244    605    54%
+TOTAL                                                27477  11798   8244    605    54%

 18 files skipped due to complete coverage.
 + '[' '!' -z 1 ']'
```
2020-10-04 16:01:32 -07:00
Ross Patterson
447881a0e0 feat(py3): Cleanup redundant string cast
I confirmed that `add_version(...)` itself calls `str(...)` on the argument that
`things_version` is passed in to under both the Python 2.7 and Python 3.6 version of the
library so this is unnecessary here.

This results in an empty diff in py3 tests output.
2020-10-04 15:57:01 -07:00
Itamar Turner-Trauring
7d8320b843 Python 3 fixes. 2020-09-16 11:13:23 -04:00
Jason R. Coombs
d74d7e733c Merge branch 'master' into 3394.py38-tests 2020-09-09 21:42:32 -04:00
Itamar Turner-Trauring
c3494f1356 Enough changes to make allmydata.test.test_storage run on Python 3.
Still lots of failures, of course.
2020-08-27 15:19:49 -04:00
Itamar Turner-Trauring
a758f32edf Try to make test_storage import on Python 3. 2020-08-26 10:53:02 -04:00
Jason R. Coombs
8b553d2045 Add Python 3 compatibility for code paths relevant to 'tahoe --version'. 2020-08-21 19:55:34 -04:00
Jean-Paul Calderone
a47463e032 Pass _Config instead of a smaller dict to get_client_resource 2019-08-19 11:21:03 -04:00
Jean-Paul Calderone
21bf7fc25c Merge remote-tracking branch 'origin/master' into integration/storage-economics 2019-08-16 15:39:31 -04:00
Jean-Paul Calderone
13409a2449 Move the remaining package checking machinery to its own module. 2019-08-13 15:11:01 -04:00
Jean-Paul Calderone
6068b6c1b2 don't reach through the tahoe-lafs config object 2019-08-03 06:19:01 -04:00
Jean-Paul Calderone
756c21c251 actually provide validating client-config-from-string function 2019-08-03 06:19:01 -04:00
Jean-Paul Calderone
fb4c5cf91f Allow for dynamic configuration validation rules 2019-08-03 05:34:21 -04:00
Jean-Paul Calderone
edba0747a3 Use listenOnUnused instead of allocate_tcp_port in create_log_tub 2019-04-16 11:32:27 -04:00
Jean-Paul Calderone
6ea1684995 use listenOnUnused instead of allocate_tcp_port in create_control_tub 2019-04-16 11:26:52 -04:00
heartsucker
129ef22185
updated instances of octal literals to use the format 0o123 for python3 compatibility 2019-03-29 11:08:44 +01:00
meejah
2b5d3be3c8 convert XXX comment to ticket 2018-09-10 21:58:28 -06:00
meejah
9a8ef9512e clarify comments 2018-09-10 21:58:28 -06:00
meejah
f488b79c71 self._portnumfile -> config.portnum_fname 2018-09-10 21:58:28 -06:00
meejah
a5287add80 clarify comments 2018-09-10 21:58:27 -06:00
meejah
714b0887dc more docstrings 2018-09-10 21:58:27 -06:00
meejah
767f4ddd5d reorder args to match other method 2018-09-10 21:58:27 -06:00
meejah
045af64c2b dead code 2018-09-10 21:58:27 -06:00
meejah
edc50f655b get rid of is_tub_listening 2018-09-10 21:58:27 -06:00
meejah
955d7abfa3 move validation code to parser-helper 2018-09-10 21:58:27 -06:00
meejah
9e34d15b90 simplify _tub_portlocation helper 2018-09-10 21:58:27 -06:00
meejah
053b494054 comment 2018-09-10 21:58:27 -06:00
meejah
2d044e1324 remove unused method 2018-09-10 21:58:27 -06:00
meejah
08e0c3b7e2 get rid of 'add_service' (just an alias to setServiceParent anyway) 2018-09-10 21:58:27 -06:00
meejah
7685fb34cd pull 'basedir' entirely into _Config
Put all config-related methods into _Config; change
code to ask config for paths instead of using basedir;
add some better docstrings
2018-09-10 21:58:27 -06:00
meejah
26007f363b pull 'StorageFarmBroker' out of __init__
This means also pulling out introducer-clients and some
related utility methods
2018-09-10 21:58:27 -06:00
meejah
739aaa3ef9 put create() methods in i2p_, tor_provider
Also Provider -> _Provider, improve docs and update tests
2018-09-10 21:58:27 -06:00
meejah
903d4afaa4 handle None for providers 2018-09-10 21:58:27 -06:00
meejah
793827f8a6 put providers back in service tree 2018-09-10 21:58:27 -06:00
meejah
c2946cc2e3 remove unused code 2018-09-10 21:58:27 -06:00
meejah
f0c3db0e5a pyflakes 2018-09-10 21:58:27 -06:00
meejah
0e51bb183e fix more things 2018-09-10 21:58:27 -06:00
meejah
279bd814fc remember info for init_client_storage_broker temporarily 2018-09-10 21:58:27 -06:00
meejah
43d857a0bd more refactor + fix some node tests 2018-09-10 21:58:27 -06:00
meejah
71484b4a12 upgrade create_introducer 2018-09-10 21:58:27 -06:00
meejah
d1a83e9be0 more tests work 2018-09-10 21:58:27 -06:00
meejah
329ef1256a refactor create_client to be async (works to run, some unit-test fails still) 2018-09-10 21:58:27 -06:00
meejah
0f22b9bad0 fixups after rebase 2018-08-25 02:23:58 -06:00
meejah
536ccf8b6d better 'file not found' handling 2018-08-25 02:10:44 -06:00
meejah
c0772cdd5f improve docstring 2018-08-25 02:10:44 -06:00
meejah
37d4b59a39 modern syntax 2018-08-25 02:10:44 -06:00
meejah
a432fc35da docstring improvements 2018-08-25 02:10:44 -06:00
meejah
d544284f92 introduce create_node_dir 2018-08-25 02:10:44 -06:00
meejah
f7f3c54f93 dead code 2018-08-25 02:10:44 -06:00