Merge remote-tracking branch 'origin/master' into 3657.util-python-3

.github/workflows/ci.yml

@@ -20,8 +20,10 @@ jobs:
         os:
           - macos-latest
           - windows-latest
+          - ubuntu-latest
         python-version:
           - 2.7
+          - 3.6
 
     steps:
 

docs/Makefile

@@ -214,3 +214,7 @@ pseudoxml:
 	$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
 	@echo
 	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
+
+.PHONY: livehtml
+livehtml:
+	sphinx-autobuild -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html

docs/proposed/http-storage-node-protocol.rst

@@ -13,6 +13,61 @@ Specifically, it should be possible to implement a Tahoe-LAFS storage server wit
 The Tahoe-LAFS client will also need to change but it is not expected that it will be noticably simplified by this change
 (though this may be the first step towards simplifying it).
 
+Motivation
+----------
+
+Foolscap
+~~~~~~~~
+
+Foolscap is a remote method invocation protocol with several distinctive features.
+At its core it allows separate processes to refer each other's objects and methods using a capability-based model.
+This allows for extremely fine-grained access control in a system that remains highly securable without becoming overwhelmingly complicated.
+Supporting this is a flexible and extensible serialization system which allows data to be exchanged between processes in carefully controlled ways.
+
+Tahoe-LAFS avails itself of only a small portion of these features.
+A Tahoe-LAFS storage server typically only exposes one object with a fixed set of methods to clients.
+A Tahoe-LAFS introducer node does roughly the same.
+Tahoe-LAFS exchanges simple data structures that have many common, standard serialized representations.
+
+In exchange for this slight use of Foolscap's sophisticated mechanisms,
+Tahoe-LAFS pays a substantial price:
+
+* Foolscap is implemented only for Python.
+  Tahoe-LAFS is thus limited to being implemented only in Python.
+* There is only one Python implementation of Foolscap.
+  The implementation is therefore the de facto standard and understanding of the protocol often relies on understanding that implementation.
+* The Foolscap developer community is very small.
+  The implementation therefore advances very little and some non-trivial part of the maintenance cost falls on the Tahoe-LAFS project.
+* The extensible serialization system imposes substantial complexity compared to the simple data structures Tahoe-LAFS actually exchanges.
+
+HTTP
+~~~~
+
+HTTP is a request/response protocol that has become the lingua franca of the internet.
+Combined with the principles of Representational State Transfer (REST) it is widely employed to create, update, and delete data in collections on the internet.
+HTTP itself provides only modest functionality in comparison to Foolscap.
+However its simplicity and widespread use have led to a diverse and almost overwhelming ecosystem of libraries, frameworks, toolkits, and so on.
+
+By adopting HTTP in place of Foolscap Tahoe-LAFS can realize the following concrete benefits:
+
+* Practically every language or runtime has an HTTP protocol implementation (or a dozen of them) available.
+  This change paves the way for new Tahoe-LAFS implementations using tools better suited for certain situations
+  (mobile client implementations, high-performance server implementations, easily distributed desktop clients, etc).
+* The simplicity of and vast quantity of resources about HTTP make it a very easy protocol to learn and use.
+  This change reduces the barrier to entry for developers to contribute improvements to Tahoe-LAFS's network interactions.
+* For any given language there is very likely an HTTP implementation with a large and active developer community.
+  Tahoe-LAFS can therefore benefit from the large effort being put into making better libraries for using HTTP.
+* One of the core features of HTTP is the mundane transfer of bulk data and implementions are often capable of doing this with extreme efficiency.
+  The alignment of this core feature with a core activity of Tahoe-LAFS of transferring bulk data means that a substantial barrier to improved Tahoe-LAFS runtime performance will be eliminated.
+
+TLS
+~~~
+
+The Foolscap-based protocol provides *some* of Tahoe-LAFS's confidentiality, integrity, and authentication properties by leveraging TLS.
+An HTTP-based protocol can make use of TLS in largely the same way to provide the same properties.
+Provision of these properties *is* dependant on implementers following Great Black Swamp's rules for x509 certificate validation
+(rather than the standard "web" rules for validation).
+
 Requirements
 ------------
 

newsfragments/3663.other

@@ -0,0 +1 @@
+You can run `make livehtml` in docs directory to invoke sphinx-autobuild.

newsfragments/3664.documentation

@@ -0,0 +1 @@
+Documentation now has its own towncrier category.

setup.py

@@ -11,6 +11,7 @@ import sys
 # See the docs/about.rst file for licensing information.
 
 import os, subprocess, re
+from io import open
 
 basedir = os.path.dirname(os.path.abspath(__file__))
 
@@ -357,7 +358,7 @@ if version:
 
 setup(name="tahoe-lafs", # also set in __init__.py
       description='secure, decentralized, fault-tolerant file store',
-      long_description=open('README.rst', 'rU').read(),
+      long_description=open('README.rst', 'r', encoding='utf-8').read(),
       author='the Tahoe-LAFS project',
       author_email='tahoe-dev@tahoe-lafs.org',
       url='https://tahoe-lafs.org/',

towncrier.pyproject.toml

@@ -37,6 +37,11 @@
         name = "Configuration Changes"
         showcontent = true
 
+    [[tool.towncrier.type]]
+        directory = "documentation"
+        name = "Documentation Changes"
+        showcontent = true
+
     [[tool.towncrier.type]]
         directory = "removed"
         name = "Removed Features"