From 9d64c881998b0cdb6962fd8abe18b86d73e31c39 Mon Sep 17 00:00:00 2001 From: Jean-Paul Calderone Date: Mon, 22 Mar 2021 20:28:18 -0400 Subject: [PATCH 1/8] news fragment --- newsfragments/3645.minor | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 newsfragments/3645.minor diff --git a/newsfragments/3645.minor b/newsfragments/3645.minor new file mode 100644 index 000000000..e69de29bb From 31c838d5848eec9a65e63a63bb01b9a24446cb42 Mon Sep 17 00:00:00 2001 From: Jean-Paul Calderone Date: Mon, 22 Mar 2021 20:28:24 -0400 Subject: [PATCH 2/8] the expanded motivation --- docs/proposed/http-storage-node-protocol.rst | 48 ++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/docs/proposed/http-storage-node-protocol.rst b/docs/proposed/http-storage-node-protocol.rst index 36c718c56..17da870b1 100644 --- a/docs/proposed/http-storage-node-protocol.rst +++ b/docs/proposed/http-storage-node-protocol.rst @@ -13,6 +13,54 @@ Specifically, it should be possible to implement a Tahoe-LAFS storage server wit The Tahoe-LAFS client will also need to change but it is not expected that it will be noticably simplified by this change (though this may be the first step towards simplifying it). +Motivation +---------- + +Foolscap +~~~~~~~~ + +Foolscap is a remote method invocation protocol with several distinctive features. +At its core it allows separate processes to refer each other's objects and methods using a capability-based model. +This allows for extremely fine-grained access control in a system that remains highly securable without becoming overwhelmingly complicated. +Supporting this is a flexible and extensible serialization system which allows data to be exchanged between processes in carefully controlled ways. + +Tahoe avails itself of only a small portion of these features. +A Tahoe storage server typically only exposes one object with a fixed set of methods to clients. +A Tahoe introducer node does roughly the same. +Tahoe exchanges simple data structures that have many common, standard serialized representations. + +In exchange for this slight use of Foolscap's sophisticated mechanisms, +Tahoe pays a substantial price: + +* Foolscap is implemented only for Python. + Tahoe is thus limited to being implemented on in Python. +* There is only one Python implementation of Foolscap. + The implementation is therefore the de facto standard and understanding of the protocol often relies on understanding that implementation. +* The Foolscap developer community is very small. + The implementation therefore advances very little and some non-trivial part of the maintenance cost falls on the Tahoe project. +* The extensible serialization system imposes substantial overhead for the simple data structures Tahoe exchanges. + Tahoe therefore presents a more sluggish experience to users and taxes servers more greatly than is necessary. + +HTTP +~~~~ + +HTTP is a request/response protocol that has become the lingua franca of the internet. +Combined with the principles of Representational state transfer (REST) it is widely employed to create, update, and delete data in collections on the internet. +HTTP itself provides only modest functionality in comparison to Foolscap. +However its simplicity and widespread use have led to a diverse and almost overwhelming ecosystem of libraries, frameworks, toolkits, and so on. + +By adopting HTTP in place of Foolscap Tahoe can realize the following concrete benefits: + +* Practically every language or runtime has an HTTP protocol implementation (or a dozen of them) available. + This change paves the way for new Tahoe implementations using tools better suited for certain situations + (mobile client implementations, high-performance server implementations, easily distributed desktop clients, etc). +* The simplicity of and vast quantity of resources about HTTP make it a very easy protocol to learn and use. + This change reduces the barrier to entry for developers to contribute improvements to Tahoe's network interactions. +* For any given language there is very likely an HTTP implementation with a large and active developer community. + Tahoe can therefore benefit from the large effort being put into making better libraries for using HTTP. +* One of the core features of HTTP is the mundane transfer of bulk data and implementions are often capable of doing this with extreme efficiency. + The alignment of this core feature with a core activity of Tahoe of transferring bulk data means that a substantial barrier to improved Tahoe runtime performance will be eliminated. + Requirements ------------ From 835c050e6c5cee9cb272bad3de3f17e2e8c9d68a Mon Sep 17 00:00:00 2001 From: Jean-Paul Calderone Date: Tue, 30 Mar 2021 12:39:22 -0400 Subject: [PATCH 3/8] fix word-o --- docs/proposed/http-storage-node-protocol.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/proposed/http-storage-node-protocol.rst b/docs/proposed/http-storage-node-protocol.rst index 8d65e121d..5e46480d3 100644 --- a/docs/proposed/http-storage-node-protocol.rst +++ b/docs/proposed/http-storage-node-protocol.rst @@ -33,7 +33,7 @@ In exchange for this slight use of Foolscap's sophisticated mechanisms, Tahoe pays a substantial price: * Foolscap is implemented only for Python. - Tahoe is thus limited to being implemented on in Python. + Tahoe is thus limited to being implemented only in Python. * There is only one Python implementation of Foolscap. The implementation is therefore the de facto standard and understanding of the protocol often relies on understanding that implementation. * The Foolscap developer community is very small. From e843206524b5904f8a49ce3ed6614da67db4ca49 Mon Sep 17 00:00:00 2001 From: Jean-Paul Calderone Date: Tue, 30 Mar 2021 12:39:27 -0400 Subject: [PATCH 4/8] expand on justification for performance complaints --- docs/proposed/http-storage-node-protocol.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/proposed/http-storage-node-protocol.rst b/docs/proposed/http-storage-node-protocol.rst index 5e46480d3..c2a68b3ee 100644 --- a/docs/proposed/http-storage-node-protocol.rst +++ b/docs/proposed/http-storage-node-protocol.rst @@ -39,7 +39,10 @@ Tahoe pays a substantial price: * The Foolscap developer community is very small. The implementation therefore advances very little and some non-trivial part of the maintenance cost falls on the Tahoe project. * The extensible serialization system imposes substantial overhead for the simple data structures Tahoe exchanges. - Tahoe therefore presents a more sluggish experience to users and taxes servers more greatly than is necessary. +* Foolscap encourages a "remote object" style of protocol design with involves many client-server interactions. + However, Foolscap does not implement "promise pipelining". + The result is that Foolscap encourages a protocol that requires many round-trips between client and server. +* The serialization overhead combined with the many round-trips result in Tahoe presenting a more sluggish experience to users and taxes servers more greatly than is necessary. HTTP ~~~~ From 0f0398aeaf24ac7d2ea54c6fc9f06661f552f45a Mon Sep 17 00:00:00 2001 From: Jean-Paul Calderone Date: Tue, 30 Mar 2021 12:39:50 -0400 Subject: [PATCH 5/8] minor typography --- docs/proposed/http-storage-node-protocol.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/proposed/http-storage-node-protocol.rst b/docs/proposed/http-storage-node-protocol.rst index c2a68b3ee..6a2f5e933 100644 --- a/docs/proposed/http-storage-node-protocol.rst +++ b/docs/proposed/http-storage-node-protocol.rst @@ -48,7 +48,7 @@ HTTP ~~~~ HTTP is a request/response protocol that has become the lingua franca of the internet. -Combined with the principles of Representational state transfer (REST) it is widely employed to create, update, and delete data in collections on the internet. +Combined with the principles of Representational State Transfer (REST) it is widely employed to create, update, and delete data in collections on the internet. HTTP itself provides only modest functionality in comparison to Foolscap. However its simplicity and widespread use have led to a diverse and almost overwhelming ecosystem of libraries, frameworks, toolkits, and so on. From 94b92202c3f448561318f03f9b070eea5f14ef2a Mon Sep 17 00:00:00 2001 From: Jean-Paul Calderone Date: Tue, 30 Mar 2021 12:45:22 -0400 Subject: [PATCH 6/8] Try to clarify requirement for some security properties --- docs/proposed/http-storage-node-protocol.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/proposed/http-storage-node-protocol.rst b/docs/proposed/http-storage-node-protocol.rst index 6a2f5e933..a760ed7d9 100644 --- a/docs/proposed/http-storage-node-protocol.rst +++ b/docs/proposed/http-storage-node-protocol.rst @@ -64,6 +64,14 @@ By adopting HTTP in place of Foolscap Tahoe can realize the following concrete b * One of the core features of HTTP is the mundane transfer of bulk data and implementions are often capable of doing this with extreme efficiency. The alignment of this core feature with a core activity of Tahoe of transferring bulk data means that a substantial barrier to improved Tahoe runtime performance will be eliminated. +TLS +~~~ + +The Foolscap-based protocol provides *some* of Tahoe's confidentiality, integrity, and authentication properties by leveraging TLS. +An HTTP-based protocol can make use of TLS in largely the same way to provide the same properties. +Provision of these properties *is* dependant on implementers following Great Black Swamp's rules for x509 certificate validation +(rather than the standard "web" rules for validation). + Requirements ------------ From c15fe70378da89e3b50fedeea7db3a5fcafb54c8 Mon Sep 17 00:00:00 2001 From: Jean-Paul Calderone Date: Thu, 1 Apr 2021 12:53:25 -0400 Subject: [PATCH 7/8] Replace plain "Tahoe" with "Tahoe-LAFS" as the name of the project/software/system/etc --- docs/proposed/http-storage-node-protocol.rst | 30 ++++++++++---------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/proposed/http-storage-node-protocol.rst b/docs/proposed/http-storage-node-protocol.rst index a760ed7d9..83b6823e7 100644 --- a/docs/proposed/http-storage-node-protocol.rst +++ b/docs/proposed/http-storage-node-protocol.rst @@ -24,25 +24,25 @@ At its core it allows separate processes to refer each other's objects and metho This allows for extremely fine-grained access control in a system that remains highly securable without becoming overwhelmingly complicated. Supporting this is a flexible and extensible serialization system which allows data to be exchanged between processes in carefully controlled ways. -Tahoe avails itself of only a small portion of these features. -A Tahoe storage server typically only exposes one object with a fixed set of methods to clients. -A Tahoe introducer node does roughly the same. -Tahoe exchanges simple data structures that have many common, standard serialized representations. +Tahoe-LAFS avails itself of only a small portion of these features. +A Tahoe-LAFS storage server typically only exposes one object with a fixed set of methods to clients. +A Tahoe-LAFS introducer node does roughly the same. +Tahoe-LAFS exchanges simple data structures that have many common, standard serialized representations. In exchange for this slight use of Foolscap's sophisticated mechanisms, -Tahoe pays a substantial price: +Tahoe-LAFS pays a substantial price: * Foolscap is implemented only for Python. - Tahoe is thus limited to being implemented only in Python. + Tahoe-LAFS is thus limited to being implemented only in Python. * There is only one Python implementation of Foolscap. The implementation is therefore the de facto standard and understanding of the protocol often relies on understanding that implementation. * The Foolscap developer community is very small. - The implementation therefore advances very little and some non-trivial part of the maintenance cost falls on the Tahoe project. -* The extensible serialization system imposes substantial overhead for the simple data structures Tahoe exchanges. + The implementation therefore advances very little and some non-trivial part of the maintenance cost falls on the Tahoe-LAFS project. +* The extensible serialization system imposes substantial overhead for the simple data structures Tahoe-LAFS exchanges. * Foolscap encourages a "remote object" style of protocol design with involves many client-server interactions. However, Foolscap does not implement "promise pipelining". The result is that Foolscap encourages a protocol that requires many round-trips between client and server. -* The serialization overhead combined with the many round-trips result in Tahoe presenting a more sluggish experience to users and taxes servers more greatly than is necessary. +* The serialization overhead combined with the many round-trips result in Tahoe-LAFS presenting a more sluggish experience to users and taxes servers more greatly than is necessary. HTTP ~~~~ @@ -52,22 +52,22 @@ Combined with the principles of Representational State Transfer (REST) it is wid HTTP itself provides only modest functionality in comparison to Foolscap. However its simplicity and widespread use have led to a diverse and almost overwhelming ecosystem of libraries, frameworks, toolkits, and so on. -By adopting HTTP in place of Foolscap Tahoe can realize the following concrete benefits: +By adopting HTTP in place of Foolscap Tahoe-LAFS can realize the following concrete benefits: * Practically every language or runtime has an HTTP protocol implementation (or a dozen of them) available. - This change paves the way for new Tahoe implementations using tools better suited for certain situations + This change paves the way for new Tahoe-LAFS implementations using tools better suited for certain situations (mobile client implementations, high-performance server implementations, easily distributed desktop clients, etc). * The simplicity of and vast quantity of resources about HTTP make it a very easy protocol to learn and use. - This change reduces the barrier to entry for developers to contribute improvements to Tahoe's network interactions. + This change reduces the barrier to entry for developers to contribute improvements to Tahoe-LAFS's network interactions. * For any given language there is very likely an HTTP implementation with a large and active developer community. - Tahoe can therefore benefit from the large effort being put into making better libraries for using HTTP. + Tahoe-LAFS can therefore benefit from the large effort being put into making better libraries for using HTTP. * One of the core features of HTTP is the mundane transfer of bulk data and implementions are often capable of doing this with extreme efficiency. - The alignment of this core feature with a core activity of Tahoe of transferring bulk data means that a substantial barrier to improved Tahoe runtime performance will be eliminated. + The alignment of this core feature with a core activity of Tahoe-LAFS of transferring bulk data means that a substantial barrier to improved Tahoe-LAFS runtime performance will be eliminated. TLS ~~~ -The Foolscap-based protocol provides *some* of Tahoe's confidentiality, integrity, and authentication properties by leveraging TLS. +The Foolscap-based protocol provides *some* of Tahoe-LAFS's confidentiality, integrity, and authentication properties by leveraging TLS. An HTTP-based protocol can make use of TLS in largely the same way to provide the same properties. Provision of these properties *is* dependant on implementers following Great Black Swamp's rules for x509 certificate validation (rather than the standard "web" rules for validation). From 89e1865f78c236b5137c3504fc4f465149bafe3e Mon Sep 17 00:00:00 2001 From: Jean-Paul Calderone Date: Fri, 2 Apr 2021 09:27:58 -0400 Subject: [PATCH 8/8] Forget about runtime performance --- docs/proposed/http-storage-node-protocol.rst | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/docs/proposed/http-storage-node-protocol.rst b/docs/proposed/http-storage-node-protocol.rst index 83b6823e7..ad9dd30bc 100644 --- a/docs/proposed/http-storage-node-protocol.rst +++ b/docs/proposed/http-storage-node-protocol.rst @@ -38,11 +38,7 @@ Tahoe-LAFS pays a substantial price: The implementation is therefore the de facto standard and understanding of the protocol often relies on understanding that implementation. * The Foolscap developer community is very small. The implementation therefore advances very little and some non-trivial part of the maintenance cost falls on the Tahoe-LAFS project. -* The extensible serialization system imposes substantial overhead for the simple data structures Tahoe-LAFS exchanges. -* Foolscap encourages a "remote object" style of protocol design with involves many client-server interactions. - However, Foolscap does not implement "promise pipelining". - The result is that Foolscap encourages a protocol that requires many round-trips between client and server. -* The serialization overhead combined with the many round-trips result in Tahoe-LAFS presenting a more sluggish experience to users and taxes servers more greatly than is necessary. +* The extensible serialization system imposes substantial complexity compared to the simple data structures Tahoe-LAFS actually exchanges. HTTP ~~~~