SFTP/FTP: merge user/account code, merge docs

docs/frontends/FTP-and-SFTP.txt

@@ -0,0 +1,213 @@
+= Tahoe FTP/SFTP Frontend =
+
+== FTP/SFTP Background ==
+
+FTP is the venerable internet file-transfer protocol, first developed in
+1971. The FTP server usually listens on port 21. A separate connection is
+used for the actual data transfers, either in the same direction as the
+initial client-to-server connection (for PORT mode), or in the reverse
+direction (for PASV) mode. Connections are unencrypted, so passwords, file
+names, and file contents are visible to eavesdroppers.
+
+SFTP is the modern replacement, developed as part of the SSH "secure shell"
+protocol, and runs as a subchannel of the regular SSH connection. The SSH
+server usually listens on port 22. All connections are encrypted.
+
+Both FTP and SFTP were developed assuming a UNIX-like server, with accounts
+and passwords, octal file modes (user/group/other, read/write/execute), and
+ctime/mtime timestamps.
+
+== Tahoe Support ==
+
+All Tahoe client nodes can run a frontend FTP server, allowing regular FTP
+clients (like /usr/bin/ftp, ncftp, and countless others) to access the
+virtual filesystem. They can also run an SFTP server, so SFTP clients (like
+/usr/bin/sftp, the sshfs FUSE plugin, and others) can too. These frontends
+sit at the same level as the webapi interface.
+
+Since Tahoe does not use user accounts or passwords, the FTP/SFTP servers
+must be configured with a way to first authenticate a user (confirm that a
+prospective client has a legitimate claim to whatever authorities we might
+grant a particular user), and second to decide what root directory cap should
+be granted to the authenticated username. FTP uses a username and password
+for this purpose. SFTP can either use a username and password, or a username
+and an RSA or DSA public key (SSH servers are frequently configured to
+require public key logins and reject passwords, to remove the threat of
+password-guessing attacks, at the expense of requiring users to carry their
+private keys around with them).
+
+Tahoe provides two mechanisms to perform this user-to-rootcap mapping. The
+first is a simple flat file with one account per line. The second is an
+HTTP-based login mechanism, backed by simple PHP script and a database. The
+latter form is used by allmydata.com to provide secure access to customer
+rootcaps.
+
+== Creating an Account File ==
+
+To use the first form, create a file (probably in
+BASEDIR/private/ftp.accounts) in which each non-comment/non-blank line is a
+space-separated line of (USERNAME, PASSWORD/PUBKEY, ROOTCAP), like so:
+
+ % cat BASEDIR/private/ftp.accounts
+ # This is a password line, (username, password, rootcap)
+ alice password URI:DIR2:ioej8xmzrwilg772gzj4fhdg7a:wtiizszzz2rgmczv4wl6bqvbv33ag4kvbr6prz3u6w3geixa6m6a
+ bob sekrit URI:DIR2:6bdmeitystckbl9yqlw7g56f4e:serp5ioqxnh34mlbmzwvkp3odehsyrr7eytt5f64we3k9hhcrcja
+
+ # and this is a public key line (username, pubkey, rootcap)
+ carol ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv2xHRVBoXnwxHLzthRD1wOWtyZ08b8n9cMZfJ58CBdBwAYP2NVNXc0XjRvswm5hnnAO+jyWPVNpXJjm9XllzYhODSNtSN+TXuJlUjhzA/T+ZwdgsgSAeHuuMQBoWt4Qc9HV6rHCdAeMhcnyqm6Q0sRAsfA/wfwiIgbvE7+cWpFa2anB6WeAnvK8+dMN0nvnkPE7GNyf/WFR1Ffuh9ifKdRB6yDNp17bQAqA3OWSFjch6fGPhp94y4g2jmTHlEUTyVsilgGqvGOutOVYnmOMnFijugU1Vu33G39GGzXWla6+fXwTk/oiVPiCYD7A7WFKes3nqMg8iVN6a6sxujrhnHQ== warner@fluxx URI:DIR2:6bdmeitystckbl9yqlw7g56f4e:serp5ioqxnh34mlbmzwvkp3odehsyrr7eytt5f64we3k9hhcrcja
+
+[TODO: the PUBKEY form is not yet supported]
+
+Note that if the second word of the line is "ssh-rsa" or "ssh-dss", the rest
+of the line is parsed differently, so users cannot have a password equal to
+either of these strings.
+
+Then add an 'accounts.file' directive to your tahoe.cfg file, as described
+in the next sections.
+
+== Configuring FTP Access ==
+
+To enable the FTP server with an accounts file, add the following lines to
+the BASEDIR/tahoe.cfg file:
+
+ [ftpd]
+ enabled = true
+ port = 8021
+ accounts.file = private/ftp.accounts
+
+The FTP server will listen on the given port number. The "accounts.file"
+pathname will be interpreted relative to the node's BASEDIR.
+
+To enable the FTP server with an account server instead, provide the URL of
+that server in an "accounts.url" directive:
+
+ [ftpd]
+ enabled = true
+ port = 8021
+ accounts.url = https://example.com/login
+
+You can provide both accounts.file and accounts.url, although it probably
+isn't very useful except for testing.
+
+== Configuring SFTP Access ==
+
+The Tahoe SFTP server requires a host keypair, just like the regular SSH
+server. It is important to give each server a distinct keypair, to prevent
+one server from masquerading as different one. The first time a client
+program talks to a given server, it will store the host key it receives, and
+will complain if a subsequent connection uses a different key. This reduces
+the opportunity for man-in-the-middle attacks to just the first connection.
+
+You will use directives in the tahoe.cfg file to tell the SFTP code where to
+find these keys. To create one, use the ssh-keygen tool (which comes with the
+normal openssl client distribution):
+
+% cd BASEDIR
+% ssh-keygen -f private/ssh_host_rsa_key
+
+Then, to enable the SFTP server with an accounts file, add the following
+lines to the BASEDIR/tahoe.cfg file:
+
+ [sftpd]
+ enabled = true
+ port = 8022
+ host_pubkey_file = private/ssh_host_rsa_key.pub
+ host_privkey_file = private/ssh_host_rsa_key
+ accounts.file = private/ftp.accounts
+
+The SFTP server will listen on the given port number. The "accounts.file"
+pathname will be interpreted relative to the node's BASEDIR.
+
+Or, to use an account server instead, do this:
+
+ [sftpd]
+ enabled = true
+ port = 8022
+ host_pubkey_file = private/ssh_host_rsa_key.pub
+ host_privkey_file = private/ssh_host_rsa_key
+ accounts.url = https://example.com/login
+
+You can provide both accounts.file and accounts.url, although it probably
+isn't very useful except for testing.
+
+
+== Dependencies ==
+
+The Tahoe SFTP server requires the Twisted "Conch" component (a "conch" is a
+twisted shell, get it?). Many Linux distributions package the Conch code
+separately: debian puts it in the "python-twisted-conch" package. Conch
+requires the "pycrypto" package, which is a Python+C implementation of many
+cryptographic functions (the debian package is named "python-crypto").
+
+Note that "pycrypto" is different than the "pycryptopp" package that Tahoe
+uses (which is a Python wrapper around the C++ -based Crypto++ library, a
+library that is frequently installed as /usr/lib/libcryptopp.a, to avoid
+problems with non-alphanumerics in filenames).
+
+The FTP server requires code in Twisted that enables asynchronous closing of
+file-upload operations. This code was not in the Twisted-8.1.0 release, and
+has not been committed to SVN trunk as of r24943. So it may be necessary to
+apply the following patch. The Tahoe node refuse to start the FTP server if
+it detects that this patch has not been applied.
+
+Index: twisted/protocols/ftp.py
+===================================================================
+--- twisted/protocols/ftp.py	(revision 24956)
++++ twisted/protocols/ftp.py	(working copy)
+@@ -1049,7 +1049,6 @@
+                 cons = ASCIIConsumerWrapper(cons)
+ 
+             d = self.dtpInstance.registerConsumer(cons)
+-            d.addCallbacks(cbSent, ebSent)
+ 
+             # Tell them what to doooo
+             if self.dtpInstance.isConnected:
+@@ -1062,6 +1061,8 @@
+         def cbOpened(file):
+             d = file.receive()
+             d.addCallback(cbConsumer)
++            d.addCallback(lambda ignored: file.close())
++            d.addCallbacks(cbSent, ebSent)
+             return d
+ 
+         def ebOpened(err):
+@@ -1434,7 +1435,14 @@
+         @rtype: C{Deferred} of C{IConsumer}
+         """
+ 
++    def close():
++        """
++        Perform any post-write work that needs to be done. This method may
++        only be invoked once on each provider, and will always be invoked
++        after receive().
+ 
++        @rtype: C{Deferred} of anything: the value is ignored
++        """
+ 
+ def _getgroups(uid):
+     """Return the primary and supplementary groups for the given UID.
+@@ -1795,6 +1803,8 @@
+         # FileConsumer will close the file object
+         return defer.succeed(FileConsumer(self.fObj))
+ 
++    def close(self):
++        return defer.succeed(None)
+ 
+ 
+ class FTPRealm:
+Index: twisted/vfs/adapters/ftp.py
+===================================================================
+--- twisted/vfs/adapters/ftp.py	(revision 24956)
++++ twisted/vfs/adapters/ftp.py	(working copy)
+@@ -295,6 +295,11 @@
+         """
+         return defer.succeed(IConsumer(self.node))
+ 
++    def close(self):
++        """
++        Perform post-write actions.
++        """
++        return defer.succeed(None)
+ 
+ 
+ class _FileToConsumerAdapter(object):

docs/frontends/ftp.txt

@@ -1,112 +0,0 @@
-= Tahoe FTP Frontend =
-
-All Tahoe client nodes can run a frontend FTP server, allowing regular FTP
-clients to access the virtual filesystem.
-
-Since Tahoe does not use user accounts or passwords, the FTP server must be
-configured with a way to translate USER+PASS into a root directory cap. Two
-mechanisms are provided. The first is a simple flat file with one account per
-line. The second is an HTTP-based login mechanism, backed by simple PHP
-script and a database. The latter form is used by allmydata.com to provide
-secure access to customer rootcaps.
-
-== Configuring an Account File ==
-
-To configure the first form, create a file (probably in
-BASEDIR/private/ftp.accounts) in which each non-comment/non-blank line is a
-space-separated line of (USERNAME, PASSWORD, ROOTCAP), like so:
-
- % cat BASEDIR/private/ftp.accounts
- # This is a password file, (username, password, rootcap)
- alice password URI:DIR2:ioej8xmzrwilg772gzj4fhdg7a:wtiizszzz2rgmczv4wl6bqvbv33ag4kvbr6prz3u6w3geixa6m6a
- bob sekrit URI:DIR2:6bdmeitystckbl9yqlw7g56f4e:serp5ioqxnh34mlbmzwvkp3odehsyrr7eytt5f64we3k9hhcrcja
-
-Then add the following lines to the BASEDIR/tahoe.cfg file:
-
- [ftpd]
- enabled = true
- ftp.port = 8021
- ftp.accounts.file = private/ftp.accounts
-
-The FTP server will listen on the given port number. The ftp.accounts.file
-pathname will be interpreted relative to the node's BASEDIR.
-
-== Configuring an Account Server ==
-
-Determine the URL of the account server, say https://example.com/login . Then
-add the following lines to BASEDIR/tahoe.cfg:
-
- [ftpd]
- enabled = true
- ftp.port = 8021
- ftp.accounts.url = https://example.com/login
-
-== Dependencies ==
-
-The FTP server requires code in Twisted that enables asynchronous closing of
-file-upload operations. This code was not in the Twisted-8.1.0 release, and
-has not been committed to SVN trunk as of r24943. So it may be necessary to
-apply the following patch. The Tahoe node refuse to start the FTP server if
-it detects that this patch has not been applied.
-
-Index: twisted/protocols/ftp.py
-===================================================================
---- twisted/protocols/ftp.py	(revision 24956)
-+++ twisted/protocols/ftp.py	(working copy)
-@@ -1049,7 +1049,6 @@
-                 cons = ASCIIConsumerWrapper(cons)
- 
-             d = self.dtpInstance.registerConsumer(cons)
--            d.addCallbacks(cbSent, ebSent)
- 
-             # Tell them what to doooo
-             if self.dtpInstance.isConnected:
-@@ -1062,6 +1061,8 @@
-         def cbOpened(file):
-             d = file.receive()
-             d.addCallback(cbConsumer)
-+            d.addCallback(lambda ignored: file.close())
-+            d.addCallbacks(cbSent, ebSent)
-             return d
- 
-         def ebOpened(err):
-@@ -1434,7 +1435,14 @@
-         @rtype: C{Deferred} of C{IConsumer}
-         """
- 
-+    def close():
-+        """
-+        Perform any post-write work that needs to be done. This method may
-+        only be invoked once on each provider, and will always be invoked
-+        after receive().
- 
-+        @rtype: C{Deferred} of anything: the value is ignored
-+        """
- 
- def _getgroups(uid):
-     """Return the primary and supplementary groups for the given UID.
-@@ -1795,6 +1803,8 @@
-         # FileConsumer will close the file object
-         return defer.succeed(FileConsumer(self.fObj))
- 
-+    def close(self):
-+        return defer.succeed(None)
- 
- 
- class FTPRealm:
-Index: twisted/vfs/adapters/ftp.py
-===================================================================
---- twisted/vfs/adapters/ftp.py	(revision 24956)
-+++ twisted/vfs/adapters/ftp.py	(working copy)
-@@ -295,6 +295,11 @@
-         """
-         return defer.succeed(IConsumer(self.node))
- 
-+    def close(self):
-+        """
-+        Perform post-write actions.
-+        """
-+        return defer.succeed(None)
- 
- 
- class _FileToConsumerAdapter(object):

docs/frontends/sftp.txt

@@ -1,74 +0,0 @@
-= Tahoe SFTP Frontend =
-
-All Tahoe client nodes can run a frontend SFTP server, allowing regular SFTP
-clients to access the virtual filesystem.
-
-Since Tahoe does not use user accounts or passwords, the FTP server must be
-configured with a way to translate a username (and either a password or
-public key) into a root directory cap. Two mechanisms are provided. The first
-is a simple flat file with one account per line. The second is an HTTP-based
-login mechanism, backed by simple PHP script and a database. The latter form
-is used by allmydata.com to provide secure access to customer rootcaps.
-
-The SFTP server must also be given a public/private host keypair.
-
-== Configuring a Keypair ==
-
-First, generate a keypair for your server:
-
-% cd BASEDIR
-% ssh-keygen -f private/ssh_host_rsa_key
-
-You will then use the following lines in the tahoe.cfg file:
-
- [sftpd]
- sftp.host_pubkey_file = private/ssh_host_rsa_key.pub
- sftp.host_privkey_file = private/ssh_host_rsa_key
-
-== Configuring an Account File ==
-
-To configure the first form, create a file (probably in
-BASEDIR/private/sftp.accounts) in which each non-comment/non-blank line is a
-space-separated line of (USERNAME, PASSWORD/PUBKEY, ROOTCAP), like so:
-
-[TODO: the PUBKEY form is not yet supported]
-
- % cat BASEDIR/private/sftp.accounts
- # This is a password file, (username, password/pubkey, rootcap)
- alice password URI:DIR2:ioej8xmzrwilg772gzj4fhdg7a:wtiizszzz2rgmczv4wl6bqvbv33ag4kvbr6prz3u6w3geixa6m6a
- bob sekrit URI:DIR2:6bdmeitystckbl9yqlw7g56f4e:serp5ioqxnh34mlbmzwvkp3odehsyrr7eytt5f64we3k9hhcrcja
- carol ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv2xHRVBoXnwxHLzthRD1wOWtyZ08b8n9cMZfJ58CBdBwAYP2NVNXc0XjRvswm5hnnAO+jyWPVNpXJjm9XllzYhODSNtSN+TXuJlUjhzA/T+ZwdgsgSAeHuuMQBoWt4Qc9HV6rHCdAeMhcnyqm6Q0sRAsfA/wfwiIgbvE7+cWpFa2anB6WeAnvK8+dMN0nvnkPE7GNyf/WFR1Ffuh9ifKdRB6yDNp17bQAqA3OWSFjch6fGPhp94y4g2jmTHlEUTyVsilgGqvGOutOVYnmOMnFijugU1Vu33G39GGzXWla6+fXwTk/oiVPiCYD7A7WFKes3nqMg8iVN6a6sxujrhnHQ== warner@fluxx URI:DIR2:6bdmeitystckbl9yqlw7g56f4e:serp5ioqxnh34mlbmzwvkp3odehsyrr7eytt5f64we3k9hhcrcja
-
-Note that if the second word of the line is "ssh-rsa" or "ssh-dss", the rest
-of the line is parsed differently, so users cannot have a password equal to
-either of these strings.
-
-Then add the following lines to the BASEDIR/tahoe.cfg file:
-
- [sftpd]
- enabled = true
- sftp.port = 8022
- sftp.host_pubkey_file = private/ssh_host_rsa_key.pub
- sftp.host_privkey_file = private/ssh_host_rsa_key
- sftp.accounts.file = private/sftp.accounts
-
-The SFTP server will listen on the given port number. The sftp.accounts.file
-pathname will be interpreted relative to the node's BASEDIR.
-
-== Configuring an Account Server ==
-
-Determine the URL of the account server, say https://example.com/login . Then
-add the following lines to BASEDIR/tahoe.cfg:
-
- [sftpd]
- enabled = true
- sftp.port = 8022
- sftp.host_pubkey_file = private/ssh_host_rsa_key.pub
- sftp.host_privkey_file = private/ssh_host_rsa_key
- sftp.accounts.url = https://example.com/login
-
-== Dependencies ==
-
-The Tahoe SFTP server requires the Twisted "Conch" component, which itself
-requires the pycrypto package (note that pycrypto is distinct from the
-pycryptopp that Tahoe uses).

src/allmydata/frontends/auth.py

@@ -0,0 +1,98 @@
+import os
+from zope.interface import implements
+from twisted.web.client import getPage
+from twisted.internet import defer
+from twisted.cred import error, checkers, credentials
+from allmydata.util import base32
+
+class FTPAvatarID:
+    def __init__(self, username, rootcap):
+        self.username = username
+        self.rootcap = rootcap
+
+class AccountFileChecker:
+    implements(checkers.ICredentialsChecker)
+    credentialInterfaces = (credentials.IUsernamePassword,
+                            credentials.IUsernameHashedPassword)
+    def __init__(self, client, accountfile):
+        self.client = client
+        self.passwords = {}
+        self.pubkeys = {}
+        self.rootcaps = {}
+        for line in open(os.path.expanduser(accountfile), "r"):
+            line = line.strip()
+            if line.startswith("#") or not line:
+                continue
+            name, passwd, rest = line.split(None, 2)
+            if passwd in ("ssh-dss", "ssh-rsa"):
+                bits = rest.split()
+                keystring = " ".join(bits[-1])
+                rootcap = bits[-1]
+                self.pubkeys[name] = keystring
+            else:
+                self.passwords[name] = passwd
+                rootcap = rest
+            self.rootcaps[name] = rootcap
+
+    def _cbPasswordMatch(self, matched, username):
+        if matched:
+            return FTPAvatarID(username, self.rootcaps[username])
+        raise error.UnauthorizedLogin
+
+    def requestAvatarId(self, credentials):
+        if credentials.username in self.passwords:
+            d = defer.maybeDeferred(credentials.checkPassword,
+                                    self.passwords[credentials.username])
+            d.addCallback(self._cbPasswordMatch, str(credentials.username))
+            return d
+        return defer.fail(error.UnauthorizedLogin())
+
+class AccountURLChecker:
+    implements(checkers.ICredentialsChecker)
+    credentialInterfaces = (credentials.IUsernamePassword,)
+
+    def __init__(self, client, auth_url):
+        self.client = client
+        self.auth_url = auth_url
+
+    def _cbPasswordMatch(self, rootcap, username):
+        return FTPAvatarID(username, rootcap)
+
+    def post_form(self, username, password):
+        sepbase = base32.b2a(os.urandom(4))
+        sep = "--" + sepbase
+        form = []
+        form.append(sep)
+        fields = {"action": "authenticate",
+                  "email": username,
+                  "passwd": password,
+                  }
+        for name, value in fields.iteritems():
+            form.append('Content-Disposition: form-data; name="%s"' % name)
+            form.append('')
+            assert isinstance(value, str)
+            form.append(value)
+            form.append(sep)
+        form[-1] += "--"
+        body = "\r\n".join(form) + "\r\n"
+        headers = {"content-type": "multipart/form-data; boundary=%s" % sepbase,
+                   }
+        return getPage(self.auth_url, method="POST",
+                       postdata=body, headers=headers,
+                       followRedirect=True, timeout=30)
+
+    def _parse_response(self, res):
+        rootcap = res.strip()
+        if rootcap == "0":
+            raise error.UnauthorizedLogin
+        return rootcap
+
+    def requestAvatarId(self, credentials):
+        # construct a POST to the login form. While this could theoretically
+        # be done with something like the stdlib 'email' package, I can't
+        # figure out how, so we just slam together a form manually.
+        d = self.post_form(credentials.username, credentials.password)
+        d.addCallback(self._parse_response)
+        d.addCallback(self._cbPasswordMatch, str(credentials.username))
+        return d
+

src/allmydata/frontends/ftpd.py

@@ -1,19 +1,16 @@
 
-import os
 import tempfile
 from zope.interface import implements
 from twisted.application import service, strports
 from twisted.internet import defer
 from twisted.internet.interfaces import IConsumer
+from twisted.cred import portal
 from twisted.protocols import ftp
-from twisted.cred import error, portal, checkers, credentials
-from twisted.web.client import getPage
 
 from allmydata.interfaces import IDirectoryNode, ExistingChildError, \
      NoSuchChildError
 from allmydata.immutable.download import ConsumerAdapter
 from allmydata.immutable.upload import FileHandle
-from allmydata.util import base32
 
 class ReadFile:
     implements(ftp.IReadFile)
@@ -270,89 +267,7 @@ class Handler:
         d.addCallback(_got_parent)
         return d
 
-
+from auth import AccountURLChecker, AccountFileChecker
-class FTPAvatarID:
-    def __init__(self, username, rootcap):
-        self.username = username
-        self.rootcap = rootcap
-
-class AccountFileChecker:
-    implements(checkers.ICredentialsChecker)
-    credentialInterfaces = (credentials.IUsernamePassword,
-                            credentials.IUsernameHashedPassword)
-    def __init__(self, client, accountfile):
-        self.client = client
-        self.passwords = {}
-        self.rootcaps = {}
-        for line in open(os.path.expanduser(accountfile), "r"):
-            line = line.strip()
-            if line.startswith("#") or not line:
-                continue
-            name, passwd, rootcap = line.split()
-            self.passwords[name] = passwd
-            self.rootcaps[name] = rootcap
-
-    def _cbPasswordMatch(self, matched, username):
-        if matched:
-            return FTPAvatarID(username, self.rootcaps[username])
-        raise error.UnauthorizedLogin
-
-    def requestAvatarId(self, credentials):
-        if credentials.username in self.passwords:
-            d = defer.maybeDeferred(credentials.checkPassword,
-                                    self.passwords[credentials.username])
-            d.addCallback(self._cbPasswordMatch, str(credentials.username))
-            return d
-        return defer.fail(error.UnauthorizedLogin())
-
-class AccountURLChecker:
-    implements(checkers.ICredentialsChecker)
-    credentialInterfaces = (credentials.IUsernamePassword,)
-
-    def __init__(self, client, auth_url):
-        self.client = client
-        self.auth_url = auth_url
-
-    def _cbPasswordMatch(self, rootcap, username):
-        return FTPAvatarID(username, rootcap)
-
-    def post_form(self, username, password):
-        sepbase = base32.b2a(os.urandom(4))
-        sep = "--" + sepbase
-        form = []
-        form.append(sep)
-        fields = {"action": "authenticate",
-                  "email": username,
-                  "passwd": password,
-                  }
-        for name, value in fields.iteritems():
-            form.append('Content-Disposition: form-data; name="%s"' % name)
-            form.append('')
-            assert isinstance(value, str)
-            form.append(value)
-            form.append(sep)
-        form[-1] += "--"
-        body = "\r\n".join(form) + "\r\n"
-        headers = {"content-type": "multipart/form-data; boundary=%s" % sepbase,
-                   }
-        return getPage(self.auth_url, method="POST",
-                       postdata=body, headers=headers,
-                       followRedirect=True, timeout=30)
-
-    def _parse_response(self, res):
-        rootcap = res.strip()
-        if rootcap == "0":
-            raise error.UnauthorizedLogin
-        return rootcap
-
-    def requestAvatarId(self, credentials):
-        # construct a POST to the login form. While this could theoretically
-        # be done with something like the stdlib 'email' package, I can't
-        # figure out how, so we just slam together a form manually.
-        d = self.post_form(credentials.username, credentials.password)
-        d.addCallback(self._parse_response)
-        d.addCallback(self._cbPasswordMatch, str(credentials.username))
-        return d
 
 
 class Dispatcher:
@@ -373,22 +288,23 @@ class FTPServer(service.MultiService):
     def __init__(self, client, accountfile, accounturl, ftp_portstr):
         service.MultiService.__init__(self)
 
-        if accountfile:
-            c = AccountFileChecker(self, accountfile)
-        elif accounturl:
-            c = AccountURLChecker(self, accounturl)
-        else:
-            # we could leave this anonymous, with just the /uri/CAP form
-            raise RuntimeError("must provide some translation")
-
         # make sure we're using a patched Twisted that uses IWriteFile.close:
-        # see docs/ftp.txt for details.
+        # see docs/frontends/ftp.txt for details.
         assert "close" in ftp.IWriteFile.names(), "your twisted is lacking"
 
         r = Dispatcher(client)
         p = portal.Portal(r)
-        p.registerChecker(c)
-        f = ftp.FTPFactory(p)
 
+        if accountfile:
+            c = AccountFileChecker(self, accountfile)
+            p.registerChecker(c)
+        if accounturl:
+            c = AccountURLChecker(self, accounturl)
+            p.registerChecker(c)
+        if not accountfile and not accounturl:
+            # we could leave this anonymous, with just the /uri/CAP form
+            raise RuntimeError("must provide some translation")
+
+        f = ftp.FTPFactory(p)
         s = strports.service(ftp_portstr, f)
         s.setServiceParent(self)

src/allmydata/frontends/sftpd.py

@@ -1,5 +1,4 @@
 
-import os
 import tempfile
 from zope.interface import implements
 from twisted.python import components
@@ -11,13 +10,11 @@ from twisted.conch.interfaces import ISFTPServer, ISFTPFile, IConchUser
 from twisted.conch.avatar import ConchUser
 from twisted.conch.openssh_compat import primes
 from twisted.conch import ls
-from twisted.cred import error, portal, checkers, credentials
+from twisted.cred import portal
-from twisted.web.client import getPage
 
 from allmydata.interfaces import IDirectoryNode, ExistingChildError, \
      NoSuchChildError
 from allmydata.immutable.upload import FileHandle
-from allmydata.util import base32
 
 class MemoryConsumer:
     implements(IConsumer)
@@ -420,101 +417,12 @@ class SFTPHandler:
         return d
 
 
-class FTPAvatarID:
-    def __init__(self, username, rootcap):
-        self.username = username
-        self.rootcap = rootcap
-
-class AccountFileChecker:
-    implements(checkers.ICredentialsChecker)
-    credentialInterfaces = (credentials.IUsernamePassword,
-                            credentials.IUsernameHashedPassword)
-    def __init__(self, client, accountfile):
-        self.client = client
-        self.passwords = {}
-        self.pubkeys = {}
-        self.rootcaps = {}
-        for line in open(os.path.expanduser(accountfile), "r"):
-            line = line.strip()
-            if line.startswith("#") or not line:
-                continue
-            name, passwd, rest = line.split(None, 2)
-            if passwd in ("ssh-dss", "ssh-rsa"):
-                bits = rest.split()
-                keystring = " ".join(bits[-1])
-                rootcap = bits[-1]
-                self.pubkeys[name] = keystring
-            else:
-                self.passwords[name] = passwd
-                rootcap = rest
-            self.rootcaps[name] = rootcap
-
-    def _cbPasswordMatch(self, matched, username):
-        if matched:
-            return FTPAvatarID(username, self.rootcaps[username])
-        raise error.UnauthorizedLogin
-
-    def requestAvatarId(self, credentials):
-        if credentials.username in self.passwords:
-            d = defer.maybeDeferred(credentials.checkPassword,
-                                    self.passwords[credentials.username])
-            d.addCallback(self._cbPasswordMatch, str(credentials.username))
-            return d
-        return defer.fail(error.UnauthorizedLogin())
-
-class AccountURLChecker:
-    implements(checkers.ICredentialsChecker)
-    credentialInterfaces = (credentials.IUsernamePassword,)
-
-    def __init__(self, client, auth_url):
-        self.client = client
-        self.auth_url = auth_url
-
-    def _cbPasswordMatch(self, rootcap, username):
-        return FTPAvatarID(username, rootcap)
-
-    def post_form(self, username, password):
-        sepbase = base32.b2a(os.urandom(4))
-        sep = "--" + sepbase
-        form = []
-        form.append(sep)
-        fields = {"action": "authenticate",
-                  "email": username,
-                  "passwd": password,
-                  }
-        for name, value in fields.iteritems():
-            form.append('Content-Disposition: form-data; name="%s"' % name)
-            form.append('')
-            assert isinstance(value, str)
-            form.append(value)
-            form.append(sep)
-        form[-1] += "--"
-        body = "\r\n".join(form) + "\r\n"
-        headers = {"content-type": "multipart/form-data; boundary=%s" % sepbase,
-                   }
-        return getPage(self.auth_url, method="POST",
-                       postdata=body, headers=headers,
-                       followRedirect=True, timeout=30)
-
-    def _parse_response(self, res):
-        rootcap = res.strip()
-        if rootcap == "0":
-            raise error.UnauthorizedLogin
-        return rootcap
-
-    def requestAvatarId(self, credentials):
-        # construct a POST to the login form. While this could theoretically
-        # be done with something like the stdlib 'email' package, I can't
-        # figure out how, so we just slam together a form manually.
-        d = self.post_form(credentials.username, credentials.password)
-        d.addCallback(self._parse_response)
-        d.addCallback(self._cbPasswordMatch, str(credentials.username))
-        return d
-
 # if you have an SFTPUser, and you want something that provides ISFTPServer,
 # then you get SFTPHandler(user)
 components.registerAdapter(SFTPHandler, SFTPUser, ISFTPServer)
 
+from auth import AccountURLChecker, AccountFileChecker
+
 class Dispatcher:
     implements(portal.IRealm)
     def __init__(self, client):
@@ -533,17 +441,18 @@ class SFTPServer(service.MultiService):
                  sftp_portstr, pubkey_file, privkey_file):
         service.MultiService.__init__(self)
 
-        if accountfile:
-            c = AccountFileChecker(self, accountfile)
-        elif accounturl:
-            c = AccountURLChecker(self, accounturl)
-        else:
-            # we could leave this anonymous, with just the /uri/CAP form
-            raise RuntimeError("must provide some translation")
-
         r = Dispatcher(client)
         p = portal.Portal(r)
+
+        if accountfile:
+            c = AccountFileChecker(self, accountfile)
             p.registerChecker(c)
+        if accounturl:
+            c = AccountURLChecker(self, accounturl)
+            p.registerChecker(c)
+        if not accountfile and not accounturl:
+            # we could leave this anonymous, with just the /uri/CAP form
+            raise RuntimeError("must provide some translation")
 
         pubkey = keys.Key.fromFile(pubkey_file)
         privkey = keys.Key.fromFile(privkey_file)