Commit Graph

1233 Commits

Author SHA1 Message Date
robk-tahoe
ccda06b061 key_generator: added a unit test
implemented a unit test of basic KeyGenService functionality,
fixed a bug in the timing of pool refreshes
2008-04-03 13:01:43 -07:00
Brian Warner
06dae8fd67 trial_figleaf.py: make our figleaf code compatible with both Twisted-8.x and Twisted-2.5.x 2008-04-02 17:48:55 -07:00
robk-tahoe
5578559b85 added offloaded key generation
this adds a new service to pre-generate RSA key pairs.  This allows
the expensive (i.e. slow) key generation to be placed into a process
outside the node, so that the node's reactor will not block when it
needs a key pair, but instead can retrieve them from a pool of already
generated key pairs in the key-generator service.

it adds a tahoe create-key-generator command which initialises an 
empty dir with a tahoe-key-generator.tac file which can then be run
via twistd.  it stashes its .pem and portnum for furl stability and
writes the furl of the key gen service to key_generator.furl, also
printing it to stdout.

by placing a key_generator.furl file into the nodes config directory
(e.g. ~/.tahoe) a node will attempt to connect to such a service, and
will use that when creating mutable files (i.e. directories) whenever
possible.  if the keygen service is unavailable, it will perform the
key generation locally instead, as before.
2008-04-01 18:45:13 -07:00
Brian Warner
f9e261d939 introducer.py: accelerate reconnection after being offline. Closes #374.
When we establish any new connection, reset the delays on all the other
Reconnectors. This will trigger a new batch of connection attempts. The idea
is to detect when we (the client) have been offline for a while, and to
connect to all servers when we get back online. By accelerating the timers
inside the Reconnectors, we try to avoid spending a long time in a
partially-connected state (which increases the chances of causing problems
with mutable files, by not updating all the shares that we ought to).
2008-03-31 15:28:45 -07:00
Brian Warner
d6be5116f5 storage: emit log messages on bucket allocate/read and mutable writev 2008-03-27 17:33:58 -07:00
Brian Warner
2c96a32633 helper: add more stats to webapi, at /helper_status 2008-03-27 16:46:08 -07:00
Brian Warner
36f5c025a6 helper: add stats for the gatherer, show some on the webish welcome page 2008-03-27 15:55:32 -07:00
Brian Warner
9b3a32d0b3 add GET /uri/URI/?t=deep-size, to compute the total size of immutable files reachable from a given directory 2008-03-27 11:33:42 -07:00
Brian Warner
3f5fed0bd4 node.py: make twistd.pid world-readable, so stats-gathering tools (specifically memory-measuring munin plugins) can see it 2008-03-26 18:37:54 -07:00
Brian Warner
04b690ac9e web-status: client methods like list_all_uploads() return Upload instances,
not status instances. Fix this. The symptom was that following a link like
'up-123' that referred to an old operation (no longer in memory) while an
upload was active would get an ugly traceback instead of a "no such resource"
message.
2008-03-26 18:20:07 -07:00
Brian Warner
4531d1e953 introweb.py: add ?t=json, to provide machine-readable subscriber counts 2008-03-25 12:56:12 -07:00
Brian Warner
4b46f1cd53 encode.py: also record the size, along with plaintext_hash and SI 2008-03-24 19:08:15 -07:00
Brian Warner
15ef9f624d encode: log a plaintext hash and SI for each upload. This will allow the log gatherer to correlate the two, to better measure the benefits of convergence 2008-03-24 18:55:37 -07:00
robk-tahoe
7f14b93341 confwiz: set a convergence domain based on root_dir upon config
when the confwiz configures a node (i.e. typically once on mac, once per
install on windows) in addition to writing the root_dir.cap retrieved from
the native_client backend into a config file, it additionally writes a hash
thereof into the 'convergence' config file.

this causes uploads from this node to use a consistent 'convergence' hashing
value matching any other nodes with the same configured root_dir, i.e. for
the most part other systems installed and configured on the same account.
2008-03-24 15:47:12 -07:00
robk-tahoe
d25d3065d1 mutable: revise a couple of error messages
at brian and zooko's suggestion, reword an error message encountered when
multiple writers are racing to make overlapping changes to a directory
2008-03-24 15:46:28 -07:00
Brian Warner
1869d7c29b upload.py: remove spurious trailing quotes that confuse syntax-highlighting 2008-03-24 15:51:19 -07:00
Brian Warner
1e097766c9 disable plaintext hashes in shares, but leave a switch to turn it back on 2008-03-24 13:39:51 -07:00
Brian Warner
7b21054c33 UNDO: upload: stop putting plaintext and ciphertext hashes in shares.
This removes the guess-partial-information attack vector, and reduces
the amount of overhead that we consume with each file. It also introduces
a forwards-compability break: older versions of the code (before the
previous download-time "make hashes optional" patch) will be unable
to read files uploaded by this version, as they will complain about the
missing hashes. This patch is experimental, and is being pushed into
trunk to obtain test coverage. We may undo it before releasing 1.0.
2008-03-23 15:35:54 -07:00
Zooko O'Whielacronx
dfdbb95648 fix check-memory to use new upload API (which requires a "convergence" argument), and change it to measure convergence instead of random-key, since convergence is the use case we care about more 2008-03-24 15:28:04 -07:00
Zooko O'Whielacronx
fc3bd0c987 use added secret to protect convergent encryption
Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key.  If you want traditional convergent encryption behavior, set the added secret to be the empty string.

This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names.  (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.)

This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 09:46:06 -07:00
Brian Warner
7996131a0a upload: stop putting plaintext and ciphertext hashes in shares.
This removes the guess-partial-information attack vector, and reduces
the amount of overhead that we consume with each file. It also introduces
a forwards-compability break: older versions of the code (before the
previous download-time "make hashes optional" patch) will be unable
to read files uploaded by this version, as they will complain about the
missing hashes. This patch is experimental, and is being pushed into
trunk to obtain test coverage. We may undo it before releasing 1.0.
2008-03-23 15:35:54 -07:00
Brian Warner
553367d567 download: make plaintext and ciphertext hashes in the UEB optional.
Removing the plaintext hashes can help with the guess-partial-information
attack. This does not affect compatibility, but if and when we actually
remove any hashes from the share, that will introduce a 
forwards-compatibility break: tahoe-0.9 will not be able to read such files.
2008-03-23 14:46:49 -07:00
robk-tahoe
f14c498e4f confwiz: refine the descriptive text on the conf wiz pages 2008-03-18 17:06:47 -07:00
robk-tahoe
605d693fa3 confwiz: reworked confwiz look and feel
this changes the confwiz to have a look and feel much more consistent
with that of the innosetup installer it is launched within the context
of.  this applies, naturally, primarily to windows.
2008-03-18 16:15:36 -07:00
robk-tahoe
e75c417bb5 webish mkdir-p: added unit test
added a test for the simple mkdir-p hack I added yesterday

checks that mkdir-p can create a directory hierarchy, and that resubmitting
a request for the same path yields the existing dir's uri
2008-03-18 13:11:08 -07:00
robk-tahoe
08fd16ef78 add a mkdir-p POST handler
this adds a t=mkdir-p call to directories (accessed by their uri as
/uri/<URI>?t=mkdir=p&path=/some/path) which returns the uri for a 
directory at a specified path before the given uri, regardless of
whether the directory exists or whether intermediate directories
need to be created to satisfy the request.

this is used by the migration code in MV to optimise the work of
path traversal which was other wise done on every file PUT
2008-03-17 18:13:01 -07:00
Brian Warner
75b52cda5c web/status.py: guard against file_size=None, closes #354 2008-03-17 13:31:10 -07:00
Zooko O'Whielacronx
7bc06872ac refactor one of the dispatch routines in webish.py
The behavior is intended to be unchanged by this refactoring.  Unit tests show no change in behavior.
2008-03-20 12:11:09 -07:00
Brian Warner
2ef70ab814 mutable.py: split replace() into update() and overwrite(). Addresses #328. 2008-03-12 18:00:43 -07:00
Zooko O'Whielacronx
0aa0efa123 storage servers announce that they will support clients as old as v0.8.0
Not that anyone pays attention to what storage servers claim about what versions they will support.
2008-03-13 09:10:11 -07:00
Brian Warner
45a1e655c7 introducer: record a timestamp with each announcement, and display it on the introducer's web page 2008-03-11 19:33:19 -07:00
Brian Warner
29e23626ad introducer: record a timestamp with each subscriber, and display it on the introducer's web page 2008-03-11 19:28:37 -07:00
Brian Warner
89be2e1bea introweb: combine announcement and subscriber information to show version+nickname for each client 2008-03-11 19:21:29 -07:00
Brian Warner
c21d30c320 client: publish a 'stub client' announcement to the introducer, to provide version/nickname information for each client 2008-03-11 19:20:10 -07:00
Brian Warner
0c40513e75 webish: put a trailing slash in the node.url file, since our unit tests use it, and some versions of twisted or nevow don't automatically add one 2008-03-11 19:01:18 -07:00
Brian Warner
fd5a922cad introweb.py: tolerate non-setLocationed client tubs 2008-03-11 18:09:13 -07:00
Brian Warner
3f5abe5cba oops, add introducer.xhtml 2008-03-11 17:41:03 -07:00
Brian Warner
810ba68343 add a webserver for the Introducer, showing service announcements and subscriber lists 2008-03-11 17:36:25 -07:00
Brian Warner
10d3ea5045 increase remote-interface size limits to 16EiB by not casually using 'int' as a constraint 2008-03-11 10:50:31 -07:00
Brian Warner
c727348d85 mutable: tolerate multiple encodings, using whichever version is recoverable first. Closes #312 2008-03-11 00:26:00 -07:00
Brian Warner
791482cf8d test_mutable: improve multiple-encodings test coverage 2008-03-10 23:47:35 -07:00
Brian Warner
9ca55b8b79 test_mutable: make test-multiple-encodings work 2008-03-10 23:16:28 -07:00
Brian Warner
57bd23f35f test_mutable: more test coverage, building up a framework to cause reads to occur in a specific order 2008-03-10 22:15:43 -07:00
Brian Warner
2464899f9a mutable: increase max segsize to 3.5MB, to allow dirnodes with about 10k entries 2008-03-10 20:39:55 -07:00
Zooko O'Whielacronx
a8a85da326 make the nodeurl start with '127.0.0.1' instead of with 'localhost'
This is because there exist in the wild computers that are misconfigured so that 'localhost' doesn't resolve to 127.0.0.1.  On those computers, using 'localhost' for the nodeurl is a security problem, because the user commonly sends valuable caps to the nodeurl.
2008-03-11 06:18:03 -07:00
Brian Warner
11445b5109 test_mutable: exercise short reads too 2008-03-10 18:08:23 -07:00
robk-tahoe
a1f2c9d624 confwiz: write accountname to a file
motivated simply by a desire to be able to identify 'noderoot' directories for
debugging and testing, the confwiz now writes an 'accountname' files based on 
what account was used when the node was configured.  this is not currently read
by or used by any code in the system, but helps identify directories from testing.
2008-03-10 17:50:44 -07:00
Brian Warner
be5a6147b4 test_mutable: test all hash-failure cases except a corrupted encrypted private key 2008-03-10 17:46:52 -07:00
Brian Warner
647734cd3b test_mutable: add Roundtrip test, suitable for new share-mangling tests 2008-03-10 16:14:08 -07:00
Brian Warner
9e460cd22e mutable: cosmetic changes 2008-03-10 15:44:05 -07:00