Commit Graph

47 Commits

Author SHA1 Message Date
meejah
1549e7eb18 simplify (for line in f.readlines() -> for line in f) 2020-05-06 15:16:34 -06:00
meejah
4c3d0ea6cc use 'with open' for more file-opens 2020-05-04 21:59:15 -06:00
Jean-Paul Calderone
72cf590320 Use IStorageServer instead of RemoteReference in a lot of places 2019-05-31 13:41:07 -04:00
tpltnt
c8f11dc2d3 ported old-style classes to new-style 2019-05-26 08:28:18 +02:00
meejah
35810a5692 pull 'basedir' entirely into _Config
Put all config-related methods into _Config; change
code to ask config for paths instead of using basedir;
add some better docstrings
2018-08-25 02:10:44 -06:00
meejah
05f48c3601 Various cleanups, fixes and improvements
Squashed all commits that were meejah's between
30d68fb499f300a393fa0ced5980229f4bb6efda
and
33c268ed3a8c63a809f4403e307ecc13d848b1ab
On the branch meejah:1382.markberger-rewrite-rebase.6 as
per review
2017-06-05 16:31:41 -06:00
meejah
49ae2c5eb7 use @implementer instead of implements
Also, derive some more things from 'object'
2017-02-27 11:01:30 -07:00
Brian Warner
341e263e03 control.py: minor improvements
refs ticket:1737
2015-04-28 11:03:12 -07:00
Brian Warner
f0e783e20e fix check-memory test, with new new (safe) control-port methods 2015-04-21 13:05:58 -07:00
Brian Warner
2eea4e9a07 remove control-port upload/download file interfaces
FURLs are unguessable, but an attacker who somehow learned this FURL
could overwrite files and read sensitive data.

This will break the memory tests. I will add a new interface to support
the memory tests soon.

refs ticket:1737
2015-04-21 11:13:17 -07:00
Brian Warner
3d771132a8 switch UploadResults to use get_uri(), hide internal ._uri
Complete the getter-based transformation, by hiding ".uri" and updating
callers to use get_uri(). Also don't set a dummy self._uri, leave it
undefined until someone calls set_uri().
2012-05-21 21:14:44 -07:00
Brian Warner
f61bdbffd7 control.py: unbreak speed-test: overwrite() wants a MutableData, not str
Really, all the upload/modify APIs should take a string or a filehandle, and
internally wrap it as needed. Callers should not need to be aware of
Uploadable() or MutableData() classes.
2011-09-23 00:37:48 -07:00
Brian Warner
0f11d35f85 replace IServer.name() with get_name(), and get_longname()
This patch was originally written by Brian, but was re-recorded by Zooko to use
darcs replace instead of hunks for any file in which it would result in fewer
total hunks.
refs #1363
2011-08-01 10:44:28 -07:00
Brian Warner
70f9f89c66 control.py: remove all uses of s.get_serverid() 2011-02-26 19:12:03 -07:00
Brian Warner
ffd296fc5a Refactor StorageFarmBroker handling of servers
Pass around IServer instance instead of (peerid, rref) tuple. Replace
"descriptor" with "server". Other replacements:

 get_all_servers -> get_connected_servers/get_known_servers
 get_servers_for_index -> get_servers_for_psi (now returns IServers)

This change still needs to be pushed further down: lots of code is now
getting the IServer and then distributing (peerid, rref) internally.
Instead, it ought to distribute the IServer internally and delay
extracting a serverid or rref until the last moment.

no_network.py was updated to retain parallelism.
2011-02-20 17:58:04 -08:00
david-sarah
6057bc02cc Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements. 2010-01-26 22:44:30 -08:00
Brian Warner
696cc000d0 control.py: fix speedtest: use download_best_version (not read) on mutable nodes 2009-12-06 22:05:12 -08:00
Brian Warner
96834da0a2 Simplify immutable download API: use just filenode.read(consumer, offset, size)
* remove Downloader.download_to_data/download_to_filename/download_to_filehandle
* remove download.Data/FileName/FileHandle targets
* remove filenode.download/download_to_data/download_to_filename methods
* leave Downloader.download (the whole Downloader will go away eventually)
* add util.consumer.MemoryConsumer/download_to_data, for convenience
  (this is mostly used by unit tests, but it gets used by enough non-test
   code to warrant putting it in allmydata.util)
* update tests
* removes about 180 lines of code. Yay negative code days!

Overall plan is to rewrite immutable/download.py and leave filenode.read() as
the sole read-side API.
2009-12-01 17:53:30 -05:00
Brian Warner
131e05b155 clean up uri-vs-cap terminology, emphasize cap instances instead of URI strings
* "cap" means a python instance which encapsulates a filecap/dircap (uri.py)
 * "uri" means a string with a "URI:" prefix
 * FileNode instances are created with (and retain) a cap instance, and
   generate uri strings on demand
 * .get_cap/get_readcap/get_verifycap/get_repaircap return cap instances
 * .get_uri/get_readonly_uri return uri strings

* add filenode.download_to_filename() for control.py, should find a better way
* use MutableFileNode.init_from_cap, not .init_from_uri
* directory URI instances: use get_filenode_cap, not get_filenode_uri
* update/cleanup bench_dirnode.py to match, add Makefile target to run it
2009-11-11 14:26:19 -08:00
Brian Warner
4a4a4f9520 immutable.Downloader: pass StorageBroker to constructor, stop being a Service
child of the client, access with client.downloader instead of
client.getServiceNamed("downloader"). The single "Downloader" instance is
scheduled for demolition anyways, to be replaced by individual
filenode.download calls.
2009-08-15 12:25:43 -07:00
Brian Warner
8df15e9f30 big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging 2009-06-22 19:10:47 -07:00
Brian Warner
c9803d5217 switch all foolscap imports to use foolscap.api or foolscap.logging 2009-05-21 17:38:23 -07:00
Brian Warner
2308440fb6 control.py: use get_buckets() instead of get_version() to measure ping time, because the latter changed recently 2008-11-22 23:13:23 -07:00
Brian Warner
0eb6b324a4 #538: add remote_get_version() to four main Referenceable objects: Introducer Service, Storage Server, Helper, CHK Upload Helper. Remove unused storage-server get_versions(). 2008-11-21 17:43:52 -07:00
Brian Warner
3dd9b79e3f control.py: removed unused testutil.PollMixin 2008-10-28 21:03:59 -07:00
Brian Warner
7394607141 move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected. 2008-07-16 13:14:39 -07:00
Zooko O'Whielacronx
fc3bd0c987 use added secret to protect convergent encryption
Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key.  If you want traditional convergent encryption behavior, set the added secret to be the empty string.

This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names.  (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.)

This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 09:46:06 -07:00
Brian Warner
2ef70ab814 mutable.py: split replace() into update() and overwrite(). Addresses #328. 2008-03-12 18:00:43 -07:00
Brian Warner
66f33ee504 upload: return an UploadResults instance (with .uri) instead of just a URI 2008-02-05 21:01:38 -07:00
Brian Warner
daecca6589 big introducer refactoring: separate publish+subscribe. Addresses #271. 2008-02-05 13:05:13 -07:00
Brian Warner
909d848071 remove upload.upload_(data,filename,filehandle) convenience functions 2008-01-30 19:03:19 -07:00
Brian Warner
492cb92dc8 speedcheck: track SSK creation time separately 2008-01-29 20:44:32 -07:00
Brian Warner
f6b2072af1 check-speed: test SSK upload/download speed too. SDMF imposes a limit on the file sizes, no 10MB or 100MB test 2007-12-14 02:05:31 -07:00
Brian Warner
406ff52cf9 control.py: hush pyflakes warning 2007-12-06 18:38:42 -07:00
Zooko O'Whielacronx
4923d8dcb5 make control.get_memory_usage() return all zeroes if reading /proc/mem/whatsit raises an exception 2007-12-06 17:27:40 -07:00
Brian Warner
8fa19fa273 check_speed: average multiple pings when measuring RTT 2007-09-26 18:16:15 -07:00
Brian Warner
c301b41f50 control: add measure_peer_response_time(), to estimate RTT for the mesh 2007-09-26 12:21:15 -07:00
Brian Warner
0cfed96796 test_system.py: add coverage for allmydata.control 2007-09-26 12:06:55 -07:00
Brian Warner
d58d794757 check_speed: do both upload and download tests 2007-09-20 18:52:44 -07:00
Brian Warner
2632c0ad8c check_speed: upload multiple files, measure Ax+B 2007-09-20 16:55:33 -07:00
Brian Warner
3774ce59ea check_memory: fix race condition for startup of in-process server nodes 2007-09-20 15:33:58 -07:00
Brian Warner
b9d5a4ead4 check_speed.py: run two 1MB uploads and measure the time it takes 2007-09-19 18:40:18 -07:00
Brian Warner
6ff94541a8 control.py: make get_memory_usage() callable from elsewhere 2007-07-08 19:07:54 -07:00
Brian Warner
ea78b4b605 check_memory: getting closer, now we have memusage numbers for uploads of 10kB and 10MB files 2007-05-29 17:39:39 -07:00
Brian Warner
228e17560a control.py: fix get_memory_usage, add a sample client tool 2007-03-07 20:31:49 -07:00
Brian Warner
d4020fa533 control: add RIControlClient, fix some pyflakes warnings 2007-03-07 19:20:27 -07:00
Brian Warner
a71d83bd8e add a local foolscap control interface, to upload/download files and check memory usage 2007-03-07 19:16:06 -07:00