96 Commits

Author SHA1 Message Date
Itamar Turner-Trauring
5cd00d6f2e And even more tests passing in Python 3. 2020-09-24 10:55:47 -04:00
Itamar Turner-Trauring
03fd566e2c Fix flake error. 2020-09-17 12:37:10 -04:00
Itamar Turner-Trauring
e8743a607f Fix failing tests. 2020-09-16 14:37:16 -04:00
Itamar Turner-Trauring
b4116239b5 Merge remote-tracking branch 'origin/master' into 3415.storage-server-python-3 2020-09-16 11:27:03 -04:00
Itamar Turner-Trauring
a552bb53a7 Merge remote-tracking branch 'origin/master' into 3409.allmydata-storage-immutable-python-3 2020-09-15 09:05:55 -04:00
Itamar Turner-Trauring
65d38055e4 Merge branch '3409.allmydata-storage-immutable-python-3' into 3415.storage-server-python-3 2020-09-14 14:48:24 -04:00
Itamar Turner-Trauring
31aa594290 Looks like float is fine for mean. 2020-09-14 14:47:26 -04:00
Itamar Turner-Trauring
d84a7a61f3 Port to Python 3. 2020-09-14 14:46:08 -04:00
Itamar Turner-Trauring
1d508c74e8 Port to Python 3. 2020-09-14 14:13:07 -04:00
Itamar Turner-Trauring
6196a1c650 Port to Python 3. 2020-09-14 14:06:27 -04:00
Itamar Turner-Trauring
aef293fbca Port to Python 3. 2020-09-14 14:04:02 -04:00
Itamar Turner-Trauring
4590c1d1f5 Port to Python 3. 2020-09-14 13:56:45 -04:00
Itamar Turner-Trauring
7de84e32a3 Port to Python 3. 2020-09-11 15:02:42 -04:00
Itamar Turner-Trauring
6957540156 Assert nodeid is bytes, to ease porting. 2020-09-08 14:10:13 -04:00
Itamar Turner-Trauring
1cfe58a52d All of test_storage passes on Python 3. 2020-08-31 13:17:52 -04:00
Itamar Turner-Trauring
5ad5b79cdd More passing tests. 2020-08-28 10:53:52 -04:00
Itamar Turner-Trauring
9ce43231b4 More passing tests. 2020-08-27 15:49:04 -04:00
Itamar Turner-Trauring
6fd8ae1cc9 Finish port to Python 3. 2020-08-20 14:55:49 -04:00
Itamar Turner-Trauring
5d2bdf5883 Explicit bytes. 2020-08-20 14:32:41 -04:00
Itamar Turner-Trauring
be9f02cb13 Should be explicitly bytes. 2020-08-20 13:15:24 -04:00
Itamar Turner-Trauring
8279be38c1 Finish porting to Python 3. 2020-08-19 13:25:11 -04:00
Itamar Turner-Trauring
35ac5a62e7 Tests now pass on Python 3 too. 2020-08-19 12:15:39 -04:00
Itamar Turner-Trauring
ff582c5129 Some progress towards running crawler on Python 3. 2020-08-19 11:38:59 -04:00
Itamar Turner-Trauring
41ad4aea91 Make imports work. 2020-08-18 13:10:57 -04:00
Itamar Turner-Trauring
77ae087c77 Remove a2b_l and b2a_l from public API, to ease likely future switch to stdlib
base64's implementation.
2020-07-13 16:12:53 -04:00
meejah
6eade15b4e put WeakKeyDictionary use back 2020-05-05 14:49:16 -06:00
meejah
4c3d0ea6cc use 'with open' for more file-opens 2020-05-04 21:59:15 -06:00
meejah
4928d62d66 use set instead of WeakKeyDictionary 2020-05-04 21:59:15 -06:00
meejah
4b0c730adf use 'with open' correctly 2020-05-04 21:59:15 -06:00
Jean-Paul Calderone
de8199d3b0 Improve the behavior of get_slot_leases when no shares exist 2019-08-15 11:08:04 -04:00
Jean-Paul Calderone
fce9657ce3 it doesn't matter if any writes happen, only if the test vectors pass 2019-08-14 13:03:58 -04:00
Jean-Paul Calderone
bba8af6be1 note in the docstring about mutable shares 2019-08-14 12:29:26 -04:00
Jean-Paul Calderone
b3741ed3b5 Factor all of the logic into a helper with a flag controlling leases 2019-08-14 12:28:05 -04:00
Jean-Paul Calderone
6d414b0794 docstrings 2019-08-14 11:54:03 -04:00
Jean-Paul Calderone
74cb121b49 Add a test for uncovered remove-not-present-share case
Also fix the implementation to not blow up if the bucket for such a share
never existed.
2019-08-14 11:44:12 -04:00
Jean-Paul Calderone
dcc8f93b4e Refactor remote_slot_testv_and_readv_and_writev into some bite-sized pieces 2019-08-14 09:18:24 -04:00
tpltnt
c8f11dc2d3 ported old-style classes to new-style 2019-05-26 08:28:18 +02:00
meejah
49ae2c5eb7 use @implementer instead of implements
Also, derive some more things from 'object'
2017-02-27 11:01:30 -07:00
Leif Ryge
335c2ed06a add "Available" column to welcome page (#648)
add get_available_space() to NativeStorageServer

It uses a new 'available-space' key in the server's v1 version dict, or falls
back to 'maximum-immutable-share-size' (which presently always has the same
value but could have a different meaning in the future).

This is a squash merge of 9773555bb87fab71145ad7a0e84785a4e92d11f7
2014-11-23 06:03:00 +00:00
Brian Warner
99479226ed hush current pyflakes warnings (list comprehensions)
closes #2245
2014-06-23 13:54:07 -07:00
Daira Hopwood
d5651a0d0e Rename 'constant_time_compare' to 'timing_safe_compare'. refs #2165
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2014-02-24 20:43:23 +00:00
david-sarah
3cb99364e6 Failing to load a crawler state pickle uses default values, but the exception clause to detect this failure is too narrow; it can also fail with EOFError or KeyError for example. 2012-07-02 18:18:47 +00:00
david-sarah
33e2d2962e Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778 2012-06-22 15:43:54 +00:00
Brian Warner
bc21726dfd new introducer: signed extensible dictionary-based messages! refs #466
This introduces new client and server halves to the Introducer (renaming the
old one with a _V1 suffix). Both have fallbacks to accomodate talking to a
different version: the publishing client switches on whether the server's
.get_version() advertises V2 support, the server switches on which
subscription method was invoked by the subscribing client.

The V2 protocol sends a three-tuple of (serialized announcement dictionary,
signature, pubkey) for each announcement. The V2 server dispatches messages
to subscribers according to the service-name, and throws errors for invalid
signatures, but does not otherwise examine the messages. The V2 receiver's
subscription callback will receive a (serverid, ann_dict) pair. The
'serverid' will be equal to the pubkey if all of the following are true:

  the originating client is V2, and was told a privkey to use
  the announcement went through a V2 server
  the signature is valid

If not, 'serverid' will be equal to the tubid portion of the announced FURL,
as was the case for V1 receivers.

Servers will create a keypair if one does not exist yet, stored in
private/server.privkey .

The signed announcement dictionary puts the server FURL in a key named
"anonymous-storage-FURL", which anticipates upcoming Accounting-related
changes in the server advertisements. It also provides a key named
"permutation-seed-base32" to tell clients what permutation seed to use. This
is computed at startup, using tubid if there are existing shares, otherwise
the pubkey, to retain share-order compatibility for existing servers.
2012-03-13 18:24:32 -07:00
Zooko O'Whielacronx
32f80625c9 storage: more paranoid handling of bounds and palimpsests in mutable share files
* storage server ignores requests to extend shares by sending a new_length
* storage server fills exposed holes (created by sending a write vector whose offset begins after the end of the current data) with 0 to avoid "palimpsest" exposure of previous contents
* storage server zeroes out lease info at the old location when moving it to a new location
ref. #1528
2011-09-12 15:26:55 -07:00
Zooko O'Whielacronx
20e2910c61 immutable: prevent clients from reading past the end of share data, which would allow them to learn the cancellation secret
Declare explicitly that we prevent this problem in the server's version dict.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)
2011-09-12 15:24:58 -07:00
Zooko O'Whielacronx
5476f67dc1 storage: remove the storage server's "remote_cancel_lease" function
We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)
2011-09-12 15:23:31 -07:00
Brian Warner
01b5124d0a improve the storage/mutable.py asserts even more 2011-09-01 09:05:43 -07:00
wilcoxjg
d2e5de3b69 storage/mutable.py: special characters in struct.foo arguments indicate standard as opposed to native sizes, we should be using these characters in these asserts 2011-09-01 01:41:44 -07:00
david-sarah
c2972e22cb src/allmydata/storage/server.py: use the filesystem of storage/shares/, rather than storage/, to calculate remaining space. fixes #1384 2011-07-18 19:27:52 -07:00