ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-27 08:22:32 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
14,387 Commits 185 Branches 90 Tags 89 MiB
e2e33933a8
Commit Graph

8462 Commits

Author SHA1 Message Date
Itamar Turner-Trauring
ac271cfb3d Merge remote-tracking branch 'origin/master' into 3877-immutable-storage-apis-continued 2022-03-09 11:23:02 -05:00
Itamar Turner-Trauring
86769c19bf Finish abort logic and tests. 2022-03-09 11:19:23 -05:00
Itamar Turner-Trauring
ef4f912a68 Less error-prone testing assertion, and fix a testing bug. 2022-03-09 11:11:39 -05:00
Itamar Turner-Trauring
4fc7ef7528 Basic HTTP test for aborts. 2022-03-09 10:57:05 -05:00
Itamar Turner-Trauring
f47741afb1 Correct behavior on timed out immutable uploads. 2022-03-09 10:45:21 -05:00
Itamar Turner-Trauring
92b952a5fe Authenticate writes! 2022-03-09 10:45:08 -05:00
Itamar Turner-Trauring
c642218173 Sketch of aborting uploads. 2022-03-08 10:41:56 -05:00
Itamar Turner-Trauring
5203873995 Refactor to unify data structure logic. 2022-03-08 10:13:37 -05:00
Itamar Turner-Trauring
5be7cbc171
Merge pull request #1182 from tahoe-lafs/3876-per-bucket-upload-secret 
Allow per-bucket upload secret

Fixes ticket:3876
 2022-03-08 10:10:48 -05:00
Itamar Turner-Trauring
7721c134f2 Change the semantics of HTTP bucket creation so that it's possible to have a 
different upload secret per upload.
 2022-03-07 09:28:21 -05:00
Itamar Turner-Trauring
40e5ab1661
Merge pull request #1178 from tahoe-lafs/3860-http-more-immutables 
More immutable support for HTTP storage API

Fixes ticket:3860
 2022-03-07 09:12:56 -05:00
Itamar Turner-Trauring
87ab56426a Validate another edge case of bad storage index. 2022-03-07 08:46:05 -05:00
Itamar Turner-Trauring
4efa65d3db Typo. 2022-03-07 08:29:26 -05:00
Itamar Turner-Trauring
60bcd5fe9f Address review comments. 2022-03-07 08:25:12 -05:00
Itamar Turner-Trauring
9f4f6668c0 Tweaks. 2022-03-07 08:21:58 -05:00
Itamar Turner-Trauring
7146cff227 Sketch of TLS listening and furl construction for the HTTP storage server. 2022-03-02 10:40:39 -05:00
Itamar Turner-Trauring
32cbc7b9df Function for getting SPKI hash. 2022-03-02 10:35:41 -05:00
Itamar Turner-Trauring
f81cd6e595 Use an option also available on PyPy3. 2022-02-23 10:57:36 -05:00
meejah
2928a480ff RSA key-size is not configurable, it's 2048bits 2022-02-16 21:46:24 -07:00
meejah
5cd95920b6 Merge branch 'master' into 3828.key-length.1 2022-02-16 21:18:50 -07:00
Itamar Turner-Trauring
5aa00abc3d Use the correct API (since direct returns break Python 2 imports) 2022-02-11 15:02:14 -05:00
Itamar Turner-Trauring
abf3048ab3 More passing HTTP IStorageServer tests. 2022-02-10 17:07:21 -05:00
Itamar Turner-Trauring
7466ee25a8 Don't send header if it makes no sense to do so. 2022-02-10 13:57:57 -05:00
Itamar Turner-Trauring
b049d4a792 Fix get_version with new API. 2022-02-10 13:52:47 -05:00
Itamar Turner-Trauring
fa2f142bc9 Another ticket. 2022-02-10 13:50:09 -05:00
Itamar Turner-Trauring
aa68be645f Return Content-Range in responses. 2022-02-10 13:48:33 -05:00
Itamar Turner-Trauring
416af7328c Support lack of Range header. 2022-02-10 13:31:09 -05:00
Itamar Turner-Trauring
7db1ddd875 Implement Range header validation. 2022-02-10 13:15:48 -05:00
Itamar Turner-Trauring
5d9e0c9bca Not found tests and implementation. 2022-02-09 13:14:27 -05:00
Itamar Turner-Trauring
45ee5e3346 Done elsewhere. 2022-02-09 13:08:34 -05:00
Itamar Turner-Trauring
bae5d58ab9 Another test. 2022-02-09 13:07:34 -05:00
Itamar Turner-Trauring
faacde4e32 Conflicting writes. 2022-02-09 12:41:32 -05:00
Itamar Turner-Trauring
8c739343f3 Reduce duplication. 2022-02-09 12:38:32 -05:00
Itamar Turner-Trauring
95d7548629 Upload to non-existent place. 2022-02-09 12:30:38 -05:00
Itamar Turner-Trauring
72bac785ee Done elsewhere. 2022-02-09 12:27:08 -05:00
Itamar Turner-Trauring
ecb1a3c5a0 Just require content-range for simplicity. 2022-02-09 12:25:47 -05:00
Itamar Turner-Trauring
d38183335e Handle bad Content-Range headers. 2022-02-08 10:46:55 -05:00
Itamar Turner-Trauring
7107a85fba Refactor client, separating low-level and high-level concerns. 2022-02-08 10:19:37 -05:00
Itamar Turner-Trauring
ce2468cdff Validate inputs automatically as part of parsing. 2022-02-04 10:35:38 -05:00
Itamar Turner-Trauring
e6f32e33bc Merge branch '3871-http-list-storage-index-shares' into 3860-http-more-immutables 2022-02-04 09:30:17 -05:00
Itamar Turner-Trauring
5e3a31166d Better explanation. 2022-02-04 09:26:58 -05:00
Itamar Turner-Trauring
7454929be0 Less code duplication. 2022-02-04 09:26:25 -05:00
Itamar Turner-Trauring
52322c25d3 Merge remote-tracking branch 'origin/master' into 3871-http-list-storage-index-shares 2022-02-04 09:19:51 -05:00
Itamar Turner-Trauring
6b3722d3f6 Avoid using possibly-private API. 2022-02-03 12:50:29 -05:00
Itamar Turner-Trauring
23c8bde9d5 Nicer cleanup. 2022-02-03 12:44:55 -05:00
Itamar Turner-Trauring
1dfc0bde36 Use better method to listen on random port. 2022-02-03 12:43:49 -05:00
Itamar Turner-Trauring
39fe48b174 More passing IStorageServer tests. 2022-02-02 12:55:41 -05:00
Itamar Turner-Trauring
bceed6e199 More bucket allocation logic. 2022-02-02 12:41:25 -05:00
Itamar Turner-Trauring
aebb5056de Don't use real reactor in these tests. 2022-02-02 11:00:16 -05:00
Itamar Turner-Trauring
70d0bd0597 Test and document what happens for non-existent storage index. 2022-02-01 10:41:12 -05:00
Itamar Turner-Trauring
35bbcad4de Merge branch '3868-istorageserver-tests-http' into 3871-http-list-storage-index-shares 2022-02-01 10:36:55 -05:00
Itamar Turner-Trauring
0fbf746e27 Skip on Python 2. 2022-02-01 10:30:27 -05:00
Itamar Turner-Trauring
48a9bf7457 Hook up more IStorageServer tests that can now pass with HTTP. 2022-02-01 10:25:13 -05:00
Itamar Turner-Trauring
c72e7b0585 Implement HTTP share listing endpoint. 2022-02-01 10:20:23 -05:00
Itamar Turner-Trauring
c2e524ddb8 Make mypy happy. 2022-02-01 09:55:13 -05:00
Itamar Turner-Trauring
5dfaa82ed2 Skip tests that don't pass. 2022-02-01 09:47:51 -05:00
Itamar Turner-Trauring
66abe5dfca First passing immutable-API-over-HTTP IStorageServer tests. 2022-01-31 12:02:52 -05:00
Itamar Turner-Trauring
ef56435578 Merge branch '3855-immutable-http-apis-part-1' into 3868-istorageserver-tests-http 2022-01-31 11:36:28 -05:00
Itamar Turner-Trauring
e672029e6d First HTTP test passes. 2022-01-31 10:49:43 -05:00
Itamar Turner-Trauring
77f6f5d176 Merge remote-tracking branch 'origin/master' into 3855-immutable-http-apis-part-1 2022-01-31 10:36:54 -05:00
Jean-Paul Calderone
640bb05a57 Merge remote-tracking branch 'origin/master' into 3788.refactor-src.test.mutable 2022-01-28 13:07:48 -05:00
Itamar Turner-Trauring
d23fdcdb8a Sketch of first IStorageServer test with HTTP server/client. 2022-01-28 12:03:17 -05:00
Itamar Turner-Trauring
2583236ad8 Fix unused import. 2022-01-25 10:56:45 -05:00
Itamar Turner-Trauring
609bd819bb Merge remote-tracking branch 'origin/master' into 3855-immutable-http-apis-part-1 2022-01-25 10:42:04 -05:00
Itamar Turner-Trauring
e1f9f7de94 Note for future improvement. 2022-01-25 10:06:23 -05:00
Itamar Turner-Trauring
0346dfea60 Note we can do this now. 2022-01-25 09:56:54 -05:00
Itamar Turner-Trauring
b64e6552a4 Fix assertion. 2022-01-24 11:30:41 -05:00
Itamar Turner-Trauring
d2e3b74098 Some progress towards upload progress result from the server. 2022-01-21 12:36:58 -05:00
Itamar Turner-Trauring
a4cb4837e6 It's a secret, compare it securely. 2022-01-21 11:43:36 -05:00
Itamar Turner-Trauring
e8e3a3e663 Expand. 2022-01-21 11:37:46 -05:00
Itamar Turner-Trauring
c4d71a4636 Use abstractions for generating headers on client, note another place we should 
generate headers.
 2022-01-20 13:12:25 -05:00
Itamar Turner-Trauring
65787e5603 Get rid of inlineCallbacks. 2022-01-20 12:57:52 -05:00
Itamar Turner-Trauring
4b5c71ffbc Bit more info. 2022-01-20 12:50:36 -05:00
Itamar Turner-Trauring
b952e738dd Try to clarify. 2022-01-20 12:43:01 -05:00
Itamar Turner-Trauring
2a2ab1ead7 Use a set, not a list, for share numbers. 2022-01-20 12:39:25 -05:00
Itamar Turner-Trauring
587a510b06 Note a better way to implement this. 2022-01-20 12:38:01 -05:00
Itamar Turner-Trauring
9a0a19c15a Reminder we might want to support JSON too. 2022-01-20 12:07:58 -05:00
Itamar Turner-Trauring
5fa8c78f97 Don't use reactor, since it's not necessary. 2022-01-20 12:04:20 -05:00
Itamar Turner-Trauring
f09aa8c796 Use pre-existing parser for Range and Content-Range headers. 2022-01-20 11:16:06 -05:00
Itamar Turner-Trauring
d5bac8e186 Make sure upload secret semantics are still supporting the security goals. 2022-01-20 10:56:08 -05:00
Itamar Turner-Trauring
1bf2b2ee5f Note follow-up issue. 2022-01-20 10:52:44 -05:00
Itamar Turner-Trauring
23368fc9d9 Move tests back into unittest module. 2022-01-20 10:34:09 -05:00
Itamar Turner-Trauring
406a06a508 Make sure we don't violate the Foolscap interface definition for this method. 2022-01-17 14:38:06 -05:00
Itamar Turner-Trauring
28dbdbe019 Make sure return type is consistent. 2022-01-17 14:31:29 -05:00
Itamar Turner-Trauring
7aed7dbd8a Make module import on Python 2 (so tests can pass). 2022-01-17 14:24:28 -05:00
Itamar Turner-Trauring
79cd9a3d6d Fix lint. 2022-01-17 14:22:15 -05:00
Itamar Turner-Trauring
d4ae7c89aa First end-to-end immutable upload then download test passes. 2022-01-17 14:20:40 -05:00
Itamar Turner-Trauring
25e2100219 Immutable writing now knows when it's finished. 2022-01-17 14:06:21 -05:00
Itamar Turner-Trauring
4ea6bf2381 A test and some progress to making it pass. 2022-01-15 12:59:23 -05:00
Itamar Turner-Trauring
3bed067828 Implement more of the writing logic. 2022-01-14 08:34:17 -05:00
Itamar Turner-Trauring
f5437d9be7 Some progress towards bucket allocation endpoint, and defining the protocol better. 2022-01-12 11:51:56 -05:00
Itamar Turner-Trauring
018f53105e Pass correct arguments. 2022-01-12 11:16:39 -05:00
Itamar Turner-Trauring
2bccb01be4 Fix bug wrapping endpoints. 2022-01-12 11:16:21 -05:00
Itamar Turner-Trauring
6e2aaa8391 Refactor more integration-y tests out. 2022-01-12 09:14:58 -05:00
Itamar Turner-Trauring
004e5fbc9d Get to point where we get failing HTTP response. 2022-01-11 15:47:32 -05:00
Itamar Turner-Trauring
2369de6873 Simple upload/download test for immutables. 2022-01-11 15:45:15 -05:00
Itamar Turner-Trauring
040569b47a Sketch of tests to write for basic HTTP immutable APIs. 2022-01-11 15:11:16 -05:00
Itamar Turner-Trauring
db68defe88 Sketch of basic immutable server-side logic. 2022-01-11 14:50:29 -05:00
Itamar Turner-Trauring
57405ea722 Finish sketch of minimal immutable HTTP client code. 2022-01-11 14:37:32 -05:00
meejah
db5c65e530 Merge branch 'master' into 3854.mime-parsing-bug 2022-01-07 12:16:25 -07:00
meejah
9644532916 don't depend on order 2022-01-07 11:28:55 -07:00
meejah
a49baf44b6 actually-reproduce 3852 2022-01-07 10:59:13 -07:00
meejah
11f2097591 docstring 2022-01-07 10:58:58 -07:00
meejah
983f90116b check differently, don't depend on order 2022-01-06 13:15:31 -07:00
Itamar Turner-Trauring
2864ff872d Another MIME type that needs to be handled by FieldStorage. 2022-01-06 13:34:56 -05:00
Itamar Turner-Trauring
9f5d7c6d22 Fix a bug where we did unnecessary parsing. 2022-01-06 13:09:25 -05:00
Itamar Turner-Trauring
2f94fdf372 Extra testing coverage, including reproducer for #3854. 2022-01-06 13:09:19 -05:00
Itamar Turner-Trauring
90a25d0109 Reorganize into shared file. 2022-01-06 12:36:46 -05:00
Itamar Turner-Trauring
9c20ac8e7b Client API sketch for basic immutable interactions. 2022-01-05 16:06:29 -05:00
Itamar Turner-Trauring
5f4db487f7 Sketch of required business logic. 2022-01-04 13:43:19 -05:00
meejah
5be5714bb3 fix; get rid of sorting 2021-12-22 15:32:40 -07:00
meejah
94b540215f args are bytes 2021-12-22 15:32:30 -07:00
meejah
e9aaaaccc4 test for json welcome page 2021-12-22 15:31:09 -07:00
Itamar Turner-Trauring
58a71517c1 Correct way to skip with testtools. 2021-12-22 13:16:43 -05:00
Itamar Turner-Trauring
7a0c83e71b Split up test. 2021-12-22 11:52:13 -05:00
Itamar Turner-Trauring
8b4d166a54 Use hypothesis for another test. 2021-12-22 11:44:45 -05:00
Itamar Turner-Trauring
776f19cbb2 Even more hypothesis, this time for secrets' contents. 2021-12-21 12:34:02 -05:00
Itamar Turner-Trauring
b1f4e82adf Switch to using hypothesis. 2021-12-21 11:55:16 -05:00
Itamar Turner-Trauring
2bda2a0127 Switch to using a fixture. 2021-12-21 11:10:53 -05:00
Itamar Turner-Trauring
1721865b20 No longer TODO. 2021-12-20 13:46:19 -05:00
Itamar Turner-Trauring
291b4e1896 Use more secure comparison to prevent timing-based side-channel attacks. 2021-12-20 11:17:27 -05:00
Itamar Turner-Trauring
a529ba7d5e More skipping on Python 2. 2021-12-17 09:14:09 -05:00
Itamar Turner-Trauring
81b95f3335 Ensure secret was validated. 2021-12-16 11:53:31 -05:00
Itamar Turner-Trauring
428a9d0573 Lint fix. 2021-12-16 11:47:40 -05:00
Itamar Turner-Trauring
fb0be6b894 Enforce length of lease secrets. 2021-12-16 11:46:35 -05:00
Itamar Turner-Trauring
816dc0c73f X-Tahoe-Authorization can be validated and are passed to server methods. 2021-12-16 11:42:06 -05:00
Itamar Turner-Trauring
da52a9aede Test for server-side secret handling. 2021-12-16 11:17:32 -05:00
Itamar Turner-Trauring
87fa9ac2a8 Infrastructure for sending secrets. 2021-12-16 11:17:11 -05:00
Itamar Turner-Trauring
b32374c8bc Secret header parsing. 2021-12-16 10:39:58 -05:00
Itamar Turner-Trauring
2b6e1285e7 Merge remote-tracking branch 'origin/master' into 3849-refactor-out-foolscap-in-storage-server 2021-12-09 13:37:37 -05:00
meejah
c3e574b2ee Merge remote-tracking branch 'origin/master' into release-1.17.0 2021-12-06 11:27:34 -07:00
Jean-Paul Calderone
5f6579d446 hew closer to security/master version of these lines 2021-12-06 12:52:31 -05:00
meejah
50cdd9bd96 unused 2021-12-04 17:20:10 -07:00
meejah
5bb6fbc51f merge errors 2021-12-04 10:14:31 -07:00
Itamar Turner-Trauring
c8f429c496 Merge remote-tracking branch 'origin/master' into 3849-refactor-out-foolscap-in-storage-server 2021-12-03 13:52:42 -05:00
Itamar Turner-Trauring
90f8480cf0 Make more of the unittests pass again with the StorageServer factoring. 2021-12-03 13:39:53 -05:00
Itamar Turner-Trauring
314b202914 Ignore another field which can change. 2021-12-03 12:58:12 -05:00
meejah
18891d9e77 Merge remote-tracking branch 'security/master' 2021-12-02 14:44:59 -07:00
Itamar Turner-Trauring
439e5f2998 Insofar as possible, switch to testing without the Foolscap API. 2021-12-02 10:52:03 -05:00
Itamar Turner-Trauring
8c3d61a94e Bit more backwards compatible. 2021-12-02 10:49:23 -05:00
Itamar Turner-Trauring
476c41e49e Split out Foolscap code from BucketReader/Writer. 2021-12-02 10:29:52 -05:00
Itamar Turner-Trauring
f7cb4d5c92 Hook up the new FoolscapStorageServer, and fix enough bugs, such that almost all 
end-to-end and integration tests pass.
 2021-12-02 10:07:14 -05:00
Itamar Turner-Trauring
6b8a42b043 Make the test more robust. 2021-12-02 09:35:37 -05:00
Jean-Paul Calderone
82099f04c4 Merge remote-tracking branch 'origin/master' into 3758.refactor.web-tests-grid-logs-root 2021-12-02 08:36:35 -05:00
meejah
4bc0df7cc1 file, not path 2021-12-02 00:14:15 -07:00
meejah
40e7be6d8d needs reason 2021-12-01 23:46:10 -07:00
meejah
e0092ededa fine, just skip tests on windows 2021-12-01 20:52:22 -07:00
Jean-Paul Calderone
8626043f61
Merge pull request #1163 from tahoe-lafs/3847.eliot-anybytes-always 
Use the "any bytes" encoder even on Python 2

Fixes: ticket:3847
 2021-12-01 15:47:16 -05:00
Jean-Paul Calderone
eee1f0975d note about how to clean this up later 2021-12-01 15:16:16 -05:00
Jean-Paul Calderone
90d1e90a14 rewrite the Eliot interaction tests to make expected behavior clearer 
and to have explicit assertions about that behavior
 2021-12-01 15:05:29 -05:00
meejah
940c6343cf consistency 2021-12-01 12:02:42 -07:00
meejah
7080ee6fc7 oops 2021-12-01 12:02:06 -07:00
meejah
25ca767095 an offering to the windows godesses 2021-12-01 11:49:52 -07:00
Itamar Turner-Trauring
f7975b9772
Merge pull request #1161 from tahoe-lafs/3843-start-http-storage-server 
A start at a HTTP-based storage server and client

Fixes ticket:3843
Fixes ticket:3762
 2021-12-01 10:18:36 -05:00
Itamar Turner-Trauring
50e21a9034 Split StorageServer into generic part and Foolscap part. 2021-12-01 09:55:44 -05:00
Itamar Turner-Trauring
6e1f6f68ca Merge branch '3843-start-http-storage-server' into 3849-refactor-out-foolscap-in-storage-server 2021-12-01 09:41:21 -05:00
Itamar Turner-Trauring
18a5966f1d Don't bother running HTTP server tests on Python 2, since it's going away any 
day now.
 2021-12-01 09:38:56 -05:00
meejah
85fa8fe32e py2/py3 glue code for json dumping 2021-11-30 23:00:59 -07:00
meejah
b47381401c flake8 2021-11-30 19:01:09 -07:00
meejah
679c464517 tests 2021-11-30 18:43:06 -07:00
meejah
fc9671a812 simplify, flake9 2021-11-30 18:25:32 -07:00
meejah
0a4bc385c5 fix tests to use migrate command 2021-11-30 18:00:58 -07:00
meejah
1b8ae8039e no auto-migrate; produce error if pickle-files exist 2021-11-30 16:01:15 -07:00
meejah
3fd1ca8acb it's an error to have pickle-format files 2021-11-30 16:00:35 -07:00
meejah
49f2489321 explicit 'migrate pickle files' command 2021-11-30 15:59:27 -07:00
Jean-Paul Calderone
c1a2964788 Merge remote-tracking branch 'origin/master' into 3527.test_create-no-mock 2021-11-30 15:06:17 -05:00
Jean-Paul Calderone
7626a02bdb remove redundant assertion 2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
20e0626e42 add capture_logging that parameterizes JSON encoder 2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
3eb1a5e7cb Add a MemoryLogger that prefers our encoder 
and use it instead of Eliot's
 2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
f40da7dc27 Put the choice of JSON encoder for Eliot into its own module 
and use it in a few places
 2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
66a0c6f3f4 add a direct test for the non-utf-8 bytestring behavior 2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
c02d8cab3a change one more assertTrue to assertEquals 2021-11-29 08:56:05 -05:00
fenn-cs
118e323b9e Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3788.refactor-src.test.mutable 2021-11-27 00:59:50 +01:00
fenn-cs
54c032d0d7 change assertTrue -> assertEquals for non bools 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-11-27 00:59:13 +01:00
meejah
5855a30e34 add docstrings 2021-11-23 14:01:43 -07:00
meejah
b0309331a5 Merge branch 'master' into 3825.json-pickle 2021-11-23 13:53:41 -07:00
Itamar Turner-Trauring
a593095dc9 Explain why it's a conditional import. 2021-11-23 10:39:53 -05:00
Itamar Turner-Trauring
c921b153f4 A better name for the API. 2021-11-23 10:39:15 -05:00
Itamar Turner-Trauring
c341a86abd Correct the comment. 2021-11-23 10:01:03 -05:00
Jean-Paul Calderone
d1839187f1 "misplaced type annotation" 2021-11-22 08:45:10 -05:00
Jean-Paul Calderone
b92343c664 some more docstrings 2021-11-22 08:36:12 -05:00
Jean-Paul Calderone
e0c0c32c13 Merge remote-tracking branch 'remediate/master' into LFS-01-010.mutable 2021-11-22 07:59:43 -05:00
Jean-Paul Calderone
04e45f065a document compare_leases_without_timestamps 2021-11-22 07:59:01 -05:00
Itamar Turner-Trauring
4c11177387 Fix a problem with typechecking. 
Using remote_write() isn't quite right given move to HTTP, but can fight that
battle another day.
 2021-11-18 15:58:55 -05:00
Itamar Turner-Trauring
e2636466b5 Fix a flake. 2021-11-18 15:47:25 -05:00
Itamar Turner-Trauring
bd645edd9e Fix flake. 2021-11-18 15:44:51 -05:00
Itamar Turner-Trauring
5d915afe1c Clean up BucketWriters on shutdown (also preventing DelayedCalls leaks in tests). 2021-11-18 15:42:54 -05:00
Itamar Turner-Trauring
1827faf36b Fix issue with leaked-past-end-of-test DelayedCalls. 2021-11-18 14:45:44 -05:00
Itamar Turner-Trauring
8c8e377466 Implement timeout and corresponding tests. 2021-11-18 14:35:04 -05:00
Itamar Turner-Trauring
5e341ad43a New tests to write. 2021-11-17 11:29:34 -05:00
Itamar Turner-Trauring
45c00e93c9 Use clock in BucketWriter. 2021-11-17 11:12:40 -05:00
Itamar Turner-Trauring
92c36a67d8 Use IReactorTime instead of ad-hoc solutions. 2021-11-17 11:01:04 -05:00
Jean-Paul Calderone
e8adca40ab give the ContainerVersionError exceptions a nice str 2021-11-16 18:36:59 -05:00
Jean-Paul Calderone
3a8432713f a note about what's happening with proxyForInterface 2021-11-16 18:29:05 -05:00
Jean-Paul Calderone
a208502e18 whitespace 2021-11-16 18:29:01 -05:00
meejah
2fe686135b rename data to appease distutils 2021-11-16 15:08:31 -07:00
meejah
23ff1b2430 noqa 2021-11-16 15:08:31 -07:00
meejah
1c93175583 cleanup 2021-11-16 15:08:31 -07:00
meejah
4f64bbaa00 data 2021-11-16 15:08:31 -07:00
meejah
9b3c55e4aa test a second deserialzation 2021-11-16 15:08:31 -07:00
meejah
069c332a68 straight assert 2021-11-16 15:08:31 -07:00
meejah
94670461f1 tests 2021-11-16 15:08:31 -07:00
meejah
a867294e00 dead 2021-11-16 15:08:31 -07:00
meejah
75410e51f0 refactor 2021-11-16 15:08:31 -07:00
meejah
d4fc14f9ad docstring 2021-11-16 15:08:31 -07:00
meejah
89c2aacadc working test of 'in the wild' data, working converters 2021-11-16 15:08:31 -07:00
meejah
bf5e682d71 test upgrade of main state works 2021-11-16 15:08:31 -07:00
meejah
f81e4e2d25 refactor to use serializers / pickle->json upgraders 2021-11-16 15:08:31 -07:00
meejah
fa6950f08d an old pickle-format lease-checker state file 2021-11-16 15:08:31 -07:00
meejah
bb70e00065 Make internal state JSON-able for lease-crawler 2021-11-16 15:08:31 -07:00
meejah
f7b385f954 play nice with subclasses 2021-11-16 15:08:31 -07:00
meejah
3b69df36b0 crawler: pickle -> json 2021-11-16 15:08:31 -07:00
Itamar Turner-Trauring
f549488bb5 Don't use a deprecated API. 2021-11-16 11:47:09 -05:00
Itamar Turner-Trauring
a64778ddb0 Flakes. 2021-11-16 11:28:13 -05:00
Itamar Turner-Trauring
c195f895db Python 2 support. 2021-11-16 11:16:26 -05:00
Itamar Turner-Trauring
171d1053ec CBOR content-type on responses. 2021-11-16 11:09:17 -05:00
Itamar Turner-Trauring
671b670154 Some type annotations. 2021-11-16 11:09:08 -05:00
Itamar Turner-Trauring
41ec63f758 Passing first tests. 2021-11-16 10:56:21 -05:00
Jean-Paul Calderone
fcd634fc43 some direct tests for the new utility function 2021-11-15 13:34:46 -05:00
Jean-Paul Calderone
84c19f5468 clarify renew_leases docs on slot_testv_and_readv_and_writev 2021-11-15 08:12:07 -05:00
Jean-Paul Calderone
ad6017e63d clarify renew_leases docs on allocate_buckets 2021-11-15 08:11:53 -05:00
Jean-Paul Calderone
6c2e85e991 put the comment back 2021-11-12 16:25:36 -05:00
Jean-Paul Calderone
dece67ee3a it is not the remote interface that varies anymore 2021-11-12 16:24:29 -05:00
Jean-Paul Calderone
85977e48a7 put this comment back and merge info from the two versions 2021-11-12 16:23:15 -05:00
Jean-Paul Calderone
c3cb0ebaea Switch to per-call parameter for controlling lease renewal behavior 
This is closer to an implementation where you could have two frontends, say a
Foolscap frontend and an HTTP frontend or even just two different HTTP
frontends, which had different opinions about what the behaviour should be.
 2021-11-12 16:20:27 -05:00
Itamar Turner-Trauring
c101dd4dc9 Closer to first passing test. 2021-11-12 13:13:19 -05:00
Itamar Turner-Trauring
12cbf8a901 First sketch of HTTP testing infrastructure. 2021-11-12 13:03:53 -05:00
Itamar Turner-Trauring
ddd2780bd2 First sketch of HTTP client. 2021-11-12 12:51:52 -05:00
Itamar Turner-Trauring
a32c6be978 A sketch of what the HTTP server will look like. 2021-11-12 12:02:58 -05:00
Jean-Paul Calderone
2742de6f7c drop some ancient cruft 
allocated_size not used anywhere, so why have it
 2021-11-10 16:08:53 -05:00
Jean-Paul Calderone
9af81d21c5 add a way to turn off implicit bucket lease renewal too 2021-11-10 16:08:40 -05:00
Jean-Paul Calderone
617a1eac9d refactor lease hashing logic to avoid mutable/immutable duplication 2021-11-05 15:30:49 -04:00
Jean-Paul Calderone
456df65a07 Add v2 of the mutable container schema 
It uses hashed lease secrets, like v2 of the immutable container schema.
 2021-11-05 14:16:43 -04:00
Jean-Paul Calderone
3de9c73b0b preserve the type when renewing HashedLeaseInfo 
does this mean immutable lease renewal is untested?  maybe
 2021-11-05 14:11:05 -04:00
Jean-Paul Calderone
5d703d9893 some type annotations 2021-11-05 14:10:27 -04:00
Jean-Paul Calderone
0cd96ed713 fix the debug tool for the hashed lease secret case 2021-11-05 14:09:46 -04:00
Jean-Paul Calderone
8adff050a7 compare without breaking out all of the fields 
HashedLeaseInfo doesn't have all of these attributes
 2021-11-05 13:51:46 -04:00
Jean-Paul Calderone
07ffde3bec
Merge pull request #1158 from tahoe-lafs/3838.more-main-tub-cleanup 
More "main tub" initialization cleanup in test code

Fixes: ticket:3838
 2021-11-05 11:13:23 -04:00
Jean-Paul Calderone
728638fe23 apply the MutableShareFile tests to all known schemas 2021-11-04 15:37:29 -04:00
Jean-Paul Calderone
931ddf85a5 introduce an explicit representation of the v1 mutable container schema 
This is only a partial representation, sufficient to express the changes that
are coming in v2.
 2021-11-04 15:26:58 -04:00
Jean-Paul Calderone
2186bfcc37 silence some mypy errors :/ 
I don't know the "right" way to make mypy happy with these things
 2021-11-04 14:40:43 -04:00
Jean-Paul Calderone
6889ab2a76 fix syntax of type hint 2021-11-04 14:16:55 -04:00
Jean-Paul Calderone
7a59aa83bb add missing import 2021-11-04 14:12:54 -04:00
Jean-Paul Calderone
b69e8d013b introduce immutable container schema version 2 
This version used on-disk hashed secrets to reduce the chance of secrets
leaking to unintended parties.
 2021-11-04 14:07:49 -04:00
Jean-Paul Calderone
234b8dcde2 Formalize LeaseInfo interface in preparation for another implementation 2021-11-04 12:07:24 -04:00
Jean-Paul Calderone
3b41419523 apply the ShareFile tests to all schema versions using hypothesis 2021-11-04 10:32:59 -04:00
Jean-Paul Calderone
10724a91f9 introduce an explicit representation of the v1 immutable container schema 
This is only a partial representation, sufficient to express the changes that
are coming in v2.
 2021-11-04 10:17:36 -04:00
Jean-Paul Calderone
274dc6e837 Introduce UnknownContainerVersionError base w/ structured args 2021-11-04 10:06:26 -04:00
Jean-Paul Calderone
a71b1d31bf Merge remote-tracking branch 'upstream/master' 2021-11-04 10:02:54 -04:00
Jean-Paul Calderone
780be2691b assign a tub.port to all system test nodes 2021-11-03 16:11:28 -04:00
Jean-Paul Calderone
5caa80fe38 use UseNode more in test_client.py 
Also make write_introducer more lenient about filesystem state
 2021-11-03 16:11:08 -04:00
Jean-Paul Calderone
5a71774bf8 use port assigner and UseNode more in test_node.py 2021-11-03 16:10:32 -04:00
Jean-Paul Calderone
31649890ef Teach UseNode to use a port assigner for tub.port 
Then use it to assign ports for tub.port unless the caller supplied their own
value.
 2021-11-03 16:08:08 -04:00
Jean-Paul Calderone
797e099459 make create_introducer_webish assign a main tub port 2021-11-03 16:05:28 -04:00
Jean-Paul Calderone
7335b2a597 remove unused import 2021-11-03 13:16:15 -04:00
Jean-Paul Calderone
8e150cce6a add explicit direct tests for the new methods 2021-11-03 13:14:55 -04:00
Jean-Paul Calderone
70a7500c56 Merge remote-tracking branch 'origin/master' into 3834.renew_secret_comparison_helper 2021-11-03 13:05:41 -04:00
Jean-Paul Calderone
e8ed711d26 Merge remote-tracking branch 'origin/master' into 3833.container-format-abstraction 2021-11-03 12:25:38 -04:00
Jean-Paul Calderone
5bfe6d5b24
Merge pull request #1156 from tahoe-lafs/3836.systemtests-introducer-main-tub-port 
Use the "port assigner" helper on the introducer main tub in SystemTestsMixin

Fixes: 3836
 2021-11-03 12:25:09 -04:00
meejah
08cf881e28 test with real-size keys 2021-11-02 22:16:14 -06:00
Jean-Paul Calderone
39c4a2c4eb tidy up some corners 2021-11-02 15:10:54 -04:00
fenn-cs
fcfc89e3ae moved new tests/update for eliotutils 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-11-02 14:32:20 +01:00
fenn-cs
11f45b5c3c Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3758.refactor.web-tests-grid-logs-root 2021-11-02 10:36:44 +01:00
fenn-cs
5ba636c7b1 removed deferred logger from basic function in test_logs 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-11-02 10:36:32 +01:00
Jean-Paul Calderone
0856218a89 Merge remote-tracking branch 'origin/master' into 3833.container-format-abstraction 2021-10-30 07:35:50 -04:00
Jean-Paul Calderone
892b468365 use the port assigner to assign a port for the main tub 2021-10-29 09:03:37 -04:00
Jean-Paul Calderone
ff511bbcf2 Merge remote-tracking branch 'origin/master' into 3832.backdated-leases 2021-10-29 07:17:23 -04:00
Jean-Paul Calderone
c05464f420
Merge pull request #1154 from tahoe-lafs/3835.document-some-testing-code 
Document some testing code

Fixes: ticket:3835
 2021-10-29 07:16:02 -04:00
Jean-Paul Calderone
b5f882ffa6 introduce and use LeaseInfo.is_cancel_secret 2021-10-28 15:21:51 -04:00
Jean-Paul Calderone
8b976b441e add LeaseInfo.is_renew_secret and use it 2021-10-28 15:21:51 -04:00
Jean-Paul Calderone
78dbe76994 remove unused import 2021-10-28 15:20:44 -04:00
Jean-Paul Calderone
f8655f149b fix the type annotations and such 2021-10-28 15:04:19 -04:00
Jean-Paul Calderone
8d5727977b it doesn't typecheck, nevermind 2021-10-28 14:47:42 -04:00
Jean-Paul Calderone
54d80222c9 switch to monkey-patching from other sources 
This is not much of an improvement to the tests themselves, unfortunately.
However, it does get us one step closer to dropping `mock` as a dependency.
 2021-10-28 14:36:30 -04:00
Jean-Paul Calderone
d0ee17d99e some docstrings 2021-10-28 12:39:01 -04:00
Jean-Paul Calderone
1b46ac7a24 add MutableShareFile.is_valid_header and use it 2021-10-28 11:38:18 -04:00
Jean-Paul Calderone
66845c9a17 Add ShareFile.is_valid_header and use it instead of manual header inspection 2021-10-28 11:12:20 -04:00
Jean-Paul Calderone
65d3ab6142 move backdating logic into mutable/immutable share files 2021-10-28 10:53:52 -04:00
Jean-Paul Calderone
3514995068 some versions of mypy don't like this so nevermind 2021-10-28 10:26:30 -04:00
Jean-Paul Calderone
76caf46347 make the alternate LeaseInfo constructors into class methods 2021-10-28 10:23:58 -04:00
Jean-Paul Calderone
70fb5d563a Get rid of the public expiration_time attribute 
LeaseInfo now has a getter and a setter for this attribute.
LeaseInfo is now also immutable by way of `attrs`.
LeaseInfo is now also comparable by way of `attrs`.
 2021-10-28 09:48:26 -04:00
meejah
eddfd244a7 code and tests to check RSA key sizes 2021-10-26 13:37:26 -06:00
Jean-Paul Calderone
f02f14a6e2 Merge remote-tracking branch 'upstream/master' 2021-10-26 09:23:49 -04:00
fenn-cs
69d335c1e1 update test overlap function docstring 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-10-26 13:14:26 +01:00
fenn-cs
9764ac740a test kwargs overlap with params in start_action 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-10-26 11:22:32 +01:00
Jean-Paul Calderone
84dfb360ce
Merge pull request #1141 from LeastAuthority/3814.remove-control-port 
Remove the "control port" and its associated Tub

Fixes: ticket:3814
 2021-10-25 20:57:23 -04:00
Jean-Paul Calderone
3de481ab6b remove unused imports 2021-10-25 20:52:35 -04:00
Jean-Paul Calderone
5878a64890 Remove password-based authentication from the SFTP frontend 2021-10-25 20:50:19 -04:00
Jean-Paul Calderone
aa6360f08e Merge remote-tracking branch 'remediate/master' 2021-10-24 07:38:42 -04:00
Jean-Paul Calderone
b51f0ac8ff storage_index is a byte string 
and Python 3 cares
 2021-10-23 08:04:19 -04:00
Jean-Paul Calderone
0ada9d93f7 remove unused local 2021-10-23 07:43:22 -04:00
Jean-Paul Calderone
470657b337 Drop corruption advisories if we don't have a matching share 2021-10-23 07:35:05 -04:00
Jean-Paul Calderone
21f848f0e3 Merge remote-tracking branch 'upstream/master' 2021-10-23 07:32:26 -04:00
Jean-Paul Calderone
cb675df48d remove unused encoding of storage index 2021-10-22 16:10:24 -04:00
Jean-Paul Calderone
194499aafe remove unused import 2021-10-22 16:09:54 -04:00
Jean-Paul Calderone
6449ad03de Do not record corruption advisories if there is no available space 2021-10-22 15:56:45 -04:00
Jean-Paul Calderone
f789339a79 Have MutableShare file only write a new lease if there is room for it 
This is analagous to the earlier ShareFile change.
 2021-10-22 15:56:45 -04:00
Jean-Paul Calderone
dd1ab2afe8 Add a helper to compute the size of a lease in a mutable share 2021-10-22 15:56:45 -04:00
Jean-Paul Calderone
e0ed04c103 use SyncTestCase to get expectThat 2021-10-22 15:56:45 -04:00
Jean-Paul Calderone
4defc641a2 Have ShareFile only write a new lease if there is room for it 
StorageServer passes available space down so it can make the decision.
ShareFile has to do it because `add_or_renew_lease` only *sometimes* adds a
lease and only ShareFile knows when that is.
 2021-10-22 15:56:45 -04:00
Jean-Paul Calderone
1264c3be1e Use _add_or_renew_leases helper consistently in StorageServer 
This will make it easier to add a new argument to the underlying
`add_or_renew_lease` call.
 2021-10-22 15:56:45 -04:00
Jean-Paul Calderone
b3aa1e224f Add a helper to LeaseInfo for computing size 
This lets some code LBYL and avoid writing if the lease won't fit in the
immutable share in the space available.
 2021-10-22 15:56:45 -04:00
Jean-Paul Calderone
c774256937 Add a test for `remote_add_lease` with respect to reserved space 2021-10-22 15:56:45 -04:00
Jean-Paul Calderone
82e4704415 Merge remote-tracking branch 'remediate/master' into LFS-01-008 2021-10-22 15:27:15 -04:00
Jean-Paul Calderone
ce30f9dd06 clean up copyediting errors 2021-10-22 15:04:45 -04:00
Jean-Paul Calderone
bb31215134 Merge remote-tracking branch 'remediate/master' into LFS-01-007 2021-10-22 12:35:57 -04:00
Jean-Paul Calderone
d8c466e9a7 try to explain lease_count_format more clearly 2021-10-22 12:35:11 -04:00
Jean-Paul Calderone
a22787e879
Merge pull request #30 from tahoe-lafs/LFS-01-001 
Fix item LFS-01-001 from the Cure53 audit

Fixes: ticket:3819
 2021-10-22 12:20:38 -04:00
Jean-Paul Calderone
6c0ca0b885 try making windows let us use longer paths 2021-10-22 11:42:13 -04:00
fenn-cs
26419c40bc Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3788.refactor-src.test.mutable 2021-10-20 11:17:41 +01:00
fenn-cs
20ad6cd9e7 iterate over args directly without indexing 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-10-19 23:57:52 +01:00
Itamar Turner-Trauring
2a5dbcb05e Tests for mutable leases. 2021-10-19 09:30:38 -04:00
Itamar Turner-Trauring
4b8b6052f3 Finish testing leases on immutables. 2021-10-19 09:05:48 -04:00
Jean-Paul Calderone
3cd9a02c81 Reject negative lengths in MutableShareFile._read_share_data and readv 2021-10-18 20:22:52 -04:00
Jean-Paul Calderone
5e58b62979 Add a test for negative offset or length to MutableShareFile.readv 2021-10-18 20:22:52 -04:00
Jean-Paul Calderone
df64bbb1e4 fail to encode the lease count *before* changing anything 
This preserves the failure behavior - `struct.error` is raised - but leaves
the actual share file contents untouched if the new lease count cannot be
encoded.

There are still two separate write operations so it is conceivable that some
other problem could cause `write_lease_record` to happen but
`write_encoded_num_leases` not to happen.  As far as I can tell we have
severely limited options for addressing that problem in general as long as
share files are backed by a POSIX filesystem.

However, by removing the failure mode that depends on user input, it may be
that this is all that is needed to close the *security* hole.
 2021-10-18 20:17:41 -04:00
Jean-Paul Calderone
f60bbbd3e2 make it possible to test this behavior of add_lease 2021-10-18 20:17:41 -04:00
Itamar Turner-Trauring
b7be91e3d0 First test for leases. 2021-10-18 13:17:07 -04:00
Itamar Turner-Trauring
2b40610a27 "Server" is extremely ambiguous, so let's just call this a client, which it is. 2021-10-18 12:55:30 -04:00
Itamar Turner-Trauring
58112ba75b Plan of implementation for lease tests. 2021-10-18 12:50:29 -04:00
Jean-Paul Calderone
1c347c5931 replace sensitive introducer fURL with path where it can be found 2021-10-18 08:05:40 -04:00
Jean-Paul Calderone
ad216e0f23 remove unused import 2021-10-15 13:13:37 -04:00
Jean-Paul Calderone
0611af6b0b Stop passing even a dummy value for control tub into Nodes 2021-10-15 13:10:18 -04:00
Jean-Paul Calderone
fe2e2cc1d6 Get rid of the control service 2021-10-15 12:12:19 -04:00
Jean-Paul Calderone
1de480dc37 Stop offering an API to create a control tub or handling the control tub 2021-10-15 12:12:03 -04:00
Jean-Paul Calderone
ddf5f461bf Stop half-pretending to have a control port 2021-10-15 12:11:53 -04:00
Jean-Paul Calderone
e0312eae57 stop creating a control tub for client nodes 2021-10-15 12:02:24 -04:00
Jean-Paul Calderone
95b765e309 stop creating a control tub for the introducer 2021-10-15 11:54:18 -04:00
fenn-cs
1a12a8acdf don't throw away unserializable parameter 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-10-15 00:50:11 +01:00
Itamar Turner-Trauring
e099bc6736 Tests for IStorageServer.advise_corrupt_share. 2021-10-14 12:52:56 -04:00
Itamar Turner-Trauring
22aab98fcf When callRemoteOnly as removed, these probably should've been changed to return 
a Deferred.
 2021-10-14 12:52:40 -04:00
Jean-Paul Calderone
efc9dc831b Revert "a stab at using setup.cfg and setuptools_scm" 
This reverts commit 68e8e0a7d5.
 2021-10-14 11:01:37 -04:00
Jean-Paul Calderone
68e8e0a7d5 a stab at using setup.cfg and setuptools_scm 2021-10-14 10:58:41 -04:00
fenn-cs
57a0f76e1f maintain list of positional arguments as tuple 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-10-13 23:41:42 +01:00
fenn-cs
bf670c0664 Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3788.refactor-src.test.mutable 2021-10-12 22:42:59 +01:00
fenn-cs
5803d9999d remove unseriable args in log_call_deferred passed to start_action 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-10-11 13:49:29 +01:00
fenn-cs
9da1456e5b Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3758.refactor.web-tests-grid-logs-root 2021-10-08 23:37:34 +01:00
fenn-cs
32e00af023 Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3758.refactor.web-tests-grid-logs-root 2021-10-07 17:46:20 +01:00
Itamar Turner-Trauring
807363adc9 Reduce duplication. 2021-10-06 16:41:50 -04:00
Itamar Turner-Trauring
3cf09ef888 Merge remote-tracking branch 'origin/master' into 3798-bucket-write-abort-tests 2021-10-06 16:07:39 -04:00
Itamar Turner-Trauring
0a605530d8
Merge pull request #1134 from tahoe-lafs/3793-persistent-bucketwriter 
Don't tie BucketWriter lifetime to Foolscap connection lifetime

Fixes ticket:3793
 2021-10-06 16:07:17 -04:00
Itamar Turner-Trauring
f3ebc71410
Merge pull request #1133 from tahoe-lafs/3805-istorageserver-tests-slot-readv 
IStorageServer tests: slot_readv

Fixes ticket:3805
 2021-10-06 15:41:29 -04:00
Itamar Turner-Trauring
bf176144c5 Handle double-disconnect, should it happen by mistake. 2021-10-06 15:18:00 -04:00
Itamar Turner-Trauring
82cbce6b7e Better explanation. 2021-10-06 15:12:22 -04:00
Itamar Turner-Trauring
17a670dfb5 Test for aborting bucket upload. 2021-10-04 10:37:37 -04:00
Itamar Turner-Trauring
b09b7fd01b Merge branch '3793-persistent-bucketwriter' into 3798-bucket-write-abort-tests 2021-10-04 10:30:41 -04:00
Itamar Turner-Trauring
23fd11be43 Expand explanation. 2021-09-29 14:13:18 -04:00
Itamar Turner-Trauring
016d6b4530 Fix spurious type checking error. 2021-09-29 14:10:14 -04:00
Itamar Turner-Trauring
f8604e2394 Fix flakes. 2021-09-29 14:00:11 -04:00
Itamar Turner-Trauring
58d7e2f627 Simplify implementation. 2021-09-29 13:58:53 -04:00
Itamar Turner-Trauring
8fb6afee1b Refactor BucketWriters such that disconnection can be limited Foolscap. 2021-09-29 13:42:17 -04:00
Itamar Turner-Trauring
a4153b7125 Implementation plan. 2021-09-29 11:56:04 -04:00
Itamar Turner-Trauring
51e8b5e197 Disconnection test works now. 2021-09-29 11:17:33 -04:00
Itamar Turner-Trauring
45c564f3c8
Merge pull request #1131 from tahoe-lafs/3797-istorageserver-tests-part-3 
IStorageServer tests, part 3: slot_testv_and_writev_and_readv

Fixes ticket:3797
 2021-09-29 10:28:42 -04:00
meejah
0a072a98c8
Merge pull request #1125 from hacklschorsch/3786.openmetrics 
3786 Add OpenMetrics (Prometheus) statistics endpoint
 2021-09-28 13:21:46 -06:00
Itamar Turner-Trauring
e64c397fc5 WIP disconnection test. 2021-09-28 13:51:31 -04:00
Itamar Turner-Trauring
a02d5f4c9c Just stick to current behavior. 2021-09-28 13:02:01 -04:00
Itamar Turner-Trauring
9970559019 Tests for slot_readv. 2021-09-28 10:46:21 -04:00
Itamar Turner-Trauring
fadfbcad99
Merge pull request #1130 from tahoe-lafs/3801-no-overlapping-writes-immutable-upload 
Disallow conflicting overlapping writes when doing an immutable upload.

Fixes ticket:3801
 2021-09-28 10:37:56 -04:00
Itamar Turner-Trauring
c668e342c2 More STARAW tests. 2021-09-28 09:54:13 -04:00
fenn-cs
759d4c85a2 avoid argument collision in call of start_action in eliotutil 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-09-28 09:56:14 +01:00
Itamar Turner-Trauring
914ca56775 TODOs. 2021-09-27 17:05:03 -04:00
Itamar Turner-Trauring
9f80435b41 Update to new interface. 2021-09-27 17:04:22 -04:00
Itamar Turner-Trauring
da363d54de Merge remote-tracking branch 'origin/master' into 3797-istorageserver-tests-part-3 2021-09-27 17:03:05 -04:00
Itamar Turner-Trauring
eb5b6c557e
Merge pull request #1127 from tahoe-lafs/3799-simplify-IStorageServer 
Rip out unnecessary operator logic, and simplify IStorageServer

Fixes ticket:3799
 2021-09-27 16:57:16 -04:00
Itamar Turner-Trauring
de1a7d7fce A more explicit test for successful overlapping. 2021-09-27 16:56:24 -04:00
Itamar Turner-Trauring
da937cef9e Correct docstring. 2021-09-27 15:04:04 -04:00
Itamar Turner-Trauring
1c10fa8c7f Merge remote-tracking branch 'origin/master' into 3799-simplify-IStorageServer 2021-09-27 15:02:09 -04:00
Itamar Turner-Trauring
b1b64c787e Add more randomness. 2021-09-27 15:00:52 -04:00
fenn-cs
49b6080097 remove depracated assert methods 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-09-25 21:03:01 +01:00
fenn-cs
88cbb7b109 remove methods that break test_filenode with AsyncBrokenTest 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-09-24 23:04:01 +01:00
Itamar Turner-Trauring
0b1082fc04 Fix lint. 2021-09-24 12:04:12 -04:00
Itamar Turner-Trauring
c1f8e9f8c7 IStorageServer test for overlapping writes. 2021-09-24 12:02:30 -04:00
Itamar Turner-Trauring
6ef3811112 Prevent conflicting overlapping writes. 2021-09-24 11:54:08 -04:00
Itamar Turner-Trauring
1ff4e61e41 Low-level tests for conflicting and non-conflicting writes. 2021-09-24 10:49:27 -04:00
fenn-cs
14076e54b5 Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3788.refactor-src.test.mutable 2021-09-24 12:39:44 +01:00
Itamar Turner-Trauring
4f7727df04 Merge remote-tracking branch 'origin/master' into 3795-istorageserver-tests-part-2 2021-09-23 14:20:44 -04:00
Itamar Turner-Trauring
ec6dfb8297 Re-enable test. 2021-09-23 14:20:34 -04:00
Jean-Paul Calderone
2f60ab300b black formatting 2021-09-23 07:48:27 -04:00
Jean-Paul Calderone
4b6d00221e protect this crazy line from black 2021-09-23 07:48:19 -04:00
Jean-Paul Calderone
f66a8ab136 formatting and explicit unicode string literals 2021-09-23 07:43:37 -04:00
Jean-Paul Calderone
cbb96bd57a one more docstring 2021-09-23 07:43:33 -04:00
Jean-Paul Calderone
4d8164773c factor helper function out to top-level 2021-09-23 07:43:18 -04:00
Jean-Paul Calderone
f8c07bfd11 add some docstrings 2021-09-23 07:42:59 -04:00
Jean-Paul Calderone
5e26f25b37 It's ported to Python 3! 2021-09-23 07:41:43 -04:00
fenn-cs
424047ba43 Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3758.refactor.web-tests-grid-logs-root 2021-09-22 23:37:47 +01:00
fenn-cs
dd8aa8a666 test if log_call_deffered decorates parametized functions correctly 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-09-22 23:37:33 +01:00
Jean-Paul Calderone
c67c0727e6
Merge pull request #1103 from LeastAuthority/3525.test_status-no-mock 
Remove usage of mock from allmydata/test/cli/test_status.py

Fixes: ticket:3525
 2021-09-22 16:48:34 -04:00
Jean-Paul Calderone
198de5c648 Hoist the sys.stdout import to top level 2021-09-22 15:53:56 -04:00
Itamar Turner-Trauring
b835a671ce Merge remote-tracking branch 'origin/master' into 3795-istorageserver-tests-part-2 2021-09-21 15:09:49 -04:00
Itamar Turner-Trauring
4a9ff82533 Merge remote-tracking branch 'origin/master' into 3799-simplify-IStorageServer 2021-09-21 15:08:42 -04:00
Itamar Turner-Trauring
460b74e171 Punt overlapping writes for now. 2021-09-21 13:29:24 -04:00
Itamar Turner-Trauring
b7ae9a675f Merge remote-tracking branch 'origin/master' into 3784-istorageserver-test-suite 2021-09-21 13:20:53 -04:00
Jean-Paul Calderone
87113d6a13 Merge remote-tracking branch 'origin/master' into 3525.test_status-no-mock 2021-09-20 07:14:02 -04:00
fenn-cs
5bd5ee580a layout for tests that check if log_call_deffered decorates parametized functions correctly 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-09-18 23:50:34 +01:00
fenn-cs
4d6f254610 Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3758.refactor.web-tests-grid-logs-root 2021-09-18 19:22:12 +01:00
fenn-cs
cd7e218a9e Merge branch 'master' of github.com:tahoe-lafs/tahoe-lafs into 3788.refactor-src.test.mutable 2021-09-18 18:50:50 +01:00
Florian Sesser
e5e0d71ef5 OpenMetrics test suite: More clean ups 
trailing whitespace
 2021-09-17 13:20:59 +00:00
Florian Sesser
fb0335cc17 OpenMetrics test suite: More clean up 
The Linter complains:
> 'testtools.matchers.Always' imported but unused
 2021-09-17 12:17:32 +00:00
Florian Sesser
a2378d0e70 OpenMetrics test suite: Make CI happy: No old style objects 2021-09-17 12:04:12 +00:00
Florian Sesser
5825b8bd42 OpenMetrics: rework test suite with exarkun 2021-09-16 15:58:04 +00:00
Itamar Turner-Trauring
e11e5dfbe6 Revert removal of length in IStorageServer. 2021-09-15 16:31:54 -04:00
Itamar Turner-Trauring
1d2073b8f8 Revert "This is unnecessary, empty vector list is fine too." 
This reverts commit f109afa3b1.
 2021-09-15 16:19:24 -04:00
Florian Sesser
b0e1cf924d OpenMetrics test: White space only: Format JSON fixture to be easier on the eyes 2021-09-15 15:14:29 +00:00
Itamar Turner-Trauring
911a5e2ed1 Rip out server-side usage of operators other than eq, because nothing ever used them. 2021-09-15 11:07:02 -04:00
Itamar Turner-Trauring
f109afa3b1 This is unnecessary, empty vector list is fine too. 2021-09-15 10:51:43 -04:00
Itamar Turner-Trauring
8633432980 Switch IStorageServer interface to be slightly different than RIStorageServer. 2021-09-15 10:33:51 -04:00
Itamar Turner-Trauring
d210062dd7 Another test for STARAW. 2021-09-15 09:47:16 -04:00
Florian Sesser
383ab4729a OpenMetrics tests: Tryfix resolve TypeError on CI 
Was:
> TypeError: unbound method render_OPENMETRICS() must be called with Statistics instance as first argument (got FakeStats instance instead)
 2021-09-15 11:53:48 +00:00
Florian Sesser
6bcff5472b OpenMetrics test suite: Add a check to see whether our stats were parsed at all. 2021-09-15 11:50:20 +00:00
Florian Sesser
21c471ed81 OpenMetrics test: Add hopefully more stable URIs to OpenMetrics spec info 2021-09-15 11:39:32 +00:00
Florian Sesser
cbe5ea1115 OpenMetrics: Add docstring 2021-09-15 11:28:39 +00:00
Florian Sesser
c66ae302c8 OpenMetrics: Extra newline at the end 2021-09-15 11:27:44 +00:00
Florian Sesser
57a3c1168e OpenMetrics: Use list of strings instead of string concatenation 2021-09-15 11:03:31 +00:00
Florian Sesser
88a2e7a4fb OpenMetrics test suite: Get rid of status mock 2021-09-15 10:09:55 +00:00
Itamar Turner-Trauring
241f4c841b Another test. 2021-09-14 13:00:29 -04:00
Itamar Turner-Trauring
98e566fc44 Expand testing scope. 2021-09-14 12:51:32 -04:00
Itamar Turner-Trauring
7b97ecfb7c More tests. 2021-09-14 12:47:03 -04:00
Itamar Turner-Trauring
aa8001edf2 Another test. 2021-09-14 12:30:45 -04:00
Itamar Turner-Trauring
5b704ff12d Another mutable test. 2021-09-14 11:36:12 -04:00
Itamar Turner-Trauring
d207c46855 First mutable test. 2021-09-14 11:26:23 -04:00
Itamar Turner-Trauring
f40b7be562 Merge branch '3784-istorageserver-test-suite' into 3797-istorageserver-tests-part-3 2021-09-14 09:42:52 -04:00
Itamar Turner-Trauring
bb626890ed Match review comment suggestions from previous PR. 2021-09-14 08:57:32 -04:00
Itamar Turner-Trauring
c2c75b5e8e Merge branch '3784-istorageserver-test-suite' into 3795-istorageserver-tests-part-2 2021-09-14 08:49:58 -04:00
fenn-cs
a3168b3844 test.mutable : refactored test_update.py 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-09-13 23:45:16 +01:00
Itamar Turner-Trauring
87555ae3e4 Add missing __future__ imports. 2021-09-13 15:51:57 -04:00
Itamar Turner-Trauring
9cea3580c8 Merge remote-tracking branch 'origin/master' into 3795-istorageserver-tests-part-2 2021-09-13 10:14:35 -04:00
Itamar Turner-Trauring
8294c2d562 Merge branch '3784-istorageserver-test-suite' into 3795-istorageserver-tests-part-2 2021-09-13 10:13:08 -04:00
Itamar Turner-Trauring
9ce7cce712 Refactor SystemTestMixin into its own module. 2021-09-13 10:10:07 -04:00
Itamar Turner-Trauring
977b6065b1 Match actual behavior. 2021-09-13 09:50:38 -04:00
Itamar Turner-Trauring
86ea3ca408 Demonstrate that last write wins. 2021-09-13 09:49:27 -04:00
Itamar Turner-Trauring
e96c22914b Fix typo. 2021-09-13 09:43:53 -04:00
Itamar Turner-Trauring
d04cd13b37 Actual test for get_version(). 2021-09-13 09:43:29 -04:00
Itamar Turner-Trauring
a482f216a1 Use more reproducible "random" numbers. 2021-09-13 09:40:32 -04:00
Itamar Turner-Trauring
b01c5c7454 Clarify argument names. 2021-09-13 09:35:52 -04:00
Itamar Turner-Trauring
2eafe41270
Merge pull request #1117 from tahoe-lafs/3779-istorageserver-with-fewer-assumptions 
Reduce implementation-dependency of `IStorageServer`

Fixes ticket:3779
 2021-09-13 09:28:46 -04:00
Itamar Turner-Trauring
1fa0f72c8e Use AsyncTestCase. 2021-09-13 09:28:03 -04:00
Itamar Turner-Trauring
b0d635c9c9 Use skip decorator. 2021-09-13 09:15:20 -04:00
Jean-Paul Calderone
7ea994776d
Merge pull request #1114 from Fenn-CS/3781.remove-incomplete-cli-warning-python3 
Python 3 support complete, so removed warning

Fixes: ticket:3781
 2021-09-10 12:15:33 -04:00
Itamar Turner-Trauring
ac9875da75 Add explanation to new error logging. 2021-09-10 11:39:48 -04:00
Itamar Turner-Trauring
570f15284a More tests for IStorageServer.get_buckets(). 2021-09-10 09:44:49 -04:00
fenn-cs
3b80b8cbe9 test.mutable : refactored test_version.py 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-09-10 14:24:20 +01:00
Florian Sesser
ad84f5df2b newline at the end. 2021-09-10 13:21:06 +00:00
Florian Sesser
339e1747e7 clean up 2021-09-10 13:15:56 +00:00
Florian Sesser
6c18983f7b OpenMetrics test: Use realistic input data 2021-09-10 13:13:13 +00:00
Florian Sesser
d04157d18a OpenMetrics test: Add parser to check against spec 2021-09-10 13:00:15 +00:00
Florian Sesser
fca1482b35 OpenMetrics Tests WIP 2021-09-10 00:10:11 +00:00
fenn-cs
61b9f15fd1 test.mutable : refactored roundtrip and servermap tests 
Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
 2021-09-10 00:59:55 +01:00
Florian Sesser
30771149fc Openmetrics: Add test case scaffold 2021-09-09 23:31:39 +00:00
Florian Sesser
d05e373d42 OpenMetrics: All strings are unicode. 2021-09-09 13:57:59 +00:00