Commit Graph

7964 Commits

Author SHA1 Message Date
meejah
50cdd9bd96 unused 2021-12-04 17:20:10 -07:00
meejah
5bb6fbc51f merge errors 2021-12-04 10:14:31 -07:00
Itamar Turner-Trauring
c8f429c496 Merge remote-tracking branch 'origin/master' into 3849-refactor-out-foolscap-in-storage-server 2021-12-03 13:52:42 -05:00
Itamar Turner-Trauring
90f8480cf0 Make more of the unittests pass again with the StorageServer factoring. 2021-12-03 13:39:53 -05:00
Itamar Turner-Trauring
314b202914 Ignore another field which can change. 2021-12-03 12:58:12 -05:00
meejah
18891d9e77 Merge remote-tracking branch 'security/master' 2021-12-02 14:44:59 -07:00
Itamar Turner-Trauring
439e5f2998 Insofar as possible, switch to testing without the Foolscap API. 2021-12-02 10:52:03 -05:00
Itamar Turner-Trauring
8c3d61a94e Bit more backwards compatible. 2021-12-02 10:49:23 -05:00
Itamar Turner-Trauring
476c41e49e Split out Foolscap code from BucketReader/Writer. 2021-12-02 10:29:52 -05:00
Itamar Turner-Trauring
f7cb4d5c92 Hook up the new FoolscapStorageServer, and fix enough bugs, such that almost all
end-to-end and integration tests pass.
2021-12-02 10:07:14 -05:00
Itamar Turner-Trauring
6b8a42b043 Make the test more robust. 2021-12-02 09:35:37 -05:00
Jean-Paul Calderone
82099f04c4 Merge remote-tracking branch 'origin/master' into 3758.refactor.web-tests-grid-logs-root 2021-12-02 08:36:35 -05:00
meejah
4bc0df7cc1 file, not path 2021-12-02 00:14:15 -07:00
meejah
40e7be6d8d needs reason 2021-12-01 23:46:10 -07:00
meejah
e0092ededa fine, just skip tests on windows 2021-12-01 20:52:22 -07:00
Jean-Paul Calderone
8626043f61
Merge pull request #1163 from tahoe-lafs/3847.eliot-anybytes-always
Use the "any bytes" encoder even on Python 2

Fixes: ticket:3847
2021-12-01 15:47:16 -05:00
Jean-Paul Calderone
eee1f0975d note about how to clean this up later 2021-12-01 15:16:16 -05:00
Jean-Paul Calderone
90d1e90a14 rewrite the Eliot interaction tests to make expected behavior clearer
and to have explicit assertions about that behavior
2021-12-01 15:05:29 -05:00
meejah
940c6343cf consistency 2021-12-01 12:02:42 -07:00
meejah
7080ee6fc7 oops 2021-12-01 12:02:06 -07:00
meejah
25ca767095 an offering to the windows godesses 2021-12-01 11:49:52 -07:00
Itamar Turner-Trauring
f7975b9772
Merge pull request #1161 from tahoe-lafs/3843-start-http-storage-server
A start at a HTTP-based storage server and client

Fixes ticket:3843
Fixes ticket:3762
2021-12-01 10:18:36 -05:00
Itamar Turner-Trauring
50e21a9034 Split StorageServer into generic part and Foolscap part. 2021-12-01 09:55:44 -05:00
Itamar Turner-Trauring
6e1f6f68ca Merge branch '3843-start-http-storage-server' into 3849-refactor-out-foolscap-in-storage-server 2021-12-01 09:41:21 -05:00
Itamar Turner-Trauring
18a5966f1d Don't bother running HTTP server tests on Python 2, since it's going away any
day now.
2021-12-01 09:38:56 -05:00
meejah
85fa8fe32e py2/py3 glue code for json dumping 2021-11-30 23:00:59 -07:00
meejah
b47381401c flake8 2021-11-30 19:01:09 -07:00
meejah
679c464517 tests 2021-11-30 18:43:06 -07:00
meejah
fc9671a812 simplify, flake9 2021-11-30 18:25:32 -07:00
meejah
0a4bc385c5 fix tests to use migrate command 2021-11-30 18:00:58 -07:00
meejah
1b8ae8039e no auto-migrate; produce error if pickle-files exist 2021-11-30 16:01:15 -07:00
meejah
3fd1ca8acb it's an error to have pickle-format files 2021-11-30 16:00:35 -07:00
meejah
49f2489321 explicit 'migrate pickle files' command 2021-11-30 15:59:27 -07:00
Jean-Paul Calderone
c1a2964788 Merge remote-tracking branch 'origin/master' into 3527.test_create-no-mock 2021-11-30 15:06:17 -05:00
Jean-Paul Calderone
7626a02bdb remove redundant assertion 2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
20e0626e42 add capture_logging that parameterizes JSON encoder 2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
3eb1a5e7cb Add a MemoryLogger that prefers our encoder
and use it instead of Eliot's
2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
f40da7dc27 Put the choice of JSON encoder for Eliot into its own module
and use it in a few places
2021-11-29 13:43:27 -05:00
Jean-Paul Calderone
66a0c6f3f4 add a direct test for the non-utf-8 bytestring behavior 2021-11-29 13:43:27 -05:00
meejah
5855a30e34 add docstrings 2021-11-23 14:01:43 -07:00
meejah
b0309331a5 Merge branch 'master' into 3825.json-pickle 2021-11-23 13:53:41 -07:00
Itamar Turner-Trauring
a593095dc9 Explain why it's a conditional import. 2021-11-23 10:39:53 -05:00
Itamar Turner-Trauring
c921b153f4 A better name for the API. 2021-11-23 10:39:15 -05:00
Itamar Turner-Trauring
c341a86abd Correct the comment. 2021-11-23 10:01:03 -05:00
Jean-Paul Calderone
d1839187f1 "misplaced type annotation" 2021-11-22 08:45:10 -05:00
Jean-Paul Calderone
b92343c664 some more docstrings 2021-11-22 08:36:12 -05:00
Jean-Paul Calderone
e0c0c32c13 Merge remote-tracking branch 'remediate/master' into LFS-01-010.mutable 2021-11-22 07:59:43 -05:00
Jean-Paul Calderone
04e45f065a document compare_leases_without_timestamps 2021-11-22 07:59:01 -05:00
Itamar Turner-Trauring
4c11177387 Fix a problem with typechecking.
Using remote_write() isn't quite right given move to HTTP, but can fight that
battle another day.
2021-11-18 15:58:55 -05:00
Itamar Turner-Trauring
e2636466b5 Fix a flake. 2021-11-18 15:47:25 -05:00
Itamar Turner-Trauring
bd645edd9e Fix flake. 2021-11-18 15:44:51 -05:00
Itamar Turner-Trauring
5d915afe1c Clean up BucketWriters on shutdown (also preventing DelayedCalls leaks in tests). 2021-11-18 15:42:54 -05:00
Itamar Turner-Trauring
1827faf36b Fix issue with leaked-past-end-of-test DelayedCalls. 2021-11-18 14:45:44 -05:00
Itamar Turner-Trauring
8c8e377466 Implement timeout and corresponding tests. 2021-11-18 14:35:04 -05:00
Itamar Turner-Trauring
5e341ad43a New tests to write. 2021-11-17 11:29:34 -05:00
Itamar Turner-Trauring
45c00e93c9 Use clock in BucketWriter. 2021-11-17 11:12:40 -05:00
Itamar Turner-Trauring
92c36a67d8 Use IReactorTime instead of ad-hoc solutions. 2021-11-17 11:01:04 -05:00
Jean-Paul Calderone
e8adca40ab give the ContainerVersionError exceptions a nice str 2021-11-16 18:36:59 -05:00
Jean-Paul Calderone
3a8432713f a note about what's happening with proxyForInterface 2021-11-16 18:29:05 -05:00
Jean-Paul Calderone
a208502e18 whitespace 2021-11-16 18:29:01 -05:00
meejah
2fe686135b rename data to appease distutils 2021-11-16 15:08:31 -07:00
meejah
23ff1b2430 noqa 2021-11-16 15:08:31 -07:00
meejah
1c93175583 cleanup 2021-11-16 15:08:31 -07:00
meejah
4f64bbaa00 data 2021-11-16 15:08:31 -07:00
meejah
9b3c55e4aa test a second deserialzation 2021-11-16 15:08:31 -07:00
meejah
069c332a68 straight assert 2021-11-16 15:08:31 -07:00
meejah
94670461f1 tests 2021-11-16 15:08:31 -07:00
meejah
a867294e00 dead 2021-11-16 15:08:31 -07:00
meejah
75410e51f0 refactor 2021-11-16 15:08:31 -07:00
meejah
d4fc14f9ad docstring 2021-11-16 15:08:31 -07:00
meejah
89c2aacadc working test of 'in the wild' data, working converters 2021-11-16 15:08:31 -07:00
meejah
bf5e682d71 test upgrade of main state works 2021-11-16 15:08:31 -07:00
meejah
f81e4e2d25 refactor to use serializers / pickle->json upgraders 2021-11-16 15:08:31 -07:00
meejah
fa6950f08d an old pickle-format lease-checker state file 2021-11-16 15:08:31 -07:00
meejah
bb70e00065 Make internal state JSON-able for lease-crawler 2021-11-16 15:08:31 -07:00
meejah
f7b385f954 play nice with subclasses 2021-11-16 15:08:31 -07:00
meejah
3b69df36b0 crawler: pickle -> json 2021-11-16 15:08:31 -07:00
Itamar Turner-Trauring
f549488bb5 Don't use a deprecated API. 2021-11-16 11:47:09 -05:00
Itamar Turner-Trauring
a64778ddb0 Flakes. 2021-11-16 11:28:13 -05:00
Itamar Turner-Trauring
c195f895db Python 2 support. 2021-11-16 11:16:26 -05:00
Itamar Turner-Trauring
171d1053ec CBOR content-type on responses. 2021-11-16 11:09:17 -05:00
Itamar Turner-Trauring
671b670154 Some type annotations. 2021-11-16 11:09:08 -05:00
Itamar Turner-Trauring
41ec63f758 Passing first tests. 2021-11-16 10:56:21 -05:00
Jean-Paul Calderone
fcd634fc43 some direct tests for the new utility function 2021-11-15 13:34:46 -05:00
Jean-Paul Calderone
84c19f5468 clarify renew_leases docs on slot_testv_and_readv_and_writev 2021-11-15 08:12:07 -05:00
Jean-Paul Calderone
ad6017e63d clarify renew_leases docs on allocate_buckets 2021-11-15 08:11:53 -05:00
Jean-Paul Calderone
6c2e85e991 put the comment back 2021-11-12 16:25:36 -05:00
Jean-Paul Calderone
dece67ee3a it is not the remote interface that varies anymore 2021-11-12 16:24:29 -05:00
Jean-Paul Calderone
85977e48a7 put this comment back and merge info from the two versions 2021-11-12 16:23:15 -05:00
Jean-Paul Calderone
c3cb0ebaea Switch to per-call parameter for controlling lease renewal behavior
This is closer to an implementation where you could have two frontends, say a
Foolscap frontend and an HTTP frontend or even just two different HTTP
frontends, which had different opinions about what the behaviour should be.
2021-11-12 16:20:27 -05:00
Itamar Turner-Trauring
c101dd4dc9 Closer to first passing test. 2021-11-12 13:13:19 -05:00
Itamar Turner-Trauring
12cbf8a901 First sketch of HTTP testing infrastructure. 2021-11-12 13:03:53 -05:00
Itamar Turner-Trauring
ddd2780bd2 First sketch of HTTP client. 2021-11-12 12:51:52 -05:00
Itamar Turner-Trauring
a32c6be978 A sketch of what the HTTP server will look like. 2021-11-12 12:02:58 -05:00
Jean-Paul Calderone
2742de6f7c drop some ancient cruft
allocated_size not used anywhere, so why have it
2021-11-10 16:08:53 -05:00
Jean-Paul Calderone
9af81d21c5 add a way to turn off implicit bucket lease renewal too 2021-11-10 16:08:40 -05:00
Jean-Paul Calderone
617a1eac9d refactor lease hashing logic to avoid mutable/immutable duplication 2021-11-05 15:30:49 -04:00
Jean-Paul Calderone
456df65a07 Add v2 of the mutable container schema
It uses hashed lease secrets, like v2 of the immutable container schema.
2021-11-05 14:16:43 -04:00
Jean-Paul Calderone
3de9c73b0b preserve the type when renewing HashedLeaseInfo
does this mean immutable lease renewal is untested?  maybe
2021-11-05 14:11:05 -04:00
Jean-Paul Calderone
5d703d9893 some type annotations 2021-11-05 14:10:27 -04:00