ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-25 23:51:07 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
14,849 Commits 185 Branches 90 Tags 89 MiB
264269f409
Commit Graph

8767 Commits

Author SHA1 Message Date
Itamar Turner-Trauring
435df4923f
Merge branch 'master' into 2916.grid-manager-proposal.6 2022-10-05 10:18:09 -04:00
Itamar Turner-Trauring
8b0ddf406e Make HTTP and Foolscap match in another edge case. 2022-10-04 11:17:19 -04:00
Itamar Turner-Trauring
8190eea489 Fix bug introduced in previous commit. 2022-10-04 11:02:36 -04:00
Itamar Turner-Trauring
ea1d248611 These objects get stored in a context where they need to be hashed, sometimes. 2022-10-04 10:51:43 -04:00
Itamar Turner-Trauring
1294baa82e LoopingCall may already have been stopped. 2022-10-04 10:30:27 -04:00
Itamar Turner-Trauring
fd07c092ed close() is called while writes are still happening. 2022-10-04 10:30:07 -04:00
Itamar Turner-Trauring
8b2884cf3a Make changes work again. 2022-10-04 09:44:30 -04:00
Itamar Turner-Trauring
1ea716c8bf Merge branch '3902-listen-storage-http' into 3783-storage-client-http 2022-10-04 09:22:49 -04:00
Itamar Turner-Trauring
c13be0c89b Try harder to cleanup. 2022-10-04 09:19:48 -04:00
Itamar Turner-Trauring
58247799c1 Fix remaining references to refactored-out-of-existence API. 2022-10-03 11:27:19 -04:00
Itamar Turner-Trauring
3034f35c7b Document type expectations. 2022-10-03 11:24:53 -04:00
Itamar Turner-Trauring
5d53cd4a17 Nicer API. 2022-10-03 11:16:30 -04:00
Itamar Turner-Trauring
d918135a0d Use parser instead of ad-hoc parser. 2022-10-03 11:10:36 -04:00
Itamar Turner-Trauring
d753bb58da Better type for storage_nurls. 2022-10-03 11:05:56 -04:00
Itamar Turner-Trauring
b0fb72e379 Link to design issue. 2022-10-03 11:02:48 -04:00
Itamar Turner-Trauring
8179ea7738 Merge remote-tracking branch 'origin/master' into 3902-listen-storage-http 2022-10-03 10:56:17 -04:00
Itamar Turner-Trauring
bd74c3d971
Merge pull request #1216 from tahoe-lafs/3904-http-storage-holistic 
Address various http storage holistic review comments
 2022-10-03 09:13:30 -04:00
meejah
d2dd211420
Merge pull request #1219 from meejah/3926.pid-time-reactor 
3926: upgrade PID files to contain start-time, do file-locking and exist on Windows
 2022-09-28 11:02:46 -06:00
meejah
bef71978b6 don't need start_new_session 2022-09-28 10:15:03 -06:00
meejah
ca522a5293 sys.argv not inline 2022-09-28 10:14:59 -06:00
meejah
8991509f8c blackslashes.... 2022-09-25 00:16:40 -06:00
meejah
f2cfd96b5e typo, longer timeout 2022-09-23 01:04:58 -06:00
meejah
7fdeb8797e hardcoding bad 2022-09-23 00:26:39 -06:00
meejah
62b92585c6 simplify 2022-09-22 23:57:19 -06:00
meejah
a182a25079 backslashes 2022-09-22 21:43:20 -06:00
meejah
8ebe331c35 maybe a newline helps 2022-09-22 00:11:20 -06:00
meejah
4f5a1ac372 naming? 2022-09-21 23:36:23 -06:00
meejah
8d8b0e6f01 cleanup 2022-09-21 20:40:25 -06:00
meejah
fedea96964 less state 2022-09-21 20:26:14 -06:00
meejah
8474ecf83d typo 2022-09-21 20:15:07 -06:00
meejah
930f4029f3 properly write pid, create-time 2022-09-21 20:07:46 -06:00
meejah
0a09d23525 more docstring 2022-09-21 19:29:40 -06:00
meejah
e111694b3e get rid of find_process= 2022-09-21 19:28:09 -06:00
meejah
56775dde19 refactor: parsing in a function 2022-09-21 19:05:30 -06:00
meejah
ea39e4ca69 docstring 2022-09-21 19:01:28 -06:00
meejah
5973196931 refactor: use filelock and test it 2022-09-21 19:00:27 -06:00
meejah
1f29cc9c29 windows special 2022-09-20 14:50:46 -06:00
meejah
77bc83d341 incorrectly removed 2022-09-20 14:45:19 -06:00
meejah
0eeb11c9cd after shutdown 2022-09-20 14:44:51 -06:00
meejah
6db1476dac comment typo 2022-09-20 14:44:21 -06:00
meejah
81c8e1c57b windows is special 2022-09-20 14:24:02 -06:00
Itamar Turner-Trauring
4b2725df00 Try to prevent leaking timeouts. 2022-09-20 10:09:43 -04:00
Itamar Turner-Trauring
3b3ed9db2b Merge remote-tracking branch 'origin/master' into 3902-listen-storage-http 2022-09-19 10:07:00 -04:00
meejah
fe80126e3f fixups 2022-09-18 22:39:25 -06:00
meejah
254a994eb5 flake8 2022-09-17 16:41:17 -06:00
meejah
8b2cb79070 cleanup via reactor 2022-09-17 16:38:25 -06:00
meejah
aef2e96139 refactor: dispatch with our reactor, pass to tahoe_run 2022-09-17 16:38:24 -06:00
meejah
114d5e1ed8 pidfile on windows now 2022-09-17 16:37:33 -06:00
meejah
228bbbc2fe new pid-file 2022-09-17 16:37:33 -06:00
meejah
82c72ddede cleanup 2022-09-17 16:37:30 -06:00
meejah
642b604753 use stdin-closing for pidfile cleanup too 2022-09-17 16:36:55 -06:00
meejah
6048d1d9a9 in case hypothesis finds the magic 2022-09-17 16:36:26 -06:00
meejah
0e0ebf6687 more testing 2022-09-17 16:36:26 -06:00
meejah
cad162bb8f should have pid-file on windows too, now 2022-09-17 16:36:26 -06:00
meejah
3bfb60c6f4 back to context-manager, simplify 2022-09-17 16:36:25 -06:00
meejah
fb532a71ef own pid-file checks 2022-09-17 16:31:49 -06:00
Itamar Turner-Trauring
444bc724c5 A better approach to MAX_UEB_SIZE: just delete the code since it's not used in practice. 2022-09-16 10:38:29 -04:00
Itamar Turner-Trauring
4a573ede34 Download the actual data we need, instead of relying on bad reading-beyond-the-end semantics. 2022-09-15 11:29:32 -04:00
Itamar Turner-Trauring
b1aa93e022 Switch prefix. 2022-09-15 10:34:59 -04:00
Itamar Turner-Trauring
8d5f08771a Minimal check on parameters' contents. 2022-09-15 09:45:46 -04:00
Itamar Turner-Trauring
373a532829 Detect corrupted UEB length more consistently. 2022-09-15 09:36:56 -04:00
Fon E. Noel NFEBE
fbc8baa238 Refactor Server class in test_storage.py 
As a follow up to commit: 48283ea6f8
this refactor adds better methods and cleans up the test to be
consistent with methods that used in classes that extend the `AsyncTestCase`.

Signed-off-by: Fon E. Noel NFEBE <fenn25.fn@gmail.com>
 2022-09-14 22:55:34 +01:00
meejah
1058e50c50 close properly 2022-09-08 16:30:30 -06:00
Itamar Turner-Trauring
c82bb5f21c Use a more meaningful constant. 2022-09-08 13:44:22 -04:00
Itamar Turner-Trauring
9975fddd88 Get rid of garbage. 2022-09-08 13:42:19 -04:00
Itamar Turner-Trauring
b72b51d22d Merge remote-tracking branch 'origin/master' into 3915-immutable-size 2022-09-08 13:41:46 -04:00
Fon E. Noel NFEBE
48283ea6f8 Refactor test_storage.py 
There are base test classes namely `SyncTestCase` and
`AsyncTestCase` which we would like all test classes in
this code base to extend.

This commit extends the listed classes in test_storage.py
to extend the above mentioned base classes:

  * UtilTests
  * BucketProxy
  * Server

Signed-off-by: Fon E. Noel NFEBE <fenn25.fn@gmail.com>
 2022-09-07 22:36:01 +01:00
meejah
711f6d39e7 missing proto 2022-09-01 22:29:19 -06:00
meejah
decb36a8f6 refactor for Windows testing 2022-09-01 22:20:07 -06:00
meejah
00c785ec76 debug windows 2022-09-01 21:47:28 -06:00
meejah
768829e993 more robust 2022-09-01 21:22:45 -06:00
meejah
488a04cb9b exit when stdin closes 2022-09-01 17:42:06 -06:00
Itamar Turner-Trauring
bdb4aac0de Pass in the missing argument. 2022-08-17 13:15:27 -04:00
Itamar Turner-Trauring
92662d802c Don't drop a Deferred on the ground. 2022-08-17 13:15:13 -04:00
Itamar Turner-Trauring
cd81e5a01c Hint for future debugging. 2022-08-17 13:13:22 -04:00
Itamar Turner-Trauring
3464637bbb Fix unit tests. 2022-08-17 12:54:26 -04:00
Itamar Turner-Trauring
9d03c476d1 Make sure we write all the bytes we say we're sending. 2022-08-17 12:49:45 -04:00
Itamar Turner-Trauring
c9084a2a45 Disable assertion we can't, sadly, enable. 2022-08-17 12:49:26 -04:00
Fon E. Noel NFEBE
7aa97336a0 Refactor FakeWebTest & MemoryConsumerTest classes 
There are base test classes namely `SyncTestCase` and
`AsyncTestCase` which we would like all test classes in
this code base to extend.

This commit refactors two test classes to use the `SyncTestCase`
with the newer assert methods.

Signed-off-by: Fon E. Noel NFEBE <fenn25.fn@gmail.com>
 2022-08-17 16:03:06 +01:00
Itamar Turner-Trauring
d50c98a1e9 Calculate URI extension size upfront, instead of hand-waving with a larger value. 2022-08-16 14:34:40 -04:00
Itamar Turner-Trauring
2e5662aa91 Temporarily enforce requirement that allocated size matches actual size of an immutable. 2022-08-16 13:11:06 -04:00
Itamar Turner-Trauring
c1bcfab7f8 Repeatedly poll status of server. 2022-08-15 11:38:02 -04:00
Itamar Turner-Trauring
6d48c1ca34 Merge branch '3902-listen-storage-http' into 3783-storage-client-http 2022-08-15 10:09:31 -04:00
Itamar Turner-Trauring
555fa6e942 Merge remote-tracking branch 'origin/master' into 3902-listen-storage-http 2022-08-15 10:09:21 -04:00
Itamar Turner-Trauring
71b7e9b643 Support comma-separated multi-location hints. 2022-08-15 10:09:04 -04:00
Itamar Turner-Trauring
78b421462d Merge remote-tracking branch 'origin/master' into 3783-storage-client-http 2022-08-15 09:51:45 -04:00
Itamar Turner-Trauring
b9ac10293a Merge remote-tracking branch 'origin/master' into 2916.grid-manager-proposal.6 2022-08-15 09:51:08 -04:00
Itamar Turner-Trauring
42e818f0a7 Refer to appropriate attributes, hopefully. 2022-08-12 11:47:08 -04:00
Itamar Turner-Trauring
3fbc4d7eea Let's make this a little clearer 2022-08-12 11:45:37 -04:00
Itamar Turner-Trauring
636b8a9e2d Fix a bytes-vs-str bug. 2022-08-12 11:28:08 -04:00
Itamar Turner-Trauring
e8609ac2df test_istorageserver passes with both Foolscap and HTTP again. 2022-08-12 11:24:41 -04:00
Itamar Turner-Trauring
09d778c2cf Allow nodes to disable the HTTPS storage protocol. 2022-08-12 11:13:09 -04:00
Itamar Turner-Trauring
f671fb04a1 A lot closer to working end-to-end. 2022-08-11 16:24:33 -04:00
Itamar Turner-Trauring
9ad4e844e8 Do status change notification. 2022-08-11 16:16:17 -04:00
Itamar Turner-Trauring
94be227aaa Hopefully don't actually need that. 2022-08-11 16:15:21 -04:00
Itamar Turner-Trauring
c3b159a3fd Continue simplified sketch of HTTPNativeStorageServer. 2022-08-11 16:12:57 -04:00
Itamar Turner-Trauring
c3e4158813 Remove duplication. 2022-08-11 15:55:14 -04:00
Itamar Turner-Trauring
709f139c85 Start refactoring to enable HTTP storage client. 2022-08-11 15:51:30 -04:00
Itamar Turner-Trauring
8b3280bf31 Simplify more. 2022-07-29 10:51:17 -04:00
Itamar Turner-Trauring
d4c73f19fe A unittest for the metaclass. 2022-07-29 10:42:56 -04:00
Itamar Turner-Trauring
533d2a7ac9 Note Tor and I2P support. 2022-07-29 10:15:23 -04:00
Itamar Turner-Trauring
1cd2185be7 More cleanups. 2022-07-29 10:12:24 -04:00
Itamar Turner-Trauring
34518f9d0d Fix lints. 2022-07-29 10:01:09 -04:00
Itamar Turner-Trauring
822b652d99 Improve factoring. 2022-07-29 09:57:18 -04:00
Itamar Turner-Trauring
21bb9e50f6 Merge remote-tracking branch 'origin/master' into 3902-listen-storage-http 2022-07-29 09:45:17 -04:00
Itamar Turner-Trauring
4cc5c4d55d Merge remote-tracking branch 'origin/master' into 2916.grid-manager-proposal.6 2022-07-25 11:55:47 -04:00
Itamar Turner-Trauring
94e0568653 Actually we do need it. 2022-07-22 11:57:32 -04:00
Itamar Turner-Trauring
83f9c0788b Use more direct API. 2022-07-22 11:56:18 -04:00
Itamar Turner-Trauring
be963e2324 Docstrings. 2022-07-22 11:55:33 -04:00
Itamar Turner-Trauring
2b3a8ddeec Docstring. 2022-07-22 11:55:00 -04:00
Itamar Turner-Trauring
36b96a8776 Fix typo. 2022-07-22 11:53:28 -04:00
Itamar Turner-Trauring
f671b47a6d Fix typo. 2022-07-22 11:53:12 -04:00
Itamar Turner-Trauring
5cd9ccfc6a Slightly nicer handling for bad edge cases. 2022-07-22 11:52:56 -04:00
Itamar Turner-Trauring
981b693402 Make HTTPS protocols work with the protocol switcher magic. 2022-07-20 15:25:22 -04:00
Itamar Turner-Trauring
11f4ebc0d9 Hook up NURL generation to the new Foolscap/HTTPS protocol switch. 2022-07-20 15:12:29 -04:00
Itamar Turner-Trauring
5e0c32708b Merge remote-tracking branch 'origin/master' into 3902-listen-storage-http 2022-07-20 14:48:26 -04:00
Itamar Turner-Trauring
d85b20b62d Fix lint. 2022-07-20 12:47:18 -04:00
Itamar Turner-Trauring
da8a36fac9 Improve test coverage. 2022-07-20 12:07:46 -04:00
Itamar Turner-Trauring
69739f5f9b Handle case where requested range results in empty response. 2022-07-20 11:42:01 -04:00
Itamar Turner-Trauring
43c6af6fde More error handling for edge cases. 2022-07-20 11:28:14 -04:00
Itamar Turner-Trauring
1b8b71b306 Content-Range headers are now checked (somewhat) and the server now sends 
correct headers when reading beyond the end.
 2022-07-19 16:10:22 -04:00
Itamar Turner-Trauring
b3aff5c43b More efficient implementations. 2022-07-19 14:37:46 -04:00
Itamar Turner-Trauring
49dfc8445c Implementation of getting length of shares (albeit inefficiently for now). 2022-07-18 14:12:12 -04:00
Itamar Turner-Trauring
1bcca7f630 Python 3 tweaks. 2022-07-18 11:29:20 -04:00
Itamar Turner-Trauring
011b942372 Python 3 tweaks. 2022-07-18 11:26:33 -04:00
Itamar Turner-Trauring
8b0941acd2 Python 3 tweaks. 2022-07-18 11:20:46 -04:00
Itamar Turner-Trauring
07a3d1ecd6 Python 3 tweaks. 2022-07-18 11:15:03 -04:00
Itamar Turner-Trauring
6cf3bc75b6 Some Python 3 cleanups. 2022-07-18 11:09:19 -04:00
Itamar Turner-Trauring
90188ce4d1 More Python 3 tweaks. 2022-07-18 11:05:03 -04:00
Itamar Turner-Trauring
d84d366c72 Some tweaks for Python 3 and modern attrs. 2022-07-18 10:50:04 -04:00
Itamar Turner-Trauring
b4703ace93 Some tweaks for Python 3. 2022-07-18 10:35:28 -04:00
Itamar Turner-Trauring
b3ab2fdb81 Fix grid manager CLI tests. 2022-07-15 15:12:31 -04:00
Itamar Turner-Trauring
1ede0e7786 Merge remote-tracking branch 'meejah/2916.grid-manager-proposal.5' into 2916.grid-manager-proposal.6 2022-07-15 14:06:10 -04:00
Itamar Turner-Trauring
a24aefaebf There can be up to 256 shares. 2022-07-15 11:35:28 -04:00
Itamar Turner-Trauring
87932e3444 Correct type. 2022-07-06 09:50:16 -04:00
Itamar Turner-Trauring
0b5132745d A nicer interface. 2022-07-06 09:47:08 -04:00
Itamar Turner-Trauring
fd8a385d1d Reformat with black. 2022-07-06 09:46:59 -04:00
Itamar Turner-Trauring
dac0080ea2 Make sure we update remaining length, and update test to catch the edge case 
this fixes.
 2022-07-06 09:40:46 -04:00
Itamar Turner-Trauring
5c5556d915 More robust usage. 2022-07-06 09:38:31 -04:00
Itamar Turner-Trauring
69c4dbf2b5 Fix tests and point to future work. 2022-07-05 17:17:38 -04:00
Itamar Turner-Trauring
6e3ca256b9 Some refactoring to handle edge cases better, in progress. 2022-07-05 15:36:21 -04:00
Itamar Turner-Trauring
3270d24c45 Slight simplification. 2022-07-05 11:30:48 -04:00
Itamar Turner-Trauring
1e6864ac01 Typo. 2022-07-05 11:30:01 -04:00
Itamar Turner-Trauring
97d0ba23eb Switch to hypothesis-based test. 2022-07-05 11:21:46 -04:00
Itamar Turner-Trauring
249f431849 Use MonkeyPatch instead of MockPatch, since we're not mocking. 2022-07-05 11:14:52 -04:00
Itamar Turner-Trauring
f2acf71998 Document next steps: NURL generation. 2022-06-30 15:58:52 -04:00
Itamar Turner-Trauring
70dfc44841 Fix for 3905. 2022-06-30 15:45:30 -04:00
Itamar Turner-Trauring
3db6080f6d Make the factories a class-level attribute. 2022-06-30 15:18:22 -04:00
Itamar Turner-Trauring
1798966f03 Store the tub on the subclass, since we'll want it (or rather its Listeners) for 
NURL construction.
 2022-06-30 14:53:37 -04:00
Itamar Turner-Trauring
d1bdce9682 A nicer API. 2022-06-30 14:26:36 -04:00
Itamar Turner-Trauring
03c515191e Better docs. 2022-06-30 14:21:43 -04:00
Itamar Turner-Trauring
451e68795c Lints, better explanation. 2022-06-30 13:54:58 -04:00
Itamar Turner-Trauring
bee46fae93 Resource limits on the client side. 2022-06-30 13:48:33 -04:00
Itamar Turner-Trauring
ab80c0f0a1 Set some length limits on various queries lengths. 2022-06-30 13:35:30 -04:00
Itamar Turner-Trauring
0e8f2aa702 More memory usage reductions. 2022-06-29 13:58:05 -04:00
Itamar Turner-Trauring
520456bdc0 Add streaming to CBOR results. 2022-06-29 11:26:25 -04:00
Itamar Turner-Trauring
efe9575d28 Nicer testing infrastructure so you don't have to switch back and forth between 
sync and async test APIs.
 2022-06-29 11:25:28 -04:00
Itamar Turner-Trauring
6dd2b2d583 More streaming, with tests passing again. 2022-06-27 17:00:24 -04:00
Itamar Turner-Trauring
06eca79263 Minimal streaming implementation. 2022-06-27 14:12:17 -04:00
Itamar Turner-Trauring
bfd54dc6ea Switch to newer attrs API, for consistency across the module. 2022-06-27 11:30:49 -04:00
Itamar Turner-Trauring
1154371d22 Clarifying comments. 2022-06-23 12:51:07 -04:00
Itamar Turner-Trauring
01d8cc7ab6 Put the attribute on the correct object. 2022-06-23 12:49:07 -04:00
Itamar Turner-Trauring
eb1e48bcc3 Add a timeout. 2022-06-23 12:47:33 -04:00
Itamar Turner-Trauring
0c99a9f7b0 Make it more accurate. 2022-06-23 12:44:17 -04:00
Itamar Turner-Trauring
d70f583172 More cleanups. 2022-06-23 12:43:46 -04:00
Itamar Turner-Trauring
026d63cd6a Fix some mypy warnings. 2022-06-23 12:41:47 -04:00
Itamar Turner-Trauring
d86f8519dc Simplify implementation. 2022-06-23 12:41:01 -04:00
Itamar Turner-Trauring
04156db74e Delay Negotiation.connectionMade so we don't create unnecessary timeouts. 2022-06-23 12:32:43 -04:00
Itamar Turner-Trauring
1579530895 Add working HTTP support. 2022-06-23 07:59:43 -04:00
Itamar Turner-Trauring
c5724c1d0a Clarify. 2022-06-22 14:20:42 -04:00
Itamar Turner-Trauring
7577d1e24c Sketch of HTTP support, still untested WIP. 2022-06-22 14:19:29 -04:00
Itamar Turner-Trauring
7910867be6 It actually works(?!) now. 2022-06-22 10:23:23 -04:00
Itamar Turner-Trauring
e1daa192fb Sketch of protocol switcher experiment. 2022-06-21 17:20:08 -04:00
Itamar Turner-Trauring
d37f187c07 Lint fix. 2022-06-08 13:56:23 -04:00
Itamar Turner-Trauring
db42651355 Be more consistent and just always write to the request in read_range. 2022-06-08 13:55:47 -04:00
Itamar Turner-Trauring
00381bc24f Correction now that it does more than what it did before. 2022-06-08 13:52:45 -04:00
Itamar Turner-Trauring
c3a304e1cc Lint and mypy fixes. 2022-06-06 11:04:34 -04:00
Itamar Turner-Trauring
85774ced95 Run shared tests on mutables too, with appropriate fixes to the tests and the server. 2022-06-06 10:57:19 -04:00
Itamar Turner-Trauring
e6efb62fd1 Refactor immutable tests so they can shared with mutables. 2022-06-06 10:25:06 -04:00
Itamar Turner-Trauring
797f34aec3 More tests. 2022-06-06 09:59:12 -04:00
Itamar Turner-Trauring
3e67d2d789 More tests. 2022-06-06 09:50:36 -04:00
Itamar Turner-Trauring
f1384096fa First unit test for mutables. 2022-06-03 13:46:23 -04:00
Itamar Turner-Trauring
fd306b9a61 Share more code across mutable and immutable reads. 2022-06-01 13:54:54 -04:00
Itamar Turner-Trauring
2313195c2b Reduce duplication. 2022-05-20 11:43:42 -04:00
Itamar Turner-Trauring
63624eedec Reduce code duplication. 2022-05-20 11:33:02 -04:00
Itamar Turner-Trauring
12927d50ba Type annotation improvements. 2022-05-20 11:10:18 -04:00
Itamar Turner-Trauring
8c8ea4927f Switch to public API. 2022-05-20 11:07:55 -04:00
Itamar Turner-Trauring
d209065a6e Fix type issue, and modernize slightly. 2022-05-16 11:22:44 -04:00
Itamar Turner-Trauring
a54b443f9d It's not an immutable client anymore. 2022-05-12 09:44:30 -04:00
Itamar Turner-Trauring
f3cf13154d Setup HTTP lease APIs for immutables too. 2022-05-11 12:00:27 -04:00
Itamar Turner-Trauring
b8735c79da Fix docstring. 2022-05-11 11:50:29 -04:00
Itamar Turner-Trauring
821bac3ddf Test another lease edge case. 2022-05-11 11:50:01 -04:00
Itamar Turner-Trauring
17fc9d0064 Merge branch '3891-mutable-http-protocol-part-2' into 3893-mutable-http-protocol-part-3 2022-05-11 11:22:03 -04:00
Itamar Turner-Trauring
457db8f992 Get rid of the "no such storage index" edge case, since it's not really necessary. 2022-05-11 11:17:57 -04:00
Itamar Turner-Trauring
4b62ec082b Match Foolscap behavior for slot_readv of unknown storage index. 2022-05-11 11:11:24 -04:00
Itamar Turner-Trauring
6d412a017c Type annotation. 2022-05-11 11:00:46 -04:00
Itamar Turner-Trauring
07e16b80b5 Better name. 2022-05-11 11:00:05 -04:00
Itamar Turner-Trauring
6f5a0e43eb Implement advise_corrupt_share for mutables. 2022-05-11 10:41:36 -04:00
Itamar Turner-Trauring
b3fed56c00 Move Foolscap compatibility to a better place. 2022-05-05 12:11:09 -04:00
Itamar Turner-Trauring
2833bec80e Unit test the new storage server backend API. 2022-05-05 12:04:45 -04:00
Itamar Turner-Trauring
06029d2878 Another end-to-end test passing (albeit with ugly implementation). 2022-05-04 11:25:13 -04:00
Itamar Turner-Trauring
852162ba06 More accurate docs. 2022-05-04 11:03:35 -04:00
Itamar Turner-Trauring
c1ce74f88d Ability to list shares, enabling more of IStorageClient to run over HTTP. 2022-05-04 11:03:14 -04:00
Itamar Turner-Trauring
5ce204ed8d Make queries run in parallel. 2022-04-28 12:18:58 -04:00
Itamar Turner-Trauring
b8b1d7515a We can at least be efficient when possible. 2022-04-28 11:59:50 -04:00
Itamar Turner-Trauring
76d0cfb770 Correct comment. 2022-04-28 11:49:21 -04:00
Itamar Turner-Trauring
e16eb6dddf Better type definitions. 2022-04-28 11:48:59 -04:00
Itamar Turner-Trauring
72c59b5f1a Unused import. 2022-04-19 15:09:02 -04:00
Itamar Turner-Trauring
f03feb0595 TODOs for later. 2022-04-19 15:08:07 -04:00
Itamar Turner-Trauring
21c3c50e37 Basic mutable read support. 2022-04-19 15:07:57 -04:00
Itamar Turner-Trauring
f5c4513cd3 A little closer to serialization and deserialization working correctly, with 
some tests passing.
 2022-04-19 14:46:13 -04:00
Itamar Turner-Trauring
898fe0bc0e Closer to running end-to-end mutable tests. 2022-04-19 13:18:31 -04:00
Itamar Turner-Trauring
b0d547ee53 Progress on implementing client side of mutable writes. 2022-04-18 14:56:20 -04:00
Itamar Turner-Trauring
24548dee0b Sketch of read/write APIs interface for mutables on client side. 2022-04-15 09:56:06 -04:00
Itamar Turner-Trauring
186aa9abc4 Make the utility reusable. 2022-04-15 09:32:15 -04:00
Itamar Turner-Trauring
58bd381202 Switch to newer attrs API. 2022-04-15 09:19:30 -04:00
Itamar Turner-Trauring
fc2807cccc Sketch of server-side read-test-write endpoint. 2022-04-15 09:08:16 -04:00
Jean-Paul Calderone
c0a84ececa
Merge pull request #1191 from tahoe-lafs/3526.test_invite-no-mock 
Remove mock usage from test_invite

Fixes: ticket:3526
 2022-04-14 12:50:17 -04:00
Itamar Turner-Trauring
4e0f912a10 Comply with license. 2022-04-14 11:52:20 -04:00
Itamar Turner-Trauring
9db5a397e1 Minor type annotation improvements. 2022-04-14 11:45:47 -04:00
Itamar Turner-Trauring
2bc8cdf852 Drop Python 2. 2022-04-14 11:40:19 -04:00
Itamar Turner-Trauring
5349f35a0b Merge remote-tracking branch 'origin/master' into 3875-http-storage-furls 2022-04-14 11:39:21 -04:00
Jean-Paul Calderone
f34e01649d some more fixes for mypy 2022-04-13 18:35:18 -04:00
Jean-Paul Calderone
03674bd452 use Tuple for type alias 
__future__.annotations only fixes py37/generic builtins in annotations syntax,
not arbitrary expressions
 2022-04-13 16:01:32 -04:00
Jean-Paul Calderone
38e1e93a75 factor the duplicate client logic out 2022-04-13 15:42:10 -04:00
Jean-Paul Calderone
ec5be01f38 more completely annotate types in the wormholetesting module 2022-04-13 14:50:38 -04:00
Jean-Paul Calderone
10f79ce8aa Use __future__.annotations in test_invite for generic builtins too 2022-04-13 14:39:52 -04:00
Jean-Paul Calderone
0f61a1dab9 Factor some duplication out of the test methods 2022-04-12 14:33:11 -04:00
Jean-Paul Calderone
71b5cd9e0d rewrite comment annotations with syntax 2022-04-12 14:13:48 -04:00
Jean-Paul Calderone
b0fffabed0 remove unnecessary module-scope wormhole 
used this during testing so the other mock() calls wouldn't explode in a
boring way
 2022-04-12 14:10:02 -04:00
Itamar Turner-Trauring
1634f137be Use sets more widely in the schema. 2022-04-12 12:54:16 -04:00
Jean-Paul Calderone
bc6dafa999 Replace monkey-patching of wormhole with a parameter to run_cli 2022-04-12 11:01:04 -04:00
Jean-Paul Calderone
dffcdf2854 Clean up the Py2/Py3 boilerplate 2022-04-11 15:05:32 -04:00
Jean-Paul Calderone
f19bf8cf86 Parameterize the options object to the run_cli helper 2022-04-11 15:04:55 -04:00
Itamar Turner-Trauring
4b20b67ce6 Client-side schema validation. 2022-04-11 14:03:48 -04:00
Itamar Turner-Trauring
dfad50b1c2 Better error. 2022-04-11 14:03:30 -04:00
Itamar Turner-Trauring
e5b0e51f72 Server-side schema validation of CBOR. 2022-04-11 13:11:45 -04:00
Itamar Turner-Trauring
bdcf054de6 Switch to generating certs on the fly since Python packaging was being a pain. 2022-04-08 13:37:18 -04:00
Itamar Turner-Trauring
423512ad00 Wait harder. 2022-04-06 11:48:17 -04:00
Itamar Turner-Trauring
22ebbba5ac Extend testing. 2022-04-06 11:35:05 -04:00
Itamar Turner-Trauring
5a82ea880b More specific methods. 2022-04-06 11:31:26 -04:00
Itamar Turner-Trauring
ab1297cdd6 Link to ticket. 2022-04-06 11:27:42 -04:00
Itamar Turner-Trauring
eda5925548 Get rid of another place where listen on port 0, and switch to FilePath only for now. 2022-04-06 11:25:37 -04:00
Itamar Turner-Trauring
710fad4f8a Support broader range of server endpoints, and switch to more robust random port 
assignment.
 2022-04-06 11:10:42 -04:00
Itamar Turner-Trauring
2e934574f0 Switch to URL-safe base64 for SPKI hash, for nicer usage in NURLs. 2022-04-06 09:37:18 -04:00
Itamar Turner-Trauring
5972a13457 Add reactor argument. 2022-04-06 09:34:17 -04:00
Itamar Turner-Trauring
30a3b006a0 Include self-signed cert in package install. 2022-03-30 10:26:26 -04:00
Itamar Turner-Trauring
da6838d6f9 Stop talking about furl, it's a NURL. 2022-03-28 11:35:45 -04:00
Itamar Turner-Trauring
119ba9468e Not needed. 2022-03-28 11:28:38 -04:00
Itamar Turner-Trauring
4e58748c4a Get constants from OpenSSL directly. 2022-03-28 11:27:32 -04:00
Itamar Turner-Trauring
ae8a7eff43 Make mypy happy. 2022-03-25 15:52:31 -04:00
Itamar Turner-Trauring
638154b2ad Cleanups. 2022-03-25 15:46:42 -04:00
Itamar Turner-Trauring
23ce581405 More tests; some still failing. 2022-03-25 14:01:06 -04:00
Itamar Turner-Trauring
bd6e537891 Hacky fix. 2022-03-25 13:52:25 -04:00
Itamar Turner-Trauring
712f4f1385 Sketch of first HTTPS client logic test. 2022-03-25 13:49:11 -04:00
Itamar Turner-Trauring
6f86675766 Split into its own file. 2022-03-25 10:59:16 -04:00
Itamar Turner-Trauring
9240d9d657 Expand the comment. 2022-03-25 10:46:14 -04:00
Itamar Turner-Trauring
e50d88f46d Technically this doesn't matter, because it's client-side, but it's good habit. 2022-03-25 10:45:54 -04:00
meejah
cc682d0413 Merge branch 'master' into 3828.key-length 2022-03-24 11:40:52 -06:00
Itamar Turner-Trauring
be0ff08275 Possibly correct, but communicating, end-to-end TLS with some amount of 
validation logic.

Still untested!
 2022-03-24 12:12:11 -04:00
Jean-Paul Calderone
f48055179f
Merge pull request #1186 from tahoe-lafs/3883.drop-cpuusagemonitor-foolscap-dep 
Drop the Foolscap dependency in allmydata.stats

Fixes: ticket:3883
 2022-03-24 08:09:42 -04:00
Jean-Paul Calderone
c737bcdb6f use the generic version of the correct types for samples 2022-03-23 18:40:12 -04:00
Itamar Turner-Trauring
5310747eaa Start hooking up end-to-end tests with TLS, fixing bugs along the way. 
At this point the issue is that the client fails certificate validation (which
is expected lacking the pinning validation logic, which should be added next).
 2022-03-23 16:33:29 -04:00
Itamar Turner-Trauring
eef99c1f22 Merge remote-tracking branch 'origin/master' into 3875-http-storage-furls 2022-03-23 15:59:07 -04:00
Jean-Paul Calderone
98634ae5cb allow the correct number of samples in 
also speed up the test with some shorter intervals
 2022-03-18 12:36:48 -04:00
Jean-Paul Calderone
b56f79c7ad read initial cpu usage value at service start time 
This removes the Foolscap dependency.
 2022-03-18 10:43:17 -04:00
Jean-Paul Calderone
ce381f3e39 pull the default initial_cpu value out to class scope 
also add python 3 syntax type annotations
 2022-03-18 10:42:40 -04:00
Jean-Paul Calderone
32e88e580b switch to a deque 
this makes more of the data structure someone else's responsibility and
probably improves performance too (but I didn't measure)
 2022-03-18 10:41:33 -04:00
Jean-Paul Calderone
b83b6adfbc remove py2 compat boilerplate 2022-03-18 10:41:07 -04:00
Itamar Turner-Trauring
7de3d93b0e Switch to TypeError. 2022-03-18 10:12:51 -04:00
Itamar Turner-Trauring
fae9556e3d Centralize client serialization logic too. 2022-03-14 11:28:54 -04:00
Itamar Turner-Trauring
0aa8089d81 Explicitly tell the server that the client accepts CBOR. 2022-03-14 11:20:23 -04:00
Itamar Turner-Trauring
106cc708a0 Use a constant. 2022-03-14 11:18:53 -04:00
Itamar Turner-Trauring
722f8e9598 Expand docs. 2022-03-14 11:17:06 -04:00
Itamar Turner-Trauring
b6073b11c2 Refactor to check HTTP content-type of request body. 2022-03-14 11:16:09 -04:00
Itamar Turner-Trauring
fef332754b Switch to shared utility so server can use it too. 2022-03-14 11:09:40 -04:00
Itamar Turner-Trauring
13fd3b3685 Get rid of Python 2 crud. 2022-03-14 11:01:20 -04:00
Itamar Turner-Trauring
1e108f8445 Don't use a custom parser. 2022-03-14 11:01:09 -04:00
Itamar Turner-Trauring
b8ab3dd6a7 Server handles Accept headers. 2022-03-14 10:53:22 -04:00
Itamar Turner-Trauring
e55c3e8acf Check for CBOR content-encoding header in client. 2022-03-14 10:35:39 -04:00
Itamar Turner-Trauring
5baf63219d Always use UTF-8 for corruption reports. 2022-03-10 17:41:28 -05:00
Itamar Turner-Trauring
7e25b43dba Direct unit tests for advising share is corrupt. 2022-03-10 11:28:48 -05:00
Itamar Turner-Trauring
922ee4feb1 Sketch of advise_corrupt_share support for immutables. 2022-03-10 11:09:45 -05:00
Itamar Turner-Trauring
f736683347 Finish testing and implementing lease renewal. 2022-03-09 13:35:50 -05:00
Itamar Turner-Trauring
aee0f7dc69 Sketch of lease renewal implementation. 2022-03-09 13:10:13 -05:00
Itamar Turner-Trauring
636ab017d4 Disconnection is purely a Foolscap concern. 2022-03-09 12:47:14 -05:00
Itamar Turner-Trauring
e598fbbc85 Get rid of redundant code. 2022-03-09 12:42:24 -05:00
Itamar Turner-Trauring
5d51aac0d3 Clarify. 2022-03-09 12:41:40 -05:00
Itamar Turner-Trauring
edb9eda53b Clarify. 2022-03-09 12:41:10 -05:00
Itamar Turner-Trauring
ac271cfb3d Merge remote-tracking branch 'origin/master' into 3877-immutable-storage-apis-continued 2022-03-09 11:23:02 -05:00
Itamar Turner-Trauring
86769c19bf Finish abort logic and tests. 2022-03-09 11:19:23 -05:00
Itamar Turner-Trauring
ef4f912a68 Less error-prone testing assertion, and fix a testing bug. 2022-03-09 11:11:39 -05:00
Itamar Turner-Trauring
4fc7ef7528 Basic HTTP test for aborts. 2022-03-09 10:57:05 -05:00
Itamar Turner-Trauring
f47741afb1 Correct behavior on timed out immutable uploads. 2022-03-09 10:45:21 -05:00
Itamar Turner-Trauring
92b952a5fe Authenticate writes! 2022-03-09 10:45:08 -05:00
Itamar Turner-Trauring
c642218173 Sketch of aborting uploads. 2022-03-08 10:41:56 -05:00
Itamar Turner-Trauring
5203873995 Refactor to unify data structure logic. 2022-03-08 10:13:37 -05:00
Itamar Turner-Trauring
5be7cbc171
Merge pull request #1182 from tahoe-lafs/3876-per-bucket-upload-secret 
Allow per-bucket upload secret

Fixes ticket:3876
 2022-03-08 10:10:48 -05:00
Itamar Turner-Trauring
7721c134f2 Change the semantics of HTTP bucket creation so that it's possible to have a 
different upload secret per upload.
 2022-03-07 09:28:21 -05:00
Itamar Turner-Trauring
40e5ab1661
Merge pull request #1178 from tahoe-lafs/3860-http-more-immutables 
More immutable support for HTTP storage API

Fixes ticket:3860
 2022-03-07 09:12:56 -05:00