ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2025-01-13 00:10:03 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
14,849 Commits 185 Branches 90 Tags 90 MiB
264269f409
Commit Graph

8767 Commits

Author SHA1 Message Date
dlee
eb26c97ef7 implicit_optional flag added and errors related to flag fixed 2023-02-06 15:29:53 -06:00
Itamar Turner-Trauring
c4114e032e Fix type signature. 2023-02-06 15:33:08 -05:00
Itamar Turner-Trauring
4576d10915 Add an explanation. 2023-02-06 13:57:33 -05:00
Itamar Turner-Trauring
5909f451e3 Use the CPU thread pool for CBOR validation. 2023-02-06 13:54:47 -05:00
Itamar Turner-Trauring
b221954946 A working thread pool. 2023-02-06 13:09:22 -05:00
dlee
31c5b78e6a Add back rsa.py accidentally removed file on website 2023-02-03 21:35:55 -06:00
Daniel
80db4a9de4
Delete rsa.py 2023-02-03 21:25:24 -06:00
dlee
e2e33933a8 Forgot to push newsfragment 2023-02-03 16:48:06 -06:00
Itamar Turner-Trauring
b477c59e15 Actually have a working run-in-thread code path 2023-01-31 15:53:16 -05:00
Itamar Turner-Trauring
921a2083dc Make sure (immutable) test vectors are run with the segment size that was used 
to generate them.
 2023-01-25 15:47:35 -05:00
Itamar Turner-Trauring
28ca5abe3c Merge remote-tracking branch 'origin/master' into 3957-mutable-over-http-speed 2023-01-25 10:53:15 -05:00
Itamar Turner-Trauring
af53f5b8b9 Merge remote-tracking branch 'origin/master' into 3946-less-chatty-downloads 2023-01-25 10:12:20 -05:00
Itamar Turner-Trauring
cf4d767535 Fix whitespace. 2023-01-24 13:28:24 -05:00
Itamar Turner-Trauring
6ccccde9e0 Increase the max default segment size. 2023-01-24 13:15:55 -05:00
Itamar Turner-Trauring
d5f5d394dd Test changes in max default segment size (both directions). 2023-01-24 13:15:40 -05:00
Itamar Turner-Trauring
ba793e2c16 Make it actually work. 2023-01-24 11:10:50 -05:00
Itamar Turner-Trauring
3e6778268f Merge remote-tracking branch 'origin/master' into 3957-mutable-over-http-speed 2023-01-23 11:39:54 -05:00
Itamar Turner-Trauring
1f3993b689 Don't block on CDDL validation. 2023-01-23 11:37:13 -05:00
Jean-Paul Calderone
e2eac5855c Remove handling for older versions of pycddl 
>=0.4 is now a hard-requirement
 2023-01-23 08:52:00 -05:00
Jean-Paul Calderone
1a807a0232 mollify the type checker 2023-01-12 16:32:32 -05:00
Jean-Paul Calderone
d5628f9c84 Merge remote-tracking branch 'origin/master' into 3962.pre-determined-rsa-keys 2023-01-12 16:17:09 -05:00
Jean-Paul Calderone
c856f1aa29 Censor private key values in the HTTP log, too. 2023-01-12 16:16:55 -05:00
Jean-Paul Calderone
47ec418f7a Test that we can also download the mutable data via the web interface 2023-01-12 16:09:29 -05:00
Jean-Paul Calderone
e6ef45d337 test that we can also download the mutable 2023-01-12 15:37:12 -05:00
Jean-Paul Calderone
2d23e2e640 some doc improvements 2023-01-12 15:37:07 -05:00
Jean-Paul Calderone
2490f0f58a some minor rationalization of the return type 2023-01-12 15:33:37 -05:00
Itamar Turner-Trauring
828fc588c5 Add minimal docstrings. 2023-01-10 11:10:31 -05:00
Itamar Turner-Trauring
ccb5956645 0 is also valid FD. 2023-01-10 11:04:22 -05:00
Itamar Turner-Trauring
7c506057b5 Merge remote-tracking branch 'origin/master' into 3956-mutable-uploads 2023-01-09 17:05:31 -05:00
Itamar Turner-Trauring
f6d9c5a1b2 Fix PyInstaller. 2023-01-09 10:46:09 -05:00
Itamar Turner-Trauring
22227c7094 Support old pycddl too so nix can keep working. 2023-01-09 10:31:48 -05:00
Jean-Paul Calderone
3ce5ee6f03 get Literal from somewhere it is more likely to be 2023-01-07 07:17:40 -05:00
Jean-Paul Calderone
e893d06cb3 RSAPrivateKey certainly does have this method 
I don't know why mypy fails to see it.
 2023-01-06 21:00:21 -05:00
Jean-Paul Calderone
8c56ccad72 fall back to *with* validation, not without 2023-01-06 21:00:10 -05:00
Jean-Paul Calderone
85234b07a0 load the right kind of key! 2023-01-06 21:00:04 -05:00
Jean-Paul Calderone
c9e23dea13 we should always be able to get these 
and we always need overload now
 2023-01-06 20:59:48 -05:00
Jean-Paul Calderone
a806b2faba Fix some more mypy errors 2023-01-06 18:11:47 -05:00
Jean-Paul Calderone
01b14fe05c Merge remote-tracking branch 'origin/master' into 3962.pre-determined-rsa-keys 2023-01-06 17:30:01 -05:00
Jean-Paul Calderone
2dc6466ef5 fix some errors reported by mypy 2023-01-06 17:12:59 -05:00
Jean-Paul Calderone
e829b891b3 important data file ... 2023-01-06 15:51:59 -05:00
Jean-Paul Calderone
3ff9c45e95 expose the private-key feature in the tahoe put cli 2023-01-06 15:40:48 -05:00
Jean-Paul Calderone
e236cc95a5 Move get_keypair to a shared location 2023-01-06 15:36:08 -05:00
Itamar Turner-Trauring
3de5ebde48 Merge branch '3956-mutable-uploads' into 3957-mutable-over-http-speed 2023-01-06 11:05:11 -05:00
Jean-Paul Calderone
3423bfb351 Expose the pre-constructed keypair functionality to the HTTP API 2023-01-03 11:31:29 -05:00
Jean-Paul Calderone
c7bb190290 Factor some SSK "signature" key handling code into a more reusable shape 
This gives the test suite access to the derivation function so it can
re-derive certain values to use as expected results to compare against actual
results.
 2023-01-03 10:38:18 -05:00
Jean-Paul Calderone
5bad92cfc5 Another Python 2 remnant cleanup 2023-01-03 10:34:39 -05:00
Jean-Paul Calderone
a58d8a567a Clean up some more Python 2 remnants 2023-01-03 10:33:37 -05:00
Jean-Paul Calderone
6b58b66677 Clean up some Python 2 remnants 2023-01-03 10:32:03 -05:00
Jean-Paul Calderone
f6d9c33526 Give slightly better error messages from rsa key validation failure 2023-01-03 10:28:59 -05:00
Jean-Paul Calderone
23f2d8b019 add some type annotations to allmydata.crypto.rsa 2023-01-03 10:28:32 -05:00
Jean-Paul Calderone
15e22dcc52 Add keypair to NodeMaker.create_mutable_file 
Previously `NodeMaker` always took responsibility for generating a keypair to
use.  Now the caller may supply one.
 2023-01-02 19:29:13 -05:00
Jean-Paul Calderone
3f8bec0ecc Merge remote-tracking branch 'origin/master' into 3914.faster-rsa-tests 2023-01-02 11:46:10 -05:00
Jean-Paul Calderone
1c643ebbaf
Merge pull request #1237 from tahoe-lafs/3954.faster-http 
Faster HTTP for immutable downloads

Fixes: ticket:3954
 2023-01-02 10:09:55 -05:00
Itamar Turner-Trauring
d1b464d0d8 Writing large files can involve many writes. 2022-12-21 09:35:10 -05:00
Itamar Turner-Trauring
54da6eb60a Remove unneeded imports. 2022-12-21 09:34:25 -05:00
Itamar Turner-Trauring
1a4dcc70e2 Support large mutable uploads in a memory-efficient manner. 2022-12-21 09:24:31 -05:00
Jean-Paul Calderone
6b57b03157 Merge remote-tracking branch 'origin/master' into 3942.missing-authorization-handling 2022-12-21 05:51:39 -05:00
Jean-Paul Calderone
80caf04966
Merge pull request #1234 from tahoe-lafs/3947.mutable-test_version-async-def 
Convert some tests for mutables to use `async def`

Fixes: ticket:3947
 2022-12-21 05:50:07 -05:00
Itamar Turner-Trauring
1d3464a430 Add end-to-end MDMF test. 2022-12-19 13:37:20 -05:00
Itamar Turner-Trauring
96347e22e2 Make a test demonstrating the problem. 2022-12-15 13:14:49 -05:00
Jean-Paul Calderone
c014ad55b1 remove Python 2 boilerplate 2022-12-14 08:48:02 -05:00
Jean-Paul Calderone
05c7450376 Try to use an upcoming python-cryptography feature to avoid some costs 
If the key is the wrong number of bits then we don't care about any other
validation results because we're just going to reject it.  So, check that
before applying other validation, if possible.

This is untested since the version of python-cryptography that supports it is
not released yet and I don't feel like setting up a Rust build tool chain at
the moment.
 2022-12-14 08:47:05 -05:00
Itamar Turner-Trauring
6ae40a932d A much more reasonable number of HTTP connections. 2022-12-13 15:54:19 -05:00
Itamar Turner-Trauring
1eba202c08
Merge pull request #1231 from tahoe-lafs/3939-faster-http-protocol 
Faster http protocol, part 1 (and maybe faster Foolscap too, while we're at it)

Fixes ticket:3939
 2022-12-05 14:05:55 -05:00
Jean-Paul Calderone
daecfc63b9
Merge pull request #1233 from tahoe-lafs/3874.fix-testing-helper-encoding 
Fix a testing helper
 2022-12-02 10:27:58 -05:00
Jean-Paul Calderone
c542b84637 remove the annotations 
everything is broken on older pythons
 2022-12-02 08:47:07 -05:00
Jean-Paul Calderone
c6cc3708f4 Fixup the annotations a bit 2022-12-02 08:38:46 -05:00
Jean-Paul Calderone
b40d882fce remove unused import 2022-12-02 08:28:22 -05:00
Jean-Paul Calderone
a84b278ecd support older pythons 2022-12-02 08:26:15 -05:00
Jean-Paul Calderone
9619e286f4 Switch the web testing double to BytesKeyDict 
This will catch more str/bytes errors by default than `dict`
 2022-12-02 08:16:02 -05:00
meejah
20b3594d12 exarkun wants a helper 2022-12-01 17:05:58 -07:00
meejah
36ed554627 proto -> transport 2022-12-01 17:03:48 -07:00
meejah
3d43cbccc9 log less-specific failures 2022-12-01 17:01:38 -07:00
meejah
c57d1aa8a9 Merge branch 'master' into 3921.exit-on-stdin-close 2022-12-01 16:58:37 -07:00
meejah
7ffcfcdb67
Update src/allmydata/test/test_runner.py 
Co-authored-by: Jean-Paul Calderone <exarkun@twistedmatrix.com>
 2022-12-01 16:47:40 -07:00
Itamar Turner-Trauring
c93ff23da7 Don't send empty string writes. 2022-12-01 14:54:28 -05:00
Itamar Turner-Trauring
8ed333b171 Correct explanation. 2022-12-01 14:45:45 -05:00
Itamar Turner-Trauring
0ba58070cd Tweaks. 2022-12-01 14:45:39 -05:00
Itamar Turner-Trauring
d4c202307c Better method name. 2022-12-01 14:43:49 -05:00
Itamar Turner-Trauring
17dfda6b5a More direct API. 2022-12-01 14:42:52 -05:00
Fon E. Noel NFEBE
b193ad3ed4 Correct addCleanup reference 
Some test_storage.py classes contain calls to cleanup methods
instead of references. This commit fixes that.

Signed-off-by: Fon E. Noel NFEBE <fenn25.fn@gmail.com>
 2022-11-30 16:53:22 +01:00
Jean-Paul Calderone
05dfa875a7 Quite a mypy warning about formatting bytes into a string 2022-11-30 09:46:13 -05:00
Jean-Paul Calderone
156954c621 no longer any need to "daisy chain" this value 2022-11-30 09:43:01 -05:00
Jean-Paul Calderone
a11eeaf240 Convert all of the asynchronous functions to use async and await 2022-11-30 09:30:37 -05:00
Jean-Paul Calderone
1acf8604ef Remove the Py2/Py3 compatibility header 2022-11-30 09:30:08 -05:00
Jean-Paul Calderone
5cebe91406 update the module docstring 2022-11-30 09:29:57 -05:00
Jean-Paul Calderone
02aeb68f17 Take care with str vs bytes in the implementation 
Also replace the intentional BAD_REQUEST with GONE for this case.
 2022-11-29 10:51:07 -05:00
Jean-Paul Calderone
d7fe25f7c7 Correct the assertion about how "not found" should be handled 
Behavior verified visually against a live client node:

```
❯ curl -v 'http://localhost:3456/uri/URI:CHK:cmtcxq7hwxvfxan34yiev6ivhy:qvcekmjtoetdcw4kmi7b3rtblvgx7544crnwaqtiewemdliqsokq:1:1:1'
*   Trying 127.0.0.1:3456...
* Connected to localhost (127.0.0.1) port 3456 (#0)
> GET /uri/URI:CHK:cmtcxq7hwxvfxan34yiev6ivhy:qvcekmjtoetdcw4kmi7b3rtblvgx7544crnwaqtiewemdliqsokq:1:1:1 HTTP/1.1
> Host: localhost:3456
> User-Agent: curl/7.83.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 410 Gone
< X-Frame-Options: DENY
< Referrer-Policy: no-referrer
< Server: TwistedWeb/22.10.0
< Date: Tue, 29 Nov 2022 15:39:47 GMT
< Content-Type: text/plain;charset=utf-8
< Accept-Ranges: bytes
< Content-Length: 294
< ETag: ui2tnwl5lltj5clzpyff42jdce-
<
NoSharesError: no shares could be found. Zero shares usually indicates a corrupt URI, or that no servers were connected, but it might also indicate severe corruption. You should perform a filecheck on this object to learn more.

The full error message is:
* Connection #0 to host localhost left intact
no shares (need 1). Last failure: None
```
 2022-11-29 10:49:20 -05:00
Jean-Paul Calderone
920467dcea Treat missing Authorization as the same as empty Authorization 2022-11-29 10:19:01 -05:00
Jean-Paul Calderone
f5b24d51e9 Add a test for missing Authorization 2022-11-29 10:18:57 -05:00
Itamar Turner-Trauring
06b57cd835 Merge remote-tracking branch 'origin/master' into 3939-faster-http-protocol 2022-11-28 11:07:01 -05:00
Itamar Turner-Trauring
aa80c9ef47 Be more robust. 2022-11-28 10:21:59 -05:00
Itamar Turner-Trauring
3ba166c2cb A bit more robust code. 2022-11-28 10:20:12 -05:00
Itamar Turner-Trauring
0f4dc91295 Refactor so internal attributes needn't leak. 2022-11-28 10:12:08 -05:00
Itamar Turner-Trauring
38d7430c57 Simplify. 2022-11-28 10:03:42 -05:00
Itamar Turner-Trauring
2ab8e3e8d2 Cancel timeout on failures too. 2022-11-28 10:02:56 -05:00
Fon E. Noel NFEBE
6b0fa64236 Clean up test_storage.py after refactor 
This PR cleans up errorneous changes resulting from
1d85a2c5cf and adds a
few improvements such as calling `super` implementations.

Making sure classes with functions returning deferreds
use `AsyncTestCase`

Signed-off-by: Fon E. Noel NFEBE <fenn25.fn@gmail.com>
 2022-11-25 13:36:22 +01:00
Itamar Turner-Trauring
41533f162e Not used anymore. 2022-11-23 10:20:32 -05:00
Itamar Turner-Trauring
3790280264 Documentation. 2022-11-23 10:16:49 -05:00
Itamar Turner-Trauring
fd9e50adf1 Simplify _WriteBuffer slightly. 2022-11-23 10:13:18 -05:00
Itamar Turner-Trauring
d1deda5fdd Unit tests for _WriteBuffer. 2022-11-23 10:09:53 -05:00
Itamar Turner-Trauring
7f1d7d4f46 Better explanation. 2022-11-23 09:53:07 -05:00
Itamar Turner-Trauring
d86d578034 Refactor to make core data structure easier to test in isolation. 2022-11-22 15:17:56 -05:00
Itamar Turner-Trauring
f638aec0af Refactor to use BytesIO. 2022-11-22 14:22:54 -05:00
Itamar Turner-Trauring
a4787ca45e Batch writes much more aggressively. 2022-11-22 14:12:14 -05:00
Itamar Turner-Trauring
c80469b50b Handle the Windows waker too. 2022-11-22 11:55:56 -05:00
Itamar Turner-Trauring
3a613aee70 Try a different approach to timeouts: dynamic, instead of hardcoded. 2022-11-21 12:24:50 -05:00
Fon E. Noel NFEBE
1d85a2c5cf Refactor more test_storage.py classes 
As a follow up to commit fbc8baa238
this refactor continues to remove deprecated methods and ensures
test classes either extend `SyncTestCase` or `AsyncTestCase`

Classes refactored:

- `MutableServer`
- `MDMFProxies`
- `Stats`
- `MutableShareFileTests`

Signed-off-by: Fon E. Noel NFEBE <fenn25.fn@gmail.com>
 2022-11-20 15:38:08 +01:00
Fon E. Noel NFEBE
4c18890e84 Merge branch 'master' into 3917-refactor-test-storage-py 2022-11-19 23:35:59 +01:00
Itamar Turner-Trauring
8cfdae2ab4 sigh 2022-11-18 15:26:02 -05:00
Itamar Turner-Trauring
aeaced848d Merge remote-tracking branch 'origin/master' into 3940-http-timeouts 2022-11-18 15:25:44 -05:00
Itamar Turner-Trauring
4c0c75a034 Fix DelayedCall leak in tests. 2022-11-18 13:56:54 -05:00
Itamar Turner-Trauring
9b21f1da90 Increase how many statuses are stored. 2022-11-17 11:35:10 -05:00
Itamar Turner-Trauring
d182a2f186 Add the delay to appropriate test. 2022-11-17 11:01:12 -05:00
Itamar Turner-Trauring
097d918a24 Sigh 2022-11-16 13:37:50 -05:00
Itamar Turner-Trauring
3531737347 Make timeouts less aggressive, CI machines are slow? 2022-11-16 10:36:11 -05:00
Itamar Turner-Trauring
2ab172ffca Try to set more aggressive timeouts when testing. 2022-11-16 10:26:29 -05:00
Itamar Turner-Trauring
9f5f287473 Nope, not helpful. 2022-11-16 09:57:39 -05:00
Itamar Turner-Trauring
a20943e10c As an experiment, see if this fixes failing CI. 2022-11-16 09:33:01 -05:00
Itamar Turner-Trauring
2a5e8e5971 Better cleanup. 2022-11-15 15:05:27 -05:00
Itamar Turner-Trauring
30a9877236 Merge branch '3783-storage-client-http' into 3940-http-timeouts 2022-11-15 14:38:48 -05:00
Itamar Turner-Trauring
f3fc426830 Switch to [storage] force_foolscap. 2022-11-15 14:36:14 -05:00
Itamar Turner-Trauring
fb52b4d302 Delete some garbage. 2022-11-15 14:22:30 -05:00
Itamar Turner-Trauring
8e4ac69032 Stop test mode when done. 2022-11-15 14:21:31 -05:00
Itamar Turner-Trauring
4aeb62b66c Use a constant. 2022-11-15 14:16:41 -05:00
Itamar Turner-Trauring
d700163aec Remove no-longer-relevant comment. 2022-11-15 14:14:29 -05:00
Itamar Turner-Trauring
6c80ad5290 Not necessary. 2022-11-15 14:13:50 -05:00
Itamar Turner-Trauring
d1287df629 The short timeout should be specific to the storage client's needs. 2022-11-15 14:02:19 -05:00
meejah
039c1d8037
Update src/allmydata/grid_manager.py 
Co-authored-by: Jean-Paul Calderone <exarkun@twistedmatrix.com>
 2022-11-09 10:52:42 -07:00
meejah
b2431f3a89
Update src/allmydata/cli/grid_manager.py 
Co-authored-by: Jean-Paul Calderone <exarkun@twistedmatrix.com>
 2022-11-09 10:46:31 -07:00
Itamar Turner-Trauring
8d678fe3de Increase timeout. 2022-11-07 11:41:50 -05:00
Itamar Turner-Trauring
65a7945fd9 Don't need a connection timeout since we have request-level timeouts. 2022-11-07 11:39:45 -05:00
Itamar Turner-Trauring
2c911eeac1 Make sure everything is using the same clock. 2022-11-07 11:28:36 -05:00
Itamar Turner-Trauring
f8b9607fc2 Finish up limited_content() timeout code. 2022-11-07 11:26:11 -05:00
Itamar Turner-Trauring
ce59ddc1ea Merge branch '3783-storage-client-http' into 3940-http-timeouts 2022-11-07 11:20:54 -05:00
Itamar Turner-Trauring
c4772482ef WIP 2022-11-07 11:19:00 -05:00
Itamar Turner-Trauring
414b463556 Use built-in treq timeout feature. 2022-11-07 09:23:04 -05:00
Itamar Turner-Trauring
1e50e96e24 Update to new test API. 2022-11-03 15:04:41 -04:00
Itamar Turner-Trauring
8bebb09edd Less test-specific way to make test_rref pass. 2022-11-03 14:38:59 -04:00
Itamar Turner-Trauring
262d9d85b9 Switch to using persistent connections in tests too. 2022-11-03 14:32:43 -04:00
Itamar Turner-Trauring
e05136c238 Less aggressive timeout, to try to make tests pass on CI. 2022-11-02 13:13:21 -04:00
Itamar Turner-Trauring
48ae729c0d Don't reuse basedir across tests. 2022-11-02 10:18:23 -04:00
Itamar Turner-Trauring
73271afa65 Merge remote-tracking branch 'origin/master' into 3783-storage-client-http 2022-11-02 09:23:29 -04:00
Itamar Turner-Trauring
1a3e3a86c3 Require latest pycddl, and work around a regression. 2022-10-14 11:27:04 -04:00
Itamar Turner-Trauring
0febc87456 Don't include reactor in comparison. 2022-10-14 10:03:06 -04:00
Itamar Turner-Trauring
e409262e86 Fix some flakes. 2022-10-14 09:50:07 -04:00
Itamar Turner-Trauring
42d3843343 Run test_system with both Foolscap and HTTP storage protocols, plus some 
resulting cleanups.
 2022-10-14 09:16:59 -04:00
Itamar Turner-Trauring
0f31e3cd4b Leave HTTP off by default for now. 2022-10-13 14:41:59 -04:00
Itamar Turner-Trauring
b80a215ae1 test_rref passes now. 2022-10-13 14:15:10 -04:00
Itamar Turner-Trauring
0d23237b11 Some progress towards passing test_rref. 2022-10-13 13:44:49 -04:00