Commit Graph

4288 Commits

Author SHA1 Message Date
meejah
9032539860 flake8; naming 2018-03-20 11:40:49 -06:00
meejah
797932244d log failure properly, add test 2018-03-20 11:40:49 -06:00
meejah
39080852cc ensure 'in progress' items return status 2018-03-20 11:40:49 -06:00
meejah
1f495ec41f cover uploads and downloads in tests 2018-03-20 11:40:49 -06:00
meejah
dbc9f7e06c remove code that sets item size (from review) 2018-03-20 11:40:49 -06:00
meejah
75e97e6c6d use QueuedItem directly instead of FakeStatusItem 2018-03-20 11:40:49 -06:00
meejah
1b16e1f792 Some documentation for QueueMixin 2018-03-20 11:40:49 -06:00
meejah
472d6ba78b cover some of get_status 2018-03-20 11:40:49 -06:00
meejah
bfabfeba64 add 'size' to tests, print unicode 2018-03-20 11:40:49 -06:00
meejah
0ebb587666 report download sizes in magic-folder status 2018-03-20 11:40:49 -06:00
meejah
0431be14fa log inotify callback errors 2018-03-20 11:40:49 -06:00
meejah
42766da553 add a 'size' to pending uploads 2018-03-20 11:40:49 -06:00
meejah
8ac63cf5bd only keep history for items that are processed 2018-03-20 11:40:49 -06:00
meejah
bcdd15394b ensure there are no gaps in magic-folder status 2018-03-20 11:40:49 -06:00
Jean-Paul Calderone
371965f7d4 Specify the new optFlag correctly 2018-03-19 15:12:59 -04:00
Jean-Paul Calderone
1f92879788 improve test coverage 2018-03-19 14:21:28 -04:00
Ruben Pollan
136de7d7f7 Add storage_dir config field
On the [storage] section of the tahoe.cfg now there is a field
'storage_dir' where the path to the storage folder can be configured.
2018-03-19 11:20:04 -04:00
meejah
a1cb401f06
Merge pull request #452 from meejah/list-aliases-json.1
Add --json option to 'tahoe list aliases'
2018-03-13 17:05:20 -06:00
tpltnt
a41d827f79 _None() as new style class 2018-03-02 17:17:30 +01:00
tpltnt
1967cd7357 removed unnecessary after return 2018-03-02 16:45:11 +01:00
tpltnt
1a14d5218b added some docstrings 2018-03-02 16:45:11 +01:00
tpltnt
3f99eceb36 minor whitespace fixes 2018-03-02 16:45:11 +01:00
tpltnt
0461cc0148 only one module import per line 2018-03-02 16:45:11 +01:00
tpltnt
f7bbc9dffe made mkstemp() call compatible with mktemp() 2018-03-02 16:45:11 +01:00
tpltnt
710ba84a02 switched to secure mkstemp() 2018-02-27 18:35:08 +01:00
meejah
0b25cad71e documentation fixups from daira's review 2018-02-06 15:28:02 -07:00
meejah
1171bf13af ticket #2882: preserve user mtime 2018-02-06 15:27:56 -07:00
meejah
68783d8b8a directories get +x, files do not 2018-01-09 01:26:37 -07:00
meejah
3466088717 add --json option for 'tahoe list-aliases'
some new tests to cover previously-uncovered code that changed
2018-01-05 18:07:52 -07:00
meejah
187eb562f2
Merge pull request #447 from tpltnt/ticket-1307
Ticket 1307
2018-01-04 17:50:21 -07:00
tpltnt
95acd57fdd removed exact version info from header 2018-01-04 22:29:57 +01:00
tpltnt
48b442f8c1 mentioned ctx 2018-01-04 22:29:57 +01:00
tpltnt
6d4f4f4a65 adjusted doc string 2018-01-04 22:29:57 +01:00
tpltnt
68b1271190 added docstring 2018-01-04 22:29:57 +01:00
tpltnt
005bcf5012 added code as described in ticket 1307 2018-01-04 22:29:57 +01:00
tpltnt
c5e8d83b73 removed old test/trial helper module 2018-01-04 22:09:52 +01:00
tpltnt
c7cf7c3956 make twisted >= 16.4.0 a requirement 2018-01-04 22:09:52 +01:00
meejah
c351a1d327
Merge pull request #453 from meejah/magic-folder-test-time
use '5 years ago' instead of absolute timestamp
2018-01-03 15:41:32 -07:00
meejah
5f00842532 use exarkun's suggestion from review 2018-01-03 14:07:38 -07:00
Brian Warner
5e62118e14 hush somewhat-real problems found by static analysis
These would have caused problems if a certain unlikely code path was taken:
mostly error-handling pathways.
2017-12-31 00:23:43 +01:00
Brian Warner
da4e7dcfbe hush a bunch of not-really-problems caught by the lgtm.com static analyzer
This is all minor stuff: unreachable debug code (that should be commented-out
instead of in an 'if False:' block), unnecessary 'pass' and 'global'
statements, redundantly-initialized variables. No behavior changes. Nothing
here was actually broken, it just looked suspicious to the static analysis at
https://lgtm.com/projects/g/tahoe-lafs/tahoe-lafs/alerts/?mode=list .
2017-12-31 00:19:41 +01:00
meejah
e5d6272b62 use '5 years ago' instead of absolute timestamp 2017-12-20 13:17:54 -07:00
meejah
6c0c90829f cheezy test for more coverage 2017-12-05 12:02:16 -07:00
meejah
87ad3cb8e9 fix 2880
- remember upload URI so we don't re-download
- account for empty files in "should_download"
- fix 'conflicted' asserts in tests
2017-12-05 12:02:16 -07:00
meejah
672475cb2b Multiple magic-folders
This moves all magic-folder configs to a single YAML
file. We load legacy config fine and don't mess with
legacy config unless you use a magic-folder command that
changes the config.

Increase test coverage
2017-12-05 10:34:48 -07:00
meejah
1b6f477549 Pull "config" handling out of Node and hide node-derivitives
This includes:

 - refactor to create_client and _Client
 - refactor to _IntroducerNode and create_introducer
2017-11-20 12:57:20 -07:00
meejah
50f8397c99 Merge branch 'invalid-pidfile' into delete-invalid-pidfile--lpirl
Conflicts:
	src/allmydata/scripts/startstop_node.py
	src/allmydata/test/cli/test_cli.py
2017-11-08 14:49:29 -07:00
Brian Warner
a1711088ed test_i2p_provider: exercise escaping of apiEndpoint attribute 2017-11-03 00:24:46 -07:00
Brian Warner
1f1afe65ec tor_provider.get_listener(): return a real endpoint, instead of a descriptor
get_listener() is allowed to return either, and the Tor provider is currently
simple enough to not really need more than a basic descriptor, but have it
return a full Endpoint for use as an example of what I2P can do later.
2017-11-03 00:24:46 -07:00
Brian Warner
d713e0e57d address review feedback: docstrings, better test class name 2017-11-03 00:24:46 -07:00
Brian Warner
097abb42fa tahoe.cfg: add tub.port=listen:i2p (and/or listen:tor)
This delegates the construction of the server Endpoint object to the i2p/tor
Provider, which can use the i2p/tor section of the config file to add options
which would be awkward to express as text in an endpoint descriptor string.

refs ticket:2889 (but note this merely makes room for a function to be
written that can process I2CP options, it does not actually handle such
options, so it does not close this ticket yet)
2017-11-03 00:24:46 -07:00
Brian Warner
d1fd43aa4f tor/i2p: rename create_onion/create_dest to create_config 2017-11-03 00:24:46 -07:00
meejah
fd90346c88 Wait up to 60s for node start; improve messaging; look for errors 2017-09-20 00:55:19 -06:00
meejah
263a3f43c1 stop chdir 2017-09-19 10:39:19 -06:00
meejah
9375056b61 Split up startstop_node and add 'tahoe daemonize'
This sets the stage for further changes to the startup
process so that "async things" are done before we create
the Client instance while still reporting early failures
to the shell where "tahoe start" is running

Also adds a bunch of test-coverage for the things that got
moved around, even though they didn't have coverage before
2017-09-19 10:39:19 -06:00
Lukas Pirl
89bacbb70a Merge branch 'master' into invalid-pidfile 2017-08-24 18:40:00 +02:00
Lukas Pirl
46305c74e1 added test for (ignoring an) invalid PID file when starting a node 2017-08-24 17:56:57 +02:00
Jean-Paul Calderone
86f79e8111 Add a test for the non-numeric case 2017-08-24 16:46:46 +02:00
Jean-Paul Calderone
354567cb85 Add a test for the non-numeric case 2017-08-24 10:34:58 -04:00
Lukas Pirl
1bf032959f delete invalid PID file on `tahoe (re)start` 2017-08-24 15:22:25 +02:00
Lukas Pirl
dd9b951c4b fix crash when stopping/restarting with an invalid pidfile
in node directory
2017-08-23 20:23:15 +02:00
Brian Warner
8dd0a5f772 dictutil: remove unused move, subtract, del_if_present 2017-08-12 21:48:05 -07:00
Brian Warner
61b85dbf78 dictutil: remove unused ValueOrderedDict 2017-08-12 21:34:24 -07:00
Brian Warner
4f493b1a03 remove unused UtilDict 2017-08-12 21:31:47 -07:00
Brian Warner
a76d9e084f dictutil: remove unused NumDict 2017-08-12 21:21:04 -07:00
Brian Warner
3afa38adb4 NummedObj: stop using dictutil.NumDict 2017-08-12 21:02:32 -07:00
Brian Warner
3f2f7dfb05 dictutil: fix bug in str(ValueOrderedDict), and improve test coverage
It looks like str() was meant to truncate the dict, but a missing i+=1 meant
that it never actually did. I also changed the format to include a clear
"..." in case we truncate it, to avoid confusion with a non-truncated dict of
the same size.

This also improves test coverage in subtract() and
NumDict.item_with_largest_value().

refs ticket:2891
2017-08-10 10:27:02 -07:00
Brian Warner
95ac5494ff test_backupdb: tolerate newer sqlite-3.20.0 error messages 2017-08-10 10:21:39 -07:00
meejah
798bf57e28 Add 'tahoe invite' and 'tahoe create-node --join' commands
This opens a wormhole and sends appropriate JSON down
it to a tahoe-gui using a wormhole server running on
tahoe-lafs.org

The other end uses the 'tahoe create-node' command (with
new --join option) to read the configuration JSON from
a 'tahoe invite' command
2017-08-08 18:27:06 -06:00
meejah
e2a5751d02 at least use random port 2017-08-08 10:42:11 -06:00
Brian Warner
a4be2dce71 avoid variable coverage by using a defaultdict
refs ticket:2891
2017-07-27 18:19:57 -07:00
Brian Warner
05317d4d12 test_web: replace all shouldHTTPError with assertHTTPError
and update to inlineCallbacks while we're in there
2017-07-27 16:31:44 -07:00
Brian Warner
2687ee90c5 test_web: remove last traces of deprecated client.getPage 2017-07-27 16:31:44 -07:00
Brian Warner
965e974ce8 test_web: fix POST(url,body) calls that should have been POST2()
and remove the followRedirect= argument from POST
2017-07-27 16:31:44 -07:00
Brian Warner
0deb903888 test_web: rewrite all POST(followRedirect=True) calls with do_http
Since POST() is about to lose followRedirect=True
2017-07-27 16:31:44 -07:00
Brian Warner
3f03367d2f test_web: remove all other uses of shouldRedirect 2017-07-27 16:31:44 -07:00
Brian Warner
76063b1c12 test_web: replace some instances of shouldRedirect with shouldRedirectTo 2017-07-27 16:31:44 -07:00
Brian Warner
6be91e369c test_web: replace shouldReplace2 with shouldReplaceTo
This removes many uses of self.POST (since shouldReplaceTo does its own
HTTP), which will make it easier to remove client.getPage from POST.
2017-07-27 16:31:44 -07:00
Brian Warner
4745239c2c test_web: use inlineCallbacks in many functions
specifically everywhere we use self.shouldRedirect2
2017-07-27 16:31:44 -07:00
Brian Warner
73d09082d7 test_web.web.Web.POST: split out build_form() helper function 2017-07-27 16:31:44 -07:00
Brian Warner
202a9714c4 test_web test_bad_method: remove a client.getPage
Add WebErrorMixin.assertHTTPError, to replace (getPage + shouldHTTPError)
2017-07-27 16:31:44 -07:00
Brian Warner
bee05e883c test_web: remove HTTPClientGETFactory
This also changes the tests to handle the new API: URL can be unicode, and
the returned Headers object is not a dictionary.
2017-07-27 16:31:44 -07:00
Brian Warner
5a895b5fb6 GridTestMixin.GET: use treq instead of HTTPClientGETFactory 2017-07-27 16:31:44 -07:00
Brian Warner
3311f9520d test.cli.test_status: replace getPage with treq 2017-07-27 16:31:44 -07:00
Brian Warner
bd1e4507ee remove unused HTTPClientHEADFactory 2017-07-27 16:31:44 -07:00
Jean-Paul Calderone
e32f581d07 Get rid of the reliance on rref
It provides information more correctly obtained from
get_connection_status().
2017-07-27 16:31:41 -04:00
Jean-Paul Calderone
1f16a173f0 flatten the result structure
flat is better than nested
2017-07-27 15:32:47 -04:00
Jean-Paul Calderone
7027547ecc fix buggy test; restructure data
scared about duplicate nodeids in the data getting lost, switch to a list
2017-07-27 15:26:30 -04:00
Jean-Paul Calderone
754304e4d5 Switch Root over to MultiFormatPage 2017-07-27 15:26:30 -04:00
Jean-Paul Calderone
f518c2a521 Switch over to stdlib json 2017-07-27 15:26:30 -04:00
Jean-Paul Calderone
e339130e63 Just make an assertion against the full value 2017-07-27 15:26:30 -04:00
Jean-Paul Calderone
e3e49632a5 use preferred spelling for new assertions 2017-07-27 15:26:30 -04:00
David Stainton
cd97f8519b web test: get_version should use a version string 2017-07-27 15:26:29 -04:00
David Stainton
923363c0d6 web test: test for server key in json welcome 2017-07-27 15:26:29 -04:00
David Stainton
16e56611f3 web: set json welcome content type to json utf8 2017-07-27 15:26:29 -04:00
David Stainton
ecb6651926 Add basic unit test for json welcome page 2017-07-27 15:26:29 -04:00
David Stainton
876a7b6602 Fix indention 2017-07-27 15:26:29 -04:00
David Stainton
ad66caeaff Make sure server.rref is not None 2017-07-27 15:26:29 -04:00
David Stainton
180975b0af Make sure version is not None 2017-07-27 15:26:29 -04:00
David Stainton
b2d4374976 Add more server info to welcome json page 2017-07-27 15:26:29 -04:00
David Stainton
32110bfc68 Add storage and introducer summaries to json welcome output 2017-07-27 15:26:29 -04:00
David Stainton
822ed78b13 Add rough sketch of json welcome page 2017-07-27 15:26:29 -04:00
Jean-Paul Calderone
1e885fa40b Convert storage.py over to MultiFormatPage 2017-07-27 13:57:02 -04:00
Jean-Paul Calderone
6e4c4b7abc Convert status.py over to MultiFormatPage 2017-07-27 13:57:02 -04:00
Jean-Paul Calderone
5a30bd6169 Convert introweb.py over to MultiFormatPage 2017-07-27 13:57:02 -04:00
Jean-Paul Calderone
0581ce7270 Convert directory.py over to MultiFormatPage 2017-07-27 13:57:02 -04:00
Jean-Paul Calderone
7c8c63a01f a helper for the ?t=json resources 2017-07-27 13:57:02 -04:00
Brian Warner
ad92b2b9b2 test_web: replace HEAD/PUT/DELETE with treq 2017-07-12 17:10:49 -07:00
Brian Warner
486fca0d5c web.test_introducer: replace getPage with treq 2017-07-12 17:10:49 -07:00
Brian Warner
8be5072fca test_system: replace getPage with treq 2017-07-12 17:10:49 -07:00
Brian Warner
2ec3791411 test_deepcheck: replace t.w.client.getPage with treq 2017-07-12 17:10:46 -07:00
Brian Warner
74e7ef4b98 test_happiness: limit string sizes to 30 chars
I think these are supposed to represent share ids or server ids, so we don't
need to exercise them being too huge.
2017-07-12 16:37:44 -07:00
Brian Warner
11c39325fb test_happiness: suppress the hypothesis 'too_slow' check
I was seeing buildbot failures where the host machine was too busy, and
hypothesis was spooked into thinking that data generation was too slow.
2017-07-12 16:27:13 -07:00
meejah
0977e52cbf Add a test for 'tahoe list-aliases --readonly-uri' 2017-06-06 18:03:05 +01:00
Daira Hopwood
d3acf82776 Rename FilesystemOptions to FileStoreOptions. refs #2345
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2017-06-06 11:20:49 +01:00
Daira Hopwood
2055a66cf2 Doc changes that require more careful review. refs #2345
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2017-06-06 11:20:49 +01:00
Daira Hopwood
4723af6e2f LAFS now stands for "Least-Authority File Store" rather than "Least-Authority File System". refs #2345
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2017-06-06 11:20:49 +01:00
meejah
05f48c3601 Various cleanups, fixes and improvements
Squashed all commits that were meejah's between
30d68fb499f300a393fa0ced5980229f4bb6efda
and
33c268ed3a8c63a809f4403e307ecc13d848b1ab
On the branch meejah:1382.markberger-rewrite-rebase.6 as
per review
2017-06-05 16:31:41 -06:00
David Stainton
19c5bbb43b Fix test test_lost_servers
Remove old hypothesis tests

Fix allmydata.test.cli.test_cli.Errors.test_get

this was broken due to differing share placements
whereas we need to allow this.

Fix test_5_overdue_immutable

This change makes the test not depend on the value
of PYTHONHASHSEED.

Revert "Fix test_5_overdue_immutable"

This reverts commit 5f3696d9a53e7df8781a2c463c7112282397cd69.

fix test to actually hang the first 5 *servers*

sort keys for stable output

use file-context-managers

remove probably-unneeded assert (that fails sometimes)

another non-deterministic test?
2017-06-05 16:26:46 -06:00
meejah
56f6dbd363 distribute only to read/write peers
correctly calculate happiness

guard with except

fix tests, and happiness calculation

remove debug

fix placements to None

happiness calc shouldn't have to filter None

WIP fixing some tests etc
2017-06-05 16:26:46 -06:00
David Stainton
a611673934 Make a correction to a hypothesis test comment
Comment out all debug print statements

Add hypothesis tests for the old servers of happiness implementation

Attempt to speed up meejah's servers of happiness

WIP

Fix test_calc_happy

WIP
2017-06-05 16:26:46 -06:00
meejah
b6d9945b95 default answer for every share
refactor hypothesis to be 'pytest style' and add another one

get rid of 'shares->set(1 thing)' in generate_mappings return

Add a unittest hypothesis came up with

fix tests since we return peers, not sets-of-1-peer

add more debug

add a unit-test that's like test_problem_layout_ticket_1128

fix bug

add a note

fix utest

unit-test for bigger numbers

re-insert markberger code for testing

results of pairing with david
2017-06-05 16:26:46 -06:00
David Stainton
e68b331bb1 Add servers of happiness hypothesis testing
Fix happiness test var names

Remove unused imports

Get rid of trailing whitespace
2017-06-05 16:26:46 -06:00
meejah
ef17ef2c62 fix happiness calculation
unit-test for happiness calculation

unused function

put old servers_of_happiness() calculation back for now

test for calculate_happiness

remove some redundant functions
2017-06-05 16:26:46 -06:00
David Stainton
42011e775d Make correction to docstring for Tahoe2ServerSelector's _handle_existing_response
Add comments 10 and 8 from the servers of happiness spec

Fix bug in _filter_g3 for servers of happiness

Remove usage of HappinessUpload class

here we modifying the PeerSelector class.
we make sure to correctly calculate the happiness value
by ignoring keys who's value are None...

Remove HappinessUpload and tests

Replace helper servers_of_happiness

we replace it's previous implementation with a new
wrapper function that uses share_placement
2017-06-05 16:26:46 -06:00
meejah
adb9a98383 WIP: refactoring (squahshed all meejah's commits) 2017-06-05 16:26:46 -06:00
Mark Berger
17cff7a176 Implements 'Servers of Happiness' algorithm for upload
This is Mark Berger's original commits, from ticket #1382
2017-06-05 16:26:46 -06:00
Brian Warner
bf0149f7d7 Merge PR412 2017-04-10 11:59:12 -07:00
Brian Warner
f24681bfdc simplify rendering of path to tahoe.cfg 2017-04-10 11:56:33 -07:00
David Stainton
366fe039e8 magic-folder: change stopService to call MultiService.stopService 2017-04-10 11:36:58 -07:00
tpltnt
63bcc07e27 explicit path to tahoe.cfg in creation message 2017-04-07 16:59:17 +02:00
meejah
4f1dd95963 Add a 'tahoe status' command
This required also fixing the JSON output from
the "recent and active operations" page.
2017-04-04 12:48:41 -06:00
Brian Warner
2ceea6df0d Merge PR406: use @implementer instead of implements
This also updates several classes to be new-style (inheriting from 'object').
2017-03-23 19:25:56 -07:00
Brian Warner
0abcbe3339 update some comments where plain cut+paste was confusing 2017-03-23 19:19:00 -07:00
meejah
49ae2c5eb7 use @implementer instead of implements
Also, derive some more things from 'object'
2017-02-27 11:01:30 -07:00
meejah
031a88c25f simplejson -> json 2017-02-23 15:21:36 -07:00
Brian Warner
bb782b0331 Merge PR400: add "tahoe list-aliases --readonly-uri"
Closes tahoe-lafs/tahoe-lafs:400
2017-02-21 10:09:31 -08:00
meejah
093945db22 set Referrer-Policy in one place 2017-02-13 16:07:41 -07:00
meejah
a9521b89a6 --readonly-uri for 'tahoe list-aliases' 2017-01-19 15:40:09 -07:00
Brian Warner
be6e0a9c29 deps: we need foolscap-0.12.6 for the new I2P API 2017-01-17 20:13:05 -05:00
Brian Warner
6db7686a8f update tests to match 2017-01-17 20:12:55 -05:00
str4d
998af5c26e Pass I2P keyfile to foolscap
If no session management is performed, txi2p starts a process-wide session the
first time a connection (client or server) is opened; all subsequent connections
use that session and its configuration properties.

This commit ensures that the same properties are passed to both client and
server endpoints, so that the correct I2P Destination is started regardless of
whether the first connection made by Tahoe-LAFS is for a client or server.

Closes #2858.
2017-01-17 20:12:32 -05:00
Brian Warner
ee58637813 fix syntax (tcp=disabled, not =none), add test 2017-01-10 16:30:04 -08:00
David Stainton
3c22a3ce50 Set tcp = tor only if txtorcon is imported
for i2p this means tcp = none as long as txtorcon is not imported
2017-01-10 16:30:04 -08:00
Brian Warner
64e344b03a introducer.yaml: ascii-fy furls before use
This fixes the foolscap-rejects-unicode-furls problem which prevented a
correctly-formatted introducers.yaml from working.

closes ticket:2862
2017-01-08 22:59:51 -08:00
Brian Warner
582931b1c2 docs: recommend proper introducers.yaml syntax
add a test to exercise that this syntax is parseable, and another to point
out that the previously-recommended syntax was not

refs ticket:2862
2017-01-08 22:59:51 -08:00
Brian Warner
79ff7ae5cb move 'shutilwhich' dependency out to the [test] extra
This is only used by integration/conftest.py, so we don't need an
unconditional dependency on it.

closes ticket:2856
2016-12-25 14:15:52 -05:00
Brian Warner
a06cf2ea4c express pypiwin32 dep via a setup.py extra
Previously this looked at sys.platform to decide what dependencies to
include. The problem with that approach is that wheels built on a unix box
won't work on windows (and vice versa), when the (pure-python) Tahoe wheels
aren't supposed to be platform-dependent.

setup.py provides a syntax to express this properly, so wheels created on
either platform will include the pypiwin32 dependency in the metadata, but
marked as only being relevant when installing on a win32 platform.

closes ticket:2763
2016-12-24 18:00:13 -05:00
Stefan Lew
481a02aee7 made tests work with new attribute rel="noreferrer" 2016-12-23 23:20:52 -05:00
Stefan Lew
12844220ec use rel="noreferrer" to prevent leakage of dircap 2016-12-23 23:20:52 -05:00