Commit Graph

7387 Commits

Author SHA1 Message Date
Brian Warner
d395a11208 travis: clean up the config file, use "language: python" on linux
The OS-X builders don't offer a python environment, so we have to build
things ourselves (installing tox/etc with --user, instead of into the
virtualenv that "language: python" gives us).
2018-03-28 19:26:26 -07:00
Brian Warner
479588d427 Merge branch '2913-travis-osx'
closes ticket:2913
2018-03-28 18:39:41 -07:00
Brian Warner
acc2b5744c tox: use newer (tox-2.4) settings, pre-install 'incremental'
* use 'extras' for our `[test]` additions instead of abusing 'deps'
* use 'deps' to pre-install 'incremental', which we couldn't do when we
  filled it with --editable to get `[test]`
* pre-install 'incremental' to work around a bug that strikes on Travis under
  OS-X-10.12 as PyPI gradually disables TLS<1.2. See ticket 2913 for details
* remove redundant configuration lines
* require tox-2.4 or newer, to get 'extras'

refs ticket:2913
https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2913
2018-03-28 17:42:01 -07:00
Brian Warner
de41ecee46 Merge branch '2912-unicode-error'
closes ticket:2912

Also adds a new travis builder (with LANG=C) to exercise the same unicode
problems that several of our buildbot workers see.
2018-03-28 17:40:29 -07:00
Brian Warner
8ae1b52070 travis: explicitly set LANG on all builds (to C or en_US.UTF-8) 2018-03-28 17:09:51 -07:00
Brian Warner
4cd8c699e2 travis: use LANG=C instead of unsetting it entirely
Thanks to exarkun for the suggestion. The failing buildbots have LANG unset,
but I'm pretty sure that defaults to LANG=C, and LANG=C triggers the failures
just as well as LANG= did.
2018-03-28 16:16:52 -07:00
Brian Warner
ce473bd5f4 cli.test_alias: move skip-unless utility to common_util.py
next to the other skip-unless function
2018-03-28 15:30:24 -07:00
Brian Warner
0616aa7de7 test_absolute_storage_dir: don't use uSNOWMAN on non-unicode platforms
Exercising the unicode possibilities are nice, but not critical to this test,
so let's just avoid the non-ascii characters when the filesystem encoding
can't handle them
2018-03-28 15:17:23 -07:00
Brian Warner
3bc099108a travis: add a LANG= linux build
to catch things like ticket:2912
(https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2912)

Also clean up other "list"-ish keys to avoid causing too many builds:

* "os": move this into "matrix"
* "python": 2.7 is the default, and we weren't running the pypy build
  anyways (not sure why, something else in this config must have disabled it,
  maybe when we moved away from language: python)
* "allow_failures" was causing a pypy build to happen even without listing it
  in "python"

We now have exactly three builds:

* linux
* linux with LANG=
* OS-X
2018-03-28 14:32:46 -07:00
Brian Warner
f155ade4ad misc/simulators/ringsim.py: fix LGTM nit
We haven't used this tool for a decade, but LGTM flagged a static-analysis
warning. Fixing it to see if the LGTM dashboard is looking at new builds.
2018-03-27 23:51:02 -07:00
Brian Warner
0964bc0d05 test_alias: skip unicode tests on non-unicode platform
This was flunking the OS-X buildbot, which runs in an environment without
$LANG being set, and thus encodingutil deduces (correctly but unhelpfully)
that we're limited to ASCII. Other tests detect this situation and raise
SkipTest, so let's do that here too.
2018-03-27 15:56:10 -07:00
Brian Warner
906c4f4f32 move tarball generation to tox.ini
and change the Makefile to delegate the "tarballs" target to tox

This should fix the ticket:2910 problem of the "tarballs" buildbot failing.
2018-03-27 14:34:32 -07:00
Brian Warner
526b97c753 tox: add 'skipsdist=True', hoping this will fix buildbot
There appears to be a bug in setuptools, triggered by running "python
setup.py sdist" with setuptools==11.3 in that python's environment, on a
project whose setup.py has a setup_requires= that requests setuptools >=
28.8.0. When setuptools is upgraded from inside setup.py, it gets into a
weird hybrid state where it's using setup() keyword-argument plugins from the
newer setuptools, but those plugins reference functions that aren't present
in the older setuptools, and the sdist command fails with an import
error (module object has no attribute 'check_specifier').

We don't actually need the sdist: all our tox test environments use
"skip_install = true", because we install tahoe via the "deps" line (so we
can get the `[test]` extra, and get a faster symlink-ish "editable" install).
That install uses "pip", which uses the pip inside the new virtualenv, which
either uses a newer version of setuptools (dependent upon what version of
"virtualenv" was installed in the parent environment, next to tox) or somehow
allows setuptools to be upgraded without exposing this weird broken hybrid
state.

Either way, skipping the sdist seems to fix this problem.

refs ticket:2910
https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2910
2018-03-27 14:08:17 -07:00
Jean-Paul Calderone
6ec5005ccd
Merge pull request #474 from exarkun/1587.basic-progress-report
Basic progress reporting for `tahoe backup`

Fixes ticket:1587
2018-03-27 07:47:01 -04:00
Jean-Paul Calderone
fa567958c3 2 blank between top-level; 1 blank between methods
Just like PEP8 says
2018-03-26 20:15:45 -04:00
Jean-Paul Calderone
bafe043b73 Explicit new-style class 2018-03-26 20:12:47 -04:00
Jean-Paul Calderone
b78c6cc5ed Implement the progress reporting 2018-03-26 11:34:31 -04:00
Jean-Paul Calderone
6690aa7337 restore, with tests, checked counters 2018-03-26 10:27:19 -04:00
Jean-Paul Calderone
c55d2823ae first pass refactoring
now collect backup work up-front instead of mixed with processing
2018-03-26 10:02:42 -04:00
Brian Warner
38da8f471c test_web: appease pyflakes 2018-03-21 00:22:31 -07:00
Brian Warner
e6ddd03338 test_web: remove noisy print statement 2018-03-21 00:13:52 -07:00
Brian Warner
57066b035b Merge branch '470-tox': upgrade setuptools to fix travis
This should also (hopefully) fix the LGTM baseline build, because they appear
to downgrade all dependencies to the lowest declared-acceptable version
before performing their static analysis, and the previous setuptools-11.3 was
too old to support some of the syntax used in zfec's setup.py (specifically
the python clause in `"argparse > 0.8 ; python <= '2.7'"`).

PR 470 updated the declared setuptools requirement, but it broke Travis on
OS-X because that platform had a really old setuptools-18.5, and apparently
upgrading from 18.5 to the current 39.0.1 during the tox process caused
internal consistency problems (probably mixing pieces of the two different
setuptools modules). This branch fixes it by telling travis to upgrade
setuptools before we run tox.

closes tahoe-lafs/tahoe-lafs#470
2018-03-20 23:14:16 -07:00
Brian Warner
40d8ad68c1 travis: add --upgrade, so we actually get a newer setuptools 2018-03-20 17:50:26 -07:00
meejah
544f87a318 need setuptools for PEP440 identifiers
(needs fixup, probably, just depending on latest setuptools)
2018-03-20 17:45:08 -07:00
Brian Warner
6f20dbc9a3 travis: install latest setuptools before running tox
The Travis OS-X worker has a very old setuptools-18.5 in /System. This is too
old to understand several important setup.py keys like `python_requires`, and
crashes when tryung to run the first invocation of tox (`tox -e codechecks`).
I think tox is using the system python (with which `tox` was invoked) to run
`setup.py egg_info` (to learn the dependencies), which gets the old
system-installed setuptools. Ideally it'd use the python from the
newly-created virtualenv, which would use whatever version of setuptools was
bundled with the `virtualenv` package (probably newer, given that
`virtualenv` itself should have been installed a moment earlier as a
dependency of `tox`.

I consider this a bug in Tox (https://github.com/tox-dev/tox/issues/507), but
the workaround is to configure Travis to install the most recent `setuptools`
along with `tox`.

refs tahoe-lafs/tahoe-lafs#470
2018-03-20 17:31:21 -07:00
Jean-Paul Calderone
3bb4c979c8 Revert "Merge PR470"
This reverts commit 7960a1b2a4, reversing
changes made to 3fa74d860f.
2018-03-20 18:10:44 -04:00
Brian Warner
7960a1b2a4 Merge PR470
closes #470
2018-03-20 14:15:43 -07:00
meejah
4eac3caa77 need setuptools for PEP440 identifiers
(needs fixup, probably, just depending on latest setuptools)
2018-03-20 14:15:33 -07:00
meejah
3fa74d860f Merge branch '2885.magic-folder-status.0' 2018-03-20 12:12:16 -06:00
meejah
643e2f9107 fix windows, maybe 2018-03-20 11:40:50 -06:00
meejah
fcfcbaa6a2 fix behavior of fake fileops test-helper 2018-03-20 11:40:49 -06:00
meejah
2fd2a9b2c7 nicer assert 2018-03-20 11:40:49 -06:00
meejah
9df84e71fe use better error-testing idiom and don't tweak internal method 2018-03-20 11:40:49 -06:00
meejah
57cc877c57 add some context to logging 2018-03-20 11:40:49 -06:00
meejah
ee0f548aa7 describe test 2018-03-20 11:40:49 -06:00
meejah
a06fe375ec describe test 2018-03-20 11:40:49 -06:00
meejah
c7cbb4a6b3 better var names 2018-03-20 11:40:49 -06:00
meejah
30811d8818 typo 2018-03-20 11:40:49 -06:00
meejah
9032539860 flake8; naming 2018-03-20 11:40:49 -06:00
meejah
797932244d log failure properly, add test 2018-03-20 11:40:49 -06:00
meejah
39080852cc ensure 'in progress' items return status 2018-03-20 11:40:49 -06:00
meejah
1f495ec41f cover uploads and downloads in tests 2018-03-20 11:40:49 -06:00
meejah
dbc9f7e06c remove code that sets item size (from review) 2018-03-20 11:40:49 -06:00
meejah
75e97e6c6d use QueuedItem directly instead of FakeStatusItem 2018-03-20 11:40:49 -06:00
meejah
1b16e1f792 Some documentation for QueueMixin 2018-03-20 11:40:49 -06:00
meejah
472d6ba78b cover some of get_status 2018-03-20 11:40:49 -06:00
meejah
bfabfeba64 add 'size' to tests, print unicode 2018-03-20 11:40:49 -06:00
meejah
0ebb587666 report download sizes in magic-folder status 2018-03-20 11:40:49 -06:00
meejah
0431be14fa log inotify callback errors 2018-03-20 11:40:49 -06:00
meejah
42766da553 add a 'size' to pending uploads 2018-03-20 11:40:49 -06:00