This introduces a py.test-based integration suite (currently just
containing magic-folder end-to-end tests). Also adds a tox environment
("integration") to run them.
The test setup is:
- a "flogtool gather" instance
- an Introducer
- five Storage nodes
- Alice and Bob client nodes
- Alice and Bob have paired magic-folders
This executes: check-debugging, check-interfaces, check-miscaptures,
find-trailing-spaces, check-umids, pyflakes.
Other changes:
* fix check-umids.py to take starting points. run it as `check-umids.py
src` instead of `check-umids.py src/allmydata/*.py`
* check-debugging: rewrite in python to run from tox: tox doesn't like
to run shell scripts.
* put check-interfaces.py last: it produces lots of warnings, but passes
anyways. The others only produce significant output if they fail.
Run with "tox -e coverage". Uses a new helper
module (allmydata.test.run_trial) to let us import+execute trial without
knowing exactly where the "trial" binary lives, which helps with using
"coverage run" under tox.
We use "--deps = --editable=.[test]" to achieve three goals:
* make tahoe and it's dependencies available for tests
* use --editable, which is faster and allows "coverage run" to get the
source filenames right
* use the [test] extra, which includes "mock"
Tox's default install command does the first, but doesn't use
--editable, so when the "deps" stage comes around, there's already a
non-editable install in place. It seems to get the [test] extra right,
but it doesn't wind up with an editable install.
So we disable the default install command and rely on the "deps" clause
instead.
One of the buildslaves (Ubuntu wily 15.10) has a very old pip-1.5.6,
which doesn't know how to "pip install" a filepath+extra (like
".[test]") unless --editable is also used.
It's convenient to have --editable set anyways (so you can do subsequent
narrow testing without re-running tox, by running ".tox/py27/bin/trial
TESTCASE" or use .tox/py27/bin/activate), so changing the dependency
from ".[test]" to "--editable=.[test]" is the easiest way to work around
that older buildslave. (I could also have upgraded the buildslave to use
a newer pip, but 15.10 is pretty recent and other people will probably
hit this too, so this way it's fixed for everybody).
refs ticket:2776
Note that Twisted doesn't do anything like Versioneer, so this will
generally show e.g. "Twisted-16.1.0" for the entire interval between
16.1.0 and 16.2.0.
Rewrote in Twisted, which lets us read/scan/print all log lines in
realtime. The output is now correctly interleaved (as well as
maintaining the stdout-vs-stderr of each message). The renamed
--warnings= logfile records all relevant lines from *both* stdout and
stderr (i.e. any that includes "DeprecationWarning"), which handles a
change (perhaps in recent Twisteds?) that emits these warnings on stdout
instead of stderr.
This runs trial against the latest (git) Twisted and Foolscap, to give
us an early warning about anything we're currently doing that will
become deprecated in their next releases.
Tox doesn't run shell pipelines, and is unhappy about running tools that
it didn't install itself (including non-python things like "make"). So
this adds misc/build_helpers/run-deprecations.py, a python script that
runs a given command (i.e. trial) and writes stdout into a separate file
where the buildbot can see it, and counts the "DeprecationWarning" lines
from the file to decide the returncode.
This ought to improve the status display on the buildbot "test
deprecations" step.
Our setup.cfg ensures that 'setup.py update_version' happens early
enough. And doing it from tox's commands= is too late anyways (because
we aren't using Versioneer, so the _version.py file must be updated
before tox installs a static copy into the virtualenv).
The buildbot's main run-trial-with-tox command has been updated to scan
for the 'tahoe --version' output and include the version string in the
buildbot results.
So use 'tox -e check-memory' instead of 'make check-memory'. The tox
version will create a virtualenv and install tahoe for you before
running the tests, removing one use of the 'tahoe @FILENAME' hack (which
was used to run a python file with a PYTHONPATH set to import tahoe's
dependencies).
This adds support for two environments: 'py27' for basic tests, and
'deprecations' to run tests with deprecation warnings turned on (the
latter is meant to be run from a buildbot step that knows how to count
and format the warnings nicely).
