Commit Graph

5409 Commits

Author SHA1 Message Date
Brian Warner
d10f260f36 retrieve.py: unconditionally check share-hash-tree. Fixes #1654.
Add Kevan's unit test, update known_issues.rst
2012-01-12 13:35:53 -08:00
Brian Warner
2efcdc93fa .gitignore: also ignore tahoe-deps and .tgz, to fix 'make tarballs'
Otherwise, the get-version-from-git code thinks the tree is dirty, and
creates SUMO tarballs with -dirty in the name.
2012-01-12 13:09:25 -08:00
Brian Warner
f62c25f419 Makefile: fix 'make-version' to use git-or-darcs, not just darcs 2012-01-12 13:06:54 -08:00
david-sarah
76e7f0ad4b _auto_deps.py: don't allow pycrypto 2.0.1. fixes #1631 2012-01-10 19:57:58 +00:00
Brian Warner
4f87e17224 MANIFEST.in: make git-based 'setup.py sdist' match darcs
Previously, tarballs generated from a git tree were lacking a lot of
important non-code files, like docs/
2012-01-09 15:46:37 -08:00
Brian Warner
96ef2a7657 restore .gitignore, stop .darcs-boringfile it 2012-01-09 02:52:43 +00:00
Brian Warner
2cccc1a3df remove setuptools_darcs.egg 2012-01-08 22:55:45 +00:00
Brian Warner
9175c2451a fix bundled data under git, remove setuptools_darcs
This uses explicitly enumerated packages= and package_data= arguments to
setup(), rather than relying upon the convenient (but darcs-specific)
functions which would determine these values by asking the revision-control
system.

Note that darcsver is still used, when building from a darcs tree.
2012-01-08 14:12:50 -08:00
Brian Warner
893eea849b mutable/retrieve.py: clean up control flow to avoid dropping errors
* replace DeferredList with gatherResults, simplify result handling
* use BadShareError to signal recoverable problems in either fetch or
  validate, catch after _validate_block
* _validate_block is thus not responsible for noticing fetch problems
* rename _validation_or_decoding_failed() to _handle_bad_share()
* _get_needed_hashes() returns two Deferreds, instead of a hard-to-unpack
  DeferredList
2012-01-08 14:12:48 -08:00
Brian Warner
c56839478e mutable/layout.py: raise BadShareError instead of assert() 2012-01-08 14:12:47 -08:00
Brian Warner
ca78e6b146 mutable: don't tell server about corruption unless it's really CorruptShareError 2012-01-08 14:12:45 -08:00
Brian Warner
f1752f54c0 mutable: simplify Retrieve._process_segment() to use a gatherDeferred 2012-01-08 14:12:44 -08:00
Brian Warner
7f0bc64325 Retrieve.decode(): simplify setup of DeferredList-like argument
make it more obviously match the expectations of _decode_blocks() and
_maybe_decode_and_decrypt_segment()
2012-01-08 14:12:40 -08:00
Brian Warner
341b8c5715 mutable: add comments about the tricky DeferredList structures in retrieve 2012-01-08 14:12:38 -08:00
Brian Warner
d887782418 add test-git-ignore.py, to port the 'clean' buildbot test to git
add .gitignore to match .darcs-boringfile, mostly
2012-01-08 14:12:32 -08:00
david-sarah
7f8bbcc155 Use a private/drop_upload_dircap file instead of the [drop_upload]upload.dircap option in tahoe.cfg. Fail if the upload.dircap option is used, or options are missing. Also updates tests and docs. fixes #1593 2011-11-20 23:24:26 +00:00
Brian Warner
7989fe21cc test_mutable: don't use 75 shares (slow), now that the bug is fixed
I missed this part of Kevan's fix-1628.darcs.2.patch .
2011-12-28 14:38:19 -08:00
Kevan Carstensen
147670fd89 mutable publish: fix not-enough-shares detection. Refs #1628.
This should match the "fix-1628.darcs.2.patch" attachment on that ticket.
2011-12-27 21:50:18 -08:00
Kevan Carstensen
e29323f68f mutable publish: track multiple servers-per-share. Fixes some of #1628.
The remaining work is to write additional tests.

src/allmydata/test/no_network.py:

 This supports tests in which servers leave the grid only to return with
 their shares intact at a later time.

src/allmydata/test/test_mutable.py:

 The UCWEs in the incident reports associated with #1628 all seem to be
 associated with shares that the servermap knows about, but which aren't
 accounted for during the publish process for whatever reason. Specifically,
 it looks like the publisher is only capable of keeping track of a single
 storage server for a given share. This makes the repair process worse than
 it was pre-MDMF at updating all of the shares of a particular file to the
 newest version, and can also cause spurious UCWEs. This test simulates such
 a layout and fails if an UCWE is thrown. We need to write another test to
 ensure that all copies of a share are updated to the latest version (or
 alter this test to do that), so that the test suite doesn't pass unless both
 regressions are fixed.

 We want the publisher to follow the existing share placement when uploading
 a new version of a mutable file, and we don't want this test to pass unless
 it does.

src/allmydata/mutable/publish.py:

 Before this commit, the publisher only kept track of a single writer for
 each share. This is insufficient to handle updates in which a single share
 may live on multiple servers. In the best case, an update will only update
 one of the existing shares instead of all of them. In some cases, the update
 will encounter the existing shares when publishing some other share,
 interpret it as a sign of an uncoordinated update, and fail. Keeping track
 of all of the writers helps ensure that all existing shares are updated, and
 helps avoid spurious uncoordinated write errors.
2011-12-27 21:33:58 -08:00
Zooko O'Whielacronx
ab9859d6a5 docs: how_to_make_a_tahoe-lafs_release.rst add Google+ page to publicity list, change to cute unicode checkboxes 2011-12-26 15:19:05 +00:00
Zooko O'Whielacronx
4a29642623 doc: about.rst: use unicode emdash, use non-embedded URIs, add clarificaiton of when a file gets its mutable-or-immutable nature
embedded URIs, although documented here:
http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#embedded-uris
generate messages like this from rst2html --verbose:

quickstart.rst:3: (INFO/1) Duplicate explicit target name: "the tahoe-dev mailing list".

Also this patch prepends a "utf-8 BOM" to the beginning of the file.
2011-12-06 17:19:08 +00:00
Brian Warner
37c0f2395d minor cleanup: remove trailing spaces in misc/ 2011-12-18 12:18:41 -08:00
david-sarah
531f562fcf Tests for ref #1592. 2011-12-17 04:31:30 +00:00
david-sarah
4618571389 test_web.py cleanup: use failUnlessIn/failIfIn in preference to 'in' operator. 2011-12-17 04:27:10 +00:00
david-sarah
054374400e Marcus Wanner's favicon patch. fixes #1592 2011-12-17 03:32:01 +00:00
Brian Warner
dd2e7a1127 setup.py: stop putting pyutil.version_class/etc in _version.py
allmydata.__version__ can just be a string, it doesn't need to be an instance
of some fancy NormalizedVersion class. Everything inside Tahoe uses
str(__version__) anyways.

Also add .dev0 when a git tree is dirty.

Closes #1466
2011-12-04 21:50:49 -08:00
Brian Warner
7d5ca407a1 setup.py: get version from git or darcs
This replaces the setup.cfg aliases that run "darcsver" before each major
command with the new "update_version". update_version is defined in setup.py,
and tries to get a version string from either darcs or git (or leaves the
existing _version.py alone if neither VC metadata is available).

Also clean up a tiny typo in verlib.py that messed up syntax hilighting.
2011-12-04 20:40:01 -08:00
david-sarah
b73aba98de docs/known_issues.rst: describe when the unauthorized access attack is known to be possible, and fix a link. 2011-11-18 00:20:13 +00:00
Brian Warner
ce8d40f31b more tiny buildbot-testing whitespace changes 2011-11-18 00:20:41 +00:00
Brian Warner
d6925dcc01 more tiny buildbot-testing whitespace changes 2011-11-18 00:18:28 +00:00
Brian Warner
10c9170f51 tiny change to exercise the buildbot hook 2011-11-18 00:15:11 +00:00
david-sarah
891069c245 Strengthen description of unauthorized access attack in known_issues.rst. 2011-11-18 00:00:30 +00:00
Brian Warner
990f294eea remove remaining uses of nevow's "formless" module
We're slowly moving away from Nevow, and marcusw's previous patch removed
uses of the formless CSS file, so now we can stop testing that nevow can find
that file, and remove the lingering unused "import formless" call.
2011-11-17 14:54:23 -08:00
Marcus Wanner
0ed6260a3c 1585-webui.darcs.patch 2011-11-17 21:49:23 +00:00
Brian Warner
1445953adc Remove duplicate tahoe_css links from manifest.xhtml and rename-form.xhtml
They were probably meant to be links to webform_css, but we aren't really
using Nevow's form-generation code anyways, so they can just be removed.
Thanks to 'marcusw' for the catch.
2011-11-16 14:42:25 -08:00
Brian Warner
1b84612fdf iputil: handle openbsd5 (just like openbsd4)
Patch by 'sickness'. Closes #1584
2011-11-15 14:04:23 -08:00
Brian Warner
5a1463d587 Makefile count-lines: let it work on OS-X (-l not --lines), add XXX
OS-X's simple-minded /usr/bin/wc doesn't understand --lines, but everyone
understands -l .
2011-11-09 10:42:27 -08:00
Brian Warner
5fc9978826 setup.py: umask=022 for 'sdist', to avoid depending on environment
The new tarball-building buildslave had a bogus umask set, causing the 1.9.0
tarballs to be non-other-user-readable (go-rwx), which is a hassle for
packaging. (The umask was correct on the old buildslave, but it was moved to
a new host shortly before the release). This should make sure tarballs are
correct despite the host's setting.

Note to others: processes run under twistd get umask=077 unless you arrange
otherwise.
2011-11-09 10:36:32 -08:00
david-sarah
5af3bfda61 _auto_deps.py: blacklist PyCrypto 2.4. 2011-11-05 02:24:57 +00:00
david-sarah
4d6260d6ad check-miscaptures.py: report the number of files that were not analysed due to syntax errors (and don't count them in the number of suspicious captures). refs #1555 2011-10-09 05:03:01 +00:00
david-sarah
efd3c8b113 check-miscaptures.py: handle corner cases around default arguments correctly. Also make a minor optimization when there are no assigned variables to consider. refs #1555 2011-10-09 04:50:23 +00:00
david-sarah
5359c24e99 check-miscaptures.py: Python doesn't really have declarations; report the topmost assignment. refs #1555 2011-10-09 04:48:00 +00:00
david-sarah
1c6fe1d230 check-miscaptures.py: handle destructuring function arguments correctly. refs #1555 2011-10-09 04:47:10 +00:00
david-sarah
d2f3ef9714 check-miscaptures.py: check while loops and list comprehensions as well as for loops. Also fix a pyflakes warning. refs #1555 2011-10-09 04:40:22 +00:00
david-sarah
be1c94893f Add misc/coding_tools/check-miscaptures.py to detect incorrect captures of variables declared in a for loop, and a 'make check-miscaptures' Makefile target to run it. (It is also run by 'make code-checks'.) This is a rewritten version that reports much fewer false positives, by determining captured variables more accurately. fixes #1555 2011-10-07 07:41:21 +00:00
david-sarah
bac24aee0d Fix pyflakes warnings in misc/ directories other than misc/build_helpers. refs #1557 2011-10-07 03:30:31 +00:00
david-sarah
0d0bbbbe0a Makefile: include misc/ directories other than misc/build_helpers in SOURCES. refs #1557 2011-10-07 03:29:58 +00:00
Brian Warner
1bbdec2a20 show-tool-versions: tolerate missing setuptools 2011-11-01 01:00:10 -07:00
Brian Warner
4baffeed1d show-tool-versions.py: condense output, hide file-not-found exceptions 2011-11-01 00:45:32 -07:00
Brian Warner
01b1244e9b relnotes.txt: fix footnotes 2011-11-01 00:19:35 -07:00