ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-20 05:28:04 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
8,839 Commits 185 Branches 90 Tags 89 MiB
4dda91df0e
Commit Graph

509 Commits

Author SHA1 Message Date
Zooko O'Whielacronx
de5cf49eba setup: add licensing declaration for setuptools (noticed by the FSF compliance folks) 2010-03-09 10:44:15 -08:00
Zooko O'Whielacronx
34a50b857a docs: a few small edits to try to guide newcomers through the docs 
These edits were suggested by my watching over Jake Appelbaum's shoulder as he completely ignored/skipped/missed install.html and also as he decided that debian.txt wouldn't help him with basic installation. Then I threw in a few docs edits that have been sitting around in my sandbox asking to be committed for months.
 2010-03-03 15:19:02 -08:00
Brian Warner
625ad78877 misc/coverage.el: improve filename matching 2010-02-23 23:47:57 -05:00
Peter Secor
8f3246430b docs running.html - "tahoe run ." does not work with the current installation, replaced with "tahoe start ." 2010-02-06 08:53:20 -08:00
Brian Warner
880f824103 code coverage: replace figleaf with coverage.py, should work on py2.6 now. 
It still lacks the right HTML report (the builtin report is very pretty, but
lacks the "lines uncovered" numbers that I want), and the half-finished
delta-from-last-run measurements.
 2010-02-03 08:54:21 -08:00
david-sarah
584c0ed487 Fix boodlegrid use of set_children 2010-01-25 22:34:14 -08:00
Zooko O'Whielacronx
7afccbdabc docs: CREDITS: add David-Sarah to the CREDITS file 2010-01-08 22:04:35 -08:00
Brian Warner
800fc176ec misc/ringsim.py: make it deterministic, more detail about grid-is-full behavior 2009-12-26 21:48:32 -05:00
Brian Warner
aa102da804 misc/ringsim.py: tool to discuss #302 2009-12-26 01:03:39 -05:00
Zooko O'Whielacronx
00a44c057a docs: a few licensing clarifications requested by Ubuntu 2009-09-26 20:32:26 -07:00
Zooko O'Whielacronx
aaaa633f18 setup: remove a convenience copy of figleaf, to ease inclusion into Ubuntu Karmic Koala 
We need to carefully document the licence of figleaf in order to get Tahoe-LAFS into Ubuntu Karmic Koala.  However, figleaf isn't really a part of Tahoe-LAFS per se -- this is just a "convenience copy" of a development tool.  The quickest way to make Tahoe-LAFS acceptable for Karmic then, is to remove figleaf from the Tahoe-LAFS tarball itself.  People who want to run figleaf on Tahoe-LAFS (as everyone should want) can install figleaf themselves.  I haven't tested this -- there may be incompatibilities between upstream figleaf and the copy that we had here...
 2009-09-23 22:32:15 -07:00
Zooko O'Whielacronx
abdf8a6f10 setup: shebang for misc/build-deb.py to fail quickly 
Without this patch, when I ran "chmod +x ./misc/build-deb.py && ./misc/build-deb.py" then it hung indefinitely.  (I wonder what it was doing.)
 2009-08-19 06:56:26 -07:00
Zooko O'Whielacronx
603e08c4a9 doc: licensing cleanups 
Use nice utf-8 © instead of "(c)". Remove licensing statements on utility modules that have been assigned to allmydata.com by their original authors. (Nattraverso was not assigned to allmydata.com -- it was LGPL'ed -- but I checked and src/allmydata/util/iputil.py was completely rewritten and doesn't contain any line of code from nattraverso.)  Add notes to misc/debian/copyright about licensing on files that aren't just allmydata.com-licensed.
 2009-09-20 10:16:31 -07:00
Brian Warner
9dd27bc7ee build-deb.py: run darcsver early, otherwise we get the wrong version later on 2009-09-17 20:36:20 -07:00
Brian Warner
7dc17aa078 new approach for debian packaging, sharing pieces across distributions. Still experimental, still only works for sid. 2009-08-18 12:05:27 -07:00
Brian Warner
24b1805ca0 new experimental deb-packaging rules. Only works for sid so far. 2009-08-17 18:40:52 -07:00
Zooko O'Whielacronx
69f0d306e7 setup: remove bundled version of darcsver-1.2.1 
That version of darcsver emits a scary error message when the darcs executable or the _darcs subdirectory is not found.
This error is hidden (unless the --loud option is passed) in darcsver >= 1.3.1.
Fixes #788.
 2009-08-16 16:34:32 -07:00
Zooko O'Whielacronx
2a63fc9159 docs: update relnotes.txt, relnotes-short.txt, and others documentation bits for v1.5.0 release! 2009-08-01 19:57:10 -07:00
Brian Warner
ba904dfeb5 debian/control: update binary package deps to pycryptopp-0.5.15 2009-07-20 23:50:25 -04:00
Zooko O'Whielacronx
6cbaaf31b5 mac: emit the stdout and stderr from a failed call to Allmydata (this is very useful diagnostic info) 2009-07-14 18:43:38 -07:00
Zooko O'Whielacronx
d2132213e3 mac: improve shutdown and clean-up after the test of the macintosh packaging 2009-07-14 05:26:55 -07:00
Zooko O'Whielacronx
4a4a47a66d setup: add "test_mac_diskimage" command which attempts to mount and use a .dmg to make sure it has a good Tahoe distribution in it 2009-07-12 16:09:40 -07:00
Zooko O'Whielacronx
7682f6b120 setup: add sys.maxunicode to the show-tool-versions output in order to investigate http://bugs.python.org/setuptools/issue78 2009-07-08 17:45:33 -07:00
Brian Warner
d79b2a02b4 clean up debian packaging: we have control files for etch/lenny/sid, and 
everything else uses one of those. Add dependency on python-pysqlite2 for
platforms that use py2.4 by default. Update foolscap dependency to 0.4.1.
 2009-07-03 00:28:04 -07:00
Brian Warner
5626e17725 * find-trailing-spaces.py: exit rc=1 if whitespace found, to be a pre-commit hook 2009-06-29 15:46:58 -07:00
Zooko O'Whielacronx
18876466f5 setup: copy in misc/show-tools-version.py from zfec -- it prints out platform and setuptools versions 2009-06-20 22:58:46 -07:00
Zooko O'Whielacronx
2a51a7bb90 setup: run the same "make quicktest" on Windows as on non-Windows 
I checked and it behaves about as well on Windows as the previous version did.
 2009-06-11 12:32:14 -07:00
Brian Warner
8e25066753 misc/run-with-pythonpath.py: exec() the child (on unix), to remove the intermediate process 2009-06-01 14:01:37 -07:00
Brian Warner
a0e3c586b2 fix quicktest: stop using setuptools, add misc/run-with-pythonpath.py, to make it run faster 2009-04-14 13:14:00 -07:00
Zooko O'Whielacronx
55de7c3fed setup: specify in the debian/control files that tahoe is compatible with Python 2.6 2009-03-11 15:59:02 -07:00
Zooko O'Whielacronx
cc9adba8a3 setup: update the debian/copyright text to reflect the current licences 2009-03-11 08:29:52 -07:00
Brian Warner
247c23904c misc/*: remove RuntimeError too 2009-02-22 17:34:01 -07:00
Brian Warner
c2a2e93089 move show-tool-versions out of setup.py and into a separate script in misc/ , since setuptools is trying to build and install a bunch of stuff first 2009-02-19 01:35:58 -07:00
Zooko O'Whielacronx
5a21cecd67 setup: add new darcsver-1.2.1 which includes Brian's and Nils Durner's patch to ignore non-ascii chars in XML output 2009-02-11 15:52:53 -07:00
Zooko O'Whielacronx
912b4ebf13 setup: removed bundled darcsver-1.2.0 
(I'm about to add a new bundled darcsver-1.2.1, but I want to see what the buildbots will do when there is no bundled darcsver present.)
 2009-02-11 14:19:04 -07:00
Zooko O'Whielacronx
4d785cfe30 setup: bundle setuptools-0.6c12dev (our own toothpick of setuptools) this version completes my patch to fix http://bugs.python.org/setuptools/issue54 , which is necessary for tahoe to build with --prefix=support without doing a lot of PYTHONPATH gymnastics around the call to setup.py 2009-02-05 09:28:18 -07:00
Zooko O'Whielacronx
c900ed8e69 setup: remove old bundled setuptools-0.6c11dev (our own toothpick of setuptools) 2009-02-05 09:27:58 -07:00
Zooko O'Whielacronx
40179a0ed8 setup: merge recent patches that change the set of bundled tools in misc/dependencies/ 2009-02-04 13:32:22 -07:00
Zooko O'Whielacronx
7d22cd4547 setup: bundle new setuptools_trial 2009-02-04 13:24:54 -07:00
Zooko O'Whielacronx
4df4efbaf0 setup: remove old bundled setuptools_trial 2009-02-04 13:24:42 -07:00
Zooko O'Whielacronx
944aa25709 setup: bundle new setuptools_darcs 2009-02-04 13:24:29 -07:00
Zooko O'Whielacronx
a2c1a93822 setup: remove old bundled setuptools_darcs 2009-02-04 13:24:18 -07:00
Zooko O'Whielacronx
759698224a roll back the upgrade of the bundled setuptools_trial from 0.5 to 0.5.1 
Upgrading setuptools trial in that way leads to a very tricky problem in setuptools's handling of recursive installation of the build-time dependencies of build-time dependencies...

rolling back:

Tue Feb  3 22:17:18 MST 2009  zooko@zooko.com
  * setup: bundle new setuptools_trial-0.5.1

    A ./misc/dependencies/setuptools_trial-0.5.1.tar
Tue Feb  3 22:17:32 MST 2009  zooko@zooko.com
  * setup: remove old bundled setuptools_trial

    R ./misc/dependencies/setuptools_trial-0.5.tar
 2009-02-04 01:38:36 -07:00
Zooko O'Whielacronx
05030112e8 setup: remove old bundled setuptools_trial 2009-02-03 23:17:32 -07:00
Zooko O'Whielacronx
313080fbf0 setup: bundle new setuptools_trial-0.5.1 2009-02-03 23:17:18 -07:00
Zooko O'Whielacronx
4af8c8cd24 setup: bundle new setuptools_darcs-1.2.5 2009-02-03 23:17:04 -07:00
Zooko O'Whielacronx
4988d1e81c setup: remove old bundled setuptools_darcs 2009-02-03 23:16:40 -07:00
Zooko O'Whielacronx
6e10a9c499 setup: remove bundled darcsver in gzipped format 2009-02-03 23:16:28 -07:00
Zooko O'Whielacronx
c613660500 setup: bundle darcsver in ungzipped format 2009-02-03 23:16:17 -07:00
Zooko O'Whielacronx
0c91a467e8 setup: remove old bundled setuptools-0.6c10dev (our own toothpick of setuptools) 2009-02-02 22:23:04 -07:00
Zooko O'Whielacronx
5c0d937eec setup: bundle setuptools-0.6c11dev (our own toothpick of setuptools) 
Hopefully this one fixes the issue with easy_install not searching the sys.path for packages that were requested to be installed, (setuptools #17), thus allowing us to do away with the "--site-dirs=" kludges, which are currently breaking some of our buildbots.
 2009-02-02 22:19:50 -07:00
Brian Warner
d8d4070e98 misc/dependencies: update setuptools_trial from 0.4 to 0.5 2009-02-02 17:36:57 -07:00
Zooko O'Whielacronx
6a4d5abc1d setup: add new bundled darcsver-1.2.0 2009-01-29 12:51:15 -07:00
Zooko O'Whielacronx
20d2136e24 setup: remove old bundled darcsver-1.1.8 2009-01-29 12:51:02 -07:00
Zooko O'Whielacronx
16f0fef785 setup: bundle darcsver-1.1.8 2009-01-28 14:03:26 -07:00
Zooko O'Whielacronx
4945dd0b0a setup: remove old bundled dependency darcsver-1.1.5 2009-01-28 12:20:12 -07:00
Brian Warner
8d6bdb5856 diskwatcher: cache growth results (which take 30s to generate) for 3 minutes, to help munin, which invokes it 6 times in a row every 5 minutes 2009-01-25 17:06:39 -07:00
Zooko O'Whielacronx
0dbc0f1194 setup: add new bundled setuptools_trial-0.4 2009-01-20 17:40:12 -07:00
Zooko O'Whielacronx
8351d9d41e setup: remove old bundled setuptools_trial-0.2 2009-01-20 17:39:18 -07:00
Zooko O'Whielacronx
4cd02cc12f setup: bundle setuptools_trial in misc/dependencies/ 2008-12-01 11:44:38 -07:00
Brian Warner
6ec352fe92 misc/cpu-watcher.tac: tolerate disk-full errors when writing the pickle, and pickle corruption from earlier disk-full errors 2008-12-05 15:54:12 -07:00
Brian Warner
995a16e3b1 misc/spacetime/diskwatcher.tac: remove dead code 2008-11-19 14:05:52 -07:00
Brian Warner
d224d22cb7 munin/tahoe_disktotal: add a 'disk used' line, since it will always be less than disktotal 2008-11-18 15:44:31 -07:00
Brian Warner
1e024d5e82 munin/tahoe_introstats: add line for distinct-storage-hosts (which counts machines instead of nodes) 2008-11-18 15:32:38 -07:00
Brian Warner
7d68e4397b munin/tahoe_diskused: new plugin to show total disk space used across the grid 2008-11-18 01:25:25 -07:00
Brian Warner
3d03e5257c munin/tahoe_disktotal: new plugin to show total disk space (used and unused) in the grid 2008-11-18 00:51:01 -07:00
Brian Warner
0a819e2912 diskwatcher: record total-space (the size of the disk as reported by df) in the db, report it to HTTP clients. This will involve a 50-item-per-second upgrade process when it is first used on old data 2008-11-17 21:45:16 -07:00
Zooko O'Whielacronx
eae5403e5a shebang: replace "/usr/bin/python" with "/usr/bin/env python" 2008-11-04 18:03:06 -07:00
Zooko O'Whielacronx
24650490ed misc/fixshebangs.py 2008-11-04 18:01:30 -07:00
Zooko O'Whielacronx
9f299ad55b setup: include _pkgutil.py in setuptools bootstrap egg so that it will work on Python 2.4 2008-10-25 08:28:39 -07:00
Zooko O'Whielacronx
8f260826b4 setup: remove old bundled setuptools-0.6c9 2008-10-25 06:49:47 -07:00
Zooko O'Whielacronx
3c974763ab setup: bundle an SVN snapshot of setuptools instead of the most recent stable release of setuptools 
This SVN snapshot fixes a problem that prevents the setting up of nevow:
http://bugs.python.org/setuptools/issue20
 2008-10-25 06:48:37 -07:00
Zooko O'Whielacronx
cd015228d5 setup: bundle setuptools-0.6c9 2008-09-30 13:04:48 -07:00
Zooko O'Whielacronx
8cad310dc8 setup: remove bundled setuptools-0.6c8 2008-09-30 13:03:36 -07:00
Brian Warner
f3d730428f misc/incident-gatherer: add classify_tahoe.py: a foolscap incident-gatherer classification plugin 2008-10-15 15:09:40 -07:00
Brian Warner
06770f720e misc/cpu-watcher.tac: use writeaside-and-rename for the history.pickle file 2008-09-30 17:30:53 -07:00
Brian Warner
3aba70bbcf misc/spacetime: use async polling so we can add a 60-second timeout, add an index to the 'url' Axiom column for 2x speedup 2008-09-30 16:34:48 -07:00
Brian Warner
1a3f154c21 munin/tahoe_doomsday: oops, tolerate 'null' in the timeleft results, to unbreak the 2wk/4wk graphs 2008-09-30 13:20:51 -07:00
Brian Warner
1f9af40fd4 misc/make-canary-files.py: tool to create 'canary files', explained in the docstring 2008-09-24 17:47:16 -07:00
Brian Warner
d4b522bedf misc/make_umid: change docs, make elisp code easier to grab 2008-09-20 11:39:33 -07:00
Brian Warner
cf27f81a88 misc/check-build.py: ignore the 'Downloading file:..' line that occurs for the setup_requires= -triggered handling of the setuptools egg 2008-09-16 22:17:25 -07:00
Brian Warner
1853020a5f #249: add 'test-desert-island', to assert that a tahoe-deps.tar.gz -enabled build does not download anything 2008-09-16 18:37:02 -07:00
Brian Warner
fafdf58e41 #249: move dependent libs out of misc/dependencies/, get them from tahoe-deps.tar.gz instead 2008-09-16 18:25:45 -07:00
Brian Warner
7da70b8197 misc/xfer-client.py: small foolscap utility to transfer a file to a waiting server 2008-09-08 16:19:03 -07:00
Brian Warner
be11bfe812 munin: add tahoe_diskleft plugin, update spacetime/diskwatcher.tac to support it 2008-08-28 13:32:36 -07:00
Brian Warner
d47d19cbfb debian: include misc/cpu-watcher.tac in the debian package 2008-08-27 15:30:26 -07:00
Brian Warner
482b3e0939 munin/tahoe_doomsday: change the graph title, 'time predictor' is more accurate than 'space predictor' 2008-08-27 14:30:13 -07:00
Brian Warner
ba6e525952 munin/tahoe_diskusage: clip the graph at zero, to prevent transient negative excursions (such as when a lot of old logfiles are deleted from a storage server's disk) from scaling the graph into unusability 2008-08-27 12:35:43 -07:00
Brian Warner
96a1ec33b8 misc/make_umid: little script and elisp fragment to insert umid= arguments 2008-08-25 18:59:18 -07:00
Brian Warner
743ca91bcd munin diskusage/doomsday: oops, fix labels, everything was reported in the 1hr column 2008-08-11 13:34:31 -07:00
Brian Warner
8828fe8769 munin/tahoe_overhead: don't emit non-sensicial numbers 2008-08-07 14:40:08 -07:00
Brian Warner
f7ad0d2f6f munin: add tahoe_overhead plugin, to measure effectiveness of GC and deleting data from inactive accounts 2008-08-07 13:39:25 -07:00
Brian Warner
39db60cc2b diskwatcher.tac: include total-bytes-used 2008-08-07 13:12:14 -07:00
Brian Warner
1c36a52db5 misc/dependencies: remove the no-longer-useful foolscap-0.2.5 tarball 2008-08-07 11:45:46 -07:00
Brian Warner
f77af25ef7 diskwatcher.tac: don't report negative timeleft 2008-08-07 10:34:33 -07:00
Brian Warner
7502971df6 diskwatcher.tac: reduce the polling rate to once per hour 2008-08-06 23:20:21 -07:00
Brian Warner
e19564df59 misc/spacetime: add munin plugins, add everything to .deb 2008-08-06 23:00:03 -07:00
Brian Warner
7fbfbcbe4d diskwatcher.tac: hush pyflakes 2008-08-06 22:04:27 -07:00
Brian Warner
730a28aa0d diskwatcher.tac: add async-GET code, but leave it commented out: urlopen() seems to work better for now 2008-08-06 22:03:27 -07:00
Brian Warner
c90bc3ad71 cpu-watcher.tac: improve error message 2008-08-06 21:38:01 -07:00
Brian Warner
46d0c9c995 disk-watcher: first draft of a daemon to use the HTTP stats interface and its new storage_server.disk_avail feature, to track changes in disk space over time 2008-08-06 21:22:22 -07:00
Brian Warner
4863b17371 misc/cpu-watcher.tac: tolerate missing pidfiles, just skip over that sample 2008-08-06 21:17:05 -07:00
Brian Warner
027408b9d2 debian/control: update dependencies to match _auto_deps: foolscap-0.3.0, pycryptopp-0.5 2008-08-05 18:32:22 -07:00
Brian Warner
315a89f9d8 munin/tahoe_spacetime: show 2wk data even if 4wk data is unavailable 2008-07-28 12:42:33 -07:00
Zooko O'Whielacronx
497aedfab9 simulate_load: put it in friendnet mode and print out a sum of doubled shares 2008-07-21 14:11:06 -07:00
Brian Warner
870fb68620 munin/tahoe_server_latency: add percentile to graph title 2008-07-16 16:08:17 -07:00
Brian Warner
af0648c0f8 munin/serverops: improve graph title 2008-07-16 12:36:42 -07:00
Brian Warner
5b3868ae2e debian/rules: put munin plugins in /usr/share/PACKAGENAME/munin/, make them +x, remove packagename from rules to make branches easier to manage 2008-07-15 18:47:41 -07:00
Brian Warner
43b54c1abb sid/control: set python versions to 2.4,2.5 , to match feisty/control, since sid has had 2.5 available forever now 2008-07-15 18:42:38 -07:00
Brian Warner
340b7add4f debian/changelog: remove all versions but 0.0.1, so debchange can always override it. The 'deb-X' Makefile targets should not be used; run 'make deb-X-head' instead 2008-07-15 15:23:41 -07:00
Brian Warner
390c5c6ca5 munin/tahoe_stats: add server operations per second (allocate/get/writev/readv 2008-07-14 13:25:35 -07:00
Brian Warner
0f231e2e66 munin/tahoe_stats: rearrange config a bit, no functional changes 2008-07-14 13:25:01 -07:00
Brian Warner
6525742b37 munin/tahoe_server_operations: fix comments 2008-07-14 13:18:47 -07:00
Brian Warner
a72dfcc00d munin: add plugin for storage-server operations per second 2008-07-14 13:18:11 -07:00
Brian Warner
d14d6d0985 copy debian/sid changes to debian/feisty 2008-07-14 12:56:38 -07:00
Brian Warner
7eaead0fcc debian/sid: add more docs to package, including munin plugins 2008-07-14 12:55:34 -07:00
Brian Warner
b2f062ab82 misc/munin: add server_latency plugin 2008-07-14 12:29:19 -07:00
Zooko O'Whielacronx
0e77fe1e92 misc/simulate_load.py: fix bug in which only K shares were uploaded instead of N, and fix the x axis in place so that successive runs can be compared 2008-07-12 16:54:29 -07:00
Zooko O'Whielacronx
f969c5f80e misc/simulate_load.py: use a more realistic distribution of file sizes, and automate the formatting of the ascii art so that it fits in 70 columns and the number of files uploaded is properly marked 2008-07-12 15:37:27 -07:00
Zooko O'Whielacronx
b483c4710d add misc/simulate_load.py in an attempt to understand why permuting the peerlist per each storage index matters 2008-07-12 14:26:22 -07:00
Brian Warner
f303670265 misc/munin: rename plugins to make munin happy: use underscores rather than hyphens, remove .py suffix 2008-07-11 15:21:13 -07:00
Brian Warner
503ce1cbd7 remove obsolete tahoe-sharesperfile munin plugin 2008-07-11 15:17:33 -07:00
Brian Warner
38252b6df1 remove tahoe-dirnodes.py munin plugin, it is way obsolete 2008-07-11 15:14:28 -07:00
Zooko O'Whielacronx
9bd1cdcdd8 setup: remove bundled darcsver-1.1.2.tar 2008-06-21 14:04:26 -07:00
Zooko O'Whielacronx
c151c13b5e setup: bundle darcsver-1.1.5 into misc/dependencies 
Once we implement #249 (move bundled dependencies out of revision control history and make them optional) then we can stop doing this.  Until then we try not to do it unless we really need to...
 2008-06-21 13:56:24 -07:00
Zooko O'Whielacronx
52a5651da2 setup: remove Nevow 0.9.18 tarball from misc/dependencies 
The 0.9.18 version of Nevow doesn't declare its dependency on Twisted in a machine-readable way ( http://divmod.org/trac/ticket/2629 ).  Neither does the current release of Nevow (0.9.31), but hopefully a future release of Nevow in the near future will fix this.
Also, we're going to be managing external dependencies like this in a separate darcs repository in the future instead of checking them into our Tahoe source tree.
 2008-06-09 11:41:56 -07:00
Zooko O'Whielacronx
b06ba64214 setup: quote the results of misc/find_trial.py so that they can be passed through a shell even if they contain spaces 2008-06-05 15:59:45 -07:00
Brian Warner
c23f675095 misc/munin/tahoe_spacetime.py: add a munin plugin to estimate how much time remains before you run out of space 2008-05-28 11:33:54 -07:00
Brian Warner
15b373ad9c misc/delete-old-helper.sh: simple script to delete old+abandoned helper files 2008-05-27 21:17:20 -07:00
Brian Warner
74df0b58b5 munin: add tahoe_cpu_watcher.py, to track the data from misc/cpu-watcher.tac 2008-05-07 13:19:08 -07:00
Brian Warner
79147a3f6a misc/cpu-watcher*: add some tools to monitor CPU usage of arbitrary processes, like tahoe nodes 2008-05-07 12:34:29 -07:00
Zooko O'Whielacronx
b1583ac6a0 setup: remove bundled pycryptopp-0.3.0.tar 2008-05-06 08:24:46 -07:00
Zooko O'Whielacronx
045e2bcdf4 setup: bundle pycryptopp-0.5.1.tar 
pycryptopp-0.3.0 incorrectly crypts AES (depending on compiler and version of Crypto++ used)
Very soon now we're going to set up an "ext" repository to hold all tarballs and no longer check them into our trunk source tree.
 2008-05-06 08:23:36 -07:00
Brian Warner
773e9abe24 debian: add python-setuptools to the debian install-time dependencies. Should close #382. 2008-04-24 16:01:04 -07:00
Brian Warner
d2a2b8a4ae munin/tahoe_estimate_files.py: tool to estimate the total number of slots (mutable and immutable combined) in the grid, from a small sample 2008-04-24 11:28:35 -07:00
robk-tahoe
5f5ed6b08d munin stats: fix typo in mutable file stats 2008-04-23 15:50:29 -07:00
robk-tahoe
f74042c75e munin stats: add mutable/uploader traffic graphs 
this adds munin graphs to present data already published by nodes to
the stats_gatherer, namely mutable files published/retrieved, and 
immutable files uploaded, and the bytes thereof
 2008-04-23 14:39:41 -07:00
Brian Warner
239c950810 munin/tahoe-stats.py: category is 'counters' not 'stats', use DERIVE 2008-04-23 14:03:23 -07:00
Zooko O'Whielacronx
1e2698713b setup: remove the misc/hatch-eggs.py script, which we no longer use since [1937] 2008-04-23 13:41:56 -07:00
Zooko O'Whielacronx
ece902a4cf setup: windows: make sub-ver.py treat None as 0 2008-04-21 06:37:13 -07:00
Brian Warner
5afc26f1d1 misc/sizes.py: update to match current defaults: 128KiB segsize, 3-of-10 2008-04-16 16:29:53 -07:00
Brian Warner
77c3c616ab misc/boodlegrid.tac: tweak sounds a bit 2008-04-15 11:01:49 -07:00
Brian Warner
500934b72f helper stats: fix the /helper_status page, the recent conflict merging missed some uses. Added tests, updated the munin plugins to match 2008-04-14 13:18:53 -07:00
robk-tahoe
766deaa9b6 stats_gatherer: reconcile helper stats gathering 
I'd implemented stats gathering hooks in the helper a while back.
Brian did the same without reference to my changes.  This reconciles
those two changes, encompassing all the stats in both changes,
implemented through the stats_provider interface.

this also provide templates for all 10 helper graphs in the 
tahoe-stats munin plugin.
 2008-04-10 17:25:44 -07:00
robk-tahoe
d8f3521a02 munin: added a series of munin graphs to report upload helper state 2008-03-25 18:30:46 -07:00
Brian Warner
623d82a130 debian: use our own /usr/bin/tahoe, remove runtime dependency on setuptools (since it required egg-aware versions of all dependencies too) 2008-04-10 16:29:59 -07:00
Brian Warner
5cfccbf44f debian: we now require setuptools at build time, and that or pkg_resources at runtime 2008-04-10 15:43:56 -07:00
Brian Warner
3f61198042 debian: use setuptools-generated support/bin/tahoe instead of bin/tahoe, to match Zooko's change that makes our in-tree bin/tahoe spawn support/bin/tahoe 2008-04-10 14:36:27 -07:00
Brian Warner
d36d731752 oops, change debian dependency on zfec to 1.1, not 1.1.0 2008-04-09 17:45:52 -07:00
Brian Warner
5385ec8fc9 make debian dependencies match _auto_deps.py ones, for foolscap and zfec 2008-04-09 10:23:01 -07:00
Brian Warner
ec77a5dc92 update our misc/dependencies to have foolscap-0.2.5 2008-04-03 16:08:22 -07:00
Brian Warner
fb51a60993 munin plugins: add 'graph_category tahoe' 2008-03-28 16:18:30 -07:00
Brian Warner
5da9f42cf8 add munin/tahoe-rootdir-space 2008-03-28 16:18:09 -07:00
Brian Warner
da6ab9f522 munin/tahoe-introstats.py: put this graph in the 'tahoe' category 2008-03-28 14:51:37 -07:00
Brian Warner
6b416fc28c helper: add another munin plugin 2008-03-27 16:50:30 -07:00
Brian Warner
2c96a32633 helper: add more stats to webapi, at /helper_status 2008-03-27 16:46:08 -07:00
Brian Warner
1ad847d4b2 boodlegrid.tac: update sound samples 2008-03-26 18:22:07 -07:00
Brian Warner
ccce37bb4a misc/boodlegrid.tac: tool to monitor a grid through its flogports 2008-03-26 16:09:34 -07:00
Zooko O'Whielacronx
5aa7986497 setup: remove bundled setuptools-0.6c7 2008-03-26 12:13:21 -07:00
Zooko O'Whielacronx
3e4b3f49b7 setup: bundle setuptools-0.6c8, we need a bugfix in it 2008-03-26 12:12:34 -07:00
Zooko O'Whielacronx
39272a1b6d setup: simplify makefile's path manipulation now that we rely on setup.py develop 2008-03-26 10:00:33 -07:00
zandr
f4d51ea366 tahoe-nodememory.py: change category to Tahoe 2008-03-25 18:42:11 -07:00
zandr
5245140a1c Copy amd-nodememory munin plugin over to tahoe and point at correct pidfile 2008-03-25 17:50:04 -07:00
Brian Warner
490f815e8f add a munin plugin to display introducer stats 2008-03-25 13:15:52 -07:00
Zooko O'Whielacronx
ec3f140824 setup: use "-rNN" with NN the "revision number" instead of "-NN" with NN the "build number" in the Windows builds 2008-03-10 20:15:04 -07:00
Zooko O'Whielacronx
d39d1661bd setup: remove old bundled pyutil-1.3.12 2008-03-10 19:59:30 -07:00
Zooko O'Whielacronx
45886cbb95 setup: remove old bundled darcsver-1.1.1 2008-03-10 19:59:20 -07:00
Zooko O'Whielacronx
7cc756eb45 setup: bundle pyutil-1.3.16.tar 2008-03-10 19:58:18 -07:00
Zooko O'Whielacronx
7bfb3ac047 setup: bundle darcsver-1.1.2.tar 2008-03-10 19:56:47 -07:00
Zooko O'Whielacronx
5108a73c6f setup: remove old bundled darcsver-1.0.1.tar 2008-03-10 19:23:34 -07:00
Zooko O'Whielacronx
ff129d648d setup: bundle darcsver-1.1.1.tar 
I forgot that we're doing uncompressed misc/deps at the moment.  We don't think this is a long-term good approach...
 2008-03-10 19:21:16 -07:00
Brian Warner
41045fba0f 'tahoe catalog-shares': add SDMF filesize to the output, update misc/find-share-anomalies.py to match 2008-02-13 15:12:06 -07:00
Zooko O'Whielacronx
98f570c3ee setup: remove older bundled pycryptopp-0.2.10 2008-02-13 15:23:15 -07:00
Zooko O'Whielacronx
fa92672e24 setup: bundle pycryptopp-0.3.0 2008-02-13 14:57:15 -07:00
Zooko O'Whielacronx
2e84230a18 setup: remove older bundled version of setuptools_darcs 2008-02-13 08:03:40 -07:00
Zooko O'Whielacronx
70b573434b setup: remove another older bundled setuptools 2008-02-13 08:01:40 -07:00
Zooko O'Whielacronx
99f609d55b setup: remove older bundled setuptools 2008-02-13 07:59:24 -07:00
Zooko O'Whielacronx
9eb2c7f5aa setup: remove older bundled version of darcsver 2008-02-13 07:57:22 -07:00
Brian Warner
95f27bb8fb misc/find-share-anomalies.py: tool to analyze 'tahoe catalog-shares' output and look for problems 2008-02-11 19:53:36 -07:00
Brian Warner
8ae6795e91 bundle foolscap-0.2.4 instead of 0.2.3 2008-02-05 15:27:41 -07:00
Brian Warner
7c7375a5ad bump foolscap dependency to 0.2.4, since we now use log.err 2008-02-05 15:27:14 -07:00
robk-tahoe
3641b7924b munin stats: don't suppress series with no current data 
having changed tahoe-stats to not report data series if there was no recent
data recorded for a node, I wound up making it hide the data series.  this
change causes it to report all data series for which stats exist in the
'config' phase, so that they show up, but only report actual data if the
stats are recent, so that they show up as missing if the node is not
reporting stats currently
 2008-02-01 18:44:57 -07:00
robk-tahoe
b80cfeb186 munin stats: suppress output of data more that 5min old 
if a node fails to report stats, the natural thing to do in re munin is to 
supress the data for that data series.  the previous tahoe-stats would output
whatever data was present in the stats_gatherer's stats.pickle, regardless of
how old.

this change means that if the gatherer hasn't received data within the last
5 min, then no data is reported to munin for that node.
 2008-01-31 21:04:23 -07:00
robk-tahoe
e910a64e7d munin stats: change truncated nodeid to 4 char (not 8) 2008-01-31 19:21:17 -07:00
Brian Warner
bf25a041f3 tarballs: stick with setup.py's sdist command, which gives us bz2/gz/zip but not rz/7z/lrz. Closes #298. 2008-01-31 13:52:48 -07:00
Brian Warner
4a363f0281 add 'tarballs' target, to generate compressed source tarballs 2008-01-30 20:45:14 -07:00
robk-tahoe
7b9f3207d0 stats: add a simple stats gathering system 
We have a desire to collect runtime statistics from multiple nodes primarily
for server monitoring purposes.   This implements a simple implementation of
such a system, as a skeleton to build more sophistication upon.

Each client now looks for a 'stats_gatherer.furl' config file.  If it has
been configured to use a stats gatherer, then it instantiates internally
a StatsProvider.  This is a central place for code which wishes to offer
stats up for monitoring to report them to, either by calling 
stats_provider.count('stat.name', value) to increment a counter, or by
registering a class as a stats producer with sp.register_producer(obj).

The StatsProvider connects to the StatsGatherer server and provides its
provider upon startup.  The StatsGatherer is then responsible for polling
the attached providers periodically to retrieve the data provided.
The provider queries each registered producer when the gatherer queries
the provider.  Both the internal 'counters' and the queried 'stats' are
then reported to the gatherer.

This provides a simple gatherer app, (c.f. make stats-gatherer-run)
which prints its furl and listens for incoming connections.  Once a
minute, the gatherer polls all connected providers, and writes the
retrieved data into a pickle file.

Also included is a munin plugin which knows how to read the gatherer's
stats.pickle and output data munin can interpret.  this plugin, 
tahoe-stats.py can be symlinked as multiple different names within
munin's 'plugins' directory, and inspects argv to determine which
data to display, doing a lookup in a table within that file.
It looks in the environment for 'statsfile' to determine the path to
the gatherer's stats.pickle.  An example plugins-conf.d file is
provided.
 2008-01-30 20:11:07 -07:00
Zooko O'Whielacronx
284c1652a9 setup: make find_trial self-contained so that we don't have a bootstrapping problem -- if allmydata can't be imported we still want to be able to run find_trial 2008-01-23 10:04:26 -07:00
Zooko O'Whielacronx
faad785859 setup: include cli.exe in the bootstrap setptools egg so that it will work on Windows (also include gui.exe just in case) 2008-01-22 12:33:55 -07:00
Zooko O'Whielacronx
b1152860cd setup: bundle setuptools_darcs-1.1.7 
fixes #263
 2008-01-22 11:01:36 -07:00
Zooko O'Whielacronx
451ca7792e setup: add a setuptools bootstrap egg that works on all versions of Python 
For versions of Python >= 2.3.
 2008-01-22 11:00:12 -07:00
Zooko O'Whielacronx
1f2e3fc912 setup: simplify the setup by removing the "tahoe dependencies" fake project 
Now we use "./setup.py develop" to ensure that changes to our source code are immediately used without requiring a "make" step.  This simplification will hopefully pave the way for easier py2exe and py2app, solving the "Unit tests test the installed version" bug (#145), and perhaps also #164 and #176.

This patch also conditionalizes the use of setuptools_darcs on the absence of a PKG-INFO file, which is part of fixing #263.
 2008-01-22 08:35:38 -07:00
robk-tahoe
3cb08209d2 windows installer build refinements 
this resolves problems of py2exe's modulefinder collection of sources from
.zipped egg files, not by using easy_install to reach the --always-unzip
option, but rather with a small tool which unpacks any zipped egg files found
in misc/dependencies.  this fixes the py2exe build given rollback of the 
easy_install stuff which had broken the unix builds.  misc/hatch-eggs.py
performs the honours.

this also includes a misc/sub-ver.py tool which substitutes elements of the
verion number for the current code base (by importing allmydata.__version__
hence make-version should be run first, and the python path carefully managed)
into template files using python's string interpolation of named args from a
dict as the templating syntax.  i.e. %(major)d %(minor)d %(point)d %(nano)d
each expand to the individual components of the version number as codified
by the pyutil.version_class.Version class.  there is also a %(build)s tag
which expands to the string form of the whole version number.  This tool is
used to interpolate the automatically generated version information into the
innosetup source file in a form consistent with innosetup/windows' restrictions
 2008-01-14 17:53:54 -07:00
Zooko O'Whielacronx
88c93469ff setup: add darcsver-1.0.1.tar to misc/dependencies/ 2008-01-16 14:08:26 -07:00
Brian Warner
16e2b3307a build-deps-setup.py: import twisted early, to make setuptools on dapper use the right version 2008-01-10 20:15:02 -07:00
robk-tahoe
2585908296 tweaks to build process to support py2exe 
py2exe is unable to handle .eggs which are packaged as zip files
in preference it will pull in other versions of libraries if they 
can be found in the environment.

this changes causes .eggs to be built as .egg directories, which
py2exe can handle.
 2008-01-09 19:02:53 -07:00
Zooko O'Whielacronx
80adf18fcc setup: remove hard import of ez_setup -- we can proceed even if ez_setup can't be imported 2008-01-10 14:01:52 -07:00
Zooko O'Whielacronx
72aa8f5cb8 setup: shebang usr bin env python 2008-01-10 14:01:31 -07:00
Zooko O'Whielacronx
1ab03fe9d2 setup: update in-line doc of get-version.py 2008-01-08 10:51:01 -07:00
Brian Warner
813706a01e debian: update dependencies to match calc-deps.py, mainly pycryptopp-0.2.8 2008-01-02 15:14:34 -07:00
Zooko O'Whielacronx
b412847485 setup: whoops, fix the use of source-tree-root-dir eggs in our Makefile 2008-01-01 01:53:31 -07:00
Zooko O'Whielacronx
97f2953cec setup: fix bin/tahoe to include .egg's from the source tree root dir as well 
This is necessary, as we can't prevent setuptools from respecting any such eggs, therefore we need to respect them in order to maintain consistency.  However, we don't normally install any "install_requires" eggs into the source tree root dir.
 2008-01-01 01:51:28 -07:00
Zooko O'Whielacronx
9b38eebcaf setup: finish renaming of docs/testnet to docs/testgrid 2007-12-31 23:36:59 -07:00
Zooko O'Whielacronx
58bedc60a2 setup: add darcsver source tarball to misc/dependencies 
In the future, we're likely to stop keeping misc/dependencies under revision control, as per

http://allmydata.org/trac/tahoe/ticket/249
 2007-12-31 23:26:51 -07:00
Brian Warner
bd187dfc50 upgrade to foolscap-0.2.3 2007-12-24 17:23:27 -07:00
Zooko O'Whielacronx
bfc69178b2 setup: fix bug in bugfix to patch to include .egg's found in CWD 2007-12-22 11:19:52 -07:00
Zooko O'Whielacronx
2ee424b3a3 setup: fix bug in previous patch to include .egg's from CWD 2007-12-22 11:14:27 -07:00
Zooko O'Whielacronx
7fdcbddbbc setup: we also need to include .egg's in the CWD in our search path, because if we install a 3rd party library into support/, and *it* installs a library that *it* requires, that one will appear in CWD 
It would be nice to figure out a way to force them to all appear in support/ where they belong.
 2007-12-22 11:04:24 -07:00
Zooko O'Whielacronx
27a0d59308 setup: zope.interface bundle .tar instead of .tar.gz because multiple bundled .tar's compress together much more nicely than multiple bundled .tar.gz's do. :-) 2007-12-22 10:50:54 -07:00
Zooko O'Whielacronx
48b13dcab8 setup: upgrade bundle zfec-1.1.tar.gz to zfec-1.3.4.tar 2007-12-22 10:50:39 -07:00
Zooko O'Whielacronx
a9545aa4b0 setup: simplejson bundle .tar instead of .tar.gz because multiple bundled .tar's compress together much more nicely than multiple bundled .tar.gz's do. :-) 2007-12-22 10:50:29 -07:00
Zooko O'Whielacronx
840fb23229 setup: upgrade bundle from setuptools_darcs-1.0.6.tar.gz to setuptools_darcs-1.1.5.tar 2007-12-22 10:50:03 -07:00
Zooko O'Whielacronx
d5bb1ac316 setup: bundle pyutil-1.3.12.tar 2007-12-22 10:49:47 -07:00
Zooko O'Whielacronx
336c482561 setup: pycryptopp: bundle .tar instead of .tar.gz because multiple bundled .tar's compress together much more nicely than multiple bundled .tar.gz's do. :-) 2007-12-22 10:49:35 -07:00
Zooko O'Whielacronx
2add4f5edb setup: foolscap bundle .tar instead of .tar.gz because multiple bundled .tar's compress together much more nicely than multiple bundled .tar.gz's do. :-) 2007-12-22 10:49:20 -07:00
Zooko O'Whielacronx
fd4880face setup: bundle argparse-0.8.0.tar 
It isn't actually needed for Tahoe, only for the command-line tools from pyutil.  Later I will make an "extras" category within pyutil to specify in a machine-readable way that argparse is not required for pyutil unless you want the command-line tools.
 2007-12-22 10:47:54 -07:00
Zooko O'Whielacronx
70b84e9298 setup: Nevow bundle .tar instead of .tar.gz because multiple bundled .tar's compress together much more nicely than multiple bundled .tar.gz's do. :-) 2007-12-22 10:47:06 -07:00
Zooko O'Whielacronx
c3a265839c setup: refactor ez_setup.py and setup.py to satisfy the Desert Island scenario, to find and use setuptools egg in-place in misc/dependencies, and make it setup_require pyutil (for darcsver) 2007-12-22 10:44:47 -07:00
Zooko O'Whielacronx
922234ec29 setup: bundle setuptools_darcs plugin in misc/dependencies 2007-12-12 19:20:42 -07:00
Zooko O'Whielacronx
cbd6881a47 setup: remove misc/make-version.py and invoke "darcsver" from the pyutil library 
misc/make-version.py has a limitation which prevents it from generating version
stamps from our current darcs history.  This limitation has been fixed in
pyutil's "darcsver".  Rather than copy the fix from there to
misc/make-version.py, I'm making it so that you have to install pyutil if you
want to automatically generate _version.py files from the current darcs
history.
 2007-12-20 18:17:55 -07:00
Brian Warner
e8c55fe85d move to foolscap-0.2.2 2007-12-12 20:21:45 -07:00
Zooko O'Whielacronx
1ac09840a4 a few documentation and naming convention updates 
Notable: the argument to make REPORTER has been renamed to TRIALARGS.
 2007-12-12 19:34:08 -07:00
Brian Warner
48e7a65882 upgrade to foolscap-0.2.1, with a new logging framework 2007-12-10 18:35:08 -07:00
Brian Warner
0618cf792d build-deps: require setuptools 0.6c4 or later, because older ones don't allow foolscap to use os.random at import time 2007-12-10 20:06:59 -07:00
Brian Warner
725621aded debian: add Depends: on python-pycryptopp, now that it's been packaged 2007-11-19 13:04:50 -07:00
Brian Warner
c86e803282 logtool: add 'gather' and 'dump' modes 2007-11-18 18:32:04 -07:00
Brian Warner
b502c3fea1 logtool: rename get-logs.py to logtool.py 2007-11-18 18:30:55 -07:00
Brian Warner
b29ce1c30a logpublisher: implement subscribe/publish for log, add a sample client 2007-11-16 20:07:50 -07:00
Zooko O'Whielacronx
59d6c3c822 decentralized directories: integration and testing 
* use new decentralized directories everywhere instead of old centralized directories
 * provide UI to them through the web server
 * provide UI to them through the CLI
 * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them
 * remove the notion of a "vdrive server" and a client thereof
 * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example)
 * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start.  The default is 1.
 * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!)
 * fix a few bugs where we used to equate "mutable" with "not read-only".  Nowadays all directories are mutable, but some might be read-only (to you).
 * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge
 * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.
 2007-12-03 14:52:42 -07:00
Zooko O'Whielacronx
211bf154d0 setup: pycryptopp: upgrade from v0.2.7 to v0.2.8 2007-11-19 15:55:17 -07:00
Zooko O'Whielacronx
cadc14fbca setup: pycryptopp: upgrade from v0.2.6 to v0.2.7 (adds AES) 2007-11-19 15:07:11 -07:00
Zooko O'Whielacronx
a82954eced setup: upgrade bundled pycryptopp from v0.2.5 to v0.2.6 2007-11-09 18:14:36 -07:00
Zooko O'Whielacronx
f0520b4e9e setup: upgrade the bundled pycryptopp tarball from pycryptopp v0.2.3 to pycryptopp v0.2.5 2007-11-09 13:02:49 -07:00
Zooko O'Whielacronx
494e07eced setup: upgrade to pycryptopp v0.2.3 2007-11-07 17:54:46 -07:00
Zooko O'Whielacronx
b699c28b67 setup: add misc/dependencies/pycryptopp-0.2.1.tar.gz 2007-11-07 09:59:51 -07:00
Brian Warner
1e8a693512 Makefile: add 'find-trailing-spaces' tool and target 2007-11-05 21:32:08 -07:00
Brian Warner
475a58fcc5 munin: add a sample file for /etc/munin/plugin-conf.d/ 2007-10-22 15:08:39 -07:00
Brian Warner
0aba63ff17 munin: oops, fix that trailing-newline fix, the previous one mangled the config 2007-10-18 12:49:22 -07:00
Brian Warner
b339c76560 munin: remove extra blank line from config output 2007-10-18 12:39:26 -07:00
Brian Warner
33a02e19f1 munin: update shareholder plugins to match current storage layout 2007-10-18 12:08:58 -07:00
Zooko O'Whielacronx
bf3f629a68 setup: use the new find_exe module to find trial 2007-10-15 11:52:26 -07:00
Zooko O'Whielacronx
aba57d9a4c setup: upgrade zfec from 1.0.4 to 1.1 2007-10-11 16:27:52 -07:00
Brian Warner
002472587e add public testnet .furls to docs/testnet/, and copy into .deb . Closes #157. 2007-10-11 14:55:23 -07:00
Brian Warner
1c7f52265f deb: add docs/* to the debian package 2007-10-11 14:37:29 -07:00
Brian Warner
dc573554c8 rename bin/allmydata-tahoe to bin/tahoe. Closes #155. 2007-10-11 03:38:24 -07:00
Zooko O'Whielacronx
be51a7e2a4 setup: bundle simplejson-1.7.3 in misc/dependencies 2007-10-04 11:23:20 -07:00
Brian Warner
7626a95636 upgrade to foolscap-0.1.7 2007-09-26 18:24:51 -07:00
Brian Warner
3b494ddf37 debian: oops, set debian/changelog version to 0.6.0-1+, to be less than everything the buildbot is creating 2007-09-26 18:15:13 -07:00
Brian Warner
7a4a31a83f debian: put a version string of '0.6.0+' in the debian changelog 2007-09-26 17:58:42 -07:00
Brian Warner
caa0a95405 debian/sid: copy rules from feisty, the same can be used for both 2007-09-26 17:55:24 -07:00