Zooko O'Whielacronx
4cd02cc12f
setup: bundle setuptools_trial in misc/dependencies/
2008-12-01 11:44:38 -07:00
Brian Warner
6ec352fe92
misc/cpu-watcher.tac: tolerate disk-full errors when writing the pickle, and pickle corruption from earlier disk-full errors
2008-12-05 15:54:12 -07:00
Brian Warner
995a16e3b1
misc/spacetime/diskwatcher.tac: remove dead code
2008-11-19 14:05:52 -07:00
Brian Warner
d224d22cb7
munin/tahoe_disktotal: add a 'disk used' line, since it will always be less than disktotal
2008-11-18 15:44:31 -07:00
Brian Warner
1e024d5e82
munin/tahoe_introstats: add line for distinct-storage-hosts (which counts machines instead of nodes)
2008-11-18 15:32:38 -07:00
Brian Warner
7d68e4397b
munin/tahoe_diskused: new plugin to show total disk space used across the grid
2008-11-18 01:25:25 -07:00
Brian Warner
3d03e5257c
munin/tahoe_disktotal: new plugin to show total disk space (used and unused) in the grid
2008-11-18 00:51:01 -07:00
Brian Warner
0a819e2912
diskwatcher: record total-space (the size of the disk as reported by df) in the db, report it to HTTP clients. This will involve a 50-item-per-second upgrade process when it is first used on old data
2008-11-17 21:45:16 -07:00
Zooko O'Whielacronx
eae5403e5a
shebang: replace "/usr/bin/python" with "/usr/bin/env python"
2008-11-04 18:03:06 -07:00
Zooko O'Whielacronx
24650490ed
misc/fixshebangs.py
2008-11-04 18:01:30 -07:00
Zooko O'Whielacronx
9f299ad55b
setup: include _pkgutil.py in setuptools bootstrap egg so that it will work on Python 2.4
2008-10-25 08:28:39 -07:00
Zooko O'Whielacronx
8f260826b4
setup: remove old bundled setuptools-0.6c9
2008-10-25 06:49:47 -07:00
Zooko O'Whielacronx
3c974763ab
setup: bundle an SVN snapshot of setuptools instead of the most recent stable release of setuptools
...
This SVN snapshot fixes a problem that prevents the setting up of nevow:
http://bugs.python.org/setuptools/issue20
2008-10-25 06:48:37 -07:00
Zooko O'Whielacronx
cd015228d5
setup: bundle setuptools-0.6c9
2008-09-30 13:04:48 -07:00
Zooko O'Whielacronx
8cad310dc8
setup: remove bundled setuptools-0.6c8
2008-09-30 13:03:36 -07:00
Brian Warner
f3d730428f
misc/incident-gatherer: add classify_tahoe.py: a foolscap incident-gatherer classification plugin
2008-10-15 15:09:40 -07:00
Brian Warner
06770f720e
misc/cpu-watcher.tac: use writeaside-and-rename for the history.pickle file
2008-09-30 17:30:53 -07:00
Brian Warner
3aba70bbcf
misc/spacetime: use async polling so we can add a 60-second timeout, add an index to the 'url' Axiom column for 2x speedup
2008-09-30 16:34:48 -07:00
Brian Warner
1a3f154c21
munin/tahoe_doomsday: oops, tolerate 'null' in the timeleft results, to unbreak the 2wk/4wk graphs
2008-09-30 13:20:51 -07:00
Brian Warner
1f9af40fd4
misc/make-canary-files.py: tool to create 'canary files', explained in the docstring
2008-09-24 17:47:16 -07:00
Brian Warner
d4b522bedf
misc/make_umid: change docs, make elisp code easier to grab
2008-09-20 11:39:33 -07:00
Brian Warner
cf27f81a88
misc/check-build.py: ignore the 'Downloading file:..' line that occurs for the setup_requires= -triggered handling of the setuptools egg
2008-09-16 22:17:25 -07:00
Brian Warner
1853020a5f
#249 : add 'test-desert-island', to assert that a tahoe-deps.tar.gz -enabled build does not download anything
2008-09-16 18:37:02 -07:00
Brian Warner
fafdf58e41
#249 : move dependent libs out of misc/dependencies/, get them from tahoe-deps.tar.gz instead
2008-09-16 18:25:45 -07:00
Brian Warner
7da70b8197
misc/xfer-client.py: small foolscap utility to transfer a file to a waiting server
2008-09-08 16:19:03 -07:00
Brian Warner
be11bfe812
munin: add tahoe_diskleft plugin, update spacetime/diskwatcher.tac to support it
2008-08-28 13:32:36 -07:00
Brian Warner
d47d19cbfb
debian: include misc/cpu-watcher.tac in the debian package
2008-08-27 15:30:26 -07:00
Brian Warner
482b3e0939
munin/tahoe_doomsday: change the graph title, 'time predictor' is more accurate than 'space predictor'
2008-08-27 14:30:13 -07:00
Brian Warner
ba6e525952
munin/tahoe_diskusage: clip the graph at zero, to prevent transient negative excursions (such as when a lot of old logfiles are deleted from a storage server's disk) from scaling the graph into unusability
2008-08-27 12:35:43 -07:00
Brian Warner
96a1ec33b8
misc/make_umid: little script and elisp fragment to insert umid= arguments
2008-08-25 18:59:18 -07:00
Brian Warner
743ca91bcd
munin diskusage/doomsday: oops, fix labels, everything was reported in the 1hr column
2008-08-11 13:34:31 -07:00
Brian Warner
8828fe8769
munin/tahoe_overhead: don't emit non-sensicial numbers
2008-08-07 14:40:08 -07:00
Brian Warner
f7ad0d2f6f
munin: add tahoe_overhead plugin, to measure effectiveness of GC and deleting data from inactive accounts
2008-08-07 13:39:25 -07:00
Brian Warner
39db60cc2b
diskwatcher.tac: include total-bytes-used
2008-08-07 13:12:14 -07:00
Brian Warner
1c36a52db5
misc/dependencies: remove the no-longer-useful foolscap-0.2.5 tarball
2008-08-07 11:45:46 -07:00
Brian Warner
f77af25ef7
diskwatcher.tac: don't report negative timeleft
2008-08-07 10:34:33 -07:00
Brian Warner
7502971df6
diskwatcher.tac: reduce the polling rate to once per hour
2008-08-06 23:20:21 -07:00
Brian Warner
e19564df59
misc/spacetime: add munin plugins, add everything to .deb
2008-08-06 23:00:03 -07:00
Brian Warner
7fbfbcbe4d
diskwatcher.tac: hush pyflakes
2008-08-06 22:04:27 -07:00
Brian Warner
730a28aa0d
diskwatcher.tac: add async-GET code, but leave it commented out: urlopen() seems to work better for now
2008-08-06 22:03:27 -07:00
Brian Warner
c90bc3ad71
cpu-watcher.tac: improve error message
2008-08-06 21:38:01 -07:00
Brian Warner
46d0c9c995
disk-watcher: first draft of a daemon to use the HTTP stats interface and its new storage_server.disk_avail feature, to track changes in disk space over time
2008-08-06 21:22:22 -07:00
Brian Warner
4863b17371
misc/cpu-watcher.tac: tolerate missing pidfiles, just skip over that sample
2008-08-06 21:17:05 -07:00
Brian Warner
027408b9d2
debian/control: update dependencies to match _auto_deps: foolscap-0.3.0, pycryptopp-0.5
2008-08-05 18:32:22 -07:00
Brian Warner
315a89f9d8
munin/tahoe_spacetime: show 2wk data even if 4wk data is unavailable
2008-07-28 12:42:33 -07:00
Zooko O'Whielacronx
497aedfab9
simulate_load: put it in friendnet mode and print out a sum of doubled shares
2008-07-21 14:11:06 -07:00
Brian Warner
870fb68620
munin/tahoe_server_latency: add percentile to graph title
2008-07-16 16:08:17 -07:00
Brian Warner
af0648c0f8
munin/serverops: improve graph title
2008-07-16 12:36:42 -07:00
Brian Warner
5b3868ae2e
debian/rules: put munin plugins in /usr/share/PACKAGENAME/munin/, make them +x, remove packagename from rules to make branches easier to manage
2008-07-15 18:47:41 -07:00
Brian Warner
43b54c1abb
sid/control: set python versions to 2.4,2.5 , to match feisty/control, since sid has had 2.5 available forever now
2008-07-15 18:42:38 -07:00
Brian Warner
340b7add4f
debian/changelog: remove all versions but 0.0.1, so debchange can always override it. The 'deb-X' Makefile targets should not be used; run 'make deb-X-head' instead
2008-07-15 15:23:41 -07:00
Brian Warner
390c5c6ca5
munin/tahoe_stats: add server operations per second (allocate/get/writev/readv
2008-07-14 13:25:35 -07:00
Brian Warner
0f231e2e66
munin/tahoe_stats: rearrange config a bit, no functional changes
2008-07-14 13:25:01 -07:00
Brian Warner
6525742b37
munin/tahoe_server_operations: fix comments
2008-07-14 13:18:47 -07:00
Brian Warner
a72dfcc00d
munin: add plugin for storage-server operations per second
2008-07-14 13:18:11 -07:00
Brian Warner
d14d6d0985
copy debian/sid changes to debian/feisty
2008-07-14 12:56:38 -07:00
Brian Warner
7eaead0fcc
debian/sid: add more docs to package, including munin plugins
2008-07-14 12:55:34 -07:00
Brian Warner
b2f062ab82
misc/munin: add server_latency plugin
2008-07-14 12:29:19 -07:00
Zooko O'Whielacronx
0e77fe1e92
misc/simulate_load.py: fix bug in which only K shares were uploaded instead of N, and fix the x axis in place so that successive runs can be compared
2008-07-12 16:54:29 -07:00
Zooko O'Whielacronx
f969c5f80e
misc/simulate_load.py: use a more realistic distribution of file sizes, and automate the formatting of the ascii art so that it fits in 70 columns and the number of files uploaded is properly marked
2008-07-12 15:37:27 -07:00
Zooko O'Whielacronx
b483c4710d
add misc/simulate_load.py in an attempt to understand why permuting the peerlist per each storage index matters
2008-07-12 14:26:22 -07:00
Brian Warner
f303670265
misc/munin: rename plugins to make munin happy: use underscores rather than hyphens, remove .py suffix
2008-07-11 15:21:13 -07:00
Brian Warner
503ce1cbd7
remove obsolete tahoe-sharesperfile munin plugin
2008-07-11 15:17:33 -07:00
Brian Warner
38252b6df1
remove tahoe-dirnodes.py munin plugin, it is way obsolete
2008-07-11 15:14:28 -07:00
Zooko O'Whielacronx
9bd1cdcdd8
setup: remove bundled darcsver-1.1.2.tar
2008-06-21 14:04:26 -07:00
Zooko O'Whielacronx
c151c13b5e
setup: bundle darcsver-1.1.5 into misc/dependencies
...
Once we implement #249 (move bundled dependencies out of revision control history and make them optional) then we can stop doing this. Until then we try not to do it unless we really need to...
2008-06-21 13:56:24 -07:00
Zooko O'Whielacronx
52a5651da2
setup: remove Nevow 0.9.18 tarball from misc/dependencies
...
The 0.9.18 version of Nevow doesn't declare its dependency on Twisted in a machine-readable way ( http://divmod.org/trac/ticket/2629 ). Neither does the current release of Nevow (0.9.31), but hopefully a future release of Nevow in the near future will fix this.
Also, we're going to be managing external dependencies like this in a separate darcs repository in the future instead of checking them into our Tahoe source tree.
2008-06-09 11:41:56 -07:00
Zooko O'Whielacronx
b06ba64214
setup: quote the results of misc/find_trial.py so that they can be passed through a shell even if they contain spaces
2008-06-05 15:59:45 -07:00
Brian Warner
c23f675095
misc/munin/tahoe_spacetime.py: add a munin plugin to estimate how much time remains before you run out of space
2008-05-28 11:33:54 -07:00
Brian Warner
15b373ad9c
misc/delete-old-helper.sh: simple script to delete old+abandoned helper files
2008-05-27 21:17:20 -07:00
Brian Warner
74df0b58b5
munin: add tahoe_cpu_watcher.py, to track the data from misc/cpu-watcher.tac
2008-05-07 13:19:08 -07:00
Brian Warner
79147a3f6a
misc/cpu-watcher*: add some tools to monitor CPU usage of arbitrary processes, like tahoe nodes
2008-05-07 12:34:29 -07:00
Zooko O'Whielacronx
b1583ac6a0
setup: remove bundled pycryptopp-0.3.0.tar
2008-05-06 08:24:46 -07:00
Zooko O'Whielacronx
045e2bcdf4
setup: bundle pycryptopp-0.5.1.tar
...
pycryptopp-0.3.0 incorrectly crypts AES (depending on compiler and version of Crypto++ used)
Very soon now we're going to set up an "ext" repository to hold all tarballs and no longer check them into our trunk source tree.
2008-05-06 08:23:36 -07:00
Brian Warner
773e9abe24
debian: add python-setuptools to the debian install-time dependencies. Should close #382 .
2008-04-24 16:01:04 -07:00
Brian Warner
d2a2b8a4ae
munin/tahoe_estimate_files.py: tool to estimate the total number of slots (mutable and immutable combined) in the grid, from a small sample
2008-04-24 11:28:35 -07:00
robk-tahoe
5f5ed6b08d
munin stats: fix typo in mutable file stats
2008-04-23 15:50:29 -07:00
robk-tahoe
f74042c75e
munin stats: add mutable/uploader traffic graphs
...
this adds munin graphs to present data already published by nodes to
the stats_gatherer, namely mutable files published/retrieved, and
immutable files uploaded, and the bytes thereof
2008-04-23 14:39:41 -07:00
Brian Warner
239c950810
munin/tahoe-stats.py: category is 'counters' not 'stats', use DERIVE
2008-04-23 14:03:23 -07:00
Zooko O'Whielacronx
1e2698713b
setup: remove the misc/hatch-eggs.py script, which we no longer use since [1937]
2008-04-23 13:41:56 -07:00
Zooko O'Whielacronx
ece902a4cf
setup: windows: make sub-ver.py treat None as 0
2008-04-21 06:37:13 -07:00
Brian Warner
5afc26f1d1
misc/sizes.py: update to match current defaults: 128KiB segsize, 3-of-10
2008-04-16 16:29:53 -07:00
Brian Warner
77c3c616ab
misc/boodlegrid.tac: tweak sounds a bit
2008-04-15 11:01:49 -07:00
Brian Warner
500934b72f
helper stats: fix the /helper_status page, the recent conflict merging missed some uses. Added tests, updated the munin plugins to match
2008-04-14 13:18:53 -07:00
robk-tahoe
766deaa9b6
stats_gatherer: reconcile helper stats gathering
...
I'd implemented stats gathering hooks in the helper a while back.
Brian did the same without reference to my changes. This reconciles
those two changes, encompassing all the stats in both changes,
implemented through the stats_provider interface.
this also provide templates for all 10 helper graphs in the
tahoe-stats munin plugin.
2008-04-10 17:25:44 -07:00
robk-tahoe
d8f3521a02
munin: added a series of munin graphs to report upload helper state
2008-03-25 18:30:46 -07:00
Brian Warner
623d82a130
debian: use our own /usr/bin/tahoe, remove runtime dependency on setuptools (since it required egg-aware versions of all dependencies too)
2008-04-10 16:29:59 -07:00
Brian Warner
5cfccbf44f
debian: we now require setuptools at build time, and that or pkg_resources at runtime
2008-04-10 15:43:56 -07:00
Brian Warner
3f61198042
debian: use setuptools-generated support/bin/tahoe instead of bin/tahoe, to match Zooko's change that makes our in-tree bin/tahoe spawn support/bin/tahoe
2008-04-10 14:36:27 -07:00
Brian Warner
d36d731752
oops, change debian dependency on zfec to 1.1, not 1.1.0
2008-04-09 17:45:52 -07:00
Brian Warner
5385ec8fc9
make debian dependencies match _auto_deps.py ones, for foolscap and zfec
2008-04-09 10:23:01 -07:00
Brian Warner
ec77a5dc92
update our misc/dependencies to have foolscap-0.2.5
2008-04-03 16:08:22 -07:00
Brian Warner
fb51a60993
munin plugins: add 'graph_category tahoe'
2008-03-28 16:18:30 -07:00
Brian Warner
5da9f42cf8
add munin/tahoe-rootdir-space
2008-03-28 16:18:09 -07:00
Brian Warner
da6ab9f522
munin/tahoe-introstats.py: put this graph in the 'tahoe' category
2008-03-28 14:51:37 -07:00
Brian Warner
6b416fc28c
helper: add another munin plugin
2008-03-27 16:50:30 -07:00
Brian Warner
2c96a32633
helper: add more stats to webapi, at /helper_status
2008-03-27 16:46:08 -07:00
Brian Warner
1ad847d4b2
boodlegrid.tac: update sound samples
2008-03-26 18:22:07 -07:00
Brian Warner
ccce37bb4a
misc/boodlegrid.tac: tool to monitor a grid through its flogports
2008-03-26 16:09:34 -07:00
Zooko O'Whielacronx
5aa7986497
setup: remove bundled setuptools-0.6c7
2008-03-26 12:13:21 -07:00
