Commit Graph

171 Commits

Author SHA1 Message Date
Zooko O'Whielacronx
1e2698713b setup: remove the misc/hatch-eggs.py script, which we no longer use since [1937] 2008-04-23 13:41:56 -07:00
Zooko O'Whielacronx
ece902a4cf setup: windows: make sub-ver.py treat None as 0 2008-04-21 06:37:13 -07:00
Brian Warner
5afc26f1d1 misc/sizes.py: update to match current defaults: 128KiB segsize, 3-of-10 2008-04-16 16:29:53 -07:00
Brian Warner
77c3c616ab misc/boodlegrid.tac: tweak sounds a bit 2008-04-15 11:01:49 -07:00
Brian Warner
500934b72f helper stats: fix the /helper_status page, the recent conflict merging missed some uses. Added tests, updated the munin plugins to match 2008-04-14 13:18:53 -07:00
robk-tahoe
766deaa9b6 stats_gatherer: reconcile helper stats gathering
I'd implemented stats gathering hooks in the helper a while back.
Brian did the same without reference to my changes.  This reconciles
those two changes, encompassing all the stats in both changes,
implemented through the stats_provider interface.

this also provide templates for all 10 helper graphs in the 
tahoe-stats munin plugin.
2008-04-10 17:25:44 -07:00
robk-tahoe
d8f3521a02 munin: added a series of munin graphs to report upload helper state 2008-03-25 18:30:46 -07:00
Brian Warner
623d82a130 debian: use our own /usr/bin/tahoe, remove runtime dependency on setuptools (since it required egg-aware versions of all dependencies too) 2008-04-10 16:29:59 -07:00
Brian Warner
5cfccbf44f debian: we now require setuptools at build time, and that or pkg_resources at runtime 2008-04-10 15:43:56 -07:00
Brian Warner
3f61198042 debian: use setuptools-generated support/bin/tahoe instead of bin/tahoe, to match Zooko's change that makes our in-tree bin/tahoe spawn support/bin/tahoe 2008-04-10 14:36:27 -07:00
Brian Warner
d36d731752 oops, change debian dependency on zfec to 1.1, not 1.1.0 2008-04-09 17:45:52 -07:00
Brian Warner
5385ec8fc9 make debian dependencies match _auto_deps.py ones, for foolscap and zfec 2008-04-09 10:23:01 -07:00
Brian Warner
ec77a5dc92 update our misc/dependencies to have foolscap-0.2.5 2008-04-03 16:08:22 -07:00
Brian Warner
fb51a60993 munin plugins: add 'graph_category tahoe' 2008-03-28 16:18:30 -07:00
Brian Warner
5da9f42cf8 add munin/tahoe-rootdir-space 2008-03-28 16:18:09 -07:00
Brian Warner
da6ab9f522 munin/tahoe-introstats.py: put this graph in the 'tahoe' category 2008-03-28 14:51:37 -07:00
Brian Warner
6b416fc28c helper: add another munin plugin 2008-03-27 16:50:30 -07:00
Brian Warner
2c96a32633 helper: add more stats to webapi, at /helper_status 2008-03-27 16:46:08 -07:00
Brian Warner
1ad847d4b2 boodlegrid.tac: update sound samples 2008-03-26 18:22:07 -07:00
Brian Warner
ccce37bb4a misc/boodlegrid.tac: tool to monitor a grid through its flogports 2008-03-26 16:09:34 -07:00
Zooko O'Whielacronx
5aa7986497 setup: remove bundled setuptools-0.6c7 2008-03-26 12:13:21 -07:00
Zooko O'Whielacronx
3e4b3f49b7 setup: bundle setuptools-0.6c8, we need a bugfix in it 2008-03-26 12:12:34 -07:00
Zooko O'Whielacronx
39272a1b6d setup: simplify makefile's path manipulation now that we rely on setup.py develop 2008-03-26 10:00:33 -07:00
zandr
f4d51ea366 tahoe-nodememory.py: change category to Tahoe 2008-03-25 18:42:11 -07:00
zandr
5245140a1c Copy amd-nodememory munin plugin over to tahoe and point at correct pidfile 2008-03-25 17:50:04 -07:00
Brian Warner
490f815e8f add a munin plugin to display introducer stats 2008-03-25 13:15:52 -07:00
Zooko O'Whielacronx
ec3f140824 setup: use "-rNN" with NN the "revision number" instead of "-NN" with NN the "build number" in the Windows builds 2008-03-10 20:15:04 -07:00
Zooko O'Whielacronx
d39d1661bd setup: remove old bundled pyutil-1.3.12 2008-03-10 19:59:30 -07:00
Zooko O'Whielacronx
45886cbb95 setup: remove old bundled darcsver-1.1.1 2008-03-10 19:59:20 -07:00
Zooko O'Whielacronx
7cc756eb45 setup: bundle pyutil-1.3.16.tar 2008-03-10 19:58:18 -07:00
Zooko O'Whielacronx
7bfb3ac047 setup: bundle darcsver-1.1.2.tar 2008-03-10 19:56:47 -07:00
Zooko O'Whielacronx
5108a73c6f setup: remove old bundled darcsver-1.0.1.tar 2008-03-10 19:23:34 -07:00
Zooko O'Whielacronx
ff129d648d setup: bundle darcsver-1.1.1.tar
I forgot that we're doing uncompressed misc/deps at the moment.  We don't think this is a long-term good approach...
2008-03-10 19:21:16 -07:00
Brian Warner
41045fba0f 'tahoe catalog-shares': add SDMF filesize to the output, update misc/find-share-anomalies.py to match 2008-02-13 15:12:06 -07:00
Zooko O'Whielacronx
98f570c3ee setup: remove older bundled pycryptopp-0.2.10 2008-02-13 15:23:15 -07:00
Zooko O'Whielacronx
fa92672e24 setup: bundle pycryptopp-0.3.0 2008-02-13 14:57:15 -07:00
Zooko O'Whielacronx
2e84230a18 setup: remove older bundled version of setuptools_darcs 2008-02-13 08:03:40 -07:00
Zooko O'Whielacronx
70b573434b setup: remove another older bundled setuptools 2008-02-13 08:01:40 -07:00
Zooko O'Whielacronx
99f609d55b setup: remove older bundled setuptools 2008-02-13 07:59:24 -07:00
Zooko O'Whielacronx
9eb2c7f5aa setup: remove older bundled version of darcsver 2008-02-13 07:57:22 -07:00
Brian Warner
95f27bb8fb misc/find-share-anomalies.py: tool to analyze 'tahoe catalog-shares' output and look for problems 2008-02-11 19:53:36 -07:00
Brian Warner
8ae6795e91 bundle foolscap-0.2.4 instead of 0.2.3 2008-02-05 15:27:41 -07:00
Brian Warner
7c7375a5ad bump foolscap dependency to 0.2.4, since we now use log.err 2008-02-05 15:27:14 -07:00
robk-tahoe
3641b7924b munin stats: don't suppress series with no current data
having changed tahoe-stats to not report data series if there was no recent
data recorded for a node, I wound up making it hide the data series.  this
change causes it to report all data series for which stats exist in the
'config' phase, so that they show up, but only report actual data if the
stats are recent, so that they show up as missing if the node is not
reporting stats currently
2008-02-01 18:44:57 -07:00
robk-tahoe
b80cfeb186 munin stats: suppress output of data more that 5min old
if a node fails to report stats, the natural thing to do in re munin is to 
supress the data for that data series.  the previous tahoe-stats would output
whatever data was present in the stats_gatherer's stats.pickle, regardless of
how old.

this change means that if the gatherer hasn't received data within the last
5 min, then no data is reported to munin for that node.
2008-01-31 21:04:23 -07:00
robk-tahoe
e910a64e7d munin stats: change truncated nodeid to 4 char (not 8) 2008-01-31 19:21:17 -07:00
Brian Warner
bf25a041f3 tarballs: stick with setup.py's sdist command, which gives us bz2/gz/zip but not rz/7z/lrz. Closes #298. 2008-01-31 13:52:48 -07:00
Brian Warner
4a363f0281 add 'tarballs' target, to generate compressed source tarballs 2008-01-30 20:45:14 -07:00
robk-tahoe
7b9f3207d0 stats: add a simple stats gathering system
We have a desire to collect runtime statistics from multiple nodes primarily
for server monitoring purposes.   This implements a simple implementation of
such a system, as a skeleton to build more sophistication upon.

Each client now looks for a 'stats_gatherer.furl' config file.  If it has
been configured to use a stats gatherer, then it instantiates internally
a StatsProvider.  This is a central place for code which wishes to offer
stats up for monitoring to report them to, either by calling 
stats_provider.count('stat.name', value) to increment a counter, or by
registering a class as a stats producer with sp.register_producer(obj).

The StatsProvider connects to the StatsGatherer server and provides its
provider upon startup.  The StatsGatherer is then responsible for polling
the attached providers periodically to retrieve the data provided.
The provider queries each registered producer when the gatherer queries
the provider.  Both the internal 'counters' and the queried 'stats' are
then reported to the gatherer.

This provides a simple gatherer app, (c.f. make stats-gatherer-run)
which prints its furl and listens for incoming connections.  Once a
minute, the gatherer polls all connected providers, and writes the
retrieved data into a pickle file.

Also included is a munin plugin which knows how to read the gatherer's
stats.pickle and output data munin can interpret.  this plugin, 
tahoe-stats.py can be symlinked as multiple different names within
munin's 'plugins' directory, and inspects argv to determine which
data to display, doing a lookup in a table within that file.
It looks in the environment for 'statsfile' to determine the path to
the gatherer's stats.pickle.  An example plugins-conf.d file is
provided.
2008-01-30 20:11:07 -07:00
Zooko O'Whielacronx
284c1652a9 setup: make find_trial self-contained so that we don't have a bootstrapping problem -- if allmydata can't be imported we still want to be able to run find_trial 2008-01-23 10:04:26 -07:00