Commit Graph

145 Commits

Author SHA1 Message Date
Zooko O'Whielacronx
ec3f140824 setup: use "-rNN" with NN the "revision number" instead of "-NN" with NN the "build number" in the Windows builds 2008-03-10 20:15:04 -07:00
Zooko O'Whielacronx
d39d1661bd setup: remove old bundled pyutil-1.3.12 2008-03-10 19:59:30 -07:00
Zooko O'Whielacronx
45886cbb95 setup: remove old bundled darcsver-1.1.1 2008-03-10 19:59:20 -07:00
Zooko O'Whielacronx
7cc756eb45 setup: bundle pyutil-1.3.16.tar 2008-03-10 19:58:18 -07:00
Zooko O'Whielacronx
7bfb3ac047 setup: bundle darcsver-1.1.2.tar 2008-03-10 19:56:47 -07:00
Zooko O'Whielacronx
5108a73c6f setup: remove old bundled darcsver-1.0.1.tar 2008-03-10 19:23:34 -07:00
Zooko O'Whielacronx
ff129d648d setup: bundle darcsver-1.1.1.tar
I forgot that we're doing uncompressed misc/deps at the moment.  We don't think this is a long-term good approach...
2008-03-10 19:21:16 -07:00
Brian Warner
41045fba0f 'tahoe catalog-shares': add SDMF filesize to the output, update misc/find-share-anomalies.py to match 2008-02-13 15:12:06 -07:00
Zooko O'Whielacronx
98f570c3ee setup: remove older bundled pycryptopp-0.2.10 2008-02-13 15:23:15 -07:00
Zooko O'Whielacronx
fa92672e24 setup: bundle pycryptopp-0.3.0 2008-02-13 14:57:15 -07:00
Zooko O'Whielacronx
2e84230a18 setup: remove older bundled version of setuptools_darcs 2008-02-13 08:03:40 -07:00
Zooko O'Whielacronx
70b573434b setup: remove another older bundled setuptools 2008-02-13 08:01:40 -07:00
Zooko O'Whielacronx
99f609d55b setup: remove older bundled setuptools 2008-02-13 07:59:24 -07:00
Zooko O'Whielacronx
9eb2c7f5aa setup: remove older bundled version of darcsver 2008-02-13 07:57:22 -07:00
Brian Warner
95f27bb8fb misc/find-share-anomalies.py: tool to analyze 'tahoe catalog-shares' output and look for problems 2008-02-11 19:53:36 -07:00
Brian Warner
8ae6795e91 bundle foolscap-0.2.4 instead of 0.2.3 2008-02-05 15:27:41 -07:00
Brian Warner
7c7375a5ad bump foolscap dependency to 0.2.4, since we now use log.err 2008-02-05 15:27:14 -07:00
robk-tahoe
3641b7924b munin stats: don't suppress series with no current data
having changed tahoe-stats to not report data series if there was no recent
data recorded for a node, I wound up making it hide the data series.  this
change causes it to report all data series for which stats exist in the
'config' phase, so that they show up, but only report actual data if the
stats are recent, so that they show up as missing if the node is not
reporting stats currently
2008-02-01 18:44:57 -07:00
robk-tahoe
b80cfeb186 munin stats: suppress output of data more that 5min old
if a node fails to report stats, the natural thing to do in re munin is to 
supress the data for that data series.  the previous tahoe-stats would output
whatever data was present in the stats_gatherer's stats.pickle, regardless of
how old.

this change means that if the gatherer hasn't received data within the last
5 min, then no data is reported to munin for that node.
2008-01-31 21:04:23 -07:00
robk-tahoe
e910a64e7d munin stats: change truncated nodeid to 4 char (not 8) 2008-01-31 19:21:17 -07:00
Brian Warner
bf25a041f3 tarballs: stick with setup.py's sdist command, which gives us bz2/gz/zip but not rz/7z/lrz. Closes . 2008-01-31 13:52:48 -07:00
Brian Warner
4a363f0281 add 'tarballs' target, to generate compressed source tarballs 2008-01-30 20:45:14 -07:00
robk-tahoe
7b9f3207d0 stats: add a simple stats gathering system
We have a desire to collect runtime statistics from multiple nodes primarily
for server monitoring purposes.   This implements a simple implementation of
such a system, as a skeleton to build more sophistication upon.

Each client now looks for a 'stats_gatherer.furl' config file.  If it has
been configured to use a stats gatherer, then it instantiates internally
a StatsProvider.  This is a central place for code which wishes to offer
stats up for monitoring to report them to, either by calling 
stats_provider.count('stat.name', value) to increment a counter, or by
registering a class as a stats producer with sp.register_producer(obj).

The StatsProvider connects to the StatsGatherer server and provides its
provider upon startup.  The StatsGatherer is then responsible for polling
the attached providers periodically to retrieve the data provided.
The provider queries each registered producer when the gatherer queries
the provider.  Both the internal 'counters' and the queried 'stats' are
then reported to the gatherer.

This provides a simple gatherer app, (c.f. make stats-gatherer-run)
which prints its furl and listens for incoming connections.  Once a
minute, the gatherer polls all connected providers, and writes the
retrieved data into a pickle file.

Also included is a munin plugin which knows how to read the gatherer's
stats.pickle and output data munin can interpret.  this plugin, 
tahoe-stats.py can be symlinked as multiple different names within
munin's 'plugins' directory, and inspects argv to determine which
data to display, doing a lookup in a table within that file.
It looks in the environment for 'statsfile' to determine the path to
the gatherer's stats.pickle.  An example plugins-conf.d file is
provided.
2008-01-30 20:11:07 -07:00
Zooko O'Whielacronx
284c1652a9 setup: make find_trial self-contained so that we don't have a bootstrapping problem -- if allmydata can't be imported we still want to be able to run find_trial 2008-01-23 10:04:26 -07:00
Zooko O'Whielacronx
faad785859 setup: include cli.exe in the bootstrap setptools egg so that it will work on Windows (also include gui.exe just in case) 2008-01-22 12:33:55 -07:00
Zooko O'Whielacronx
b1152860cd setup: bundle setuptools_darcs-1.1.7
fixes 
2008-01-22 11:01:36 -07:00
Zooko O'Whielacronx
451ca7792e setup: add a setuptools bootstrap egg that works on all versions of Python
For versions of Python >= 2.3.
2008-01-22 11:00:12 -07:00
Zooko O'Whielacronx
1f2e3fc912 setup: simplify the setup by removing the "tahoe dependencies" fake project
Now we use "./setup.py develop" to ensure that changes to our source code are immediately used without requiring a "make" step.  This simplification will hopefully pave the way for easier py2exe and py2app, solving the "Unit tests test the installed version" bug (), and perhaps also  and .

This patch also conditionalizes the use of setuptools_darcs on the absence of a PKG-INFO file, which is part of fixing .
2008-01-22 08:35:38 -07:00
robk-tahoe
3cb08209d2 windows installer build refinements
this resolves problems of py2exe's modulefinder collection of sources from
.zipped egg files, not by using easy_install to reach the --always-unzip
option, but rather with a small tool which unpacks any zipped egg files found
in misc/dependencies.  this fixes the py2exe build given rollback of the 
easy_install stuff which had broken the unix builds.  misc/hatch-eggs.py
performs the honours.

this also includes a misc/sub-ver.py tool which substitutes elements of the
verion number for the current code base (by importing allmydata.__version__
hence make-version should be run first, and the python path carefully managed)
into template files using python's string interpolation of named args from a
dict as the templating syntax.  i.e. %(major)d %(minor)d %(point)d %(nano)d
each expand to the individual components of the version number as codified
by the pyutil.version_class.Version class.  there is also a %(build)s tag
which expands to the string form of the whole version number.  This tool is
used to interpolate the automatically generated version information into the
innosetup source file in a form consistent with innosetup/windows' restrictions
2008-01-14 17:53:54 -07:00
Zooko O'Whielacronx
88c93469ff setup: add darcsver-1.0.1.tar to misc/dependencies/ 2008-01-16 14:08:26 -07:00
Brian Warner
16e2b3307a build-deps-setup.py: import twisted early, to make setuptools on dapper use the right version 2008-01-10 20:15:02 -07:00
robk-tahoe
2585908296 tweaks to build process to support py2exe
py2exe is unable to handle .eggs which are packaged as zip files
in preference it will pull in other versions of libraries if they 
can be found in the environment.

this changes causes .eggs to be built as .egg directories, which
py2exe can handle.
2008-01-09 19:02:53 -07:00
Zooko O'Whielacronx
80adf18fcc setup: remove hard import of ez_setup -- we can proceed even if ez_setup can't be imported 2008-01-10 14:01:52 -07:00
Zooko O'Whielacronx
72aa8f5cb8 setup: shebang usr bin env python 2008-01-10 14:01:31 -07:00
Zooko O'Whielacronx
1ab03fe9d2 setup: update in-line doc of get-version.py 2008-01-08 10:51:01 -07:00
Brian Warner
813706a01e debian: update dependencies to match calc-deps.py, mainly pycryptopp-0.2.8 2008-01-02 15:14:34 -07:00
Zooko O'Whielacronx
b412847485 setup: whoops, fix the use of source-tree-root-dir eggs in our Makefile 2008-01-01 01:53:31 -07:00
Zooko O'Whielacronx
97f2953cec setup: fix bin/tahoe to include .egg's from the source tree root dir as well
This is necessary, as we can't prevent setuptools from respecting any such eggs, therefore we need to respect them in order to maintain consistency.  However, we don't normally install any "install_requires" eggs into the source tree root dir.
2008-01-01 01:51:28 -07:00
Zooko O'Whielacronx
9b38eebcaf setup: finish renaming of docs/testnet to docs/testgrid 2007-12-31 23:36:59 -07:00
Zooko O'Whielacronx
58bedc60a2 setup: add darcsver source tarball to misc/dependencies
In the future, we're likely to stop keeping misc/dependencies under revision control, as per

http://allmydata.org/trac/tahoe/ticket/249
2007-12-31 23:26:51 -07:00
Brian Warner
bd187dfc50 upgrade to foolscap-0.2.3 2007-12-24 17:23:27 -07:00
Zooko O'Whielacronx
bfc69178b2 setup: fix bug in bugfix to patch to include .egg's found in CWD 2007-12-22 11:19:52 -07:00
Zooko O'Whielacronx
2ee424b3a3 setup: fix bug in previous patch to include .egg's from CWD 2007-12-22 11:14:27 -07:00
Zooko O'Whielacronx
7fdcbddbbc setup: we also need to include .egg's in the CWD in our search path, because if we install a 3rd party library into support/, and *it* installs a library that *it* requires, that one will appear in CWD
It would be nice to figure out a way to force them to all appear in support/ where they belong.
2007-12-22 11:04:24 -07:00
Zooko O'Whielacronx
27a0d59308 setup: zope.interface bundle .tar instead of .tar.gz because multiple bundled .tar's compress together much more nicely than multiple bundled .tar.gz's do. :-) 2007-12-22 10:50:54 -07:00
Zooko O'Whielacronx
48b13dcab8 setup: upgrade bundle zfec-1.1.tar.gz to zfec-1.3.4.tar 2007-12-22 10:50:39 -07:00
Zooko O'Whielacronx
a9545aa4b0 setup: simplejson bundle .tar instead of .tar.gz because multiple bundled .tar's compress together much more nicely than multiple bundled .tar.gz's do. :-) 2007-12-22 10:50:29 -07:00
Zooko O'Whielacronx
840fb23229 setup: upgrade bundle from setuptools_darcs-1.0.6.tar.gz to setuptools_darcs-1.1.5.tar 2007-12-22 10:50:03 -07:00
Zooko O'Whielacronx
d5bb1ac316 setup: bundle pyutil-1.3.12.tar 2007-12-22 10:49:47 -07:00
Zooko O'Whielacronx
336c482561 setup: pycryptopp: bundle .tar instead of .tar.gz because multiple bundled .tar's compress together much more nicely than multiple bundled .tar.gz's do. :-) 2007-12-22 10:49:35 -07:00