Improve KiwiEx 2013 utilities

Rhizome mirror daemon and Serval Maps push script handle errors better

Added README.md

utilities/README.md

@@ -0,0 +1,36 @@
+Serval DNA Utilities
+====================
+[Serval Project][], February 2013
+
+This directory contains utilities that accompany [Serval DNA][]:
+
+ * [`rhizome_mirrord`][] is a Python 2.7 script that continuously extracts
+   Rhizome bundles from a local Rhizome store into a mirror directory, and
+   optionally unpacks Zip and Tar payloads into a separate directory.
+
+ * [`serval_maps_push.sh`][] is a Shell script designed to be invoked by
+   [`rhizome_mirrord`][] whenever in unpacks a Zip or Tar bundle.  It copies the
+   newly unpacked contents to the [Serval Maps testing server][] using
+   [rsync(1)][], then prods the testing server to process them by making an
+   HTTP request to a particular URL using [curl(1)][].
+
+These two scripts were created to inject Rhizome traffic from New Zealand Red
+Cross's KiwiEx 2013 field trial exercise into the Serval Maps visualisation
+server, as a demonstration of how Rhizome can be used to transmit situational
+awareness field reports back to base.
+
+In deployment, the [`serval_maps_push.sh`][] script was edited to use the
+KiwiEx2013 rsync destination directory and URL by changing the `TARGET`
+variable from `testing` to `kiwiex-2013`.
+
+The [`rhizome_mirrord`][] script has been designed in a general fashion, and is
+suitable for use in other deployments that require a similar mirror directory
+of Rhizome content.
+
+
+[Serval Project]: http://www.servalproject.org/
+[`rhizome_mirrord`]: ./rhizome_mirrord
+[`serval_maps_push.sh`]: ./serval_maps_push.sh
+[Serval Maps testing server]: http://maps.servalproject.org/testing/
+[rsync(1)]: http://rsync.samba.org/ftp/rsync/rsync.html
+[curl(1)]: http://curl.haxx.se/docs/manpage.html

utilities/rhizome_mirrord

@@ -348,12 +348,11 @@ class RhizomeMirror(object):
             log("execute " + ' '.join(args))
             try:
                 stat = subprocess.call(args, stdin= open('/dev/null'), env= env)
-                if stat == 0:
-                    self.unpacked.clear()
-                else:
+                if stat != 0:
                     error('exec-on-unpack script %r failed - error status %d' % (self.opts.exec_on_unpack, stat))
             except OSError, e:
                 error('cannot execute %r - %s' % (self.opts.exec_on_unpack, e))
+            self.unpacked.clear()
 
     def mkdirs(self, path):
         if os.path.isdir(path):
@@ -666,10 +665,12 @@ def log(msg):
         print >>log_output, '+ %s' % (msg,)
 
 def error(msg):
+    if log_output:
+        print >>log_output, '+ error(%r)' % (str(msg),)
     print >>sys.stderr, '%s: %s' % (os.path.basename(sys.argv[0]), msg)
 
 def fatal(msg):
-    error(msg)
+    error('FATAL: %s' % (msg,))
     sys.exit(1)
 
 if __name__ == '__main__':

utilities/serval_maps_push.sh

@@ -19,6 +19,7 @@
 
 set -e
 
+TARGET=testing
 UNPACK_DIR="${1?}"
 shift
 
@@ -30,9 +31,22 @@ fi
 
 rsync -a \
    "$UNPACK_DIR/" \
-   servalp@servalproject.org:/home/servalp/maps.servalproject.org/testing/admin/data/instances/
+   "servalp@servalproject.org:/home/servalp/maps.servalproject.org/$TARGET/admin/data/instances/"
 
 curl -s \
    -o serval_maps_push_result.html \
    -D serval_maps_push_headers.txt \
-   http://maps.servalproject.org/testing/admin/cache-update/instances/2798a6651e9caecd3d30fdc5e6a0e0f5
+   "http://maps.servalproject.org/$TARGET/admin/cache-update/instances/2798a6651e9caecd3d30fdc5e6a0e0f5"
+
+response=`sed -n -e '1s/^HTTP\/1\.. //p' serval_maps_push_headers.txt` 
+case $response in
+2[0-9][0-9]\ *) exit 0;;
+3[0-9][0-9]\ *) echo "Unexpected HTTP response: $response" >&2;;
+[45][0-9][0-9]\ *) echo "HTTP error response: $response" >&2;;
+*) echo "Malformed HTTP response" >&2;;
+esac
+if [ -n "$RHIZOME_MIRRORD_LOG_STDOUT" ]; then
+   cat serval_maps_push_headers.txt
+   cat serval_maps_push_result.html
+fi
+exit 1