Fleet monitoring: sysstat 28d + perfsnap/PSI/iotop timers + on-demand toolbox container (#826)
ci / audit (push) Successful in 26s

- monitoring/node-install.sh: idempotent per-node installer (sar HISTORY=28,
  three oneshot timers, per-node conf with qemu.slice/lxc cgroup globs)
- monitoring/collectors + systemd units: iotop-snap 15min, perfsnap 60s,
  psi-textfile 15s (now sources /etc/knelperf/perfsnap.conf; multi-glob)
- monitoring/toolbox: pct LXC (vmid 950, privileged, apparmor unconfined,
  stopped at rest) + knelperf-toolbox wrapper + Dockerfile variant
- monitoring/deploy-fleet.sh: workstation-side rollout driver
- piloted on pfv-tsys1: per-VM PSI gauges verified live

Redmine: https://projects.knownelement.com/issues/826
This commit is contained in:
2026-09-06 18:50:07 -05:00
parent f1729868fa
commit 3680f1dba1
16 changed files with 407 additions and 1 deletions
@@ -1,9 +1,11 @@
#!/usr/bin/env bash
# KNELPerf PSI (pressure stall info) avg60 -> node_exporter textfile collector.
# Ported from ultix staged/ukrrs-psi-textfile.sh (#826); generalizations:
# - cgroup glob is env-driven (any slice namespace, not just ukrrs-*)
# - cgroup globs are env-driven (space-separated LIST, any slice namespace)
# - metric prefix is env-driven (default knelperf_)
set -u
CONF=/etc/knelperf/perfsnap.conf
[ -r "$CONF" ] && . "$CONF"
OUT_DIR=${KNELPERF_TEXTFILE_DIR:-/var/lib/node_exporter/textfile}
CGROOT=${KNELPERF_CGROOT:-/sys/fs/cgroup}
CG_GLOB=${KNELPERF_CG_GLOB:-"$CGROOT"/system.slice/knelperf-*.slice}
@@ -27,6 +29,7 @@ emit() { # name cgroup_path
echo "# HELP ${PREFIX}_slice_pressure_some60 PSI some avg60 (percent) per slice" >>"$tmp"
echo "# TYPE ${PREFIX}_slice_pressure_some60 gauge" >>"$tmp"
for cg in $CG_GLOB; do
case $cg in "$CGROOT"/*) ;; *) continue ;; esac # skip unmatched glob literals
[ -d "$cg" ] && emit "${cg##*/}" "$cg"
done
[ -d "$CGROOT/user.slice" ] && emit "user.slice" "$CGROOT/user.slice"