feat(oam): codify garage-pdu-relay snmpd extend + timeout hardening [#733]

Deployed manually on tsys-librenms 2026-09-02 (founder relay ruling);
codified same-day rule. snmpget -t2 -r1 so a dark PDU answers in ~2s.
Deployed target: tsys-librenms:/usr/local/bin/garage-pdu-relay.

Detail: https://projects.knownelement.com/issues/733
This commit is contained in:
2026-09-03 12:55:31 -05:00
parent d3b9099d5f
commit e27e3a06b8
2 changed files with 99 additions and 0 deletions
+30
View File
@@ -0,0 +1,30 @@
#!/bin/bash
# garage-pdu-relay: expose APC AP7830 metered output current via snmpd extend.
#
# Relay pattern (founder ruling 2026-09-02): the garage PDU speaks SNMPv1
# only, walks die on its old AOS, and its ACL admits the LibreNMS VM vantage.
# This relay runs ON tsys-librenms; HA polls the relay via the fleet snmpd
# extend (same rails as drac-extend). Output: "output_amps <x.y>".
#
# Deploy target: tsys-librenms:/usr/local/bin/garage-pdu-relay ONLY (relay
# host — not a per-host agent). snmpd.conf line on the relay host:
# extend garagepdu /usr/local/bin/garage-pdu-relay
# snmpd ACL must admit the pfv-bms LAN + Tailscale IPs as query sources
# (rocommunity kn3lmgmt 192.168.3.12 / 100.67.108.125 as of 2026-09-03).
#
# Codified 2026-09-03 [#733]: was deployed manually 2026-09-02; hardened
# with explicit snmpget timeout/retry so a dark PDU answers in ~2s instead
# of ~6s and cannot pile up long-running children under HA's poll cadence.
set -euo pipefail
PDU_HOST="${PDU_HOST:-pfv-garage-pdu-1.knel.net}"
COMMUNITY="${PDU_COMMUNITY:-kn3lmgmt}"
OID=".1.3.6.1.4.1.318.1.1.12.2.3.1.1.2.1"
raw=$(snmpget -v1 -c "$COMMUNITY" -t 2 -r 1 -On -Ov "$PDU_HOST" "$OID" 2>/dev/null \
| grep -oE '[0-9]+$') || true
if [ -z "${raw:-}" ]; then
echo "output_amps unavailable"
exit 0
fi
awk -v t="$raw" 'BEGIN { printf "output_amps %.1f\n", t/10 }'
+69
View File
@@ -0,0 +1,69 @@
# Questions — v9 (current round)
v8's open items are now settled by the founder's 2026-09-03 chat
ruling: **OUTBOUND ONLY** — all PFV hosts (k8s included) send via the
gateway; no inbound ever; no DNAT, no MX cut-over (that scope is
dropped from #696). Robustness is a stated requirement (VIP pair,
queueing, cluster, monitoring). Map + target architecture updated in
place: [Discourse t/322](https://community.turnsys.com/t/322).
## PMG round — resolved
**PQ1 (backend).** Cloudron VPS `mail.knownelement.com`
(152.53.37.179); public MX already points there; fleet delivers
direct today; knel.net zone has no MX (and never needs one now).
**PQ2 (inbound path).** ✅ N/A by ruling — outbound only. No DNAT, no
MX work, ever.
**PQ3 (outbound smarthost).** All hosts through the gateway VIP.
**PQ4 (placement/sizing).** Pair already exists: VM 604 tsys6
(192.168.1.11) + VM 711 tsys7 (192.168.1.10), PMG 9.1.0, unclustered.
**"Old relay in postfix?"** None — `relayhost` empty fleet-wide.
## PMG round — still open (one-line answers fine)
**PQ5 (native PMG cluster?).** Recommended: yes — config + quarantine
replication between .1.11/.1.10, managed strictly serial. OK?
**PQ6 (TLS from fleet CA #697?).** Recommended: yes for the two admin
UIs; SMTP stays opportunistic (all flows are internal + to Cloudron).
**PQ7 (notify address).** Pair already forwards to `coo@turnsys.com`.
Confirm for PMG alerts + any held-mail notices?
**PQ8 (VIP specifics).** Confirm **VIP 192.168.3.249** + name
`smtp.knel.net` + dhcpd host-decl pin (VRRP MAC 00:00:5e:00:01:0a)?
Or name your preferred slot/name.
**PQ9 (outbound cutover staging).** Recommended: serial staged — PVE
hosts + PBS first (verify a week), then VM postfix set. OK?
**PQ11 (Cloudron relay user — founder side).** Recommended pattern:
PMG authenticates to a dedicated Cloudron mail user (e.g.
`pfv-relay@turnsys.com`) over :587 submission, so fleet mail exits
with Cloudron's rDNS/reputation. Will you create that user (or point
me at who does)? Fallback if you'd rather not: PMG delivers direct to
recipient MXs (same egress as today, no deliverability gain).
**PQ12 (k8s submission port).** Recommended: :25 unauthenticated from
trusted LAN (pod traffic SNATs to node IPs) as the default; optional
:587+auth later if you want app-level credentials. OK as default?
**PQ10 (side findings — sub-ticket placement).** (a) tsys-ucs-02 root
alias blackholes to `systemmail@unassigned-hostname...`; (b) Wazuh
alerting never configured (stock example smtp). Ride-alongs in #696
or separate sub-tickets? They get tickets either way — just say where.
## Carried from v6 — still open
**Q4. Second human approver in gitea?** "Planned for very near
future"; until then KNEL/access-roster merges are impossible by
design. Does not block code/deploy work.
## Superseded
v7 PQ wording and v8 PQ10 ordering are superseded by the rulings
above; v7/v8 files remain in git history.