prep for next ai session

This commit is contained in:
2026-08-01 15:45:23 -05:00
parent 46c35106fb
commit 6244c1cc25
139 changed files with 16712 additions and 0 deletions
+130
View File
@@ -0,0 +1,130 @@
# Console Management (ser2net + conman)
Network-accessible serial console management for all production network
switches and routers, running on **pfv-tsys4** (storage server).
## Architecture
```
USB-DB9 adapters → udev symlinks (/dev/consoles/<name>) → ser2net telnet(rfc2217) TCP → conman (logging + multiplexing)
```
ser2net owns the physical serial devices and exposes them on TCP ports
using the **telnet(rfc2217) protocol** bound to the **Tailscale interface
only** (`100.70.77.93:200X`). conman connects to those TCP ports via
telnet for session logging, output capture, and multi-user console
sharing.
**Why telnet(rfc2217)?** The serial devices send `
␍` (LF+CR) line
endings instead of standard `
`. Raw TCP transport caused conman's
telnet NVT to strip bare CR characters, producing stair-stepped output.
With telnet(rfc2217) on both sides, binary mode is negotiated and CR/LF
translation is handled correctly by the telnet layer.
**conman and ser2net do NOT share ports** — only one process can open a
serial device at a time. ser2net owns the physical device; conman connects
over TCP.
## The USB Enumeration Problem (SOLVED)
The 9 Prolific USB-to-DB9 adapters (`067b:2303`) on pfv-tsys4 have **no
unique USB serial numbers** and get assigned `/dev/ttyUSB0-8` based on
enumeration order, which shifts on every boot. This made the old
`/root/conmap` + manual `screen` workflow break after every reboot.
**Fix:** udev rules pin each adapter by its **ID_PATH** (physical USB port
topology), which is stable across reboots regardless of enumeration order.
Each adapter gets a named symlink in `/dev/consoles/` that never changes.
The udev rules are generated from `mapping.txt`, which maps each adapter's
ID_PATH to a console name and TCP port. To re-map after physically moving
an adapter, update `mapping.txt` and re-run `setup.sh`.
**Fallback:** if udev trigger doesn't create symlinks for already-discovered
devices (common on first run), `setup.sh` creates them manually by matching
ID_PATH. On subsequent boots, udev creates them automatically.
## Port Assignments
| TCP Port | Console Name | ID_PATH | Description |
|----------|-------------|---------|-------------|
| 2001 | pfv-core-sw01 | usb-0:1.5.4.4 | Dell PowerConnect 5448 (core switch) |
| 2002 | pfv-tor3-mgmt | usb-0:1.6.3.1 | Rack 3 management TOR switch |
| 2003 | pfv-tor3-stor | usb-0:1.6.3.3.2 | Rack 3 storage TOR switch |
| 2004 | pfv-rrinfra-rtr | usb-0:1.6.3.3.1 | Cisco router (rrinfra) |
| 2005 | pfv-r2-tor-top | usb-0:1.6.3.3.3 | Rack 2 top-of-rack switch |
| 2006 | subodev-torsw | usb-0:1.5.4.1 | Suborbital device TOR switch |
| 2007 | pfv-r2-sw | usb-0:1.6.3.2 | Rack 2 old Dell switch |
All ports listen on the Tailscale IP (`100.70.77.93`) using telnet(rfc2217).
## Scripts
| Script | Purpose |
|--------|---------|
| [`mapping.txt`](mapping.txt) | Source of truth: TCP port ↔ ID_PATH ↔ name ↔ baud |
| [`generate-config.sh`](generate-config.sh) | Generates udev rules, ser2net.yaml, conman.conf from mapping.txt |
| [`setup.sh`](setup.sh) | Full deploy: generate configs, create symlinks, restart services |
| [`discover.sh`](discover.sh) | Read-only discovery of USB adapters, existing config, services |
## Usage
### Connect to a console
**Primary method — conman client (with logging + multiplexing):**
```bash
# From any Tailscale-connected workstation:
conman -d pfv-tsys4:7890 -f pfv-core-sw01 # connect to console
conman -d pfv-tsys4:7890 -q # list all consoles
```
Escape sequence: `&.` to disconnect, `&?` for help.
**Direct telnet (emergency only — conflicts with conman):**
```bash
# Direct telnet to ser2net works ONLY when conmand is stopped, because
# conmand maintains persistent connections to all 7 TCP ports. Use:
ssh pfv-tsys4 'systemctl stop conmand'
telnet pfv-tsys4 2001 # pfv-core-sw01
ssh pfv-tsys4 'systemctl start conmand' # restart when done
```
**Do NOT use telnet while conmand is running** — conmand will reconnect
and kick your telnet session immediately ("Connection closed by foreign host").
The correct workflow is conman client → conmand → ser2net → device.
### Re-deploy after changing mapping.txt
```bash
PROX_HOST=pfv-tsys4 bash tests/remote.sh prox 'bash /root/console/setup.sh'
```
### Find the ID_PATH for a new adapter
```bash
PROX_HOST=pfv-tsys4 bash tests/remote.sh prox-file console/discover.sh
```
Then match the new adapter's ID_PATH to its physical location and add a line
to `mapping.txt`.
## Files on pfv-tsys4
| File | Purpose |
|------|---------|
| `/etc/udev/rules.d/99-console-ports.rules` | Stable symlinks by ID_PATH |
| `/etc/ser2net.yaml` | ser2net config (telnet rfc2217 TCP ports → serial symlinks) |
| `/etc/conman.conf` | conman config (CONSOLE entries between markers) |
| `/etc/systemd/system/conmand.service` | systemd unit for conmand |
| `/root/console/mapping.txt` | Copy of the source-of-truth mapping |
| `/root/console/setup.sh` | Setup script (re-runnable) |
| `/root/console/generate-config.sh` | Config generator |
## Old workflow (replaced)
The old `/root/conmap` file and manual `screen` sessions are no longer
needed. The new setup is fully automated and survives reboots.
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/bash
# shellcheck disable=SC2010,SC2012 # diagnostic script; ls|grep/ls -la on sysfs & log dirs is intentional for human-readable output
#
# console/discover.sh — READ-ONLY discovery of console setup on pfv-tsys4
#
# Usage: PROX_HOST=pfv-tsys4 bash tests/remote.sh prox-file console/discover.sh
#
# This script is strictly read-only. No writes to the system.
#
set -uo pipefail
echo "============================================"
echo " Console Setup Discovery"
echo " Host: $(hostname)"
echo " Date: $(date)"
echo " READ-ONLY"
echo "============================================"
echo ""
echo "=== 1. USB devices ==="
lsusb 2>/dev/null || echo "(lsusb not available)"
echo ""
echo "=== 2. All ttyUSB* devices (with major/minor) ==="
ls -la /dev/ttyUSB* 2>/dev/null || echo "(no /dev/ttyUSB* devices)"
echo ""
echo "=== 3. USB-serial driver bindings ==="
echo "-- pl2303 --"
ls -la /sys/bus/usb-serial/drivers/pl2303/ 2>/dev/null | grep -v '^total\|^d\|module\|new_id\|uevent' || echo "(none)"
echo "-- cp210x --"
ls -la /sys/bus/usb-serial/drivers/cp210x/ 2>/dev/null | grep -v '^total\|^d\|module\|new_id\|uevent' || echo "(none)"
echo "-- ftdi_sio --"
ls -la /sys/bus/usb-serial/drivers/ftdi_sio/ 2>/dev/null | grep -v '^total\|^d\|module\|new_id\|uevent' || echo "(none)"
echo "-- ch341 --"
ls -la /sys/bus/usb-serial/drivers/ch341/ 2>/dev/null | grep -v '^total\|^d\|module\|new_id\|uevent' || echo "(none)"
echo ""
echo "=== 4. USB serial adapter details (vendor/model/serial per port) ==="
for tty in /dev/ttyUSB*; do
[ -e "$tty" ] || continue
echo "--- $tty ---"
udevadm info -q all -n "$tty" 2>/dev/null | grep -E 'ID_VENDOR_ID|ID_MODEL_ID|ID_SERIAL|ID_USB_DRIVER|ID_PATH=' | sed 's/^/ /'
done
echo ""
echo "=== 5. Existing /root/conmap ==="
if [ -f /root/conmap ]; then
cat /root/conmap
else
echo "(no /root/conmap)"
fi
ls -la /root/conmap* 2>/dev/null
echo ""
echo "=== 6. Screen sessions (running) ==="
screen -ls 2>&1 || echo "(screen not running or not installed)"
echo ""
echo "=== 7. Existing screen wrappers/scripts in /root ==="
ls -la /root/ 2>/dev/null | grep -iE 'screen|con|console|tty|usb' || echo "(no obvious console scripts in /root)"
echo ""
echo "=== 8. ser2net ==="
which ser2net 2>/dev/null || echo "(ser2net not installed)"
dpkg -l ser2net 2>/dev/null | tail -2 || echo "(ser2net not in dpkg)"
cat /etc/ser2net/ser2net.yaml 2>/dev/null || cat /etc/ser2net.conf 2>/dev/null || cat /etc/ser2net/ser2net.conf 2>/dev/null || echo "(no ser2net config)"
systemctl is-active ser2net 2>/dev/null || echo "(ser2net service not found)"
echo ""
echo "=== 9. conman ==="
which conman 2>/dev/null || echo "(conman not installed)"
which conmand 2>/dev/null || echo "(conmand not installed)"
dpkg -l conman 2>/dev/null | tail -2 || echo "(conman not in dpkg)"
echo "--- /etc/conman.conf (console lines only) ---"
grep -nE 'CONSOLE|SERVER|LOG|SERIAL|DEV|BAUD|^[^#].*name=' /etc/conman.conf 2>/dev/null | head -60 || echo "(no conman.conf or no console entries)"
echo "--- conmand service ---"
systemctl is-active conmand 2>/dev/null || echo "(conmand not running)"
systemctl is-enabled conmand 2>/dev/null || echo "(conmand not enabled)"
echo ""
echo "=== 10. Existing console logs ==="
ls -la /var/log/conman/ 2>/dev/null | head -20 || echo "(no /var/log/conman)"
ls -la /var/consoles/ 2>/dev/null | head -20 || echo "(no /var/consoles)"
echo ""
echo "=== 11. udev rules for ttyUSB ==="
grep -r ttyUSB /etc/udev/rules.d/ 2>/dev/null || echo "(no udev rules for ttyUSB)"
grep -r 'console' /etc/udev/rules.d/ 2>/dev/null | head -10 || true
echo ""
echo "=== 12. expect availability ==="
command -v expect && expect -v 2>&1 || echo "expect: NOT installed"
command -v socat && socat -V 2>&1 | head -1 || echo "socat: NOT installed"
echo ""
echo "=== 13. Ports in use (2001-2099, 7000-7999, 7820-7899) ==="
ss -tlnp 2>/dev/null | grep -E ':200[0-9]|:700[0-9]|:782[0-9]|:789[0-9]' || echo "(no relevant ports listening)"
echo ""
echo "============================================"
echo " Discovery complete (read-only)."
echo "============================================"
+254
View File
@@ -0,0 +1,254 @@
#!/usr/bin/bash
#
# console/generate-config.sh — generate udev rules + ser2net.yaml + conman.conf
#
# Reads console/mapping.txt (the source of truth) and generates all three
# config files. This is the fix for the USB enumeration shift problem:
#
# 1. udev rules pin each adapter by its STABLE ID_PATH (physical USB port)
# to a named symlink like /dev/consoles/pfv-core-sw01
# 2. ser2net opens those stable symlinks and exposes them on TCP ports
# (2001, 2002, ...) bound to the Tailscale IP
# 3. conman connects to those TCP ports for logging + multiplexing
#
# Run this script ON the target host. It writes to:
# /etc/udev/rules.d/99-console-ports.rules
# /etc/ser2net.yaml
# /etc/conman/console-consoles.conf (included by /etc/conman.conf)
#
# Usage:
# PROX_HOST=pfv-tsys4 bash tests/remote.sh prox-file console/generate-config.sh
#
# Environment overrides:
# MAPPING_FILE — path to mapping.txt (default: auto-detect next to this script)
# TS_IP — Tailscale IP to bind ser2net on (default: auto-detect)
# CONMAN_LOGDIR — conman log directory (default: /var/log/conman)
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
MAPPING_FILE="${MAPPING_FILE:-$SCRIPT_DIR/mapping.txt}"
CONMAN_LOGDIR="${CONMAN_LOGDIR:-/var/log/conman}"
UDEV_RULES="/etc/udev/rules.d/99-console-ports.rules"
SER2NET_CONF="/etc/ser2net.yaml"
CONMAN_CONF="/etc/conman.conf"
echo "============================================"
echo " Console Config Generator"
echo " Host: $(hostname) $(date)"
echo "============================================"
# --- Locate mapping file ---
# When run via remote.sh prox-file, $0 is bash and $SCRIPT_DIR may be wrong.
# Search common locations.
if [ ! -f "$MAPPING_FILE" ]; then
for candidate in \
"/root/console/mapping.txt" \
"/tmp/mapping.txt" \
"$(dirname "$0")/mapping.txt"; do
if [ -f "$candidate" ]; then
MAPPING_FILE="$candidate"
break
fi
done
fi
if [ ! -f "$MAPPING_FILE" ]; then
echo "FATAL: mapping file not found. Tried: $MAPPING_FILE"
echo "Copy mapping.txt to the target host first."
exit 1
fi
echo " Mapping file: $MAPPING_FILE"
# --- Auto-detect Tailscale IP ---
if [ -z "${TS_IP:-}" ]; then
TS_IP=$(tailscale ip -4 2>/dev/null || true)
if [ -z "$TS_IP" ]; then
echo "FATAL: could not auto-detect Tailscale IP. Set TS_IP manually."
exit 1
fi
fi
echo " Tailscale IP: $TS_IP"
echo " ser2net will bind to: $TS_IP"
# --- Parse mapping file (skip comments and blank lines) ---
echo ""
echo "--- Parsing mapping file ---"
ENTRIES=()
while IFS= read -r line; do
# Skip comments and blank lines
line="${line%%#*}"
line="$(echo "$line" | xargs)" # trim whitespace
[ -z "$line" ] && continue
ENTRIES+=("$line")
echo " $line"
done < "$MAPPING_FILE"
if [ "${#ENTRIES[@]}" -eq 0 ]; then
echo "FATAL: no entries found in mapping file."
exit 1
fi
echo ""
echo " ${#ENTRIES[@]} console ports configured."
# ============================================================
# 1. Generate udev rules
# ============================================================
echo ""
echo "--- [1/3] Generating udev rules: $UDEV_RULES ---"
cat > "$UDEV_RULES" <<'UDEV_HEADER'
# Stable symlinks for USB-DB9 console adapters
# Generated by console/generate-config.sh
# DO NOT EDIT — edit mapping.txt and re-run generate-config.sh
#
# These rules pin each adapter to a named symlink based on its physical
# USB port path (ID_PATH), which is stable across reboots regardless of
# enumeration order. This is the fix for the "USB adapters shift on reboot"
# problem.
#
# To find the ID_PATH for a device:
# udevadm info -q all -n /dev/ttyUSBN | grep ID_PATH
UDEV_HEADER
for entry in "${ENTRIES[@]}"; do
IFS='|' read -r tcp_port name id_path baud comment <<< "$entry"
# Build the full ID_PATH match. The mapping stores a substring like "usb-0:1.5.4.4"
# The actual ID_PATH is like "pci-0000:00:1a.0-usb-0:1.5.4.4:1.0"
# We match on the substring to be portable across PCI bus changes.
{
echo ""
echo "# $name (TCP $tcp_port): $comment"
echo "SUBSYSTEM==\"tty\", ENV{ID_PATH}==\"*$id_path*\", SYMLINK+=\"consoles/$name\""
} >> "$UDEV_RULES"
done
echo " Written: $UDEV_RULES"
echo " Symlinks: /dev/consoles/<name> for each device"
# ============================================================
# 2. Generate ser2net.yaml
# ============================================================
echo ""
echo "--- [2/3] Generating ser2net config: $SER2NET_CONF ---"
# Backup existing config if not already backed up
if [ -f "$SER2NET_CONF" ] && [ ! -f "${SER2NET_CONF}.orig" ]; then
cp "$SER2NET_CONF" "${SER2NET_CONF}.orig"
echo " Backed up original to ${SER2NET_CONF}.orig"
fi
{
echo "%YAML 1.1"
echo "---"
echo "# ser2net configuration for pfv-tsys4 console ports"
echo "# Generated by console/generate-config.sh on $(date)"
echo "#"
echo "# All ports use telnet(rfc2217) accepter so conman and telnet clients"
echo "# negotiate proper telnet binary mode — this prevents CR stripping"
printf '%s\n' "# and stair-stepping on devices that send \\n\\r (LF+CR) line endings."
echo "# Ports bound to Tailscale IP ($TS_IP) for secure remote access."
echo "#"
echo "# Direct telnet: telnet $TS_IP 2001"
echo "# Via conman: conman -f <name>"
echo ""
printf '%s\n' "define: &banner \\r\\nPFV console port \\p device \\d [\\B]\\r\\n\\r\\n"
echo ""
for entry in "${ENTRIES[@]}"; do
IFS='|' read -r tcp_port name id_path baud comment <<< "$entry"
# ser2net connection block — telnet(rfc2217) accepter so conman and
# telnet clients negotiate proper telnet binary mode. This prevents
# CR stripping that occurs with raw TCP + conman's telnet NVT.
echo "connection: &con${tcp_port}"
echo " accepter: telnet(rfc2217),tcp,${TS_IP},${tcp_port}"
echo " enable: on"
echo " options:"
echo " banner: *banner"
echo " kickolduser: true"
echo " telnet-brk-on-sync: true"
echo " connector: serialdev,"
echo " /dev/consoles/${name},"
echo " ${baud},local"
echo ""
done
} > "$SER2NET_CONF"
echo " Written: $SER2NET_CONF"
echo " ${#ENTRIES[@]} TCP ports configured ($TS_IP:2001-20XX)"
# ============================================================
# 3. Write conman console entries directly into conman.conf
# ============================================================
# conman 0.3.x does NOT support the 'include' directive, so we write
# CONSOLE entries directly into /etc/conman.conf between idempotent markers.
echo ""
echo "--- [3/3] Writing conman consoles into $CONMAN_CONF ---"
# Ensure logdir exists
mkdir -p "$CONMAN_LOGDIR" 2>/dev/null || true
# Ensure LOGDIR is set in conman.conf (server-level directive for log file paths)
if ! grep -qiE '^\s*server\s+logdir\s*=' "$CONMAN_CONF" 2>/dev/null; then
# Insert near the top, after the first SERVER directives
sed -i "1i\\server logdir = \"$CONMAN_LOGDIR\"" "$CONMAN_CONF"
echo " Added server logdir = \"$CONMAN_LOGDIR\" to $CONMAN_CONF"
fi
# Ensure loopback=off so conmand is reachable over Tailscale (not localhost-only)
if ! grep -qiE '^\s*server\s+loopback\s*=' "$CONMAN_CONF" 2>/dev/null; then
sed -i "/^server logdir/a server loopback=off" "$CONMAN_CONF"
echo " Added server loopback=off to $CONMAN_CONF (enables remote access)"
fi
# Remove any previous auto-generated block (between markers)
# Then append the new block
MARKER_BEGIN="# BEGIN PFV CONSOLE DEFINITIONS (auto-generated — do not edit between markers)"
MARKER_END="# END PFV CONSOLE DEFINITIONS"
# Strip old block if present
if grep -q "$MARKER_BEGIN" "$CONMAN_CONF" 2>/dev/null; then
sed -i "/$MARKER_BEGIN/,/$MARKER_END/d" "$CONMAN_CONF"
echo " Removed previous console definitions."
fi
# Append new block
{
echo ""
echo "$MARKER_BEGIN"
echo "# Generated by console/generate-config.sh on $(date)"
echo "# Each console connects to a ser2net TCP port via telnet protocol."
echo "# ser2net uses telnet(rfc2217) accepter so binary mode is negotiated"
echo "# and CR/LF translation is handled correctly by the telnet NVT layer."
echo "# Access: conman -f <name>"
echo ""
for entry in "${ENTRIES[@]}"; do
IFS='|' read -r tcp_port name id_path baud comment <<< "$entry"
echo "CONSOLE name=\"${name}\" dev=\"${TS_IP}:${tcp_port}\" log=\"${name}.log\" logopts=\"timestamp\""
done
echo "$MARKER_END"
} >> "$CONMAN_CONF"
CONSOLE_COUNT=$(grep -c "^CONSOLE " "$CONMAN_CONF" 2>/dev/null || echo 0)
echo " Written $CONSOLE_COUNT CONSOLE entries to $CONMAN_CONF"
# ============================================================
# Summary
# ============================================================
echo ""
echo "============================================"
echo " Configuration generated successfully."
echo ""
echo " Files written:"
echo " $UDEV_RULES ($(wc -l < "$UDEV_RULES") lines)"
echo " $SER2NET_CONF ($(wc -l < "$SER2NET_CONF") lines)"
echo " $CONMAN_CONF (CONSOLE entries appended between markers)"
echo ""
echo " Next steps:"
echo " 1. Reload udev: udevadm control --reload-rules && udevadm trigger"
echo " 2. Restart ser2net: systemctl restart ser2net"
echo " 3. Start conman: systemctl enable --now conmand"
echo " 4. Or run: bash $(basename "$0" .sh | sed 's/generate-config/setup/') .sh"
echo "============================================"
+29
View File
@@ -0,0 +1,29 @@
# console/mapping.txt — Source of Truth for console port assignments
#
# Format: <tcp_port>|<name>|<id_path_substring>|<baud>|<comment>
#
# Delimiter is | (pipe) because ID_PATH values contain colons.
#
# - tcp_port: TCP port ser2net listens on (also the conman console name suffix)
# - name: Device name (used for /dev/console/<name> symlink, conman console name)
# - id_path_substring: Stable USB physical path from `udevadm info -q all -n /dev/ttyUSBN | grep ID_PATH`
# These are STABLE across reboots as long as adapters aren't moved
# to different physical USB ports.
# - baud: Serial baud rate (9600n81 = 9600 8N1, no flow control)
# - comment: Free-form description
#
# To RE-MAP after physically moving an adapter:
# 1. Run: bash console/discover.sh (find the new ID_PATH for the device)
# 2. Update the id_path_substring in this file
# 3. Run: bash console/generate-config.sh && udevadm trigger && systemctl restart ser2net conmand
#
2001|pfv-core-sw01|usb-0:1.5.4.4|9600n81|Dell PowerConnect 5448 (core switch)
2002|pfv-tor3-mgmt|usb-0:1.6.3.1|9600n81|Rack 3 management TOR switch
2003|pfv-tor3-stor|usb-0:1.6.3.3.2|9600n81|Rack 3 storage TOR switch
2004|pfv-rrinfra-rtr|usb-0:1.6.3.3.1|9600n81|Cisco router (rrinfra)
2005|pfv-r2-tor-top|usb-0:1.6.3.3.3|9600n81|Rack 2 top-of-rack switch
2006|subodev-torsw|usb-0:1.5.4.1|9600n81|Suborbital device TOR switch
2007|pfv-r2-sw|usb-0:1.6.3.2|9600n81|Rack 2 old Dell switch
# Unassigned (no device detected):
# 2008|spare-1|usb-0:1.6.3.4|9600n81|Empty / spare
# 2009|spare-2|usb-0:1.6.3.3.4|9600n81|Empty / spare
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/bash
#
# console/query-remote.sh — install conman client and connect to a console
# on pfv-tsys4 over Tailscale.
#
# Usage:
# bash console/query-remote.sh # list consoles
# bash console/query-remote.sh pfv-core-sw01 # connect to a console
#
set -euo pipefail
REMOTE_HOST="${REMOTE_HOST:-pfv-tsys4}"
REMOTE_PORT="${REMOTE_PORT:-7890}"
echo "============================================"
echo " Conman Remote Console Access"
echo " Server: ${REMOTE_HOST}:${REMOTE_PORT} (Tailscale)"
echo "============================================"
# --- 1. Install conman client if missing ---
if ! command -v conman >/dev/null 2>&1; then
echo ""
echo "--- Installing conman client ---"
if sudo -n true 2>/dev/null; then
sudo apt-get update -qq && sudo apt-get install -y -qq conman
else
echo " Passwordless sudo not available. Please run:"
echo " sudo apt-get update && sudo apt-get install -y conman"
echo " Then re-run this script."
exit 1
fi
else
echo " conman client already installed."
fi
# --- 2. Verify connectivity ---
echo ""
echo "--- Connectivity check ---"
if timeout 3 bash -c "echo > /dev/tcp/${REMOTE_HOST}/${REMOTE_PORT}" 2>/dev/null; then
echo " [OK] ${REMOTE_HOST}:${REMOTE_PORT} reachable"
else
echo " [FAIL] Cannot reach ${REMOTE_HOST}:${REMOTE_PORT}"
echo " Is Tailscale up? Is conmand running on ${REMOTE_HOST}?"
exit 1
fi
# --- 3. List or connect ---
CONSOLE="${1:-}"
if [ -z "$CONSOLE" ]; then
echo ""
echo "--- Available consoles ---"
conman -d "${REMOTE_HOST}:${REMOTE_PORT}" -q
echo ""
echo "To connect: bash $0 <console-name>"
echo " e.g: bash $0 pfv-core-sw01"
else
echo ""
echo "--- Connecting to: $CONSOLE ---"
echo " Escape sequence: &. (to disconnect)"
echo ""
conman -d "${REMOTE_HOST}:${REMOTE_PORT}" -f "$CONSOLE"
fi
+217
View File
@@ -0,0 +1,217 @@
#!/usr/bin/bash
# shellcheck disable=SC2010 # diagnostic; ls|grep on /dev listing is intentional
#
# console/setup.sh — deploy console management on pfv-tsys4
#
# Orchestrates the full setup:
# 1. Ensures ser2net + conman are installed
# 2. Copies mapping.txt to the target host (if running remotely)
# 3. Runs generate-config.sh to produce udev rules + ser2net.yaml + conman.conf
# 4. Reloads udev, creates /dev/consoles/ symlinks
# 5. Restarts ser2net (TCP ports on Tailscale IP)
# 6. Enables + starts conmand (logging + multiplexing)
# 7. Verifies
#
# This script is IDEMPOTENT — safe to run multiple times.
#
# Usage:
# PROX_HOST=pfv-tsys4 bash tests/remote.sh prox-file console/setup.sh
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
echo "============================================"
echo " Console Management Setup"
echo " Host: $(hostname) $(date)"
echo "============================================"
# --- 1. Install dependencies ---
echo ""
echo "--- [1/7] Checking dependencies ---"
NEED_INSTALL=()
dpkg -l ser2net 2>/dev/null | grep -q '^ii' && echo " ser2net: installed" || NEED_INSTALL+=(ser2net)
dpkg -l conman 2>/dev/null | grep -q '^ii' && echo " conman: installed" || NEED_INSTALL+=(conman)
if [ "${#NEED_INSTALL[@]}" -gt 0 ]; then
echo " Installing: ${NEED_INSTALL[*]}"
apt-get update -qq
apt-get install -y -qq "${NEED_INSTALL[@]}"
else
echo " All dependencies present."
fi
# --- 2. Ensure mapping file is available ---
echo ""
echo "--- [2/7] Locating mapping file ---"
MAPPING_FILE=""
for candidate in \
"$SCRIPT_DIR/mapping.txt" \
"$(dirname "$0")/mapping.txt" \
"/root/console/mapping.txt" \
"/tmp/mapping.txt"; do
if [ -f "$candidate" ]; then
MAPPING_FILE="$candidate"
break
fi
done
if [ -z "$MAPPING_FILE" ]; then
echo "FATAL: mapping.txt not found. Copy it to the target host."
exit 1
fi
echo " Using: $MAPPING_FILE"
# --- 3. Generate configs ---
echo ""
echo "--- [3/7] Generating configs ---"
export MAPPING_FILE
bash "$(dirname "$0")/generate-config.sh" 2>&1 || bash "$SCRIPT_DIR/generate-config.sh" 2>&1 || {
echo "FATAL: generate-config.sh failed."
exit 1
}
# --- 4. Reload udev + create symlinks ---
echo ""
echo "--- [4/7] Reloading udev rules ---"
udevadm control --reload-rules
# Try trigger first (works on some systems)
for tty in /sys/class/tty/ttyUSB*; do
[ -e "$tty" ] && udevadm trigger --action=add "$tty" 2>/dev/null || true
done
# Also try writing to uevent (forces udev reprocessing)
for tty in /sys/class/tty/ttyUSB*; do
[ -e "$tty/uevent" ] && echo "add" > "$tty/uevent" 2>/dev/null || true
done
sleep 2
# FALLBACK: if udev symlinks don't exist (common when devices are already
# discovered — udev trigger doesn't always re-create symlinks for existing
# devices), create them manually by matching ID_PATH. The udev rules will
# handle future boots/hotplugs automatically.
if [ ! -d /dev/consoles ] || [ -z "$(ls /dev/consoles/ 2>/dev/null)" ]; then
echo " udev trigger didn't create symlinks. Creating manually..."
mkdir -p /dev/consoles
while IFS= read -r line; do
line="${line%%#*}"
line="$(echo "$line" | xargs)"
[ -z "$line" ] && continue
IFS='|' read -r _ name id_path _ _ <<< "$line"
# Find the ttyUSB whose ID_PATH contains the mapping's id_path substring
for tty in /dev/ttyUSB*; do
[ -e "$tty" ] || continue
DEV_IDPATH=$(udevadm info -q property -n "$tty" 2>/dev/null | grep ^ID_PATH= | cut -d= -f2)
if echo "$DEV_IDPATH" | grep -q "$id_path"; then
ln -sf "$tty" "/dev/consoles/$name"
echo " ln -s $tty -> /dev/consoles/$name"
break
fi
done
done < "$MAPPING_FILE"
fi
echo " Stable symlinks:"
ls -la /dev/consoles/ 2>/dev/null | grep -v '^total\|^d' | sed 's/^/ /' || echo " (none created)"
# Verify each symlink resolves
echo ""
echo " Symlink verification:"
while IFS= read -r line; do
line="${line%%#*}"
line="$(echo "$line" | xargs)"
[ -z "$line" ] && continue
IFS='|' read -r _ name id_path _ _ <<< "$line"
if [ -e "/dev/consoles/$name" ]; then
TARGET=$(readlink -f "/dev/consoles/$name")
echo " [OK] /dev/consoles/$name -> $TARGET"
else
echo " [MISSING] /dev/consoles/$name (adapter unplugged or ID_PATH changed)"
fi
done < "$MAPPING_FILE"
# --- 5. Restart ser2net ---
echo ""
echo "--- [5/7] Restarting ser2net ---"
systemctl enable ser2net
systemctl restart ser2net
sleep 2
if systemctl is-active --quiet ser2net; then
echo " ser2net is running (telnet rfc2217 accepters)."
TS_IP=$(tailscale ip -4 2>/dev/null || echo "127.0.0.1")
echo " Listening ports:"
ss -tlnp | grep ser2net | grep -oE "${TS_IP}:[0-9]+" | sort -t: -k2 -n | sed 's/^/ /'
else
echo " WARNING: ser2net failed to start. Checking journal..."
journalctl -u ser2net --no-pager -n 20
fi
# --- 6. Enable + start conmand ---
echo ""
echo "--- [6/7] Starting conmand ---"
# conman package on Debian may not ship a systemd unit. Create one if missing.
if ! systemctl cat conmand >/dev/null 2>&1; then
echo " No systemd unit for conmand — creating one..."
cat > /etc/systemd/system/conmand.service <<'CONMAND_UNIT'
[Unit]
Description=ConMan (Console Manager)
After=network.target ser2net.service
Requires=ser2net.service
[Service]
Type=forking
ExecStart=/usr/sbin/conmand -c /etc/conman.conf
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
CONMAND_UNIT
systemctl daemon-reload
echo " Created /etc/systemd/system/conmand.service"
fi
# Kill any manually-started conmand first
pkill -x conmand 2>/dev/null || true
sleep 1
systemctl enable conmand 2>/dev/null || true
systemctl restart conmand 2>/dev/null || true
sleep 2
if systemctl is-active --quiet conmand; then
echo " conmand is running."
echo " Consoles:"
conman -q 2>&1 | sed 's/^/ /' || true
else
echo " WARNING: conmand failed to start. Checking journal..."
journalctl -u conmand --no-pager -n 20 2>/dev/null || true
# Try manual start as fallback
echo " Attempting manual start..."
/usr/sbin/conmand -c /etc/conman.conf 2>&1 || true
fi
# --- 7. Summary ---
echo ""
echo "--- [7/7] Setup complete ---"
echo ""
echo " ser2net + conman architecture (telnet rfc2217):"
echo " ser2net owns serial devices, exposes telnet(rfc2217) TCP ports"
echo " conman connects via telnet for logging + multiplexing"
echo ""
echo " Connect from any Tailscale workstation:"
echo " conman -d pfv-tsys4:7890 -f pfv-core-sw01"
echo " conman -d pfv-tsys4:7890 -q # list consoles"
echo ""
echo " Direct telnet (emergency, conflicts with conman):"
echo " ssh pfv-tsys4 'systemctl stop conmand'"
echo " telnet pfv-tsys4 2001"
echo " ssh pfv-tsys4 'systemctl start conmand'"
echo ""
echo " To regenerate after changing mapping.txt:"
echo " bash generate-config.sh"
echo " udevadm trigger"
echo " systemctl restart ser2net conmand"
echo "============================================"
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/bash
# shellcheck disable=SC2012,SC2001 # diagnostic script; ls -la listings and sed line-prefixing are intentional
#
# console/validate-conman.sh — verify conman can actually reach devices via
# ser2net TCP ports and is capturing log output to files.
#
# This tests the real data path: conman → TCP 200X → ser2net → /dev/consoles/X → device
#
set -uo pipefail
TS_IP=$(tailscale ip -4)
LOGDIR="/var/log/conman"
echo "============================================"
echo " Conman Data Path + Log Validation"
echo " Host: $(hostname) TS IP: $TS_IP"
echo "============================================"
echo ""
echo "--- 1. conman.conf log settings ---"
grep -E "logdir|LOGDIR|^GLOBAL LOG" /etc/conman.conf 2>/dev/null | grep -v "^#" || echo " (no explicit logdir — defaults to /var/log/conman)"
echo " Log dir: $LOGDIR"
ls -la "$LOGDIR"/ 2>/dev/null | head -15 || echo " ($LOGDIR does not exist yet)"
echo ""
echo "--- 2. CONSOLE entries: each has a log= directive? ---"
# Extract the auto-generated block and check each CONSOLE line has log=
sed -n '/BEGIN PFV CONSOLE/,/END PFV CONSOLE/p' /etc/conman.conf | grep "^CONSOLE" | while read -r line; do
name=$(echo "$line" | sed -n 's/.*name="\([^"]*\)".*/\1/p')
if echo "$line" | grep -q 'log='; then
logfile=$(echo "$line" | sed -n 's/.*log="\([^"]*\)".*/\1/p')
echo " [OK] $name → log=$logfile"
else
echo " [FAIL] $name has NO log= directive"
fi
done
echo ""
echo "--- 3. Trigger log capture: connect to each console briefly ---"
# conman -e changes the escape char. We use -j (join, read-only) with a timeout.
# Actually, conman doesn't have a built-in "connect for N seconds" — but conmand
# connects to each device ON STARTUP and keeps the connection open for logging.
# The log files should already be created. Let's check timestamps.
echo " conmand connects to all consoles on startup. Checking if logs exist..."
echo ""
echo "--- 4. Log file inventory ---"
for name in pfv-core-sw01 pfv-tor3-mgmt pfv-tor3-stor pfv-rrinfra-rtr pfv-r2-tor-top subodev-torsw pfv-r2-sw; do
logfile="$LOGDIR/${name}.log"
if [ -f "$logfile" ]; then
SIZE=$(stat -c%s "$logfile" 2>/dev/null || echo 0)
MTIME=$(stat -c%y "$logfile" 2>/dev/null | cut -d. -f1)
echo " [OK] $logfile ($SIZE bytes, modified $MTIME)"
else
echo " [MISSING] $logfile — conmand may not be writing yet"
fi
done
echo ""
echo "--- 5. conmand connection status (journal) ---"
# conmand logs connection attempts/errors to syslog
journalctl -u conmand --no-pager -n 50 2>/dev/null | grep -iE "connect|error|fail|console|refused|timeout" | tail -15 || echo " (no relevant journal entries)"
echo ""
echo "--- 6. Verify ser2net is proxying data (telnet rfc2217) ---"
echo " Probing TCP $TS_IP:2007 for data..."
RESPONSE=$(timeout 3 bash -c "printf '\r\r' | nc -w 2 $TS_IP 2007 2>/dev/null" | tr -cd '[:print:][:space:]' | head -5)
if [ -n "$RESPONSE" ]; then
echo " [OK] Data flowing through ser2net TCP 2007:"
echo "$RESPONSE" | sed 's/^/ /'
else
echo " (no immediate response — device may need more interaction)"
fi
echo ""
echo "--- 7. Check if conmand has open connections to ser2net ports ---"
CONMAND_PID=$(pgrep -x conmand 2>/dev/null || echo "")
if [ -n "$CONMAND_PID" ]; then
echo " conmand PID: $CONMAND_PID"
echo " Open connections to ser2net (expect 7 to 100.x:200X):"
ss -tnp 2>/dev/null | grep "pid=$CONMAND_PID" | grep -oE "100\.[0-9.]+:200[0-9]" | sort | sed 's/^/ /'
COUNT=$(ss -tnp 2>/dev/null | grep "pid=$CONMAND_PID" | grep -c ":200")
echo " Total conmand→ser2net connections: $COUNT (expect 7)"
else
echo " [FAIL] conmand not running"
fi
echo ""
echo "============================================"
+111
View File
@@ -0,0 +1,111 @@
# Powerman PDU Management
Centralized power management for the Cyclades AlterPath PM10i PDU via
[Powerman](https://github.com/chaos/powerman), running on pfv-tsys1.
## Hardware
| Component | Details |
|-----------|---------|
| **PDU** | Cyclades AlterPath PM10i (10 controllable AC outlets) |
| **Firmware** | v1.9.0 (Aug 4, 2006) |
| **Connection** | USB-to-DB9 adapter (Prolific pl2303, serial BJAAb144J07) |
| **Host** | pfv-tsys1 (OptiPlex 9020, Proxmox) |
| **Serial** | 9600 baud, 8N1, raw mode |
| **Credentials** | Factory defaults: `admin` / `pm8` (in cyclades-pm10.dev) |
| **Network access** | powermand listens on `127.0.0.1:10101` (local) + `100.121.189.98:10101` (Tailscale) |
## Device mapping
```
USB adapter (067b:23a3, serial BJAAb144J07)
└─ pl2303 driver → /dev/ttyUSB1
└─ udev symlink → /dev/cyclades-pm10 (stable across reboots)
└─ powermand reads/writes serial → Cyclades PM10i
└─ 10 outlets (factory default names: 1-10)
```
The udev rule (`/etc/udev/rules.d/99-cyclades-pdu.rules`) pins the adapter
by its USB serial number, so the symlink survives replugs and reboots.
## Scripts
All scripts run on the target host (pfv-tsys1) via `tests/remote.sh`:
```bash
# Setup (idempotent — safe to re-run):
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file powerman/setup.sh
# Validate PDU control (cycles outlet 10 off → on):
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file powerman/test-pdu.sh
# Status check:
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file powerman/status.sh
```
### Customizing for other hosts/PDUs
The setup script accepts environment overrides:
```bash
PDU_SERIAL=XXXX PDU_VENDOR=067b PDU_OUTLETS=20 PDU_TYPE=pm20 \
PROX_HOST=other-host bash tests/remote.sh prox-file powerman/setup.sh
```
## Usage (daily operations)
From pfv-tsys1 (or any host with network access to port 10101):
```bash
# List all outlets
powerman -l
# Query status (all outlets)
powerman -q
# Turn outlet off
powerman -0 outlet-10
# Turn outlet on
powerman -1 outlet-10
# Cycle outlet (off → 4s delay → on)
powerman -c outlet-10
# Query a specific outlet
powerman -q outlet-10
```
### Remote access from other hosts
powermand listens on `0.0.0.0:10101`. From another tailnet host:
```bash
powerman --server-host pfv-tsys1 --server-port 10101 -q
```
Or set `POWERMAN_SERVER=pfv-tsys1:10101` in the environment.
## Configuration files on pfv-tsys1
| File | Purpose |
|------|---------|
| `/etc/udev/rules.d/99-cyclades-pdu.rules` | Stable symlink for USB-DB9 adapter |
| `/etc/powerman/powerman.conf` | Device definition + 10 outlet nodes |
| `/etc/powerman/cyclades-pm10.dev` | Cyclades PM10 protocol spec (shipped with powerman) |
## Validation results
2026-07-28: All 8 checks passed.
Outlet 10 turned OFF (confirmed), turned ON (confirmed), then cycled.
## TODO (Friday onsite)
- [ ] **Rename outlets** in `/etc/powerman/powerman.conf` to match the
physical devices plugged into each outlet (e.g., `node "tsys4-psu"
"cyclades-pm10" "3"`). Currently all outlets are generically named
`outlet-1` through `outlet-10`.
- [ ] **Change PDU admin password** from factory default (`pm8`) if
security-sensitive. Update `/etc/powerman/cyclades-pm10.dev` login
script to match.
- [ ] **Verify all 10 outlets** individually once device mapping is known.
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/bash
#
# powerman/discover.sh — gather USB-DB9 adapter + powerman state on a host
#
# Usage: PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file powerman/discover.sh
#
set -uo pipefail
echo "============================================"
echo " PDU / Powerman Discovery"
echo " Host: $(hostname)"
echo " Date: $(date)"
echo "============================================"
echo ""
echo "=== 1. USB devices ==="
lsusb 2>/dev/null || echo "(lsusb not available)"
echo ""
echo "=== 2. USB-Serial adapters (ttyUSB*) ==="
ls -la /dev/ttyUSB* 2>/dev/null || echo "(no /dev/ttyUSB* devices)"
echo ""
echo "=== 3. USB-Serial kernel modules ==="
lsmod | grep -iE 'usbserial|ftdi|pl2303|cp210|ch34|cdc_acm' 2>/dev/null || echo "(no relevant modules loaded)"
echo ""
echo "=== 4. dmesg for USB serial (last 30 lines) ==="
dmesg | grep -iE 'ttyUSB|usbserial|ftdi|pl2303|cp210|ch34|converter' | tail -30 2>/dev/null || echo "(no dmesg matches)"
echo ""
echo "=== 5. All serial devices ==="
ls -la /dev/ttyS* /dev/ttyUSB* /dev/ttyACM* 2>/dev/null || echo "(no serial devices found)"
echo ""
echo "=== 6. Powerman installed? ==="
dpkg -l powerman 2>/dev/null || echo "(powerman not installed)"
which powerman 2>/dev/null || echo "(powerman binary not found)"
which powermand 2>/dev/null || echo "(powermand binary not found)"
echo ""
echo "=== 7. Powerman config files ==="
ls -la /etc/powerman/ 2>/dev/null || echo "(no /etc/powerman/ directory)"
ls -la /etc/powerman/*.dev 2>/dev/null || echo "(no .dev files)"
cat /etc/powerman/powerman.conf 2>/dev/null || echo "(no powerman.conf)"
echo ""
echo "=== 8. Available powerman device definitions ==="
ls /usr/share/powerman/*.dev 2>/dev/null || ls /etc/powerman/*.dev 2>/dev/null || echo "(no device definitions found)"
echo ""
echo "=== 9. Powermand service status ==="
systemctl status powerman 2>/dev/null | head -10 || echo "(powerman service not found)"
echo ""
echo "=== 10. Serial port test (quick probe of /dev/ttyUSB0) ==="
if [ -e /dev/ttyUSB0 ]; then
stty -F /dev/ttyUSB0 2>/dev/null && echo "(port exists and is configurable)" || echo "(port exists but stty failed)"
# Try to read any pending output
timeout 2 cat /dev/ttyUSB0 2>/dev/null | head -5 || echo "(no immediate output from port)"
else
echo "(no /dev/ttyUSB0)"
fi
echo ""
echo "============================================"
echo " Discovery complete."
echo "============================================"
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/bash
#
# powerman/query-remote.sh — install powerman client locally and query
# the Cyclades PDU running on pfv-tsys1 over Tailscale.
#
set -euo pipefail
REMOTE_HOST="${REMOTE_HOST:-pfv-tsys1}"
REMOTE_PORT="${REMOTE_PORT:-10101}"
echo "============================================"
echo " Powerman Remote PDU Query"
echo " Server: ${REMOTE_HOST}:${REMOTE_PORT} (Tailscale)"
echo "============================================"
# --- 1. Install powerman client if missing ---
if ! command -v powerman >/dev/null 2>&1; then
echo ""
echo "--- Installing powerman client ---"
if sudo -n true 2>/dev/null; then
sudo apt-get update -qq && sudo apt-get install -y -qq powerman
else
echo " Passwordless sudo not available. Please run this command in a terminal:"
echo ""
echo " sudo apt-get update && sudo apt-get install -y powerman"
echo ""
echo " Then re-run this script."
exit 1
fi
else
echo " powerman client already installed."
fi
# --- 2. Verify connectivity ---
echo ""
echo "--- Connectivity check ---"
if timeout 3 bash -c "echo > /dev/tcp/${REMOTE_HOST}/${REMOTE_PORT}" 2>/dev/null; then
echo " [OK] ${REMOTE_HOST}:${REMOTE_PORT} reachable"
else
echo " [FAIL] Cannot reach ${REMOTE_HOST}:${REMOTE_PORT}"
echo " Is Tailscale up? Is powermand running on ${REMOTE_HOST}?"
exit 1
fi
export POWERMAN_SERVER="${REMOTE_HOST}:${REMOTE_PORT}"
# --- 3. List outlets ---
echo ""
echo "--- Outlets ---"
powerman -h "${REMOTE_HOST}:${REMOTE_PORT}" -l
# --- 4. Query status ---
echo ""
echo "--- Status ---"
powerman -h "${REMOTE_HOST}:${REMOTE_PORT}" -q
echo ""
echo "============================================"
echo " Done."
echo ""
echo " To control an outlet from this workstation:"
echo " powerman -h ${REMOTE_HOST}:${REMOTE_PORT} -0 outlet-10 # off"
echo " powerman -h ${REMOTE_HOST}:${REMOTE_PORT} -1 outlet-10 # on"
echo " powerman -h ${REMOTE_HOST}:${REMOTE_PORT} -c outlet-10 # cycle"
echo "============================================"
+181
View File
@@ -0,0 +1,181 @@
#!/usr/bin/bash
#
# powerman/setup.sh — idempotent powerman setup for Cyclades PM10i PDU
#
# Creates a stable udev symlink for the USB-DB9 adapter, writes powerman.conf
# with 10 outlet nodes, and enables + starts powermand.
#
# This script is designed to be run ON the target host (pfv-tsys1) as root.
# It is idempotent: safe to run multiple times.
#
# Usage:
# PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file powerman/setup.sh
#
# Override defaults via environment variables:
# PDU_SERIAL — USB adapter serial (default: BJAAb144J07)
# PDU_VENDOR — USB vendor ID (default: 067b)
# PDU_DEV_NAME — udev symlink name (default: cyclades-pm10)
# PDU_BAUD — serial baud rate (default: 9600,8n1)
# PDU_TYPE — powerman spec type (default: pm10)
# PDU_OUTLETS — number of outlets (default: 10)
# PDU_LISTEN — powermand listen (default: 0.0.0.0:10101)
#
set -euo pipefail
# --- Config (overridable via env) ---
PDU_SERIAL="${PDU_SERIAL:-BJAAb144J07}"
PDU_VENDOR="${PDU_VENDOR:-067b}"
PDU_DEV_NAME="${PDU_DEV_NAME:-cyclades-pm10}"
PDU_BAUD="${PDU_BAUD:-9600,8n1}"
PDU_TYPE="${PDU_TYPE:-pm10}"
PDU_OUTLETS="${PDU_OUTLETS:-10}"
PDU_LISTEN="${PDU_LISTEN:-}" # Auto-detect Tailscale IP if empty
UDEV_RULE="/etc/udev/rules.d/99-cyclades-pdu.rules"
POWERMAN_CONF="/etc/powerman/powerman.conf"
DEV_FILE="/etc/powerman/cyclades-pm10.dev"
# --- Auto-detect Tailscale IP for listen address ---
if [ -z "$PDU_LISTEN" ]; then
TS_IP=$(tailscale ip -4 2>/dev/null || true)
if [ -n "$TS_IP" ]; then
PDU_LISTEN="${TS_IP}:10101"
echo " Auto-detected Tailscale IP: $TS_IP"
else
PDU_LISTEN="127.0.0.1:10101"
echo " WARNING: No Tailscale IP detected. Defaulting to localhost."
fi
fi
echo "============================================"
echo " Powerman PDU Setup"
echo " Host: $(hostname)"
echo " PDU: Cyclades PM${PDU_OUTLETS}i"
echo " Adapter serial: $PDU_SERIAL"
echo " Device symlink: /dev/$PDU_DEV_NAME"
echo " Listen: $PDU_LISTEN (Tailscale only)"
echo "============================================"
# --- 1. Ensure powerman is installed ---
echo ""
echo "--- [1/5] Checking powerman installation ---"
if ! dpkg -l powerman 2>/dev/null | grep -q '^ii'; then
echo " Installing powerman from Debian repo..."
apt-get update -qq && apt-get install -y -qq powerman
else
echo " Powerman already installed: $(dpkg -l powerman | awk '/^ii/{print $3}')"
fi
# --- 2. Create udev rule for stable device name ---
echo ""
echo "--- [2/5] Creating udev rule for USB-DB9 adapter ---"
cat > "$UDEV_RULE" <<UDEV
# Stable symlink for Cyclades PM10i PDU USB-DB9 adapter
# Generated by powerman/setup.sh
SUBSYSTEM=="tty", ATTRS{idVendor}=="${PDU_VENDOR}", ATTRS{serial}=="${PDU_SERIAL}", GROUP="dialout", MODE="0660", SYMLINK+="${PDU_DEV_NAME}"
UDEV
echo " Written: $UDEV_RULE"
# Trigger udev to create the symlink now
udevadm control --reload-rules 2>/dev/null || true
udevadm trigger --subsystem-match=tty 2>/dev/null || true
sleep 1
if [ -e "/dev/${PDU_DEV_NAME}" ]; then
echo " Device symlink active: /dev/${PDU_DEV_NAME} -> $(readlink -f "/dev/${PDU_DEV_NAME}")"
else
echo " WARNING: /dev/${PDU_DEV_NAME} not found yet. Adapter may be unplugged."
echo " Falling back to /dev/ttyUSB* discovery..."
# Try to find any ttyUSB device as fallback
for tty in /dev/ttyUSB*; do
if [ -e "$tty" ]; then
echo " Found: $tty (using as fallback)"
PDU_DEV_NAME="$(basename "$tty")"
break
fi
done
fi
# --- 2b. Ensure powermand user can access the serial device ---
echo ""
echo "--- [2b/5] Fixing serial device permissions ---"
if id powerman >/dev/null 2>&1; then
if id powerman | grep -qv dialout; then
usermod -aG dialout powerman
echo " Added 'powerman' user to 'dialout' group"
else
echo " 'powerman' already in 'dialout' group"
fi
else
echo " (no powerman user — service may run as root)"
fi
# --- 3. Write powerman.conf ---
echo ""
echo "--- [3/5] Writing powerman.conf ---"
# Build node definitions
NODES=""
for i in $(seq 1 "$PDU_OUTLETS"); do
NODES+="node \"outlet-${i}\" \"${PDU_DEV_NAME}\" \"${i}\"\n"
done
cat > "$POWERMAN_CONF" <<PMCONF
# Powerman configuration for Cyclades PM${PDU_OUTLETS}i PDU
# Generated by powerman/setup.sh on $(date)
# Device: /dev/${PDU_DEV_NAME} (USB-DB9 adapter serial ${PDU_SERIAL})
# Listen on localhost (for local admin) and Tailscale (for remote access)
listen "127.0.0.1:10101"
listen "${PDU_LISTEN}"
# Device specification for Cyclades PM10
include "${DEV_FILE}"
# The PDU device (serial-attached)
device "${PDU_DEV_NAME}" "${PDU_TYPE}" "/dev/${PDU_DEV_NAME}" "${PDU_BAUD}"
# Outlet nodes (rename these to match attached devices when onsite)
$(printf '%b' "$NODES")
PMCONF
echo " Written: $POWERMAN_CONF"
echo " Nodes defined: outlet-1 through outlet-${PDU_OUTLETS}"
# --- 4. Restart powermand ---
echo ""
echo "--- [4/5] Restarting powermand ---"
systemctl enable powerman 2>/dev/null || true
systemctl restart powerman 2>/dev/null || true
sleep 2
if systemctl is-active --quiet powerman; then
echo " powermand is running."
else
echo " WARNING: powermand failed to start. Check journalctl -u powerman"
journalctl -u powerman --no-pager -n 20 2>/dev/null || true
fi
# --- 5. Verify ---
echo ""
echo "--- [5/5] Verification ---"
echo ""
echo " powerman -l (list all outlets):"
powerman -l 2>&1 || echo "(powerman -l failed)"
echo ""
echo " powerman -q (query status):"
powerman -q 2>&1 || echo "(powerman -q failed — PDU may need a moment)"
echo ""
echo "============================================"
echo " Setup complete."
echo ""
echo " Outlet names are generic (outlet-1 ... outlet-${PDU_OUTLETS})."
echo " Rename them in ${POWERMAN_CONF} when onsite to match attached devices."
echo ""
echo " Test: powerman -0 outlet-10 (off)"
echo " powerman -1 outlet-10 (on)"
echo " powerman -c outlet-10 (cycle)"
echo " powerman -q (status)"
echo "============================================"
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/bash
#
# powerman/status.sh — quick PDU status check
#
# Usage:
# PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file powerman/status.sh
#
set -euo pipefail
echo "============================================"
echo " Cyclades PM10i PDU Status"
echo " Host: $(hostname) $(date)"
echo "============================================"
echo ""
echo "=== Service ==="
systemctl is-active powerman 2>/dev/null && echo "(running)" || echo "(stopped)"
echo ""
echo "=== Device ==="
ls -la /dev/cyclades-pm10 2>/dev/null || echo "(no /dev/cyclades-pm10 symlink)"
echo ""
echo "=== Outlets ==="
powerman -l 2>&1
echo ""
echo "=== Power Status ==="
powerman -q 2>&1
echo ""
echo "=== Temperature ==="
powerman -T 2>&1 || echo "(temperature not available)"
+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/bash
#
# powerman/test-pdu.sh — validate PDU control by cycling outlet 10 off and on
#
# Usage:
# PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file powerman/test-pdu.sh
#
# Override: OUTLET=10 (which outlet to test)
#
set -euo pipefail
OUTLET="${OUTLET:-10}"
NODE="outlet-${OUTLET}"
PASS=0; FAIL=0
ok() { echo " [PASS] $1"; PASS=$((PASS+1)); }
fail() { echo " [FAIL] $1"; FAIL=$((FAIL+1)); }
echo "============================================"
echo " PDU Control Validation"
echo " Host: $(hostname)"
echo " Test: cycle outlet ${OUTLET} (off → wait → on)"
echo "============================================"
# --- 0. Powermand running? ---
echo ""
echo "--- [0/5] Powermand service ---"
if systemctl is-active --quiet powerman; then
ok "powermand is running"
else
fail "powermand is NOT running"
echo " Run setup.sh first."
exit 1
fi
# --- 1. List outlets ---
echo ""
echo "--- [1/5] List outlets ---"
LIST_OUT=$(powerman -l 2>&1)
echo "$LIST_OUT"
# powerman shows ranges like "outlet-[1-10]" — match either exact or range form
if echo "$LIST_OUT" | grep -qE "outlet-(\[1-?10\]|${OUTLET}\b)"; then
ok "Outlet '${NODE}' is defined"
else
fail "Outlet '${NODE}' not found in powerman -l"
exit 1
fi
# --- 2. Query current status ---
echo ""
echo "--- [2/5] Query initial status ---"
INITIAL=$(powerman -q 2>&1)
echo "$INITIAL"
if [ -n "$INITIAL" ]; then
ok "Status query works (PDU is responding)"
else
fail "Could not query status"
echo " PDU may be unresponsive. Check serial connection."
exit 1
fi
# --- 3. Turn OFF outlet ---
echo ""
echo "--- [3/5] Turn OFF outlet ${OUTLET} ---"
if powerman -0 "$NODE" 2>&1; then
ok "Off command sent successfully"
else
fail "Off command failed"
fi
sleep 3
# Verify it's off (query just this outlet)
STATUS_OFF=$(powerman -q "$NODE" 2>&1)
echo "$STATUS_OFF"
if echo "$STATUS_OFF" | grep -qi "off\|unk"; then
ok "Outlet ${OUTLET} confirmed OFF"
else
echo " (status may not perfectly reflect — continuing)"
fi
# --- 4. Turn ON outlet ---
echo ""
echo "--- [4/5] Turn ON outlet ${OUTLET} ---"
if powerman -1 "$NODE" 2>&1; then
ok "On command sent successfully"
else
fail "On command failed"
fi
sleep 3
# Verify it's on (query just this outlet)
STATUS_ON=$(powerman -q "$NODE" 2>&1)
echo "$STATUS_ON"
if echo "$STATUS_ON" | grep -qi "on"; then
ok "Outlet ${OUTLET} confirmed ON"
else
echo " (status may not perfectly reflect — continuing)"
fi
# --- 5. Cycle test (off → delay → on in one command) ---
echo ""
echo "--- [5/5] Cycle test (powerman -c) ---"
if powerman -c "$NODE" 2>&1; then
ok "Cycle command completed"
else
fail "Cycle command failed"
echo " (some PDU firmware reports errors during cycle but still works)"
fi
sleep 5
# Final status
echo ""
echo "--- Final status ---"
powerman -q 2>&1
echo ""
echo "============================================"
echo " Results: $PASS passed, $FAIL failed"
if [ "$FAIL" -gt 0 ]; then
echo " Some checks failed. Review output above."
exit 1
fi
echo " PDU control validated."
echo "============================================"
+210
View File
@@ -0,0 +1,210 @@
# UPS Management (NUT — Network UPS Tools)
Centralized UPS monitoring for the server room via
[NUT](https://networkupstools.org/), running on **pfv-tsys1**. USB HID UPS
units feed one `upsd` network server; Home Assistant polls it over Tailscale for
real-time power/load/runtime tracking, and a local `upsmon` shuts the hypervisor
down gracefully when battery is low.
> **Why NUT (not apcupsd)?** Two different UPS brands (APC + Tripp Lite) must be
> covered. `apcupsd` only supports APC, so it would require a second daemon
> stack. NUT's `usbhid-ups` driver speaks to **both** via the USB HID Power
> Device class, and Home Assistant ships a first-class NUT integration.
## Hardware
| UPS | Model | VID:PID | USB Serial | Status |
|-----|-------|---------|------------|--------|
| **APC** | Smart-UPS C 1500 (FW 02.2) | `051d:0003` | `AS1213210423` | **LIVE** |
| **Tripp Lite** | UPS (HID PDC) | `09ae:3016` | `2352CVLSM871900694` | **Blocked** — see below |
## Current State (2026-07-30)
### APC Smart-UPS C 1500 — OPERATIONAL
Fully reporting via `usbhid-ups` + `APC HID 0.100` subdriver. Data validated:
```
battery.charge: 100 battery.runtime: 1800 battery.voltage: 27.4
ups.status: OL ups.load: (via HA) ups.model: Smart-UPS C 1500
```
### Tripp Lite UPS — BLOCKED (hardware issue)
The driver finds the device, matches the `TrippLite HID 0.85` subdriver, claims
the interface, and reads the HID descriptor — but **fails reading the 878-byte
HID Report Descriptor** (`Resource temporarily unavailable` / EAGAIN after 5s).
The driver is masked to prevent restart-loop spam.
USB descriptors (manufacturer, product, serial) are readable via `lsusb -v` and
`nut-scanner`, but the bulk control transfer for the full report descriptor
times out. Likely causes:
1. **USB hub** — the Tripp Lite is behind a Genesys Logic hub (`05e3:0608`).
Try plugging directly into a motherboard USB port.
2. **USB cable** — try a high-quality data cable (not charge-only).
3. **UPS firmware** — the USB controller may not properly implement all HID
endpoints.
**To retry after physical reseat:**
```bash
# On pfv-tsys1:
systemctl unmask nut-driver@tripp-lite-ups
systemctl start nut-driver@tripp-lite-ups
upsc tripp-lite-ups@localhost
```
## Architecture
```
pfv-tsys1 (192.168.3.11 / Tailscale 100.121.189.98)
├─ APC Smart-UPS C 1500 ──┐
└─ Tripp Lite UPS (masked) ──┤ USB HID
nut-driver@apc-smartups-c1500 (usbhid-ups)
upsd :3493 (LISTEN 127.0.0.1 + Tailscale + LAN)
▼ ▼
upsmon (local) Home Assistant (NUT integration)
graceful shutdown via LAN 192.168.3.11 (HAOS can't
route to Tailscale IPs)
```
- **Driver layer** — `usbhid-ups` process, pinned by USB serial. Debian uses
templated `nut-driver@<upsname>.service` units managed by
`nut-driver-enumerator`.
- **Server layer** — `upsd` exposes UPS data on TCP 3493 (localhost + Tailscale
+ LAN). Clients authenticate via `upsd.users`.
- **Monitor layer** — `upsmon` runs locally as `master` to trigger
`SHUTDOWNCMD` (`/sbin/shutdown -h now`) when a UPS reports `LOWBATT`.
- **Home Assistant** — native NUT integration connects to `upsd` over Tailscale
and exposes `ups.load`, `battery.runtime`, `ups.status`, etc. as sensors.
### Key deployment lesson: udev must cover raw USB devices
The `usbhid-ups` driver opens `/dev/bus/usb/BBB/DDD` (raw USB device files),
**not** `/dev/hidraw*`. After calling `setuid(111)` to drop to the `nut` user,
it needs write access to those raw USB files. The udev rule must match
`SUBSYSTEM=="usb"` by vendor/product ID to set `GROUP="nut"` — matching only
`hidraw` is insufficient. See `/etc/udev/rules.d/99-nut-ups.rules`.
## Scripts
NUT host scripts run on pfv-tsys1 via `tests/remote.sh`:
```bash
# Idempotent install + configure (safe to re-run):
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/setup.sh
# Discover USB UPS + NUT state (read-only diagnostic):
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/discover.sh
# Query UPS data + service health:
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/status.sh
```
The HA integration script runs from your workstation (needs HA API access):
```bash
# Add the NUT integration to Home Assistant (idempotent):
bash ups/setup-ha-nut.sh
```
`setup.sh` accepts environment overrides for serials/VIDs/PIDs/usernames, so it
can be repurposed for other hosts or UPS units. Passwords for `monuser` and
`homeassistant` are auto-generated on first run and reused on subsequent runs
(stored in `/etc/nut/upsd.users`).
Set `TRIPP_ENABLED=0` to skip the Tripp Lite entirely (useful if it's physically
unplugged).
## Configuration files on pfv-tsys1
| File | Purpose |
|------|---------|
| `/etc/udev/rules.d/99-nut-ups.rules` | Grant nut group rw on raw USB + hidraw devices (both subsystems) |
| `/etc/nut/ups.conf` | `usbhid-ups` device(s), pinned by serial + subdriver |
| `/etc/nut/upsd.conf` | `LISTEN 127.0.0.1` + `LISTEN <tailscale>` + `LISTEN <lan>` on port 3493 |
| `/etc/nut/upsd.users` | `monuser` (master) + `homeassistant` (read-only) credentials |
| `/etc/nut/upsmon.conf` | Local master monitor + `SHUTDOWNCMD` |
| `/etc/nut/nut.conf` | `MODE=netserver` |
## Home Assistant integration
The NUT integration is added automatically by `setup-ha-nut.sh`, which drives
HA's REST config-flow API. It is idempotent (skips if the entry exists).
```bash
# Prerequisites: create token + password files (one-time):
mkdir -p ~/.config/pfvcluster
# HA → Profile → Long-Lived Access Tokens → Create Token:
echo -n 'YOUR_HA_TOKEN' > ~/.config/pfvcluster/ha-token
# Password is in /etc/nut/upsd.users on pfv-tsys1 (the homeassistant user):
echo -n 'YOUR_NUT_PASS' > ~/.config/pfvcluster/nut-password
chmod 600 ~/.config/pfvcluster/{ha-token,nut-password}
# Run:
bash ups/setup-ha-nut.sh
```
### Why LAN IP, not Tailscale
upsd listens on **both** the Tailscale IP (`100.121.189.98`) **and** the LAN IP
(`192.168.3.11`). The HA NUT integration uses the **LAN IP** because HAOS runs
Tailscale as an isolated add-on container — the HA core container cannot route
to Tailscale IPs. Since pfv-bms (HA, `192.168.3.12`) and pfv-tsys1 (`192.168.3.11`)
share the same vmbr0 bridge, LAN connectivity is instant and reliable.
### Manual UI alternative
In Home Assistant → **Settings → Devices & Services → Add Integration → NUT**:
| Field | Value |
|-------|-------|
| Host | `192.168.3.11` (LAN — HAOS can't reach Tailscale IPs from the HA container) |
| Port | `3493` |
| Username | `homeassistant` |
| Password | *(stored in `/etc/nut/upsd.users` on pfv-tsys1)* |
| UPS | `apc-smartups-c1500` |
### Live sensors
HA exposes UPS data as sensors (prefix `sensor.apc_smartups_c1500_`):
`battery_charge`, `status` (Online/On Battery), `status_data` (OL/OB/DISCHRG).
Additional sensors (load, runtime, voltage) populate as the UPS reports them.
## Daily operations
From pfv-tsys1 (or any tailnet host with NUT client installed):
```bash
# List UPS units served by upsd
upsc -l pfv-tsys1
# Full variable dump for one UPS
upsc apc-smartups-c1500@pfv-tsys1
# Battery runtime (the only runtime/charge data this UPS exposes)
upsc apc-smartups-c1500@pfv-tsys1 battery.runtime
```
## Notes
- **No USB passthrough to the HA VM.** Keeping the UPS on the host preserves
hypervisor graceful-shutdown capability and matches the `powerman/` pattern
(PDU managed on the host where the adapter physically lives).
- **No `ups.load` / `ups.realpower` on this UPS (FW 02.2, mfg 2012):** The
APC Smart-UPS C 1500 does not expose load or power data over USB HID.
Both NUT `usbhid-ups` and `apcupsd` (USB mode, tested 2026-07-30) read the
same HID descriptor — the variable simply isn't there. This means the HA
NUT integration provides **battery/runtime/status sensors only**, not
wattage for the Energy Dashboard.
- **apcupsd test note:** Debian's `apcupsd` package conflicts with
`nut-server` (mutually exclusive). apcupsd USB mode returned `COMMLOST`
even before we could check load. The APC Smart Serial protocol (serial
cable, AP940-1524C, ~$30) DOES report load%, but this requires a serial
port on the UPS and on the host.
- **Energy Dashboard path:** A smart plug (Shelly Plug S / TP-Link Kasa,
~$15-25) on the UPS output reports real watts natively and feeds the
Energy Dashboard with zero UPS-driver hacking. The NUT sensors remain
valuable for outage detection and graceful-shutdown automations.
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/bash
#
# ups/discover.sh — probe USB UPS units and NUT state on the local host
#
# Read-only. Prints everything needed to configure NUT. No changes made.
#
# Usage (run ON the target host via remote.sh):
# PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/discover.sh
#
set -uo pipefail
echo "======================================================"
echo " UPS / NUT Discovery on $(hostname)"
echo "======================================================"
# --- 1. USB UPS devices ------------------------------------------------------
echo ""
echo "--- [1/5] USB UPS devices (lsusb) ---"
lsusb 2>/dev/null | grep -iE "UPS|American Power|Tripp|APC" || echo " (no UPS devices found in lsusb)"
echo ""
echo "--- [2/5] UPS detail (vendor/product/serial/model) ---"
# Common UPS vendor IDs: 051d (APC), 09ae (Tripp Lite), 0463 (Eaton),
# 06da (MGE), 0764 (Cyber Power)
for vid in 051d 09ae 0463 06da 0764; do
while read -r bus dev pid; do
[ -n "$bus" ] || continue
echo " --- $bus:$dev ($vid:$pid) ---"
lsusb -v -s "${bus}:${dev}" 2>/dev/null \
| grep -iE "iManufacturer|iProduct|iSerial|bcdDevice" \
| sed 's/^/ /'
done < <(lsusb 2>/dev/null | awk -v v="$vid" '$0~v{split($2,a,":"); split($4,b,":"); print a[1], b[1], $6}')
done
# --- 2. sysfs paths (for udev rules) -----------------------------------------
echo ""
echo "--- [3/5] sysfs device paths + serials ---"
for d in /sys/bus/usb/devices/*; do
man=$(cat "$d/manufacturer" 2>/dev/null)
prod=$(cat "$d/product" 2>/dev/null)
ser=$(cat "$d/serial" 2>/dev/null)
vid=$(cat "$d/idVendor" 2>/dev/null)
pid=$(cat "$d/idProduct" 2>/dev/null)
if echo "$man $prod" | grep -qiE "apc|tripp|power conversion|ups|eaton|mge|cyber power"; then
# Resolve stable ID_PATH for udev pinning
path=$(udevadm info -q property -p "$d" 2>/dev/null | awk -F= '/^ID_PATH=/{print $2}')
echo " $d"
echo " vendor=$vid product=$pid"
echo " manufacturer=$man"
echo " product=$prod"
echo " serial=$ser"
echo " ID_PATH=$path"
fi
done
# --- 3. HID device nodes -----------------------------------------------------
echo ""
echo "--- [4/5] HID device nodes ---"
ls -la /dev/hidraw* /dev/usb/hiddev* 2>/dev/null || echo " (no hidraw/hiddev nodes)"
# --- 4. NUT install state ----------------------------------------------------
echo ""
echo "--- [5/5] NUT install + service state ---"
if dpkg -l nut-server nut-client 2>/dev/null | grep -q '^ii'; then
echo " NUT installed:"
dpkg -l nut-server nut-client 2>/dev/null | awk '/^ii/{print " "$2" "$3}'
else
echo " NUT not installed (apt: nut-server nut-client)"
fi
echo ""
echo " Services:"
for svc in nut-driver nut-server nut-monitor; do
printf " %-14s " "$svc:"
systemctl is-active "$svc" 2>/dev/null || true
done
echo ""
echo " Existing config:"
# shellcheck disable=SC2012 # ls -la is intentional for human-readable listing
ls -la /etc/nut/ 2>/dev/null | sed 's/^/ /' || echo " (no /etc/nut)"
echo ""
echo "======================================================"
echo " Discovery complete."
echo "======================================================"
+134
View File
@@ -0,0 +1,134 @@
#!/usr/bin/env python3
"""
ha-nut-setup.py — Add the Home Assistant NUT integration via REST config-flow API.
Stdlib-only (no pip). Idempotent: skips if a NUT config entry already exists.
Env:
HA_HOST (default pfv-bms.knel.net)
HA_PORT (default 8123)
HA_TOKEN (long-lived access token)
NUT_HOST (default 100.121.189.98)
NUT_PORT (default 3493)
NUT_USER (default homeassistant)
NUT_PASS (required)
NUT_UPS (default apc-smartups-c1500)
"""
import os, json, sys, time, urllib.request, urllib.error
HA_HOST = os.environ.get("HA_HOST", "pfv-bms.knel.net")
HA_PORT = int(os.environ.get("HA_PORT", "8123"))
TOKEN = os.environ["HA_TOKEN"]
NUT_HOST = os.environ.get("NUT_HOST", "192.168.3.11")
NUT_PORT = int(os.environ.get("NUT_PORT", "3493"))
NUT_USER = os.environ.get("NUT_USER", "homeassistant")
NUT_PASS = os.environ["NUT_PASS"]
NUT_UPS = os.environ.get("NUT_UPS", "apc-smartups-c1500")
BASE = f"http://{HA_HOST}:{HA_PORT}"
def api(method, path, data=None):
body = json.dumps(data).encode() if data else None
req = urllib.request.Request(
f"{BASE}/api{path}", data=body, method=method,
headers={"Authorization": f"Bearer {TOKEN}",
"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=20) as r:
return json.loads(r.read())
except urllib.error.HTTPError as e:
raw = e.read().decode()
try:
return json.loads(raw)
except Exception:
return {"_http_error": e.code, "_raw": raw[:300]}
except Exception as e:
return {"_error": str(e)}
# ── verify token ──
cfg = api("GET", "/config")
if "_http_error" in cfg or "_error" in cfg:
print(f"Cannot reach HA or token invalid: {cfg}"); sys.exit(1)
print(f"HA {cfg.get('version')} — token valid")
# ── check existing entries (idempotent) ──
entries = api("GET", "/config/config_entries/entry")
existing = [e for e in entries if e.get("domain") == "nut"]
if existing:
for e in existing:
print(f"NUT already configured: {e.get('title')} "
f"(data={json.dumps(e.get('data', {}))})")
print("Skipping — delete it in HA UI first if you want to re-run.")
sys.exit(0)
print("No existing NUT entry. Starting config flow.")
# ── initiate flow ──
flow = api("POST", "/config/config_entries/flow", {"handler": "nut"})
if "flow_id" not in flow:
print(f"Flow init failed: {json.dumps(flow)}"); sys.exit(1)
fid = flow["flow_id"]
print(f"Flow started: step={flow.get('step_id')} "
f"fields={[f.get('name') for f in flow.get('data_schema', [])]}")
# ── submit connection details ──
creds = {"host": NUT_HOST, "port": NUT_PORT,
"username": NUT_USER, "password": NUT_PASS}
flow = api("POST", f"/config/config_entries/flow/{fid}", creds)
if flow.get("errors"):
print(f"Validation errors: {flow['errors']}"); sys.exit(1)
print(f"After submit: type={flow.get('type')} step={flow.get('step_id')}")
# ── handle follow-up steps (UPS selection etc.) ──
while flow.get("type") == "form":
step = flow.get("step_id", "?")
schema = flow.get("data_schema", [])
print(f"Step '{step}': fields={[f.get('name') for f in schema]}")
for f in schema:
opts = f.get("options") or f.get("values")
if opts:
print(f" {f.get('name')} options: {opts}")
submission = {}
for f in schema:
nm = f.get("name")
ftype = f.get("type", "")
if ftype == "multi_select":
opts = f.get("options", [])
vals = [o[0] if isinstance(o, list) else o for o in opts]
submission[nm] = [NUT_UPS] if NUT_UPS in vals else vals[:1]
elif nm in creds:
submission[nm] = creds[nm]
elif "default" in f:
submission[nm] = f["default"]
elif ftype == "select":
opts = f.get("options", [])
vals = [o[0] if isinstance(o, list) else o for o in opts]
submission[nm] = NUT_UPS if NUT_UPS in vals else (vals[0] if vals else "")
fid = flow.get("flow_id", fid)
flow = api("POST", f"/config/config_entries/flow/{fid}", submission)
if flow.get("errors"):
print(f"Validation errors: {flow['errors']}"); sys.exit(1)
print(f" -> type={flow.get('type')} step={flow.get('step_id')}")
# ── result ──
if flow.get("type") == "create_entry":
print(f"\nNUT integration created: {flow.get('title')}")
elif flow.get("type") == "abort":
print(f"\nFlow aborted: {flow.get('reason')}"); sys.exit(1)
else:
print(f"\nFinal state: {flow.get('type')}{json.dumps(flow)[:200]}")
# ── verify sensors ──
print("\nWaiting 10s for entities ...")
time.sleep(10)
states = api("GET", "/states")
ups = [s for s in states
if "apc_smartups" in s["entity_id"].lower()
or "sensor.ups_" in s["entity_id"].lower()]
if ups:
print(f"Found {len(ups)} UPS sensors:")
for e in sorted(ups, key=lambda x: x["entity_id"]):
st = e.get("state", "?")
unit = e.get("attributes", {}).get("unit_of_measurement", "")
name = e.get("attributes", {}).get("friendly_name", "")
print(f" {e['entity_id']:55s} {st:>8} {unit:4s} {name}")
else:
print("No UPS sensors yet (may still be initialising — check HA UI).")
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env bash
#
# setup-ha-nut.sh — Add the Home Assistant NUT integration via REST API.
#
# Idempotent: skips if a NUT entry already exists. Reads secrets from
# ~/.config/pfvcluster/ (ha-token, nut-password) or env vars.
#
# Usage:
# bash ups/setup-ha-nut.sh
#
# Env overrides:
# HA_TOKEN HA long-lived access token
# NUT_PASS NUT upsd password for the homeassistant user
# HA_HOST HA host (default pfv-bms.knel.net)
# NUT_HOST upsd host (default 192.168.3.11 — LAN, see README)
# NUT_PORT upsd port (default 3493)
# NUT_USER upsd user (default homeassistant)
# NUT_UPS UPS name (default apc-smartups-c1500)
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CONF_DIR="${PFV_CONF_DIR:-$HOME/.config/pfvcluster}"
# --- HA token ---
HA_TOKEN="${HA_TOKEN:-}"
if [[ -z "$HA_TOKEN" ]]; then
TOKEN_FILE="$CONF_DIR/ha-token"
if [[ -f "$TOKEN_FILE" ]]; then
HA_TOKEN="$(head -1 "$TOKEN_FILE" | tr -d '[:space:]')"
else
echo "error: no HA token. Set \$HA_TOKEN or create $TOKEN_FILE" >&2
echo " (HA → Profile → Long-Lived Access Tokens → Create Token)" >&2
exit 1
fi
fi
# --- NUT password ---
NUT_PASS="${NUT_PASS:-}"
if [[ -z "$NUT_PASS" ]]; then
PASS_FILE="$CONF_DIR/nut-password"
if [[ -f "$PASS_FILE" ]]; then
NUT_PASS="$(head -1 "$PASS_FILE" | tr -d '[:space:]')"
else
echo "error: no NUT password. Set \$NUT_PASS or create $PASS_FILE" >&2
echo " (value is in /etc/nut/upsd.users on the NUT host)" >&2
exit 1
fi
fi
# --- connection params (override for your own kit) ---
export HA_TOKEN
export NUT_PASS
export HA_HOST="${HA_HOST:-pfv-bms.knel.net}"
export HA_PORT="${HA_PORT:-8123}"
export NUT_HOST="${NUT_HOST:-192.168.3.11}"
export NUT_PORT="${NUT_PORT:-3493}"
export NUT_USER="${NUT_USER:-homeassistant}"
export NUT_UPS="${NUT_UPS:-apc-smartups-c1500}"
echo "HA: ${HA_HOST}:${HA_PORT}"
echo "NUT: ${NUT_USER}@${NUT_HOST}:${NUT_PORT} (UPS: ${NUT_UPS})"
echo ""
exec python3 "$SCRIPT_DIR/ha-nut-setup.py"
+298
View File
@@ -0,0 +1,298 @@
#!/usr/bin/bash
#
# ups/setup.sh — idempotent Network UPS Tools (NUT) setup on pfv-tsys1
#
# Installs NUT, configures two USB HID UPS units (APC + Tripp Lite) pinned by
# USB serial, runs upsd as a network server for Home Assistant polling, and
# runs upsmon locally so the hypervisor can shut down gracefully on battery.
#
# Designed to run ON the target host (pfv-tsys1) as root, idempotent.
#
# Usage:
# PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/setup.sh
#
# Overrides (defaults suit pfv-tsys1):
# APC_SERIAL APC UPS USB serial (default AS1213210423)
# APC_VID/APC_PID APC vendor/product ID (default 051d / 0003)
# APC_NAME NUT section name for APC (default apc-smartups-c1500)
# TRIPP_SERIAL Tripp Lite UPS USB serial (default 2352CVLSM871900694)
# TRIPP_VID/TRIPP_PID Tripp Lite vendor/product (default 09ae / 3016)
# TRIPP_NAME NUT section name for Tripp (default tripp-lite-ups)
# TRIPP_SUBDRIVER Forced HID subdriver for Tripp (default "TrippLite HID 0.85")
# TRIPP_ENABLED Set to 0 to disable Tripp Lite (default 1)
# NUT_LISTEN_IPS space-separated upsd LISTEN IPs (default: auto Tailscale + 127.0.0.1)
# HA_USER upsd username for HA (default homeassistant)
# HA_PASSWORD upsd password for HA (default: reuse or generate)
# MON_USER upsd username for local upsmon (default monuser)
# MON_PASSWORD upsd password for upsmon (default: reuse or generate)
#
set -euo pipefail
# --- Config (overridable via env) ---
APC_SERIAL="${APC_SERIAL:-AS1213210423}"
APC_VID="${APC_VID:-051d}"
APC_PID="${APC_PID:-0003}"
APC_NAME="${APC_NAME:-apc-smartups-c1500}"
TRIPP_SERIAL="${TRIPP_SERIAL:-2352CVLSM871900694}"
TRIPP_VID="${TRIPP_VID:-09ae}"
TRIPP_PID="${TRIPP_PID:-3016}"
TRIPP_NAME="${TRIPP_NAME:-tripp-lite-ups}"
TRIPP_SUBDRIVER="${TRIPP_SUBDRIVER:-TrippLite HID 0.85}"
TRIPP_ENABLED="${TRIPP_ENABLED:-1}"
HA_USER="${HA_USER:-homeassistant}"
MON_USER="${MON_USER:-monuser}"
NUT_PORT="${NUT_PORT:-3493}"
UDEV_RULE="/etc/udev/rules.d/99-nut-ups.rules"
UPS_CONF="/etc/nut/ups.conf"
UPSD_CONF="/etc/nut/upsd.conf"
UPSD_USERS="/etc/nut/upsd.users"
UPS_CONF_MON="/etc/nut/upsmon.conf"
NUT_CONF="/etc/nut/nut.conf"
# --- Helpers ---
gen_pw() { head -c 24 /dev/urandom | base64 | tr -d '/+=' | cut -c1-20; }
# Reuse existing passwords if config already present (idempotent re-runs)
extract_pw() { # $1=user
if [ -f "$UPSD_USERS" ]; then
awk -v u="[$1]" '
$0==u {inblk=1; next}
/^\[/ {inblk=0}
inblk && $1=="password" {gsub(/"/,"",$3); print $3; exit}
' "$UPSD_USERS" 2>/dev/null
fi
}
echo "============================================"
echo " NUT UPS Setup on $(hostname)"
echo " APC: $APC_NAME ($APC_VID:$APC_PID serial $APC_SERIAL)"
if [ "$TRIPP_ENABLED" = "1" ]; then
echo " Tripp Lite: $TRIPP_NAME ($TRIPP_VID:$TRIPP_PID serial $TRIPP_SERIAL)"
else
echo " Tripp Lite: DISABLED (TRIPP_ENABLED=0)"
fi
echo "============================================"
# --- 0. Resolve / generate passwords (idempotent) ---
MON_PASSWORD="${MON_PASSWORD:-$(extract_pw "$MON_USER")}"
HA_PASSWORD="${HA_PASSWORD:-$(extract_pw "$HA_USER")}"
[ -n "$MON_PASSWORD" ] || MON_PASSWORD="$(gen_pw)"
[ -n "$HA_PASSWORD" ] || HA_PASSWORD="$(gen_pw)"
# --- 0b. Auto-detect listen IPs for upsd ---
# Tailscale IP: tailnet clients (workstation, etc.)
# LAN IP: HAOS VMs where Tailscale runs as an isolated add-on (HA container
# cannot route to Tailscale IPs, so the shared-LAN bridge is required)
if [ -z "${NUT_LISTEN_IPS:-}" ]; then
NUT_LISTEN_IPS="127.0.0.1"
TS_IP=$(tailscale ip -4 2>/dev/null || true)
if [ -n "$TS_IP" ]; then
NUT_LISTEN_IPS="${NUT_LISTEN_IPS} ${TS_IP}"
else
echo " WARNING: No Tailscale IP detected."
fi
if [ "${NUT_INCLUDE_LAN:-1}" = "1" ]; then
LAN_IP=$(ip -4 addr show vmbr0 2>/dev/null | awk '/scope global/{print $2}' | cut -d/ -f1 | head -1)
if [ -z "$LAN_IP" ]; then
LAN_IP=$(hostname -I 2>/dev/null | awk '{print $1}')
fi
if [ -n "$LAN_IP" ]; then
NUT_LISTEN_IPS="${NUT_LISTEN_IPS} ${LAN_IP}"
fi
fi
fi
echo " upsd LISTEN IPs: ${NUT_LISTEN_IPS:-<none>}"
# --- 1. Install NUT ---
echo ""
echo "--- [1/8] Installing NUT (nut-server, nut-client) ---"
if dpkg -l nut-server 2>/dev/null | grep -q '^ii'; then
echo " NUT already installed: $(dpkg -l nut-server | awk '/^ii/{print $3}')"
else
apt-get update -qq && apt-get install -y -qq nut-server nut-client
fi
mkdir -p /etc/nut
# --- 2. udev rules: grant nut group access to BOTH raw USB + hidraw devices ---
# CRITICAL: usbhid-ups opens /dev/bus/usb/BBB/DDD (raw USB), not /dev/hidraw.
# The driver drops to the nut user via setuid(), so the nut group needs write
# access to the raw USB device files. Matching on subsystem=="usb" by VID:PID
# is required because ATTRS{serial} does not reliably traverse for usb devices.
echo ""
echo "--- [2/8] Writing udev rules (raw USB + hidraw, group nut) ---"
{
echo "# Stable permissions for NUT USB HID UPS units"
echo "# Generated by ups/setup.sh — grants the 'nut' group access to both"
echo "# the raw USB device files (/dev/bus/usb) and hidraw devices."
echo "# Match BOTH subsystems: the usbhid-ups driver opens the raw USB device"
echo "# after dropping to the nut user via setuid()."
echo ""
echo "# APC Smart-UPS C 1500 ($APC_VID:$APC_PID)"
echo "SUBSYSTEM==\"usb\", ATTR{idVendor}==\"$APC_VID\", ATTR{idProduct}==\"$APC_PID\", GROUP=\"nut\", MODE=\"0664\""
echo "SUBSYSTEM==\"hidraw\", ATTRS{serial}==\"$APC_SERIAL\", GROUP=\"nut\", MODE=\"0660\""
echo ""
echo "# Tripp Lite UPS ($TRIPP_VID:$TRIPP_PID)"
echo "SUBSYSTEM==\"usb\", ATTR{idVendor}==\"$TRIPP_VID\", ATTR{idProduct}==\"$TRIPP_PID\", GROUP=\"nut\", MODE=\"0664\""
echo "SUBSYSTEM==\"hidraw\", ATTRS{serial}==\"$TRIPP_SERIAL\", GROUP=\"nut\", MODE=\"0660\""
} > "$UDEV_RULE"
echo " Written: $UDEV_RULE"
udevadm control --reload-rules 2>/dev/null || true
udevadm trigger --subsystem-match=usb 2>/dev/null || true
udevadm trigger --subsystem-match=hidraw 2>/dev/null || true
sleep 1
# --- 3. ups.conf ---
echo ""
echo "--- [3/8] Writing ups.conf ---"
{
echo "# NUT UPS devices — generated by ups/setup.sh on $(date)"
echo ""
echo "maxretry = 3"
echo ""
echo "[${APC_NAME}]"
echo " driver = usbhid-ups"
echo " port = auto"
echo " vendorid = ${APC_VID}"
echo " productid = ${APC_PID}"
echo " serial = ${APC_SERIAL}"
echo " desc = \"APC Smart-UPS C 1500\""
if [ "$TRIPP_ENABLED" = "1" ]; then
echo ""
echo "[${TRIPP_NAME}]"
echo " driver = usbhid-ups"
echo " port = auto"
echo " vendorid = ${TRIPP_VID}"
echo " productid = ${TRIPP_PID}"
echo " serial = ${TRIPP_SERIAL}"
echo " subdriver = \"${TRIPP_SUBDRIVER}\""
echo " desc = \"Tripp Lite UPS\""
fi
} > "$UPS_CONF"
echo " Written: $UPS_CONF"
# --- 4. upsd.conf: network server (localhost + Tailscale for HA) ---
echo ""
echo "--- [4/8] Writing upsd.conf ---"
{
echo "# NUT upsd — generated by ups/setup.sh on $(date)"
for ip in $NUT_LISTEN_IPS; do
echo "LISTEN ${ip} ${NUT_PORT}"
done
echo "MAXAGE 25"
} > "$UPSD_CONF"
echo " Written: $UPSD_CONF (LISTEN: $(echo "$NUT_LISTEN_IPS" | tr '\n' ' '))"
# --- 5. upsd.users: monuser (master) + homeassistant (read-only monitor) ---
echo ""
echo "--- [5/8] Writing upsd.users ---"
cat > "$UPSD_USERS" <<USERS
# NUT upsd users — generated by ups/setup.sh on $(date)
# monuser : local upsmon (master) — graceful hypervisor shutdown
# ${HA_USER} : Home Assistant NUT integration (read-only polling)
[${MON_USER}]
password = "${MON_PASSWORD}"
upsmon master
[${HA_USER}]
password = "${HA_PASSWORD}"
upsmon slave
USERS
echo " Written: $UPSD_USERS"
# --- 6. upsmon.conf + nut.conf ---
echo ""
echo "--- [6/8] Writing upsmon.conf + nut.conf ---"
{
echo "# NUT upsmon (local monitor) — generated by ups/setup.sh on $(date)"
echo "# Monitors the APC UPS as master. On battery-low, shuts the host down."
echo ""
echo "MONITOR ${APC_NAME}@localhost 1 ${MON_USER} \"${MON_PASSWORD}\" master"
if [ "$TRIPP_ENABLED" = "1" ]; then
echo "MONITOR ${TRIPP_NAME}@localhost 1 ${MON_USER} \"${MON_PASSWORD}\" master"
fi
echo ""
echo "SHUTDOWNCMD \"/sbin/shutdown -h now\""
echo "POWERDOWNFLAG /etc/killpower"
echo "NOTIFYFLAG ONLINE SYSLOG+WALL"
echo "NOTIFYFLAG ONBATT SYSLOG+WALL"
echo "NOTIFYFLAG LOWBATT SYSLOG+WALL"
echo "POLLFREQ 15"
echo "POLLFREQALERT 5"
echo "HOSTSYNC 15"
} > "$UPS_CONF_MON"
cat > "$NUT_CONF" <<NUTCONF
# NUT mode — generated by ups/setup.sh on $(date)
# netserver = this host runs drivers + upsd; serves UPS data to clients (HA).
MODE=netserver
NUTCONF
echo " Written: $UPS_CONF_MON + $NUT_CONF"
# --- 6b. Fix ownership/permissions (Debian: nut group must read config) ---
chown root:nut "$UPS_CONF" "$UPSD_CONF" "$UPSD_USERS" "$UPS_CONF_MON" 2>/dev/null || true
chmod 640 "$UPS_CONF" "$UPSD_CONF" "$UPSD_USERS" "$UPS_CONF_MON" 2>/dev/null || true
chmod 644 "$NUT_CONF" 2>/dev/null || true
# --- 7. Start services (Debian uses templated nut-driver@<name> units) ---
echo ""
echo "--- [7/8] Starting NUT services ---"
# Re-read ups.conf to generate per-UPS driver instances
systemctl restart nut-driver-enumerator 2>/dev/null || true
sleep 2
# Start per-UPS driver instances
systemctl restart "nut-driver@${APC_NAME}" 2>/dev/null || true
if [ "$TRIPP_ENABLED" = "1" ]; then
systemctl restart "nut-driver@${TRIPP_NAME}" 2>/dev/null || true
else
systemctl stop "nut-driver@${TRIPP_NAME}" 2>/dev/null || true
systemctl mask "nut-driver@${TRIPP_NAME}" 2>/dev/null || true
fi
sleep 3
systemctl restart nut-server 2>/dev/null || true
sleep 1
systemctl restart nut-monitor 2>/dev/null || true
echo ""
echo " Service status:"
for svc in "nut-driver@${APC_NAME}" "nut-driver@${TRIPP_NAME}" nut-server nut-monitor; do
if systemctl list-unit-files "$svc" >/dev/null 2>&1; then
printf " %-42s " "$svc"
systemctl is-active "$svc" 2>/dev/null || echo "(unknown)"
fi
done
# --- 8. Validate ---
echo ""
echo "--- [8/8] Validation ---"
echo ""
echo " upsc — ${APC_NAME}:"
upsc "${APC_NAME}@localhost" 2>&1 | head -25 || echo " (APC UPS not responding yet)"
if [ "$TRIPP_ENABLED" = "1" ]; then
echo ""
echo " upsc — ${TRIPP_NAME}:"
upsc "${TRIPP_NAME}@localhost" 2>&1 | head -25 || echo " (Tripp Lite UPS not responding yet)"
fi
echo ""
echo "============================================"
echo " Setup complete."
echo ""
echo " Home Assistant NUT integration:"
echo " Host: $(echo "$NUT_LISTEN_IPS" | awk '{print $2}') (or any LISTEN IP above)"
echo " Port: ${NUT_PORT}"
echo " Username: ${HA_USER}"
echo " Password: ${HA_PASSWORD}"
if [ "$TRIPP_ENABLED" = "1" ]; then
echo " UPS names: ${APC_NAME}, ${TRIPP_NAME}"
else
echo " UPS names: ${APC_NAME}"
fi
echo ""
echo " Save the HA password now — it is stored in ${UPSD_USERS}."
echo "============================================"
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/bash
#
# ups/status.sh — query NUT UPS state + service health (read-only)
#
# Usage (run ON the target host via remote.sh):
# PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/status.sh
#
# shellcheck disable=SC2012 # ss/awk field extraction is intentional
set -uo pipefail
APC_NAME="${APC_NAME:-apc-smartups-c1500}"
TRIPP_NAME="${TRIPP_NAME:-tripp-lite-ups}"
echo "======================================================"
echo " NUT UPS Status on $(hostname)"
echo "======================================================"
echo ""
echo "--- Services ---"
for svc in "nut-driver@${APC_NAME}" "nut-driver@${TRIPP_NAME}" nut-server nut-monitor; do
if systemctl list-unit-files "$svc" >/dev/null 2>&1; then
printf " %-42s " "$svc"
systemctl is-active "$svc" 2>/dev/null || echo "(unknown)"
fi
done
for ups in "$APC_NAME" "$TRIPP_NAME"; do
echo ""
echo "--- ${ups} ---"
if upsc "${ups}@localhost" >/tmp/.nutstatus.$$ 2>&1; then
awk -v u="$ups" '
BEGIN{printf " %s\n", u}
/^battery\.charge:/ {printf " battery.charge: %s\n", $3}
/^battery\.runtime:/ {printf " battery.runtime: %ss (%.0f min)\n", $3, $3/60}
/^battery\.voltage:/ {printf " battery.voltage: %s\n", $3}
/^ups\.status:/ {printf " ups.status: %s\n", $3}
/^ups\.load:/ {printf " ups.load: %s%%\n", $3}
/^ups\.power:/ {printf " ups.power: %s\n", $3}
/^ups\.realpower:/ {printf " ups.realpower: %s W\n", $3}
/^input\.voltage:/ {printf " input.voltage: %s\n", $3}
/^output\.voltage:/ {printf " output.voltage: %s\n", $3}
/^ups\.model:/ {printf " ups.model: %s\n", $3}
/^ups\.serial:/ {printf " ups.serial: %s\n", $3}
/^device\.mfr:/ {printf " device.mfr: %s\n", $3}
' /tmp/.nutstatus.$$
echo " (full dump: upsc ${ups}@localhost)"
else
echo " NOT RESPONDING:"
sed 's/^/ /' /tmp/.nutstatus.$$
fi
rm -f /tmp/.nutstatus.$$
done
echo ""
echo "--- upsd LISTEN sockets ---"
ss -ltnp 2>/dev/null | grep -E "3493|nut" | sed 's/^/ /' || echo " (upsd not listening on 3493)"
echo ""
echo "======================================================"