Files
PFVCluster/dcinfra/ups/README.md
T
2026-08-01 15:45:23 -05:00

211 lines
8.6 KiB
Markdown

# UPS Management (NUT — Network UPS Tools)
Centralized UPS monitoring for the server room via
[NUT](https://networkupstools.org/), running on **pfv-tsys1**. USB HID UPS
units feed one `upsd` network server; Home Assistant polls it over Tailscale for
real-time power/load/runtime tracking, and a local `upsmon` shuts the hypervisor
down gracefully when battery is low.
> **Why NUT (not apcupsd)?** Two different UPS brands (APC + Tripp Lite) must be
> covered. `apcupsd` only supports APC, so it would require a second daemon
> stack. NUT's `usbhid-ups` driver speaks to **both** via the USB HID Power
> Device class, and Home Assistant ships a first-class NUT integration.
## Hardware
| UPS | Model | VID:PID | USB Serial | Status |
|-----|-------|---------|------------|--------|
| **APC** | Smart-UPS C 1500 (FW 02.2) | `051d:0003` | `AS1213210423` | **LIVE** |
| **Tripp Lite** | UPS (HID PDC) | `09ae:3016` | `2352CVLSM871900694` | **Blocked** — see below |
## Current State (2026-07-30)
### APC Smart-UPS C 1500 — OPERATIONAL
Fully reporting via `usbhid-ups` + `APC HID 0.100` subdriver. Data validated:
```
battery.charge: 100 battery.runtime: 1800 battery.voltage: 27.4
ups.status: OL ups.load: (via HA) ups.model: Smart-UPS C 1500
```
### Tripp Lite UPS — BLOCKED (hardware issue)
The driver finds the device, matches the `TrippLite HID 0.85` subdriver, claims
the interface, and reads the HID descriptor — but **fails reading the 878-byte
HID Report Descriptor** (`Resource temporarily unavailable` / EAGAIN after 5s).
The driver is masked to prevent restart-loop spam.
USB descriptors (manufacturer, product, serial) are readable via `lsusb -v` and
`nut-scanner`, but the bulk control transfer for the full report descriptor
times out. Likely causes:
1. **USB hub** — the Tripp Lite is behind a Genesys Logic hub (`05e3:0608`).
Try plugging directly into a motherboard USB port.
2. **USB cable** — try a high-quality data cable (not charge-only).
3. **UPS firmware** — the USB controller may not properly implement all HID
endpoints.
**To retry after physical reseat:**
```bash
# On pfv-tsys1:
systemctl unmask nut-driver@tripp-lite-ups
systemctl start nut-driver@tripp-lite-ups
upsc tripp-lite-ups@localhost
```
## Architecture
```
pfv-tsys1 (192.168.3.11 / Tailscale 100.121.189.98)
├─ APC Smart-UPS C 1500 ──┐
└─ Tripp Lite UPS (masked) ──┤ USB HID
nut-driver@apc-smartups-c1500 (usbhid-ups)
upsd :3493 (LISTEN 127.0.0.1 + Tailscale + LAN)
▼ ▼
upsmon (local) Home Assistant (NUT integration)
graceful shutdown via LAN 192.168.3.11 (HAOS can't
route to Tailscale IPs)
```
- **Driver layer** — `usbhid-ups` process, pinned by USB serial. Debian uses
templated `nut-driver@<upsname>.service` units managed by
`nut-driver-enumerator`.
- **Server layer** — `upsd` exposes UPS data on TCP 3493 (localhost + Tailscale
+ LAN). Clients authenticate via `upsd.users`.
- **Monitor layer** — `upsmon` runs locally as `master` to trigger
`SHUTDOWNCMD` (`/sbin/shutdown -h now`) when a UPS reports `LOWBATT`.
- **Home Assistant** — native NUT integration connects to `upsd` over Tailscale
and exposes `ups.load`, `battery.runtime`, `ups.status`, etc. as sensors.
### Key deployment lesson: udev must cover raw USB devices
The `usbhid-ups` driver opens `/dev/bus/usb/BBB/DDD` (raw USB device files),
**not** `/dev/hidraw*`. After calling `setuid(111)` to drop to the `nut` user,
it needs write access to those raw USB files. The udev rule must match
`SUBSYSTEM=="usb"` by vendor/product ID to set `GROUP="nut"` — matching only
`hidraw` is insufficient. See `/etc/udev/rules.d/99-nut-ups.rules`.
## Scripts
NUT host scripts run on pfv-tsys1 via `tests/remote.sh`:
```bash
# Idempotent install + configure (safe to re-run):
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/setup.sh
# Discover USB UPS + NUT state (read-only diagnostic):
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/discover.sh
# Query UPS data + service health:
PROX_HOST=pfv-tsys1 bash tests/remote.sh prox-file ups/status.sh
```
The HA integration script runs from your workstation (needs HA API access):
```bash
# Add the NUT integration to Home Assistant (idempotent):
bash ups/setup-ha-nut.sh
```
`setup.sh` accepts environment overrides for serials/VIDs/PIDs/usernames, so it
can be repurposed for other hosts or UPS units. Passwords for `monuser` and
`homeassistant` are auto-generated on first run and reused on subsequent runs
(stored in `/etc/nut/upsd.users`).
Set `TRIPP_ENABLED=0` to skip the Tripp Lite entirely (useful if it's physically
unplugged).
## Configuration files on pfv-tsys1
| File | Purpose |
|------|---------|
| `/etc/udev/rules.d/99-nut-ups.rules` | Grant nut group rw on raw USB + hidraw devices (both subsystems) |
| `/etc/nut/ups.conf` | `usbhid-ups` device(s), pinned by serial + subdriver |
| `/etc/nut/upsd.conf` | `LISTEN 127.0.0.1` + `LISTEN <tailscale>` + `LISTEN <lan>` on port 3493 |
| `/etc/nut/upsd.users` | `monuser` (master) + `homeassistant` (read-only) credentials |
| `/etc/nut/upsmon.conf` | Local master monitor + `SHUTDOWNCMD` |
| `/etc/nut/nut.conf` | `MODE=netserver` |
## Home Assistant integration
The NUT integration is added automatically by `setup-ha-nut.sh`, which drives
HA's REST config-flow API. It is idempotent (skips if the entry exists).
```bash
# Prerequisites: create token + password files (one-time):
mkdir -p ~/.config/pfvcluster
# HA → Profile → Long-Lived Access Tokens → Create Token:
echo -n 'YOUR_HA_TOKEN' > ~/.config/pfvcluster/ha-token
# Password is in /etc/nut/upsd.users on pfv-tsys1 (the homeassistant user):
echo -n 'YOUR_NUT_PASS' > ~/.config/pfvcluster/nut-password
chmod 600 ~/.config/pfvcluster/{ha-token,nut-password}
# Run:
bash ups/setup-ha-nut.sh
```
### Why LAN IP, not Tailscale
upsd listens on **both** the Tailscale IP (`100.121.189.98`) **and** the LAN IP
(`192.168.3.11`). The HA NUT integration uses the **LAN IP** because HAOS runs
Tailscale as an isolated add-on container — the HA core container cannot route
to Tailscale IPs. Since pfv-bms (HA, `192.168.3.12`) and pfv-tsys1 (`192.168.3.11`)
share the same vmbr0 bridge, LAN connectivity is instant and reliable.
### Manual UI alternative
In Home Assistant → **Settings → Devices & Services → Add Integration → NUT**:
| Field | Value |
|-------|-------|
| Host | `192.168.3.11` (LAN — HAOS can't reach Tailscale IPs from the HA container) |
| Port | `3493` |
| Username | `homeassistant` |
| Password | *(stored in `/etc/nut/upsd.users` on pfv-tsys1)* |
| UPS | `apc-smartups-c1500` |
### Live sensors
HA exposes UPS data as sensors (prefix `sensor.apc_smartups_c1500_`):
`battery_charge`, `status` (Online/On Battery), `status_data` (OL/OB/DISCHRG).
Additional sensors (load, runtime, voltage) populate as the UPS reports them.
## Daily operations
From pfv-tsys1 (or any tailnet host with NUT client installed):
```bash
# List UPS units served by upsd
upsc -l pfv-tsys1
# Full variable dump for one UPS
upsc apc-smartups-c1500@pfv-tsys1
# Battery runtime (the only runtime/charge data this UPS exposes)
upsc apc-smartups-c1500@pfv-tsys1 battery.runtime
```
## Notes
- **No USB passthrough to the HA VM.** Keeping the UPS on the host preserves
hypervisor graceful-shutdown capability and matches the `powerman/` pattern
(PDU managed on the host where the adapter physically lives).
- **No `ups.load` / `ups.realpower` on this UPS (FW 02.2, mfg 2012):** The
APC Smart-UPS C 1500 does not expose load or power data over USB HID.
Both NUT `usbhid-ups` and `apcupsd` (USB mode, tested 2026-07-30) read the
same HID descriptor — the variable simply isn't there. This means the HA
NUT integration provides **battery/runtime/status sensors only**, not
wattage for the Energy Dashboard.
- **apcupsd test note:** Debian's `apcupsd` package conflicts with
`nut-server` (mutually exclusive). apcupsd USB mode returned `COMMLOST`
even before we could check load. The APC Smart Serial protocol (serial
cable, AP940-1524C, ~$30) DOES report load%, but this requires a serial
port on the UPS and on the host.
- **Energy Dashboard path:** A smart plug (Shelly Plug S / TP-Link Kasa,
~$15-25) on the UPS output reports real watts natively and feeds the
Energy Dashboard with zero UPS-driver hacking. The NUT sensors remain
valuable for outage detection and graceful-shutdown automations.