release PR template: CR deep-link line per CR-gating ruling

GLPI CR reference now a required field (dev-only PRs declare no-CR).
Standard: https://projects.knownelement.com/issues/820
Ticket: https://projects.knownelement.com/issues/811

.gitea/pull_request_template.md

@@ -0,0 +1,21 @@
+## Release PR (dev -> release)
+
+Merging this deploys to production (pfv-bms restarts if packages/** or
+configuration.yaml changed).
+
+### Change management (ruling 2026-09-06 — mandatory)
+- GLPI CR deep link (required): https://cmdb.knownelement.com/front/change.form.php?id=N
+  - Dev-only PRs: state `no-CR (dev-only)` explicitly.
+
+### Gate checklist (branch protection enforces the mechanical parts)
+- [ ] CI green on the head commit (yamllint + secret scan + SDLC suite)
+- [ ] CR filed in GLPI and referenced above (production changes)
+- [ ] Adversarial pass done: what breaks if this is wrong? what watches for it?
+- [ ] post-deploy validation (#778) will run on restart — expected entity changes noted below
+
+### Reviewer (founder)
+- [ ] Diff reviewed
+- [ ] Approve to release
+
+Expected entity/automation changes: <list or "none">
+Rollback: git revert <sha> on dev, re-PR, or push previous release sha directly.

.gitea/workflows/ci.yml

@@ -0,0 +1,28 @@
+# pfv-bms CI [#784 #778] — fleet standard per Discourse "CI/CD: Gitea Actions".
+# Runs on dev (pre-release gate) and release (final gate before live pull).
+name: ci
+on:
+  push:
+    branches: [dev, release]
+jobs:
+  lint:
+    runs-on: ultix
+    container:
+      image: node:20-bookworm
+    steps:
+      - uses: actions/checkout@v4
+      - name: yamllint
+        run: |
+          apt-get update -qq >/dev/null && apt-get install -qq -y python3-pip >/dev/null && pip install --quiet --break-system-packages yamllint
+          yamllint -c .yamllint .
+      - name: secret scan
+        run: |
+          if grep -rInE "BEGIN (RSA |OPENSSH |EC |DSA )?PRIVATE KEY|aws_secret_access_key *=|AKIA[0-9A-Z]{16}" --exclude-dir=.git .; then
+            echo "::error::secret material committed"; exit 1
+          else
+            echo "secret scan clean"
+          fi
+      - name: sdlc regression tests
+        run: |
+          chmod +x tests/test_sdlc.sh
+          bash tests/test_sdlc.sh

.yamllint

@@ -0,0 +1,22 @@
+# HA-config-aware yamllint [#784]: keep the checks that catch real bugs
+# (duplicates, structure), relax the stylistic ones HA templates trip.
+extends: default
+ignore: |
+  docs/
+  secrets.yaml
+  *.db
+rules:
+  line-length: disable
+  comments: disable
+  comments-indentation: disable
+  document-start: disable
+  truthy:
+    check-keys: false
+  indentation:
+    spaces: consistent
+    indent-sequences: consistent
+  key-duplicates: enable
+  new-line-at-end-of-file: disable
+  trailing-spaces: disable
+  empty-lines:
+    max-end: 2

README.md

@@ -134,16 +134,23 @@ apps.
 
 ## 🚀 Deployment pipeline
 
+**Branch model (2026-09-04):** `dev` = WIP integration — the pipeline
+IGNORES it. `release` = what the box runs — pushing release IS the deploy.
+`master` is frozen (pre-branch-model history). A PR process with AI +
+human reviewers will gate dev → release (founder, upcoming).
+
 ```mermaid
 flowchart LR
-    PUSH[📝 git push] --> HOOK[gitea webhook<br/>fast path]
+    PUSH[📝 push to dev] --> NOOP[🚫 pipeline ignores dev]
-    PUSH --> SHA[sha-watch<br/>safety net · 300 s poll]
+    REL[🚀 merge dev → release<br/>and push] --> HOOK[gitea webhook<br/>fast path]
+    REL --> SHA[sha-watch<br/>release sha · 300 s poll]
     HOOK & SHA --> CHECK{changed files?}
-    CHECK -->|packages/**| RESTART[♻️ core restart<br/>webhook triggers need it]
+    CHECK -->|configuration.yaml,<br/>packages/**| RESTART[♻️ core restart<br/>webhook triggers need it]
     CHECK -->|other YAML| RELOAD[⚡ reload only<br/>~5 min to live]
 ```
 
-Gate before pushing `packages/**`: `ha core check` must pass on-box.
+Gate before pushing `release` (and after any `packages/**` change on
+`dev`): `ha core check` must pass on-box.
 Measured end-to-end: 4 m 52 s commit → live.
 
 ## 🗺️ Repo map

STATUS.md

@@ -0,0 +1,18 @@
+# STATUS — point-in-time agent state (sparse; history lives in Redmine)
+
+Updated: 2026-09-06 ~07:10 CDT · dev @ aa8d788 · release @ 83b32d5
+
+| ticket | tier | state | item |
+|---|---|---|---|
+| #811 | — | ⛔ | NeedsInput: merge PR #3, PVE tokens (#790), beta onboarding click, rotations |
+| #344 | prod | 🔄 | release PR #3 open, CI green, review gate passed (FAIL→fixed→dispositioned) |
+| #790 | dev | ✅ | shutdown chain DRY RUN, level-based re-polls; exec wiring waits on PVE tokens |
+| #628 | docs | ✅ | restore runbook + baseline backup 34a05d70 off-box on ultix |
+| #781 | beta | 🔄 | VM 53110 @ 192.168.3.240, HA up, owner created; onboarding form + config sync left |
+| #813 | — | ⏳ | CI check_config job parked — needs UI job log, recipe in ticket |
+| #345 | prod | 🔄 | rest_command verified registered; swap-back = founder re-test call |
+
+## Inbox
+- 2026-09-06: tsys6 trend false-positive fix — watch trend sensors 24h post-deploy
+- 2026-09-06: stlpc-artroom dead all morning — policy ask in #811 item 8
+- 2026-09-06: runner container got external SIGTERM 11:35 UTC, restarted 11:37 — OAM check

automations.yaml

@@ -4,7 +4,7 @@
   - trigger: state
     entity_id:
     - sensor.tsys1_ups_status
-    - sensor.tsys9_ups_status
+    - sensor.tsys1_triplite_status
   conditions:
   - condition: template
     value_template: '{{ ''On Battery'' in states(trigger.entity_id) }}'
@@ -16,7 +16,6 @@
       - notify.allthes_ipad
     data:
       title: POWER OUTAGE
-      clickAction: /lovelace-pfv-server-room/ups
       url: /lovelace-pfv-server-room/ups
       message: '{{ state_attr(trigger.entity_id, ''friendly_name'') }} lost utility
         power and is ON BATTERY at {{ states(''sensor.'' ~ trigger.entity_id.split(''.'')[1]
@@ -28,6 +27,7 @@
   - trigger: numeric_state
     entity_id:
     - sensor.tsys1_ups_battery_charge
+    - sensor.tsys1_triplite_battery_charge
     below: 40
   conditions:
   - condition: template
@@ -41,7 +41,6 @@
       - notify.allthes_ipad
     data:
       title: UPS BATTERY LOW
-      clickAction: /lovelace-pfv-server-room/ups
       url: /lovelace-pfv-server-room/ups
       message: '{{ trigger.entity_id }} at {{ states(trigger.entity_id) }}% while
         on battery. Start graceful shutdown of VMs before hypervisors (#455 ordering).'
@@ -52,7 +51,7 @@
   - trigger: state
     entity_id:
     - sensor.tsys1_ups_status
-    - sensor.tsys9_ups_status
+    - sensor.tsys1_triplite_status
     to: unavailable
     for: 00:05:00
   actions:
@@ -63,22 +62,42 @@
       - notify.allthes_ipad
     data:
       title: UPS COMMS LOST
-      clickAction: /lovelace-pfv-server-room/ups
       url: /lovelace-pfv-server-room/ups
       message: '{{ trigger.entity_id }} unreachable for 5 minutes (nut-server or network
         path down; tsys1 also hosts pfv-bms itself).'
   mode: single
 - id: pfv_plant_ups_runtime_low
   alias: PFV plant - UPS runtime low while on battery
-  description: On battery and runtime burned below 10 min — the graceful
+  description: >-
-    shutdown window is closing. VMs before hypervisors per #455.
+    On battery and runtime burned below 10 min — the graceful shutdown
+    window is closing. VMs before hypervisors per #455 ordering. Covers
+    BOTH UPSes (APC + Tripp Lite — the Tripp Lite gap closed 2026-09-06);
+    the status check maps each runtime sensor to its own UPS status.
   triggers:
   - trigger: numeric_state
-    entity_id: sensor.pfv_ups_runtime_minutes
+    entity_id:
+    - sensor.pfv_ups_runtime_minutes
+    - sensor.pfv_tripp_lite_runtime_minutes
     below: 10
+  # 5-min level re-check: numeric_state alone fires only on the threshold
+  # CROSSING — a blip at that minute (or an HA restart mid-outage) would
+  # silence the page for the whole discharge. Level-based polling turns
+  # this into an escalation ladder while the outage persists.
+  - trigger: time_pattern
+    minutes: /5
   conditions:
   - condition: template
-    value_template: "{{ states('sensor.tsys1_ups_status') not in ['Online', 'unknown', 'unavailable'] }}"
+    value_template: >-
+      {% set ns = namespace(hit=false) %}
+      {% for rt, st in [
+           ('sensor.pfv_ups_runtime_minutes', 'sensor.tsys1_ups_status'),
+           ('sensor.pfv_tripp_lite_runtime_minutes', 'sensor.tsys1_triplite_status')] %}
+        {% if is_number(states(rt)) and (states(rt) | float) < 10
+              and states(st) not in ['Online', 'OL', 'unknown', 'unavailable'] %}
+          {% set ns.hit = true %}
+        {% endif %}
+      {% endfor %}
+      {{ ns.hit }}
   actions:
   - action: notify.send_message
     target:
@@ -87,9 +106,8 @@
       - notify.allthes_ipad
     data:
       title: UPS RUNTIME LOW
-      clickAction: /lovelace-pfv-server-room/ups
       url: /lovelace-pfv-server-room/ups
-      message: On battery {{ states('sensor.pfv_ups_runtime_minutes') }} min left — start graceful shutdown now (#455 ordering).
+      message: '{{ state_attr(trigger.entity_id, ''friendly_name'') }}: {{ states(trigger.entity_id) }} min left — start graceful shutdown now (#455 ordering).'
   mode: single
 - id: pfv_plant_ups_power_restored
   alias: PFV plant - utility power restored
@@ -97,7 +115,7 @@
   - trigger: state
     entity_id:
     - sensor.tsys1_ups_status
-    - sensor.tsys9_ups_status
+    - sensor.tsys1_triplite_status
   conditions:
   - condition: template
     value_template: "{{ trigger.from_state is not none and 'On Battery' in trigger.from_state.state\n
@@ -110,7 +128,6 @@
       - notify.allthes_ipad
     data:
       title: Power restored
-      clickAction: /lovelace-pfv-server-room/ups
       url: /lovelace-pfv-server-room/ups
       message: '{{ state_attr(trigger.entity_id, ''friendly_name'') }} is back ONLINE.'
   mode: single
@@ -188,39 +205,61 @@
       - notify.allthes_ipad
     data:
       title: HIGH TEMPERATURE
-      clickAction: /lovelace-pfv-server-room/temps
       url: /lovelace-pfv-server-room/temps
       message: '{{ trigger.entity_id }} is {{ states(trigger.entity_id) }}°F (class
         threshold exceeded). Check cooling/airflow.'
   mode: single
 - id: pfv_plant_sensor_watchdog
   alias: PFV plant - sensor watchdog (unavailable sensors)
-  description: HA is the environmental brain (#344) — a silent sensor is an outage.
+  description: >
-    Re-alert suppressed by 4h cooldown via last_notified input_text if present.
+    HA is the environmental brain (#344) — a silent sensor is an outage.
+    Fleet-wide sweep (every pfv_* sensor, broader than the 5-min critical
+    watchdog). PUSHES ONLY when sensors are actually dead, and re-pages a
+    still-dead fleet at most every 4h — the unconditional 30-min page was
+    retired 2026-09-06 after live-fire alert fatigue. All-OK cycles log
+    quietly; the Kuma dead-man heartbeat (#792) owns liveness signaling.
+  mode: single
+  max_exceeded: silent
   triggers:
   - trigger: time_pattern
     minutes: /30
   actions:
-  - action: notify.send_message
+  - variables:
-    target:
+      fleet: "{{ states.sensor | selectattr('entity_id', 'match', 'sensor\\.pfv_') | list }}"
-      entity_id:
+      dead: "{{ fleet | selectattr('state', 'in', ['unavailable', 'unknown']) | list }}"
-      - notify.ultix_sidecar_new
+      # 4h cooldown: page again only if the last page is older than 4h
-      - notify.allthes_ipad
+      # (or we never paged). Fresh HA start (no restore) counts as never.
-    data:
+      # as_timestamp math — aware/naive datetime subtraction throws in Jinja.
-      title: SENSOR WATCHDOG
+      notify_gate: >-
-      clickAction: /lovelace-pfv-server-room/overview
+        {{ states('input_datetime.pfv_plant_watchdog_last_notify') in ['unknown', 'unavailable', 'none']
-      url: /lovelace-pfv-server-room/overview
+           or as_timestamp(now()) - as_timestamp(states('input_datetime.pfv_plant_watchdog_last_notify')) > 14400 }}
-      message: >-
+  - if:
-        {% set fleet = states.sensor | selectattr('entity_id', 'match', 'sensor\.pfv_')
+    - condition: template
-          | list %}
+      value_template: "{{ dead | length > 0 and notify_gate }}"
-        {% set dead = fleet | selectattr('state', 'in', ['unavailable', 'unknown']) | list %}
+    then:
-        {% if dead | length > 0 %}
+    - action: notify.send_message
+      target:
+        entity_id:
+        - notify.ultix_sidecar_new
+        - notify.allthes_ipad
+      data:
+        title: SENSOR WATCHDOG
+        url: /lovelace-pfv-server-room/overview
+        message: >-
           {{ dead | length }} of {{ fleet | length }} fleet sensors silent:
           {{ dead | map(attribute='entity_id') | join(', ') }}
-        {% else %}
+    - action: input_datetime.set_datetime
-          all {{ fleet | length }} fleet sensors OK
+      target:
-        {% endif %}
+        entity_id: input_datetime.pfv_plant_watchdog_last_notify
-  mode: single
+      data:
+        datetime: "{{ now().strftime('%Y-%m-%d %H:%M:%S') }}"
+    else:
+    - action: logbook.log
+      data:
+        name: Plant watchdog
+        message: >-
+          Sweep: all {{ fleet | length }} fleet sensors answering
+          ({{ dead | length }} silent, cooldown {{ 'active' if not notify_gate else 'n/a' }}).
 - id: pfv_plant_temp_rising_fast
   alias: PFV plant - temperature rising fast (rate-of-change)
   description: Catches dead AC minutes before the 104F threshold trips.
@@ -242,11 +281,10 @@
       - notify.allthes_ipad
     data:
       title: TEMP RISING FAST
-      clickAction: /lovelace-pfv-server-room/temps
       url: /lovelace-pfv-server-room/temps
       message: '{{ trigger.entity_id }} climbing ~10F/15min — check cooling
         before the 104F threshold. (rate-of-change early warning)'
-# Deploy pipeline [#734]: gitea repo webhook (master pushes) -> forced pull
+# Deploy pipeline [#734]: gitea repo webhook (release pushes) -> forced pull
 # -> minimal reload. Core restarts ONLY when configuration.yaml or a
 # packages platform-sensor file changed; everything else reloads with no
 # bounce. The git_pull add-on's auto_restart is disabled - this webhook
@@ -254,7 +292,7 @@
 - id: pfv_deploy_git_webhook
   alias: PFV deploy - git push to pull + minimal reload
   description: >-
-    Gitea webhook on master pushes. Forces a git pull now, then
+    Gitea webhook on release pushes. Forces a git pull now, then
     homeassistant.reload_all unless files that require a core restart
     changed. Replaces the git_pull add-on timer auto-restart [#734].
   mode: queued
@@ -269,9 +307,9 @@
   actions:
   - if:
     - condition: template
-      value_template: "{{ trigger.json.ref | default('') != 'refs/heads/master' }}"
+      value_template: "{{ trigger.json.ref | default('') != 'refs/heads/release' }}"
     then:
-    - stop: "deploy webhook: non-master ref {{ trigger.json.ref | default('') }} ignored"
+    - stop: "deploy webhook: non-release ref {{ trigger.json.ref | default('') }} ignored"
   - variables:
       changed_files: >-
         {{ (trigger.json.commits | default([]) | map(attribute='added') | flatten | list)
@@ -337,7 +375,7 @@
       data:
         title: PFV deploy complete
         message: "Pulled + reloaded YAML at {{ now().strftime('%H:%M:%S') }} - no core restart. Files: {{ changed_files | join(', ') | truncate(120) }}"
-# Deploy pipeline safety net [#734]: polls gitea master HEAD (rest sensor
+# Deploy pipeline safety net [#734]: polls gitea release HEAD (rest sensor
 # in packages/deploy_pipeline.yaml) and applies pulls reload-only. The
 # gitea webhook is the fast path once gitea's SSRF allowlist permits
 # pfv-bms.knel.net; until then this net catches every push within 5 min
@@ -346,7 +384,7 @@
 - id: pfv_deploy_sha_watch
   alias: PFV deploy - gitea sha watch safety net
   description: >-
-    Master HEAD sha changed -> pull + CORE RESTART. Interim deploy
+    Release HEAD sha changed -> pull + CORE RESTART. Interim deploy
     trigger while gitea webhook delivery is blocked by its SSRF host
     allowlist [#734]. Restart (not reload) is REQUIRED: automation
     reload loses webhook-trigger registration - found live 2026-09-03
@@ -354,12 +392,12 @@
     scan ingest, this pipeline's own fast path) silently stop firing
     while the endpoint still returns HTTP 200. Core restart
     re-registers them. Accepted trade-off: ~1 min HA outage per
-    master deploy.
+    release deploy.
   mode: single
   max_exceeded: silent
   triggers:
   - trigger: state
-    entity_id: sensor.pfv_deploy_gitea_master_head
+    entity_id: sensor.pfv_deploy_gitea_release_head
   conditions:
   - condition: template
     value_template: >-
@@ -383,7 +421,7 @@
     data:
       title: PFV deploy (sha watch)
       message: >-
-        New master HEAD {{ states('sensor.pfv_deploy_gitea_master_head')[:7] }}
+        New release HEAD {{ states('sensor.pfv_deploy_gitea_release_head')[:7] }}
         pulled. Restarting HA core (webhook re-registration) - back in
         ~1 min.
   - action: homeassistant.restart
@@ -518,10 +556,193 @@
       - notify.allthes_ipad
     data:
       title: TEMPERATURE RECOVERED
-      clickAction: /lovelace-pfv-server-room/temps
       url: /lovelace-pfv-server-room/temps
       message: >-
         {{ state_attr(trigger.entity_id, 'friendly_name') | default(trigger.entity_id) }}
         back to {{ states(trigger.entity_id) }}°F and holding — was
         {{ trigger.from_state.state }}°F when the alert cleared (tier fire line {{ fire }}°F).
   mode: single
+- id: pfv_watchdog_stale_entities
+  alias: PFV watchdog - stale entity auto-heal
+  description: >
+    Detects silent poller wedges (2026-09-04: snmp tsys9 went 12h
+    undetected; NUT APC 3h) and auto-heals config-entry platforms via
+    homeassistant.reload_config_entry — no restart needed. Escalates to
+    notify when reload does not recover. Auto core restart ONLY while
+    input_boolean.pfv_watchdog_auto_restart is ON and entities stay
+    stale — restart costs ~1 min downtime and re-registers webhooks
+    (#345, #789). v1 notify repeats each cycle while stale (visible
+    alarm); throttle in v2.
+  mode: single
+  max_exceeded: silent
+  triggers:
+  - trigger: time_pattern
+    minutes: /5
+  variables:
+    checks:
+    - [sensor.pfv_tsys1_rack_temperature, 20]
+    - [sensor.pfv_tsys1_drive_temperature, 30]
+    - [sensor.pfv_tsys3_cpu_temperature, 20]
+    - [sensor.pfv_tsys4_ambient_temperature, 20]
+    - [sensor.pfv_tsys6_cpu_temperature, 20]
+    - [sensor.pfv_tsys6_drac_ambient, 30]
+    - [sensor.pfv_tsys7_cpu_temperature, 20]
+    - [sensor.pfv_tsys7_drac_inlet, 30]
+    - [sensor.pfv_tsys9_cpu_temperature, 20]
+    - [sensor.pfv_tsys9_drive_temperature, 30]
+    - [sensor.tsys1_ups_status, 15]
+    - [sensor.tsys1_triplite_status, 15]
+    - [sensor.pfv_deploy_gitea_release_head, 30]
+  actions:
+  - variables:
+      stale_list: >-
+        {% set ns = namespace(stale=[]) %}
+        {% for pair in checks %}
+          {% set s = states[pair[0]] %}
+          {% if s is not none and s.last_updated is defined
+                and (now() - s.last_updated).total_seconds() > pair[1] * 60 %}
+            {% set ns.stale = ns.stale + [pair[0]] %}
+          {% endif %}
+        {% endfor %}
+        {{ ns.stale }}
+  - condition: template
+    value_template: "{{ stale_list | length > 0 }}"
+  - repeat:
+      for_each: "{{ stale_list }}"
+      sequence:
+      - action: homeassistant.reload_config_entry
+        data:
+          entity_id: "{{ repeat.item }}"
+  - delay:
+      hours: 0
+      minutes: 2
+      seconds: 0
+  - variables:
+      still_stale: >-
+        {% set ns = namespace(stale=[]) %}
+        {% for pair in checks %}
+          {% set s = states[pair[0]] %}
+          {% if s is not none and s.last_updated is defined
+                and (now() - s.last_updated).total_seconds() > pair[1] * 60 %}
+            {% set ns.stale = ns.stale + [pair[0]] %}
+          {% endif %}
+        {% endfor %}
+        {{ ns.stale }}
+  - action: notify.send_message
+    target:
+      entity_id:
+      - notify.ultix_sidecar_new
+      - notify.allthes_ipad
+    data:
+      title: WATCHDOG
+      url: /lovelace-pfv-server-room/temps
+      message: >-
+        Stale after auto-reload: {{ still_stale | join(', ') }}.
+        Auto-restart is {{ 'ON' if is_state('input_boolean.pfv_watchdog_auto_restart','on')
+        else 'OFF (arm it on the watchdog switch if you want unattended restarts)' }}.
+  - if:
+    - condition: template
+      value_template: "{{ is_state('input_boolean.pfv_watchdog_auto_restart','on') }}"
+    then:
+    - delay:
+        hours: 0
+        minutes: 30
+        seconds: 0
+    - variables:
+        hard_stale: >-
+          {% set ns = namespace(stale=[]) %}
+          {% for pair in checks %}
+            {% set s = states[pair[0]] %}
+            {% if s is not none and s.last_updated is defined
+                  and (now() - s.last_updated).total_seconds() > pair[1] * 60 %}
+              {% set ns.stale = ns.stale + [pair[0]] %}
+            {% endif %}
+          {% endfor %}
+          {{ ns.stale }}
+    - if:
+      - condition: template
+        value_template: "{{ hard_stale | length > 0 }}"
+      then:
+      - action: notify.send_message
+        target:
+          entity_id:
+          - notify.ultix_sidecar_new
+          - notify.allthes_ipad
+        data:
+          title: WATCHDOG — restarting core
+          message: "Still stale after reload + 30 min: {{ hard_stale | join(', ') }}. Restarting HA core now."
+      - action: homeassistant.restart
+- id: pfv_post_deploy_validation
+  alias: PFV post-deploy validation
+  description: >
+    Runs after EVERY HA start (deploy restarts included): after a settle
+    delay, verifies the critical sensor set actually materialized and is
+    fresh. Catches bad deploys — missing templates, broken packages,
+    wedged pollers — and pages with specifics instead of waiting for a
+    human to notice (#778, #777). The watchdog then handles recovery.
+  mode: single
+  max_exceeded: silent
+  triggers:
+  - trigger: event
+    event_type: homeassistant_start
+  actions:
+  - delay:
+      hours: 0
+      minutes: 3
+      seconds: 0
+  - variables:
+      required:
+      - [sensor.pfv_tsys1_rack_temperature, 20]
+      - [sensor.pfv_tsys9_cpu_temperature, 20]
+      - [sensor.tsys1_ups_status, 15]
+      - [sensor.tsys1_triplite_status, 15]
+      - [sensor.pfv_ups_runtime_minutes, 30]
+      - [sensor.pfv_tripp_lite_runtime_minutes, 30]
+      - [sensor.pfv_deploy_gitea_release_head, 30]
+      - [automation.pfv_watchdog_stale_entities, 0]
+  - variables:
+      bad_list: >-
+        {% set ns = namespace(bad=[]) %}
+        {% for pair in required %}
+          {% set s = states[pair[0]] %}
+          {% if s is none or s.state in ['unknown', 'unavailable', 'none'] %}
+            {% set ns.bad = ns.bad + [pair[0] ~ ' (missing)'] %}
+          {% elif pair[1] > 0 and s.last_updated is defined
+                and (now() - s.last_updated).total_seconds() > pair[1] * 60 %}
+            {% set ns.bad = ns.bad + [pair[0] ~ ' (stale)'] %}
+          {% endif %}
+        {% endfor %}
+        {{ ns.bad }}
+  - if:
+    - condition: template
+      value_template: "{{ bad_list | length > 0 }}"
+    then:
+    - action: notify.send_message
+      target:
+        entity_id:
+        - notify.ultix_sidecar_new
+        - notify.allthes_ipad
+      data:
+        title: POST-DEPLOY VALIDATION FAILED
+        url: /lovelace-pfv-thermal
+        message: >-
+          After restart, these criticals are missing/stale:
+          {{ bad_list | join(', ') }}. Watchdog will retry recovery;
+          if this repeats, revert the last release (git revert + push).
+    - action: persistent_notification.create
+      data:
+        title: POST-DEPLOY VALIDATION FAILED
+        message: "Missing/stale: {{ bad_list | join(', ') }}"
+- id: pfv_kuma_heartbeat
+  alias: PFV Kuma heartbeat (dead-man)
+  description: >
+    Pushes OK to the out-of-band Uptime Kuma every 5 minutes (#792).
+    Kuma pages the founder directly if heartbeats stop — the alert path
+    that works when HA itself is the failure.
+  mode: single
+  max_exceeded: silent
+  triggers:
+  - trigger: time_pattern
+    minutes: /5
+  actions:
+  - action: shell_command.kuma_ha_heartbeat

configuration.yaml

@@ -20,6 +20,21 @@ prometheus:
 lovelace:
   mode: storage
   dashboards:
+    pfv-overview:
+      mode: yaml
+      title: Ops Overview
+      filename: dashboards/overview.yaml
+      icon: mdi:view-dashboard-outline
+    pfv-power:
+      mode: yaml
+      title: Power
+      filename: dashboards/power.yaml
+      icon: mdi:flash
+    pfv-thermal:
+      mode: yaml
+      title: Thermal
+      filename: dashboards/thermal.yaml
+      icon: mdi:thermometer-lines
     pfv-server-room:
       mode: yaml
       title: Server Room
@@ -34,3 +49,19 @@ lovelace:
       title: STLP
       filename: dashboards/stlp.yaml
       icon: mdi:home
+
+# Watchdog arm switch (#789): auto core restart on unresolved staleness.
+# Default OFF — reload_config_entry heals config-entry wedges without it.
+input_boolean:
+  pfv_watchdog_auto_restart:
+    name: Watchdog auto core restart
+    icon: mdi:restart-alert
+
+# 30-min plant watchdog cooldown (#344, 2026-09-06): re-pages a dead sensor
+# at most every 4h instead of every cycle. Set by the automation on notify.
+input_datetime:
+  pfv_plant_watchdog_last_notify:
+    name: Plant watchdog last notify
+    has_date: true
+    has_time: true
+    icon: mdi:bell-cancel

dashboards/overview.yaml

@@ -0,0 +1,79 @@
+title: Ops Overview
+views:
+  - title: Overview
+    type: sections
+    sections:
+      - type: grid
+        cards:
+          - type: markdown
+            content: >
+              ## Ops Overview
+
+              One screen: is anything on fire? Details live in
+              [Power](/lovelace-pfv-power), [Thermal](/lovelace-pfv-thermal),
+              [Server Room](/lovelace-pfv-server-room) and
+              [STLP](/lovelace-pfv-stlp).
+      - type: grid
+        cards:
+          - type: heading
+            heading: Room & cooling
+          - type: gauge
+            entity: sensor.pfv_tsys1_rack_temperature
+            name: Server room
+            min: 50
+            max: 110
+            severity:
+              green: 60
+              yellow: 85
+              red: 95
+          - type: entities
+            entities:
+              - sensor.pfv_garage_temperature
+              - sensor.pfv_artroom_temperature
+      - type: grid
+        cards:
+          - type: heading
+            heading: Power
+          - type: gauge
+            entity: sensor.tsys1_ups_battery_charge
+            name: APC charge
+            min: 0
+            max: 100
+            severity:
+              green: 60
+              yellow: 40
+              red: 20
+          - type: gauge
+            entity: sensor.tsys1_triplite_battery_charge
+            name: Tripp Lite charge
+            min: 0
+            max: 100
+            severity:
+              green: 60
+              yellow: 40
+              red: 20
+          - type: entities
+            entities:
+              - sensor.tsys1_ups_status
+              - sensor.tsys1_triplite_status
+              - sensor.pfv_ups_runtime_minutes
+              - sensor.pfv_tripp_lite_runtime_minutes
+      - type: grid
+        cards:
+          - type: heading
+            heading: Alerts & self-healing
+          - type: entities
+            entities:
+              - automation.pfv_plant_ups_on_battery
+              - automation.pfv_plant_ups_battery_low
+              - automation.pfv_plant_ups_communication_lost
+              - automation.pfv_plant_high_temperature
+              - automation.pfv_watchdog_stale_entity_auto_heal
+              - input_boolean.pfv_watchdog_auto_restart
+      - type: grid
+        cards:
+          - type: markdown
+            content: >
+              **Pipeline:** deploy sensor tracks `release` on KNEL/KNELBMS.
+              CI runs on every dev/release push (Actions tab in gitea).
+              Out-of-band paging: Uptime Kuma (Cloudron).

dashboards/power.yaml

@@ -0,0 +1,114 @@
+title: Power
+views:
+  - title: UPS detail
+    type: sections
+    sections:
+      - type: grid
+        cards:
+          - type: heading
+            heading: APC Smart-UPS C 1500 (battery-side USB HID)
+          - type: gauge
+            entity: sensor.tsys1_ups_battery_charge
+            name: Charge
+            min: 0
+            max: 100
+            severity:
+              green: 60
+              yellow: 40
+              red: 20
+          - type: gauge
+            entity: sensor.pfv_ups_runtime_minutes
+            name: Runtime (min)
+            min: 0
+            max: 60
+            severity:
+              green: 25
+              yellow: 15
+              red: 5
+      - type: grid
+        cards:
+          - type: entities
+            title: APC status
+            entities:
+              - sensor.tsys1_ups_status
+              - sensor.tsys1_ups_status_data
+              - sensor.tsys1_ups_battery_charge
+              - sensor.pfv_ups_runtime_minutes
+              - sensor.tsys1_ups_battery_runtime
+              - sensor.tsys1_ups_battery_voltage
+              - sensor.tsys1_ups_nominal_battery_voltage
+              - sensor.tsys1_ups_low_battery_setpoint
+              - sensor.tsys1_ups_beeper_status
+          - type: markdown
+            content: >
+              Serial AS1213210423 · mfr 2012-03 · fw UPS 02.2 · 24 V
+              nominal. USB HID exposes battery-side metrics only — no
+              input/output voltage or watts on this unit.
+      - type: grid
+        cards:
+          - type: heading
+            heading: Tripp Lite 1500 VA (moved from tsys9, #372)
+          - type: gauge
+            entity: sensor.tsys1_triplite_battery_charge
+            name: Charge
+            min: 0
+            max: 100
+            severity:
+              green: 60
+              yellow: 40
+              red: 20
+          - type: gauge
+            entity: sensor.tsys1_triplite_load
+            name: Load %
+            min: 0
+            max: 100
+            severity:
+              green: 70
+              yellow: 85
+              red: 95
+      - type: grid
+        cards:
+          - type: entities
+            title: Tripp Lite status
+            entities:
+              - sensor.tsys1_triplite_status
+              - sensor.tsys1_triplite_status_data
+              - sensor.tsys1_triplite_battery_charge
+              - sensor.tsys1_triplite_apparent_power
+              - sensor.tsys1_triplite_output_current
+              - sensor.tsys1_triplite_load
+              - sensor.pfv_tripp_lite_runtime_minutes
+              - sensor.tsys1_triplite_battery_runtime
+              - sensor.tsys1_triplite_input_voltage
+              - sensor.tsys1_triplite_output_voltage
+              - sensor.tsys1_triplite_nominal_power
+  - title: Trends
+    type: sections
+    sections:
+      - type: grid
+        cards:
+          - type: history-graph
+            title: UPS charge (7 days)
+            entities:
+              - sensor.tsys1_ups_battery_charge
+              - sensor.tsys1_triplite_battery_charge
+            hours_to_show: 168
+      - type: grid
+        cards:
+          - type: history-graph
+            title: Tripp Lite apparent power (24h)
+            entities:
+              - sensor.tsys1_triplite_apparent_power
+            hours_to_show: 24
+          - type: history-graph
+            title: UPS load % (24h)
+            entities:
+              - sensor.tsys1_triplite_load
+            hours_to_show: 24
+      - type: grid
+        cards:
+          - type: markdown
+            content: >
+              **Planned sections** (land with their data sources):
+              Emporia CT rental-bank feed (#623), whole-house kWh vs
+              solar production (#760), garage PDU circuits (#733).

dashboards/server-room.yaml

@@ -103,6 +103,46 @@ views:
               fw UPS 02.2 · PbAc, 24 V nominal · usbhid-ups via pfv-tsys1
               (NUT :3493). This unit reports no input/output voltage over
               USB HID — battery-side metrics only.
+      - type: grid
+        cards:
+          - type: gauge
+            entity: sensor.tsys1_triplite_battery_charge
+            name: Tripp Lite charge
+            min: 0
+            max: 100
+            severity:
+              green: 60
+              yellow: 40
+              red: 20
+          - type: gauge
+            entity: sensor.tsys1_triplite_load
+            name: Tripp Lite load %
+            min: 0
+            max: 100
+            severity:
+              green: 70
+              yellow: 85
+              red: 95
+      - type: grid
+        cards:
+          - type: entities
+            title: Tripp Lite status
+            entities:
+              - sensor.tsys1_triplite_status
+              - sensor.tsys1_triplite_status_data
+              - sensor.tsys1_triplite_battery_charge
+              - sensor.tsys1_triplite_apparent_power
+              - sensor.tsys1_triplite_output_current
+              - sensor.tsys1_triplite_load
+              - sensor.tsys1_triplite_input_voltage
+              - sensor.tsys1_triplite_output_voltage
+              - sensor.tsys1_triplite_battery_runtime
+          - type: markdown
+            content: >
+              **Tripp Lite UPS** · usbhid-ups via pfv-tsys1 (NUT :3493);
+              moved from tsys9 2026-09-04 (#372). Carries ~46% load
+              (~414 VA apparent / 1500 VA nominal) — mind runtime
+              before adding gear.
       - type: grid
         cards:
           - type: entities

dashboards/thermal.yaml

@@ -0,0 +1,119 @@
+title: Thermal
+views:
+  - title: Rooms
+    type: sections
+    sections:
+      - type: grid
+        cards:
+          - type: gauge
+            entity: sensor.pfv_tsys1_rack_temperature
+            name: Server room
+            min: 50
+            max: 110
+            severity:
+              green: 60
+              yellow: 85
+              red: 95
+          - type: history-graph
+            title: Server room (24h)
+            entities:
+              - sensor.pfv_tsys1_rack_temperature
+            hours_to_show: 24
+      - type: grid
+        cards:
+          - type: gauge
+            entity: sensor.pfv_garage_temperature
+            name: Garage
+            min: 30
+            max: 110
+            severity:
+              green: 40
+              yellow: 90
+              red: 100
+          - type: history-graph
+            title: Garage (24h)
+            entities:
+              - sensor.pfv_garage_temperature
+            hours_to_show: 24
+      - type: grid
+        cards:
+          - type: gauge
+            entity: sensor.pfv_artroom_temperature
+            name: Art room
+            min: 40
+            max: 110
+            severity:
+              green: 60
+              yellow: 85
+              red: 95
+          - type: history-graph
+            title: Art room (24h)
+            entities:
+              - sensor.pfv_artroom_temperature
+            hours_to_show: 24
+      - type: grid
+        cards:
+          - type: markdown
+            content: >
+              **Minisplit** — climate entity lands with the Carrier dongle
+              commissioning (#629). Manual setpoint tonight: 62°F, with the
+              oscillating fan improving distribution. Alert tiers:
+              ambient 104°F · drives 140°F · NVMe 150°F · PCH 160°F ·
+              GPU 185°F.
+  - title: Racks & silicon
+    type: sections
+    sections:
+      - type: grid
+        cards:
+          - type: entities
+            title: R3 · PowerEdge (DRAC)
+            entities:
+              - sensor.pfv_tsys6_cpu_temperature
+              - sensor.pfv_tsys6_drac_ambient
+              - sensor.pfv_tsys7_cpu_temperature
+              - sensor.pfv_tsys7_drac_inlet
+              - sensor.pfv_tsys7_drac_exhaust
+              - sensor.pfv_tsys7_drac_cpu1
+              - sensor.pfv_tsys7_drac_cpu2
+      - type: grid
+        cards:
+          - type: entities
+            title: R3 · Pis + Jetson
+            entities:
+              - sensor.pfv_svrpi_temperature
+              - sensor.pfv_subopi3_temperature
+              - sensor.pfv_subopi_dev_3_temperature
+              - sensor.pfv_subopi_dev_4_temperature
+              - sensor.pfv_jetson_temperature
+              - sensor.pfv_jetson_gpu_temperature
+      - type: grid
+        cards:
+          - type: entities
+            title: R5 · Laptops
+            entities:
+              - sensor.pfv_tsys3_cpu_temperature
+              - sensor.pfv_tsys3_gpu_temperature
+              - sensor.pfv_tsys3_nvme_temperature
+              - sensor.pfv_tsys5_nvme_temperature
+      - type: grid
+        cards:
+          - type: entities
+            title: R5 · Desktops / OptiPlex-class
+            entities:
+              - sensor.pfv_tsys1_rack_temperature
+              - sensor.pfv_tsys4_ambient_temperature
+              - sensor.pfv_tsys1_cpu_temperature
+              - sensor.pfv_tsys4_cpu_temperature
+              - sensor.pfv_tsys9_cpu_temperature
+              - sensor.pfv_tsys9_pch_temperature
+      - type: grid
+        cards:
+          - type: history-graph
+            title: CPU temperatures (24h)
+            entities:
+              - sensor.pfv_tsys1_cpu_temperature
+              - sensor.pfv_tsys3_cpu_temperature
+              - sensor.pfv_tsys6_cpu_temperature
+              - sensor.pfv_tsys7_cpu_temperature
+              - sensor.pfv_tsys9_cpu_temperature
+            hours_to_show: 24

docs/restore-runbook.md

@@ -0,0 +1,78 @@
+# pfv-bms restore runbook (#628)
+
+Three restore layers exist. Work **inside-out**: usually only layer 1 is
+needed. Restore drills run on the **dev lane only** — never on hosts
+running prod VMs (standing rule after the 2026-09-03 PBS drill that
+I/O-starved prod HA, #684 / t/298).
+
+## Layer 0 — what is NOT in git
+
+| Artifact | Where it lives | Covered by |
+|----------|----------------|------------|
+| `secrets.yaml` (tokens, webhook ids, unlock URL) | on-box `/config/secrets.yaml` | Layer 1 + 2 backups |
+| `.storage/` (dashboards-in-storage, registry, auth) | on-box `/config/.storage/` | Layer 1 + 2 backups |
+| Zigbee/HomeKit/BT pairings | on-box | Layer 1 + 2 backups |
+
+Git (this repo, `release` branch) is the source of truth for
+configuration **YAML only**. A fresh box restored from git still needs a
+`secrets.yaml` re-provisioned from the keeper's records (~/.creds notes
++ badge/webhook ids) before HA will start.
+
+## Layer 1 — HA-level backup (config + settings, small, fast)
+
+Taken on-box via the `ha` CLI; produces a self-contained `.tar`:
+
+```bash
+ssh -p 22222 root@pfv-bms.knel.net 'ha backups new --name "pfv-bms-$(date +%F)"'
+ssh -p 22222 root@pfv-bms.knel.net 'ha backups list'          # find slug
+ssh -p 22222 root@pfv-bms.knel.net 'ha backups info <slug>'   # verify size/state
+# OFF-BOX COPY (the on-box copy dies with the VM disk):
+scp -P 22222 'root@pfv-bms.knel.net:/mnt/data/supervisor/backup/<slug>.tar' /backup/destination/
+```
+
+HAOS automatic backups are CONFIRMED LIVE (daily "Automatic backup"
+entries observed 2026-09-05/06) — layer 1 has a baseline; the off-box
+copy remains the manual step.
+
+Restore (same or fresh HAOS box):
+
+```bash
+scp -P 22222 backup.tar root@pfv-bms.knel.net:/mnt/data/supervisor/backup/
+ssh -p 22222 root@pfv-bms.knel.net 'ha backups restore <slug> --type full'
+```
+
+Baseline on record: slug `34a05d70`, name
+`pfv-bms-baseline-2026-09-06-post-790`, 788 MB compressed, taken after
+the alert-hygiene + #790 batches (dev `c4ebb8a`). Off-box copy:
+`~/projects/KNEL/PhysicalPlant/pfv-bms-backups/` on ultix-streaming
+(sha256 6ff2fa6bd5f3dd2f2a47…).
+
+## Layer 2 — VM-level backup (prod PBS, nightly)
+
+- All prod VMs — **including VM 100 (pfv-bms) on pfv-tsys1** — back up
+  nightly to prod PBS, staggered 21:00–04:30 Central, throttled
+  60/30 MiB/s + job bwlimit 50 MiB/s (founder ruling t/321).
+- Restore: PVE web UI or `qmrestore` — **target a scratch VM on the dev
+  lane** (verify the restore boots there), never restore in place over
+  the live VM without a Kuma maintenance window.
+- Gap (open, OAM lane): pfv-tsys5 has no PBS job yet — beta/test VMs on
+  tsys5 are unprotected; fine for beta, not for anything promoted.
+
+## Layer 3 — config-from-git (last resort, fresh HAOS)
+
+1. Flash/install HAOS, complete onboarding, join Tailscale.
+2. `git clone git.knownelement.com:KNEL/KNELBMS` → copy tree to `/config`
+   at the `release` HEAD the fleet expects.
+3. Re-provision `secrets.yaml` (Layer 0 table) — from keeper records.
+4. `ha core check` before starting; then watch the first boot: the
+   post-deploy validation automation pages if criticals fail to
+   materialize (#778).
+5. Verify the trail: deploy sensor matches `release` HEAD, Kuma
+   heartbeat green, doorman roster sensors populated.
+
+## Related
+
+- Deploy pipeline + gate: README "Deployment pipeline"; `ha core check`
+  on-box before any release push.
+- Incident history: #684 (PBS drill), #780 (tsys1 storage), t/298
+  (Sep 3 wedge), t/320 (reload vs restart deploy semantics).

packages/deploy_pipeline.yaml

@@ -1,4 +1,4 @@
-# Deploy pipeline [#734]: gitea master-HEAD sha watcher. Feeds the
+# Deploy pipeline [#734]: gitea release-branch sha watcher. Feeds the
 # sha-watch deploy automation (reload-only apply). The gitea webhook
 # (gitea hook #10 -> pfv_deploy_git_webhook automation) is the fast path
 # with full restart mapping, but gitea cannot deliver webhooks to
@@ -6,16 +6,23 @@
 # (founder action via Cloudron file manager); this watcher is the
 # always-works safety net.
 rest:
-  - resource: https://git.knownelement.com/api/v1/repos/KNEL/pfv-bms/commits?limit=1
+  - resource: https://git.knownelement.com/api/v1/repos/KNEL/KNELBMS/commits?sha=release&limit=1
     headers:
       Authorization: !secret gitea_auth_header
       User-Agent: pfv-bms-deploy-watch
     scan_interval: 300
     verify_ssl: true
     sensor:
-      - name: PFV deploy gitea master head
+      - name: PFV deploy gitea release head
-        unique_id: pfv_deploy_gitea_master_head
+        unique_id: pfv_deploy_gitea_release_head
         value_template: "{{ value_json[0].sha if value_json is list and value_json | length > 0 else (value_json | string | truncate(90)) }}"
 
 # E2E self-deploy proof commit [#734] - if you can read this in /config,
 # the sha-watch deployed it autonomously.
+
+# Dead-man heartbeat (#792): pushes OK to the out-of-band Uptime Kuma
+# (Cloudron, status.knownelement.com) every 5 minutes. If HA stops
+# pushing — crash, wedge, power loss, network partition — Kuma pages
+# the founder's phone directly. No HA in the alert loop.
+shell_command:
+  kuma_ha_heartbeat: 'curl -sS -m 15 "{{ kuma_push_url }}"'

packages/doorman.yaml

@@ -7,6 +7,7 @@
 # PR gate). The roster is fetched live from the gitea API — NO git
 # mechanics, NO submodule (git_pull add-on lacks submodule support),
 # NO local copy: access-roster remains the single source of truth.
+# Reader/door contract (webhook JSON + unlock endpoint): KNEL/doorman README
 #
 # UNLOCK PATH (founder-approved 2026-09-03): valid badge + the
 # doorman_unlock_enabled input_boolean ON -> rest_command fires the
@@ -81,6 +82,18 @@ input_boolean:
     name: Doorman unlock armed
     icon: mdi:lock-open-variant
 
+shell_command:
+  # Unlock dispatch via curl — swapped from rest_command 2026-09-04: the
+  # rest_command integration silently failed to register on this box
+  # (404 on /api/services/rest_command despite valid config + secret).
+  # NOT reproducible: 2026-09-06 the integration + doorman_unlock service
+  # are registered again (suspect: transient state after a reload-only
+  # deploy window; rest_command now also exposes a reload service).
+  # curl stays as the verified-live dispatch path — swapping back to
+  # rest_command (drops the token from the process list) is gated on one
+  # armed badge re-test (founder call, questions-v1).
+  doorman_unlock: "curl -sS -m 10 '{{ doorman_unlock_url }}'"
+
 rest_command:
   # Fires the doorctl endpoint on pfvsvrpi (systemd socket-activated,
   # Tailscale-bound, token+source allowlisted; self-securing relay).
@@ -173,7 +186,7 @@ automation:
                       {{ is_state('input_boolean.doorman_unlock_enabled', 'on')
                          and not (trigger.json.local_unlock | default(false) | bool(false)) }}
                 then:
-                  - action: rest_command.doorman_unlock
+                  - action: shell_command.doorman_unlock
                     continue_on_error: true
                   - action: logbook.log
                     data:

packages/plant_snmp.yaml

@@ -542,27 +542,65 @@ sensor:
     method: left
     unique_id: pfv_garage_pdu_energy
 
-# --- Rate-of-change early warning (#682): +10°F/15min = AC trouble [#344] ---
+  # --- Smoothed ambient feeds for the trend sensors (2026-09-06) ---
+  # DRAC sensors quantize to whole °C and FLAP ~1.8°F between polls; raw-fed
+  # trend sensors false-tripped TEMP RISING FAST (05:02 CDT, tsys6). Trend
+  # inputs are 5-min time-weighted moving averages (legacy filter SENSOR
+  # platform — a top-level `filter:` key cannot be merged from packages).
+  # Real climbs survive the smoothing, quantization flapping does not.
+  - platform: filter
+    name: "PFV tsys1 rack temperature smoothed"
+    unique_id: pfv_tsys1_rack_temperature_smoothed
+    entity_id: sensor.pfv_tsys1_rack_temperature
+    filters:
+      - filter: time_simple_moving_average
+        window: "00:05"
+        precision: 2
+  - platform: filter
+    name: "PFV tsys6 DRAC ambient smoothed"
+    unique_id: pfv_tsys6_drac_ambient_smoothed
+    entity_id: sensor.pfv_tsys6_drac_ambient
+    filters:
+      - filter: time_simple_moving_average
+        window: "00:05"
+        precision: 2
+  - platform: filter
+    name: "PFV tsys7 DRAC inlet smoothed"
+    unique_id: pfv_tsys7_drac_inlet_smoothed
+    entity_id: sensor.pfv_tsys7_drac_inlet
+    filters:
+      - filter: time_simple_moving_average
+        window: "00:05"
+        precision: 2
+  - platform: filter
+    name: "PFV garage temperature smoothed"
+    unique_id: pfv_garage_temperature_smoothed
+    entity_id: sensor.pfv_garage_temperature
+    filters:
+      - filter: time_simple_moving_average
+        window: "00:05"
+        precision: 2
+
 binary_sensor:
   - platform: trend
     sensors:
       server_room_temp_rising:
-        entity_id: sensor.pfv_tsys1_rack_temperature
+        entity_id: sensor.pfv_tsys1_rack_temperature_smoothed
         sample_duration: 900
         min_gradient: 0.011
         unique_id: trend_server_room_rising
       tsys6_ambient_rising:
-        entity_id: sensor.pfv_tsys6_drac_ambient
+        entity_id: sensor.pfv_tsys6_drac_ambient_smoothed
         sample_duration: 900
         min_gradient: 0.011
         unique_id: trend_tsys6_ambient_rising
       tsys7_inlet_rising:
-        entity_id: sensor.pfv_tsys7_drac_inlet
+        entity_id: sensor.pfv_tsys7_drac_inlet_smoothed
         sample_duration: 900
         min_gradient: 0.011
         unique_id: trend_tsys7_inlet_rising
       garage_temp_rising:
-        entity_id: sensor.pfv_garage_temperature
+        entity_id: sensor.pfv_garage_temperature_smoothed
         sample_duration: 900
         min_gradient: 0.011
         unique_id: trend_garage_rising
@@ -800,6 +838,13 @@ template:
         state: >-
           {% set v = states('sensor.tsys1_ups_battery_runtime') %}
           {{ (v | float / 60) | round(0) if is_number(v) else none }}
+      - name: "PFV Tripp Lite runtime minutes"
+        unique_id: pfv_triplite_runtime_minutes
+        device_class: duration
+        unit_of_measurement: "min"
+        state: >-
+          {% set v = states('sensor.tsys1_triplite_battery_runtime') %}
+          {{ (v | float / 60) | round(0) if is_number(v) else none }}
       - name: "PFV tsys1 drive temperature"
         unique_id: pfv_tsys1_drive_temperature
         device_class: temperature

packages/ups_shutdown.yaml

@@ -0,0 +1,150 @@
+# ============================================================================
+# UPS graceful-shutdown chain (#790) — the #455 ordering, automated
+# ============================================================================
+# DESIGN (unchanged from the 2026-09-05 session spec):
+#   DRY_RUN is the DEFAULT and the shipped mode. The chain LOGS + PAGES the
+#   exact shutdown plan (#455: VMs before hypervisors, pfv-bms last of the
+#   VMs — the brain stays alive as long as possible). Execution (real
+#   qmshutdown / PVE API calls) is double-gated:
+#     1. input_boolean.pfv_ups_shutdown_armed   — founder arm, default OFF
+#     2. input_boolean.pfv_ups_shutdown_dry_run — default ON
+#   Arming for the Sept 11-12 live test needs, in order: founder review of
+#   the VM ordering below (edit this file via PR), PVE API tokens provisioned
+#   into on-box secrets.yaml (NeedsInput ticket), then a supervised live test.
+#
+# PLAN DATA — captured from LIVE inventory 2026-09-06 (qm list, all hosts):
+#   tsys1: 100 pfv-bms, 101 tsys-ca, 102 k8s-cnode1, 103 netinfra-01,
+#          104 librenms, 105 pve-datacenter | tsys3: 313 k8s-wnode
+#   tsys4: 400 PBS | tsys5: test/preprod bed (5xxx) | tsys6: 600-605
+#   tsys7: 701-711, 902 | tsys9: 904, 905
+# ============================================================================
+
+input_boolean:
+  # NOTE: `initial:` wins over restored state at EVERY HA start — a deploy
+  # mid-test silently reverts these to the safe defaults (armed OFF,
+  # dry-run ON). Safe direction, but re-arm after any restart during the
+  # supervised Sept 11-12 live test.
+  # (no description: keys — input_boolean rejects them, 2026-09-06)
+  pfv_ups_shutdown_armed:
+    # Founder arm. ON + dry-run OFF = real shutdown dispatch.
+    name: UPS shutdown chain ARMED
+    icon: mdi:power-standby
+    initial: false
+  pfv_ups_shutdown_dry_run:
+    # ON (default) = log/page the plan only, touch nothing.
+    name: UPS shutdown DRY RUN
+    icon: mdi:numeric-1-box-outline
+    initial: true
+
+input_datetime:
+  pfv_ups_shutdown_last_page:
+    name: UPS shutdown last plan page
+    has_date: true
+    has_time: true
+    icon: mdi:clock-alert-outline
+
+automation:
+  - id: pfv_ups_shutdown_plan
+    alias: PFV UPS shutdown - plan (dry-run by default)
+    description: >-
+      On battery with the shutdown window opening (runtime < 20 min or
+      charge < 50%), log + page the #455 ordered plan. Re-pages at most
+      every 6h per outage. When armed AND dry-run OFF (founder + supervised
+      test only), the plan page says EXECUTE — real dispatch wiring lands
+      with PVE API tokens (#790 NeedsInput); until then nothing but text
+      can be emitted by this automation.
+    mode: single
+    max_exceeded: silent
+    triggers:
+      - trigger: numeric_state
+        entity_id:
+          - sensor.pfv_ups_runtime_minutes
+          - sensor.pfv_tripp_lite_runtime_minutes
+        below: 20
+      - trigger: numeric_state
+        entity_id:
+          - sensor.tsys1_ups_battery_charge
+          - sensor.tsys1_triplite_battery_charge
+        below: 50
+      # 5-min level re-check — same edge-vs-level reasoning as the
+      # runtime-low alert: never let a threshold-crossing blip silence
+      # the plan page for an entire discharge.
+      - trigger: time_pattern
+        minutes: /5
+    conditions:
+      - condition: template
+        value_template: >-
+          {% set ns = namespace(hit=false) %}
+          {% for rt, st in [
+               ('sensor.pfv_ups_runtime_minutes', 'sensor.tsys1_ups_status'),
+               ('sensor.pfv_tripp_lite_runtime_minutes', 'sensor.tsys1_triplite_status')] %}
+            {% if is_number(states(rt)) and (states(rt) | float) < 20
+                  and states(st) not in ['Online', 'OL', 'unknown', 'unavailable'] %}
+              {% set ns.hit = true %}
+            {% endif %}
+          {% endfor %}
+          {% for ch, st in [
+               ('sensor.tsys1_ups_battery_charge', 'sensor.tsys1_ups_status'),
+               ('sensor.tsys1_triplite_battery_charge', 'sensor.tsys1_triplite_status')] %}
+            {% if is_number(states(ch)) and (states(ch) | float) < 50
+                  and states(st) not in ['Online', 'OL', 'unknown', 'unavailable'] %}
+              {% set ns.hit = true %}
+            {% endif %}
+          {% endfor %}
+          {{ ns.hit }}
+      # Status 'unavailable' suppresses (anti-noise); UPS COMMS LOST covers
+      # that failure mode separately after 5 min.
+      - condition: template
+        value_template: >-
+          {{ states('input_datetime.pfv_ups_shutdown_last_page') in ['unknown', 'unavailable', 'none']
+             or as_timestamp(now()) - as_timestamp(states('input_datetime.pfv_ups_shutdown_last_page')) > 21600 }}
+    actions:
+      - variables:
+          execute_mode: >-
+            {{ is_state('input_boolean.pfv_ups_shutdown_armed', 'on')
+               and not is_state('input_boolean.pfv_ups_shutdown_dry_run', 'on') }}
+          plan: >-
+            UPS ON BATTERY — #455 shutdown plan (VMs before hypervisors):
+            1. tsys5 test bed: qmshutdown 5105 awx, 5101 siem, 51017 sec onion,
+               53100-53102 preprod, 5108-5109, 513 sectestbed k8s, 5000 sandbox
+            2. k8s workers: tsys9 905, tsys7 701, tsys6 601, tsys3 313
+               (drain + shutdown; cnodes 102/603/705 after workers)
+            3. LOB/infra VMs: tsys6 600 awx, 602/605, tsys7 702 uisp, 703, 706-711,
+               902 ucs-2, 604/711 mailgw (after queue drain), tsys9 904 netinfra-02
+            4. tsys1: 104 librenms, 105 pve-dc, 103 netinfra-01, 101 tsys-ca
+               (ca LAST — signing path for everything above)
+            5. pfv-bms VM 100 LAST of the VMs — the brain stays up until now
+            6. Hypervisors: iDRAC graceful off tsys6/tsys7 (PowerEdge);
+               tsys1, tsys3, tsys4, tsys5, tsys9 idle out on UPS exhaustion
+               (OptiPlex/Precision, no BMC)
+          mode_note: >-
+            {{ 'MODE: EXECUTE (armed, live dispatch)' if execute_mode
+               else 'MODE: DRY RUN (plan only — nothing will be shut down)' }}
+      - action: logbook.log
+        data:
+          name: UPS shutdown chain
+          message: "{{ mode_note }} Plan: {{ plan | truncate(250) }}"
+      - action: notify.send_message
+        target:
+          entity_id:
+            - notify.ultix_sidecar_new
+            - notify.allthes_ipad
+        data:
+          title: "{{ 'UPS SHUTDOWN EXECUTING' if execute_mode else 'UPS SHUTDOWN PLAN (dry run)' }}"
+          url: /lovelace-pfv-server-room/ups
+          message: >-
+            {{ mode_note }} Runtime APC {{ states('sensor.pfv_ups_runtime_minutes') }}
+            min / TrippLite {{ states('sensor.pfv_tripp_lite_runtime_minutes') }} min.
+            Order: test bed → k8s workers → LOB VMs → tsys1 infra → CA →
+            pfv-bms (100) last → hypervisors. Full plan in the logbook.
+      - action: persistent_notification.create
+        data:
+          title: "{{ 'UPS SHUTDOWN EXECUTING' if execute_mode else 'UPS SHUTDOWN PLAN (dry run)' }}"
+          message: "{{ plan }}"
+      # Suppression stamp set LAST — a failed notify must not suppress the
+      # next 5-min re-page cycle.
+      - action: input_datetime.set_datetime
+        target:
+          entity_id: input_datetime.pfv_ups_shutdown_last_page
+        data:
+          datetime: "{{ now().strftime('%Y-%m-%d %H:%M:%S') }}"

scenes.yaml

@@ -0,0 +1 @@
+# intentionally empty — scenes are authored in the HA UI if ever needed

scripts.yaml

@@ -0,0 +1 @@
+# intentionally empty — scripts are authored in the HA UI if ever needed

tests/fixtures/secrets-stub.yaml

@@ -0,0 +1,7 @@
+# CI-only stub for `homeassistant check_config` (see .gitea/workflows/ci.yml).
+# NEVER copied to the live box. Values are syntactically-valid placeholders.
+deploy_webhook_id: ci-stub-deploy-webhook
+doorman_scan_webhook_id: ci-stub-scan-webhook
+doorman_unlock_url: http://127.0.0.1:1/unlock-stub
+gitea_auth_header: token ci-stub-gitea-token
+kuma_push_url: http://127.0.0.1:1/push?status=up&msg=ci-stub

tests/test_sdlc.sh

@@ -0,0 +1,92 @@
+#!/bin/bash
+# SDLC regression tests (#778) — run in CI (see .gitea/workflows/ci.yml).
+# Each test guards a rule that exists because of a real incident or ruling.
+set -uo pipefail
+cd "$(dirname "$0")/.." || exit 1
+
+FAIL=0
+t() {
+  # $1 = label, $2 = shell command string
+  if eval "$2"; then
+    echo "  OK   $1"
+  else
+    echo "  FAIL $1"
+    FAIL=$((FAIL + 1))
+  fi
+}
+
+echo "=== alert twins (every alert ships a recovery twin) ==="
+t "utility power restored twin" \
+  "grep -q 'id: pfv_plant_ups_power_restored' automations.yaml"
+t "temperature recovered twin" \
+  "grep -q 'id: pfv_plant_temperature_recovered' automations.yaml"
+
+echo "=== alert deep links (taps land on dashboards, #344) ==="
+ALERTS="POWER OUTAGE|UPS BATTERY LOW|UPS COMMS LOST|UPS RUNTIME LOW|Power restored|HIGH TEMPERATURE|SENSOR WATCHDOG|TEMP RISING FAST|TEMPERATURE RECOVERED"
+DL_OK=1
+IFS='|' read -ra ALERT_ARR <<< "$ALERTS"
+for a in "${ALERT_ARR[@]}"; do
+  if ! grep -A2 "title: $a$" automations.yaml | grep -q "      url: "; then
+    echo "  (missing deep link: $a)"
+    DL_OK=0
+  fi
+done
+t "all alerts deep-linked" "[ \"$DL_OK\" -eq 1 ]"
+
+echo "=== watchdog coverage (#789) ==="
+t "watchdog watches the deploy sensor" \
+  "grep -q 'sensor.pfv_deploy_gitea_release_head, 30' automations.yaml"
+t "watchdog automation exists" \
+  "grep -q 'id: pfv_watchdog_stale_entities' automations.yaml"
+
+echo "=== ghost entities (dead references never return) ==="
+t "no tsys9_ups ghost refs" \
+  "! grep -rq 'tsys9_ups' automations.yaml packages/ dashboards/"
+
+echo "=== secrets ==="
+t "no private keys in tree" \
+  "! grep -rInE 'BEGIN (RSA |OPENSSH |EC |DSA )?PRIVATE KEY' --exclude-dir=.git --exclude-dir=docs ."
+
+echo "=== alert hygiene: push only when something is wrong (2026-09-06 live-fire) ==="
+# The 30-min sweep used to page every cycle even when the fleet was all-OK
+# (48 pages/day of noise). Notify must sit behind the dead>0 + 4h-cooldown gate.
+t "plant watchdog notify gated (dead-only + cooldown input)" \
+  "grep -q 'pfv_plant_watchdog_last_notify' configuration.yaml && grep -q 'notify_gate' automations.yaml"
+
+# tsys6 DRAC ambient quantizes to whole degrees C and flaps ~2F; the raw-fed
+# trend sensor tripped TEMP RISING FAST with zero real climb (2026-09-06 05:02 CDT).
+t "trend sensors fed from smoothed sources" \
+  "grep -q 'time_simple_moving_average' packages/plant_snmp.yaml && sed -n '/^binary_sensor:/,/^utility_meter:/p' packages/plant_snmp.yaml | grep -q 'entity_id: sensor.pfv_tsys6_drac_ambient_smoothed'"
+
+# Tripp Lite had runtime sensors but no runtime-low alert — half the plant's
+# shutdown window was unmonitored.
+t "runtime-low covers Tripp Lite" \
+  "sed -n '/id: pfv_plant_ups_runtime_low/,/^  mode:/p' automations.yaml | grep -q 'sensor.pfv_tripp_lite_runtime_minutes'"
+t "runtime-low status check maps per-UPS" \
+  "sed -n '/id: pfv_plant_ups_runtime_low/,/^  mode:/p' automations.yaml | grep -q \"sensor.pfv_tripp_lite_runtime_minutes', 'sensor.tsys1_triplite_status'\""
+
+echo "=== UPS shutdown chain (#790 — dry-run is the shipped mode) ==="
+t "shutdown chain package present, dry-run default ON" \
+  "grep -q 'id: pfv_ups_shutdown_plan' packages/ups_shutdown.yaml && grep -A4 'pfv_ups_shutdown_dry_run:' packages/ups_shutdown.yaml | grep -q 'initial: true'"
+t "armed switch default OFF" \
+  "grep -A4 'pfv_ups_shutdown_armed:' packages/ups_shutdown.yaml | grep -q 'initial: false'"
+# Host coverage must hold in the PLAN BODY, not the header comment —
+# scope the grep to the plan template variable.
+PLAN_OK=1
+for h in tsys1 tsys3 tsys4 tsys5 tsys6 tsys7 tsys9; do
+  # full-read grep (not -q): the script runs with pipefail — -q's early
+  # exit SIGPIPEs sed and flips the pipeline status to 141
+  sed -n '/plan: >-/,/mode_note:/p' packages/ups_shutdown.yaml | grep "$h" >/dev/null || PLAN_OK=0
+done
+t "plan covers all seven hypervisors (plan body)" "[ \$PLAN_OK -eq 1 ]"
+t "pfv-bms VM 100 ordered last of the VMs (#455)" \
+  "grep -q 'pfv-bms VM 100 LAST' packages/ups_shutdown.yaml"
+
+echo "=== level-based re-checks (review finding: edge-only triggers miss pages) ==="
+t "runtime-low re-polls every 5 min" \
+  "sed -n '/id: pfv_plant_ups_runtime_low/,/^  mode:/p' automations.yaml | grep -q 'minutes: /5'"
+t "shutdown chain re-polls every 5 min" \
+  "grep -A30 'id: pfv_ups_shutdown_plan' packages/ups_shutdown.yaml | grep -q 'minutes: /5'"
+
+echo "=== SDLC SUITE: $FAIL failures ==="
+exit "$FAIL"

themes/.gitkeep

@@ -0,0 +1,2 @@
+# intentional placeholder: themes dir must exist for the
+# !include_dir_merge_named in configuration.yaml [#784 gardening]