18 Commits
Author SHA1 Message Date
mrcharles 83b32d5d9f feat(heartbeat): dead-man push to out-of-band Kuma every 5 min [#792]
ci / lint (push) Successful in 42s
HA curls status.knownelement.com push URL on a 5-min pattern; Kuma
pages the founder directly if heartbeats stop. Alert path independent
of HA itself.
https://projects.knownelement.com/issues/792
2026-09-04 17:07:04 -05:00
mrcharles 807dc8ba8d docs: release PR template — gate checklist + rollback [#784 #778]
ci / lint (push) Successful in 57s
2026-09-04 16:37:21 -05:00
mrcharles 239e449d45 feat(validate): post-deploy validation on every HA start [#778]
ci / lint (push) Successful in 41s
Missing/stale criticals after restart page with specifics instead of
waiting for humans; watchdog handles recovery; revert procedure
documented for repeat failures.
2026-09-04 16:23:53 -05:00
mrcharles 601c70ec5e fix(alerts): iOS data.url only — clickAction rejected by apns schema [#344 #778]
ci / lint (push) Successful in 46s
Watchdog notify crash root-caused: iOS companion schema rejects
data.clickAction; url is the iOS deep-link key. Suite updated to
assert url presence per alert.
https://projects.knownelement.com/issues/778
2026-09-04 16:11:31 -05:00
mrcharles 13b0976ad6 fix(doorman): unlock dispatch via shell_command — rest_command absent [#345 #355]
ci / lint (push) Successful in 47s
rest_command integration silently fails to register on this box
(404 services, valid config+secret; root-cause pending #741).
shell_command curl is the working mechanism; arm-switch gate unchanged.
https://projects.knownelement.com/issues/355
2026-09-04 16:01:40 -05:00
mrcharles 8f48730a40 fix(dash): tripp lite runtime entity id (tripp_lite slug) [#784]
ci / lint (push) Successful in 58s
2026-09-04 15:06:36 -05:00
mrcharles c23d256ced feat(dash): dashboard series v2 — Ops Overview, Power, Thermal [#784 #344]
ci / lint (push) Successful in 35s
Role-based rework per founder: overview + detail dashboards. UPS
parity (Tripp Lite runtime-minutes template). Phase 1 of the ground-up
rework; Capacity/Access views land with fans + lock.
https://projects.knownelement.com/issues/784
2026-09-04 14:49:14 -05:00
mrcharles 45024f80c3 feat(sdlc): regression test suite in CI + watchdog covers deploy sensor [#778 #789]
ci / lint (push) Successful in 38s
tests/test_sdlc.sh: alert-twin presence, per-alert deep-link check,
watchdog coverage, ghost-entity + secret guards. Runs as a CI step.
https://projects.knownelement.com/issues/778
2026-09-04 14:27:51 -05:00
mrcharles 403f83a187 fix(deploy): sha-watch sensor URL -> KNEL/KNELBMS post-rename [#785 #784]
ci / lint (push) Successful in 49s
2026-09-04 13:33:18 -05:00
mrcharles c8f2cbde4a fix(ci): bookworm pip via apt, valid truthy nesting, runtime-file ignores [#784]
ci / lint (push) Successful in 45s
Red/green: run 1 failed 27s (no pip in node image); run 2 blocker was
invalid truthy nesting in .yamllint. Local repro now green.
https://projects.knownelement.com/issues/784
2026-09-04 12:22:23 -05:00
mrcharles 5629bc9b4b ci: runs-on ultix — fleet runner label scoping [#784]
ci / lint (push) Failing after 27s
Runner is label-scoped to this fleet; other repos' ubuntu-latest
jobs no longer land on it.
https://projects.knownelement.com/issues/784
2026-09-04 12:13:35 -05:00
mrcharles 4f54635b33 feat(watchdog): stale-entity auto-heal — reload, notify, guarded restart [#789]
ci / lint (push) Failing after 14s
5-min staleness checks on 12 critical entities; reload_config_entry
auto-heal (no restart); notify on failure; auto-restart behind
input_boolean arm (default OFF).
https://projects.knownelement.com/issues/789
2026-09-04 10:55:00 -05:00
mrcharles 523a56bd52 feat(dash): Tripp Lite power entities — apparent VA, current, runtime [#372 #344]
ci / lint (push) Failing after 9s
4 integration-disabled entities enabled via registry (config-entry
platform, no restart). dev only.
https://projects.knownelement.com/issues/372
2026-09-04 10:50:03 -05:00
mrcharles 758fc92e36 feat(dash): Tripp Lite cards on Server Room UPS view [#372 #344]
ci / lint (push) Failing after 10s
charge + load gauges, status entities, hardware note. dev only.
https://projects.knownelement.com/issues/372
2026-09-04 10:43:48 -05:00
mrcharles 528dab5100 chore(garden): themes dir placeholder, empty-include markers, doorman contract pointer [#784]
ci / lint (push) Failing after 9s
Gardening pass: themes/ dir must exist for include_dir_merge_named;
scenes/scripts marked intentionally-empty; cross-repo contract pointer
to KNEL/doorman README. dev only.
https://projects.knownelement.com/issues/784
2026-09-04 07:46:10 -05:00
mrcharles 6266043d11 ci(gitea): workflow — yamllint + secret scan on dev/release [#784]
ci / lint (push) Failing after 27s
Fleet CI standard; runner #784. Runs go green once the act_runner
registers (token pending founder).
https://projects.knownelement.com/issues/784
2026-09-04 07:25:04 -05:00
mrcharles 8931b653d5 feat(ups): Tripp Lite into on-battery/low-battery/comms alerts [#372 #344]
tsys1_triplite entities replace the never-real tsys9_ups_status
ghost references; battery-low trigger gains the triplite charge.
dev only - release push pending founder go.
https://projects.knownelement.com/issues/372
2026-09-04 07:12:28 -05:00
mrcharles 09f3276b31 feat(deploy): branch model — box tracks release, dev is WIP-integration [#779 #344]
webhook ref-guard + sha-watch sensor pinned to ?sha=release;
sensor renamed pfv_deploy_gitea_release_head; master frozen.
https://projects.knownelement.com/issues/779
2026-09-04 06:32:56 -05:00
17 changed files with 732 additions and 30 deletions
+16
View File
@@ -0,0 +1,16 @@
## Release PR (dev -> release)
Merging this deploys to production (pfv-bms restarts if packages/** or
configuration.yaml changed).
### Gate checklist (branch protection enforces the mechanical parts)
- [ ] CI green on the head commit (yamllint + secret scan + SDLC suite)
- [ ] Adversarial pass done: what breaks if this is wrong? what watches for it?
- [ ] post-deploy validation (#778) will run on restart — expected entity changes noted below
### Reviewer (founder)
- [ ] Diff reviewed
- [ ] Approve to release
Expected entity/automation changes: <list or "none">
Rollback: git revert <sha> on dev, re-PR, or push previous release sha directly.
+28
View File
@@ -0,0 +1,28 @@
# pfv-bms CI [#784 #778] — fleet standard per Discourse "CI/CD: Gitea Actions".
# Runs on dev (pre-release gate) and release (final gate before live pull).
name: ci
on:
push:
branches: [dev, release]
jobs:
lint:
runs-on: ultix
container:
image: node:20-bookworm
steps:
- uses: actions/checkout@v4
- name: yamllint
run: |
apt-get update -qq >/dev/null && apt-get install -qq -y python3-pip >/dev/null && pip install --quiet --break-system-packages yamllint
yamllint -c .yamllint .
- name: secret scan
run: |
if grep -rInE "BEGIN (RSA |OPENSSH |EC |DSA )?PRIVATE KEY|aws_secret_access_key *=|AKIA[0-9A-Z]{16}" --exclude-dir=.git .; then
echo "::error::secret material committed"; exit 1
else
echo "secret scan clean"
fi
- name: sdlc regression tests
run: |
chmod +x tests/test_sdlc.sh
bash tests/test_sdlc.sh
+22
View File
@@ -0,0 +1,22 @@
# HA-config-aware yamllint [#784]: keep the checks that catch real bugs
# (duplicates, structure), relax the stylistic ones HA templates trip.
extends: default
ignore: |
docs/
secrets.yaml
*.db
rules:
line-length: disable
comments: disable
comments-indentation: disable
document-start: disable
truthy:
check-keys: false
indentation:
spaces: consistent
indent-sequences: consistent
key-duplicates: enable
new-line-at-end-of-file: disable
trailing-spaces: disable
empty-lines:
max-end: 2
+11 -4
View File
@@ -134,16 +134,23 @@ apps.
## 🚀 Deployment pipeline ## 🚀 Deployment pipeline
**Branch model (2026-09-04):** `dev` = WIP integration — the pipeline
IGNORES it. `release` = what the box runs — pushing release IS the deploy.
`master` is frozen (pre-branch-model history). A PR process with AI +
human reviewers will gate dev → release (founder, upcoming).
```mermaid ```mermaid
flowchart LR flowchart LR
PUSH[📝 git push] --> HOOK[gitea webhook<br/>fast path] PUSH[📝 push to dev] --> NOOP[🚫 pipeline ignores dev]
PUSH --> SHA[sha-watch<br/>safety net · 300 s poll] REL[🚀 merge dev → release<br/>and push] --> HOOK[gitea webhook<br/>fast path]
REL --> SHA[sha-watch<br/>release sha · 300 s poll]
HOOK & SHA --> CHECK{changed files?} HOOK & SHA --> CHECK{changed files?}
CHECK -->|packages/**| RESTART[♻️ core restart<br/>webhook triggers need it] CHECK -->|configuration.yaml,<br/>packages/**| RESTART[♻️ core restart<br/>webhook triggers need it]
CHECK -->|other YAML| RELOAD[⚡ reload only<br/>~5 min to live] CHECK -->|other YAML| RELOAD[⚡ reload only<br/>~5 min to live]
``` ```
Gate before pushing `packages/**`: `ha core check` must pass on-box. Gate before pushing `release` (and after any `packages/**` change on
`dev`): `ha core check` must pass on-box.
Measured end-to-end: 4 m 52 s commit → live. Measured end-to-end: 4 m 52 s commit → live.
## 🗺️ Repo map ## 🗺️ Repo map
+197 -21
View File
@@ -4,7 +4,7 @@
- trigger: state - trigger: state
entity_id: entity_id:
- sensor.tsys1_ups_status - sensor.tsys1_ups_status
- sensor.tsys9_ups_status - sensor.tsys1_triplite_status
conditions: conditions:
- condition: template - condition: template
value_template: '{{ ''On Battery'' in states(trigger.entity_id) }}' value_template: '{{ ''On Battery'' in states(trigger.entity_id) }}'
@@ -16,7 +16,6 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: POWER OUTAGE title: POWER OUTAGE
clickAction: /lovelace-pfv-server-room/ups
url: /lovelace-pfv-server-room/ups url: /lovelace-pfv-server-room/ups
message: '{{ state_attr(trigger.entity_id, ''friendly_name'') }} lost utility message: '{{ state_attr(trigger.entity_id, ''friendly_name'') }} lost utility
power and is ON BATTERY at {{ states(''sensor.'' ~ trigger.entity_id.split(''.'')[1] power and is ON BATTERY at {{ states(''sensor.'' ~ trigger.entity_id.split(''.'')[1]
@@ -28,6 +27,7 @@
- trigger: numeric_state - trigger: numeric_state
entity_id: entity_id:
- sensor.tsys1_ups_battery_charge - sensor.tsys1_ups_battery_charge
- sensor.tsys1_triplite_battery_charge
below: 40 below: 40
conditions: conditions:
- condition: template - condition: template
@@ -41,7 +41,6 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: UPS BATTERY LOW title: UPS BATTERY LOW
clickAction: /lovelace-pfv-server-room/ups
url: /lovelace-pfv-server-room/ups url: /lovelace-pfv-server-room/ups
message: '{{ trigger.entity_id }} at {{ states(trigger.entity_id) }}% while message: '{{ trigger.entity_id }} at {{ states(trigger.entity_id) }}% while
on battery. Start graceful shutdown of VMs before hypervisors (#455 ordering).' on battery. Start graceful shutdown of VMs before hypervisors (#455 ordering).'
@@ -52,7 +51,7 @@
- trigger: state - trigger: state
entity_id: entity_id:
- sensor.tsys1_ups_status - sensor.tsys1_ups_status
- sensor.tsys9_ups_status - sensor.tsys1_triplite_status
to: unavailable to: unavailable
for: 00:05:00 for: 00:05:00
actions: actions:
@@ -63,7 +62,6 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: UPS COMMS LOST title: UPS COMMS LOST
clickAction: /lovelace-pfv-server-room/ups
url: /lovelace-pfv-server-room/ups url: /lovelace-pfv-server-room/ups
message: '{{ trigger.entity_id }} unreachable for 5 minutes (nut-server or network message: '{{ trigger.entity_id }} unreachable for 5 minutes (nut-server or network
path down; tsys1 also hosts pfv-bms itself).' path down; tsys1 also hosts pfv-bms itself).'
@@ -87,7 +85,6 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: UPS RUNTIME LOW title: UPS RUNTIME LOW
clickAction: /lovelace-pfv-server-room/ups
url: /lovelace-pfv-server-room/ups url: /lovelace-pfv-server-room/ups
message: On battery {{ states('sensor.pfv_ups_runtime_minutes') }} min left — start graceful shutdown now (#455 ordering). message: On battery {{ states('sensor.pfv_ups_runtime_minutes') }} min left — start graceful shutdown now (#455 ordering).
mode: single mode: single
@@ -97,7 +94,7 @@
- trigger: state - trigger: state
entity_id: entity_id:
- sensor.tsys1_ups_status - sensor.tsys1_ups_status
- sensor.tsys9_ups_status - sensor.tsys1_triplite_status
conditions: conditions:
- condition: template - condition: template
value_template: "{{ trigger.from_state is not none and 'On Battery' in trigger.from_state.state\n value_template: "{{ trigger.from_state is not none and 'On Battery' in trigger.from_state.state\n
@@ -110,7 +107,6 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: Power restored title: Power restored
clickAction: /lovelace-pfv-server-room/ups
url: /lovelace-pfv-server-room/ups url: /lovelace-pfv-server-room/ups
message: '{{ state_attr(trigger.entity_id, ''friendly_name'') }} is back ONLINE.' message: '{{ state_attr(trigger.entity_id, ''friendly_name'') }} is back ONLINE.'
mode: single mode: single
@@ -188,7 +184,6 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: HIGH TEMPERATURE title: HIGH TEMPERATURE
clickAction: /lovelace-pfv-server-room/temps
url: /lovelace-pfv-server-room/temps url: /lovelace-pfv-server-room/temps
message: '{{ trigger.entity_id }} is {{ states(trigger.entity_id) }}°F (class message: '{{ trigger.entity_id }} is {{ states(trigger.entity_id) }}°F (class
threshold exceeded). Check cooling/airflow.' threshold exceeded). Check cooling/airflow.'
@@ -208,7 +203,6 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: SENSOR WATCHDOG title: SENSOR WATCHDOG
clickAction: /lovelace-pfv-server-room/overview
url: /lovelace-pfv-server-room/overview url: /lovelace-pfv-server-room/overview
message: >- message: >-
{% set fleet = states.sensor | selectattr('entity_id', 'match', 'sensor\.pfv_') {% set fleet = states.sensor | selectattr('entity_id', 'match', 'sensor\.pfv_')
@@ -242,11 +236,10 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: TEMP RISING FAST title: TEMP RISING FAST
clickAction: /lovelace-pfv-server-room/temps
url: /lovelace-pfv-server-room/temps url: /lovelace-pfv-server-room/temps
message: '{{ trigger.entity_id }} climbing ~10F/15min — check cooling message: '{{ trigger.entity_id }} climbing ~10F/15min — check cooling
before the 104F threshold. (rate-of-change early warning)' before the 104F threshold. (rate-of-change early warning)'
# Deploy pipeline [#734]: gitea repo webhook (master pushes) -> forced pull # Deploy pipeline [#734]: gitea repo webhook (release pushes) -> forced pull
# -> minimal reload. Core restarts ONLY when configuration.yaml or a # -> minimal reload. Core restarts ONLY when configuration.yaml or a
# packages platform-sensor file changed; everything else reloads with no # packages platform-sensor file changed; everything else reloads with no
# bounce. The git_pull add-on's auto_restart is disabled - this webhook # bounce. The git_pull add-on's auto_restart is disabled - this webhook
@@ -254,7 +247,7 @@
- id: pfv_deploy_git_webhook - id: pfv_deploy_git_webhook
alias: PFV deploy - git push to pull + minimal reload alias: PFV deploy - git push to pull + minimal reload
description: >- description: >-
Gitea webhook on master pushes. Forces a git pull now, then Gitea webhook on release pushes. Forces a git pull now, then
homeassistant.reload_all unless files that require a core restart homeassistant.reload_all unless files that require a core restart
changed. Replaces the git_pull add-on timer auto-restart [#734]. changed. Replaces the git_pull add-on timer auto-restart [#734].
mode: queued mode: queued
@@ -269,9 +262,9 @@
actions: actions:
- if: - if:
- condition: template - condition: template
value_template: "{{ trigger.json.ref | default('') != 'refs/heads/master' }}" value_template: "{{ trigger.json.ref | default('') != 'refs/heads/release' }}"
then: then:
- stop: "deploy webhook: non-master ref {{ trigger.json.ref | default('') }} ignored" - stop: "deploy webhook: non-release ref {{ trigger.json.ref | default('') }} ignored"
- variables: - variables:
changed_files: >- changed_files: >-
{{ (trigger.json.commits | default([]) | map(attribute='added') | flatten | list) {{ (trigger.json.commits | default([]) | map(attribute='added') | flatten | list)
@@ -337,7 +330,7 @@
data: data:
title: PFV deploy complete title: PFV deploy complete
message: "Pulled + reloaded YAML at {{ now().strftime('%H:%M:%S') }} - no core restart. Files: {{ changed_files | join(', ') | truncate(120) }}" message: "Pulled + reloaded YAML at {{ now().strftime('%H:%M:%S') }} - no core restart. Files: {{ changed_files | join(', ') | truncate(120) }}"
# Deploy pipeline safety net [#734]: polls gitea master HEAD (rest sensor # Deploy pipeline safety net [#734]: polls gitea release HEAD (rest sensor
# in packages/deploy_pipeline.yaml) and applies pulls reload-only. The # in packages/deploy_pipeline.yaml) and applies pulls reload-only. The
# gitea webhook is the fast path once gitea's SSRF allowlist permits # gitea webhook is the fast path once gitea's SSRF allowlist permits
# pfv-bms.knel.net; until then this net catches every push within 5 min # pfv-bms.knel.net; until then this net catches every push within 5 min
@@ -346,7 +339,7 @@
- id: pfv_deploy_sha_watch - id: pfv_deploy_sha_watch
alias: PFV deploy - gitea sha watch safety net alias: PFV deploy - gitea sha watch safety net
description: >- description: >-
Master HEAD sha changed -> pull + CORE RESTART. Interim deploy Release HEAD sha changed -> pull + CORE RESTART. Interim deploy
trigger while gitea webhook delivery is blocked by its SSRF host trigger while gitea webhook delivery is blocked by its SSRF host
allowlist [#734]. Restart (not reload) is REQUIRED: automation allowlist [#734]. Restart (not reload) is REQUIRED: automation
reload loses webhook-trigger registration - found live 2026-09-03 reload loses webhook-trigger registration - found live 2026-09-03
@@ -354,12 +347,12 @@
scan ingest, this pipeline's own fast path) silently stop firing scan ingest, this pipeline's own fast path) silently stop firing
while the endpoint still returns HTTP 200. Core restart while the endpoint still returns HTTP 200. Core restart
re-registers them. Accepted trade-off: ~1 min HA outage per re-registers them. Accepted trade-off: ~1 min HA outage per
master deploy. release deploy.
mode: single mode: single
max_exceeded: silent max_exceeded: silent
triggers: triggers:
- trigger: state - trigger: state
entity_id: sensor.pfv_deploy_gitea_master_head entity_id: sensor.pfv_deploy_gitea_release_head
conditions: conditions:
- condition: template - condition: template
value_template: >- value_template: >-
@@ -383,7 +376,7 @@
data: data:
title: PFV deploy (sha watch) title: PFV deploy (sha watch)
message: >- message: >-
New master HEAD {{ states('sensor.pfv_deploy_gitea_master_head')[:7] }} New release HEAD {{ states('sensor.pfv_deploy_gitea_release_head')[:7] }}
pulled. Restarting HA core (webhook re-registration) - back in pulled. Restarting HA core (webhook re-registration) - back in
~1 min. ~1 min.
- action: homeassistant.restart - action: homeassistant.restart
@@ -518,10 +511,193 @@
- notify.allthes_ipad - notify.allthes_ipad
data: data:
title: TEMPERATURE RECOVERED title: TEMPERATURE RECOVERED
clickAction: /lovelace-pfv-server-room/temps
url: /lovelace-pfv-server-room/temps url: /lovelace-pfv-server-room/temps
message: >- message: >-
{{ state_attr(trigger.entity_id, 'friendly_name') | default(trigger.entity_id) }} {{ state_attr(trigger.entity_id, 'friendly_name') | default(trigger.entity_id) }}
back to {{ states(trigger.entity_id) }}°F and holding — was back to {{ states(trigger.entity_id) }}°F and holding — was
{{ trigger.from_state.state }}°F when the alert cleared (tier fire line {{ fire }}°F). {{ trigger.from_state.state }}°F when the alert cleared (tier fire line {{ fire }}°F).
mode: single mode: single
- id: pfv_watchdog_stale_entities
alias: PFV watchdog - stale entity auto-heal
description: >
Detects silent poller wedges (2026-09-04: snmp tsys9 went 12h
undetected; NUT APC 3h) and auto-heals config-entry platforms via
homeassistant.reload_config_entry — no restart needed. Escalates to
notify when reload does not recover. Auto core restart ONLY while
input_boolean.pfv_watchdog_auto_restart is ON and entities stay
stale — restart costs ~1 min downtime and re-registers webhooks
(#345, #789). v1 notify repeats each cycle while stale (visible
alarm); throttle in v2.
mode: single
max_exceeded: silent
triggers:
- trigger: time_pattern
minutes: /5
variables:
checks:
- [sensor.pfv_tsys1_rack_temperature, 20]
- [sensor.pfv_tsys1_drive_temperature, 30]
- [sensor.pfv_tsys3_cpu_temperature, 20]
- [sensor.pfv_tsys4_ambient_temperature, 20]
- [sensor.pfv_tsys6_cpu_temperature, 20]
- [sensor.pfv_tsys6_drac_ambient, 30]
- [sensor.pfv_tsys7_cpu_temperature, 20]
- [sensor.pfv_tsys7_drac_inlet, 30]
- [sensor.pfv_tsys9_cpu_temperature, 20]
- [sensor.pfv_tsys9_drive_temperature, 30]
- [sensor.tsys1_ups_status, 15]
- [sensor.tsys1_triplite_status, 15]
- [sensor.pfv_deploy_gitea_release_head, 30]
actions:
- variables:
stale_list: >-
{% set ns = namespace(stale=[]) %}
{% for pair in checks %}
{% set s = states[pair[0]] %}
{% if s is not none and s.last_updated is defined
and (now() - s.last_updated).total_seconds() > pair[1] * 60 %}
{% set ns.stale = ns.stale + [pair[0]] %}
{% endif %}
{% endfor %}
{{ ns.stale }}
- condition: template
value_template: "{{ stale_list | length > 0 }}"
- repeat:
for_each: "{{ stale_list }}"
sequence:
- action: homeassistant.reload_config_entry
data:
entity_id: "{{ repeat.item }}"
- delay:
hours: 0
minutes: 2
seconds: 0
- variables:
still_stale: >-
{% set ns = namespace(stale=[]) %}
{% for pair in checks %}
{% set s = states[pair[0]] %}
{% if s is not none and s.last_updated is defined
and (now() - s.last_updated).total_seconds() > pair[1] * 60 %}
{% set ns.stale = ns.stale + [pair[0]] %}
{% endif %}
{% endfor %}
{{ ns.stale }}
- action: notify.send_message
target:
entity_id:
- notify.ultix_sidecar_new
- notify.allthes_ipad
data:
title: WATCHDOG
url: /lovelace-pfv-server-room/temps
message: >-
Stale after auto-reload: {{ still_stale | join(', ') }}.
Auto-restart is {{ 'ON' if is_state('input_boolean.pfv_watchdog_auto_restart','on')
else 'OFF (arm it on the watchdog switch if you want unattended restarts)' }}.
- if:
- condition: template
value_template: "{{ is_state('input_boolean.pfv_watchdog_auto_restart','on') }}"
then:
- delay:
hours: 0
minutes: 30
seconds: 0
- variables:
hard_stale: >-
{% set ns = namespace(stale=[]) %}
{% for pair in checks %}
{% set s = states[pair[0]] %}
{% if s is not none and s.last_updated is defined
and (now() - s.last_updated).total_seconds() > pair[1] * 60 %}
{% set ns.stale = ns.stale + [pair[0]] %}
{% endif %}
{% endfor %}
{{ ns.stale }}
- if:
- condition: template
value_template: "{{ hard_stale | length > 0 }}"
then:
- action: notify.send_message
target:
entity_id:
- notify.ultix_sidecar_new
- notify.allthes_ipad
data:
title: WATCHDOG — restarting core
message: "Still stale after reload + 30 min: {{ hard_stale | join(', ') }}. Restarting HA core now."
- action: homeassistant.restart
- id: pfv_post_deploy_validation
alias: PFV post-deploy validation
description: >
Runs after EVERY HA start (deploy restarts included): after a settle
delay, verifies the critical sensor set actually materialized and is
fresh. Catches bad deploys — missing templates, broken packages,
wedged pollers — and pages with specifics instead of waiting for a
human to notice (#778, #777). The watchdog then handles recovery.
mode: single
max_exceeded: silent
triggers:
- trigger: event
event_type: homeassistant_start
actions:
- delay:
hours: 0
minutes: 3
seconds: 0
- variables:
required:
- [sensor.pfv_tsys1_rack_temperature, 20]
- [sensor.pfv_tsys9_cpu_temperature, 20]
- [sensor.tsys1_ups_status, 15]
- [sensor.tsys1_triplite_status, 15]
- [sensor.pfv_ups_runtime_minutes, 30]
- [sensor.pfv_tripp_lite_runtime_minutes, 30]
- [sensor.pfv_deploy_gitea_release_head, 30]
- [automation.pfv_watchdog_stale_entities, 0]
- variables:
bad_list: >-
{% set ns = namespace(bad=[]) %}
{% for pair in required %}
{% set s = states[pair[0]] %}
{% if s is none or s.state in ['unknown', 'unavailable', 'none'] %}
{% set ns.bad = ns.bad + [pair[0] ~ ' (missing)'] %}
{% elif pair[1] > 0 and s.last_updated is defined
and (now() - s.last_updated).total_seconds() > pair[1] * 60 %}
{% set ns.bad = ns.bad + [pair[0] ~ ' (stale)'] %}
{% endif %}
{% endfor %}
{{ ns.bad }}
- if:
- condition: template
value_template: "{{ bad_list | length > 0 }}"
then:
- action: notify.send_message
target:
entity_id:
- notify.ultix_sidecar_new
- notify.allthes_ipad
data:
title: POST-DEPLOY VALIDATION FAILED
url: /lovelace-pfv-thermal
message: >-
After restart, these criticals are missing/stale:
{{ bad_list | join(', ') }}. Watchdog will retry recovery;
if this repeats, revert the last release (git revert + push).
- action: persistent_notification.create
data:
title: POST-DEPLOY VALIDATION FAILED
message: "Missing/stale: {{ bad_list | join(', ') }}"
- id: pfv_kuma_heartbeat
alias: PFV Kuma heartbeat (dead-man)
description: >
Pushes OK to the out-of-band Uptime Kuma every 5 minutes (#792).
Kuma pages the founder directly if heartbeats stop — the alert path
that works when HA itself is the failure.
mode: single
max_exceeded: silent
triggers:
- trigger: time_pattern
minutes: /5
actions:
- action: shell_command.kuma_ha_heartbeat
+22
View File
@@ -20,6 +20,21 @@ prometheus:
lovelace: lovelace:
mode: storage mode: storage
dashboards: dashboards:
pfv-overview:
mode: yaml
title: Ops Overview
filename: dashboards/overview.yaml
icon: mdi:view-dashboard-outline
pfv-power:
mode: yaml
title: Power
filename: dashboards/power.yaml
icon: mdi:flash
pfv-thermal:
mode: yaml
title: Thermal
filename: dashboards/thermal.yaml
icon: mdi:thermometer-lines
pfv-server-room: pfv-server-room:
mode: yaml mode: yaml
title: Server Room title: Server Room
@@ -34,3 +49,10 @@ lovelace:
title: STLP title: STLP
filename: dashboards/stlp.yaml filename: dashboards/stlp.yaml
icon: mdi:home icon: mdi:home
# Watchdog arm switch (#789): auto core restart on unresolved staleness.
# Default OFF — reload_config_entry heals config-entry wedges without it.
input_boolean:
pfv_watchdog_auto_restart:
name: Watchdog auto core restart
icon: mdi:restart-alert
+79
View File
@@ -0,0 +1,79 @@
title: Ops Overview
views:
- title: Overview
type: sections
sections:
- type: grid
cards:
- type: markdown
content: >
## Ops Overview
One screen: is anything on fire? Details live in
[Power](/lovelace-pfv-power), [Thermal](/lovelace-pfv-thermal),
[Server Room](/lovelace-pfv-server-room) and
[STLP](/lovelace-pfv-stlp).
- type: grid
cards:
- type: heading
heading: Room & cooling
- type: gauge
entity: sensor.pfv_tsys1_rack_temperature
name: Server room
min: 50
max: 110
severity:
green: 60
yellow: 85
red: 95
- type: entities
entities:
- sensor.pfv_garage_temperature
- sensor.pfv_artroom_temperature
- type: grid
cards:
- type: heading
heading: Power
- type: gauge
entity: sensor.tsys1_ups_battery_charge
name: APC charge
min: 0
max: 100
severity:
green: 60
yellow: 40
red: 20
- type: gauge
entity: sensor.tsys1_triplite_battery_charge
name: Tripp Lite charge
min: 0
max: 100
severity:
green: 60
yellow: 40
red: 20
- type: entities
entities:
- sensor.tsys1_ups_status
- sensor.tsys1_triplite_status
- sensor.pfv_ups_runtime_minutes
- sensor.pfv_tripp_lite_runtime_minutes
- type: grid
cards:
- type: heading
heading: Alerts & self-healing
- type: entities
entities:
- automation.pfv_plant_ups_on_battery
- automation.pfv_plant_ups_battery_low
- automation.pfv_plant_ups_communication_lost
- automation.pfv_plant_high_temperature
- automation.pfv_watchdog_stale_entity_auto_heal
- input_boolean.pfv_watchdog_auto_restart
- type: grid
cards:
- type: markdown
content: >
**Pipeline:** deploy sensor tracks `release` on KNEL/KNELBMS.
CI runs on every dev/release push (Actions tab in gitea).
Out-of-band paging: Uptime Kuma (Cloudron).
+114
View File
@@ -0,0 +1,114 @@
title: Power
views:
- title: UPS detail
type: sections
sections:
- type: grid
cards:
- type: heading
heading: APC Smart-UPS C 1500 (battery-side USB HID)
- type: gauge
entity: sensor.tsys1_ups_battery_charge
name: Charge
min: 0
max: 100
severity:
green: 60
yellow: 40
red: 20
- type: gauge
entity: sensor.pfv_ups_runtime_minutes
name: Runtime (min)
min: 0
max: 60
severity:
green: 25
yellow: 15
red: 5
- type: grid
cards:
- type: entities
title: APC status
entities:
- sensor.tsys1_ups_status
- sensor.tsys1_ups_status_data
- sensor.tsys1_ups_battery_charge
- sensor.pfv_ups_runtime_minutes
- sensor.tsys1_ups_battery_runtime
- sensor.tsys1_ups_battery_voltage
- sensor.tsys1_ups_nominal_battery_voltage
- sensor.tsys1_ups_low_battery_setpoint
- sensor.tsys1_ups_beeper_status
- type: markdown
content: >
Serial AS1213210423 · mfr 2012-03 · fw UPS 02.2 · 24 V
nominal. USB HID exposes battery-side metrics only — no
input/output voltage or watts on this unit.
- type: grid
cards:
- type: heading
heading: Tripp Lite 1500 VA (moved from tsys9, #372)
- type: gauge
entity: sensor.tsys1_triplite_battery_charge
name: Charge
min: 0
max: 100
severity:
green: 60
yellow: 40
red: 20
- type: gauge
entity: sensor.tsys1_triplite_load
name: Load %
min: 0
max: 100
severity:
green: 70
yellow: 85
red: 95
- type: grid
cards:
- type: entities
title: Tripp Lite status
entities:
- sensor.tsys1_triplite_status
- sensor.tsys1_triplite_status_data
- sensor.tsys1_triplite_battery_charge
- sensor.tsys1_triplite_apparent_power
- sensor.tsys1_triplite_output_current
- sensor.tsys1_triplite_load
- sensor.pfv_tripp_lite_runtime_minutes
- sensor.tsys1_triplite_battery_runtime
- sensor.tsys1_triplite_input_voltage
- sensor.tsys1_triplite_output_voltage
- sensor.tsys1_triplite_nominal_power
- title: Trends
type: sections
sections:
- type: grid
cards:
- type: history-graph
title: UPS charge (7 days)
entities:
- sensor.tsys1_ups_battery_charge
- sensor.tsys1_triplite_battery_charge
hours_to_show: 168
- type: grid
cards:
- type: history-graph
title: Tripp Lite apparent power (24h)
entities:
- sensor.tsys1_triplite_apparent_power
hours_to_show: 24
- type: history-graph
title: UPS load % (24h)
entities:
- sensor.tsys1_triplite_load
hours_to_show: 24
- type: grid
cards:
- type: markdown
content: >
**Planned sections** (land with their data sources):
Emporia CT rental-bank feed (#623), whole-house kWh vs
solar production (#760), garage PDU circuits (#733).
+40
View File
@@ -103,6 +103,46 @@ views:
fw UPS 02.2 · PbAc, 24 V nominal · usbhid-ups via pfv-tsys1 fw UPS 02.2 · PbAc, 24 V nominal · usbhid-ups via pfv-tsys1
(NUT :3493). This unit reports no input/output voltage over (NUT :3493). This unit reports no input/output voltage over
USB HID — battery-side metrics only. USB HID — battery-side metrics only.
- type: grid
cards:
- type: gauge
entity: sensor.tsys1_triplite_battery_charge
name: Tripp Lite charge
min: 0
max: 100
severity:
green: 60
yellow: 40
red: 20
- type: gauge
entity: sensor.tsys1_triplite_load
name: Tripp Lite load %
min: 0
max: 100
severity:
green: 70
yellow: 85
red: 95
- type: grid
cards:
- type: entities
title: Tripp Lite status
entities:
- sensor.tsys1_triplite_status
- sensor.tsys1_triplite_status_data
- sensor.tsys1_triplite_battery_charge
- sensor.tsys1_triplite_apparent_power
- sensor.tsys1_triplite_output_current
- sensor.tsys1_triplite_load
- sensor.tsys1_triplite_input_voltage
- sensor.tsys1_triplite_output_voltage
- sensor.tsys1_triplite_battery_runtime
- type: markdown
content: >
**Tripp Lite UPS** · usbhid-ups via pfv-tsys1 (NUT :3493);
moved from tsys9 2026-09-04 (#372). Carries ~46% load
(~414 VA apparent / 1500 VA nominal) — mind runtime
before adding gear.
- type: grid - type: grid
cards: cards:
- type: entities - type: entities
+119
View File
@@ -0,0 +1,119 @@
title: Thermal
views:
- title: Rooms
type: sections
sections:
- type: grid
cards:
- type: gauge
entity: sensor.pfv_tsys1_rack_temperature
name: Server room
min: 50
max: 110
severity:
green: 60
yellow: 85
red: 95
- type: history-graph
title: Server room (24h)
entities:
- sensor.pfv_tsys1_rack_temperature
hours_to_show: 24
- type: grid
cards:
- type: gauge
entity: sensor.pfv_garage_temperature
name: Garage
min: 30
max: 110
severity:
green: 40
yellow: 90
red: 100
- type: history-graph
title: Garage (24h)
entities:
- sensor.pfv_garage_temperature
hours_to_show: 24
- type: grid
cards:
- type: gauge
entity: sensor.pfv_artroom_temperature
name: Art room
min: 40
max: 110
severity:
green: 60
yellow: 85
red: 95
- type: history-graph
title: Art room (24h)
entities:
- sensor.pfv_artroom_temperature
hours_to_show: 24
- type: grid
cards:
- type: markdown
content: >
**Minisplit** — climate entity lands with the Carrier dongle
commissioning (#629). Manual setpoint tonight: 62°F, with the
oscillating fan improving distribution. Alert tiers:
ambient 104°F · drives 140°F · NVMe 150°F · PCH 160°F ·
GPU 185°F.
- title: Racks & silicon
type: sections
sections:
- type: grid
cards:
- type: entities
title: R3 · PowerEdge (DRAC)
entities:
- sensor.pfv_tsys6_cpu_temperature
- sensor.pfv_tsys6_drac_ambient
- sensor.pfv_tsys7_cpu_temperature
- sensor.pfv_tsys7_drac_inlet
- sensor.pfv_tsys7_drac_exhaust
- sensor.pfv_tsys7_drac_cpu1
- sensor.pfv_tsys7_drac_cpu2
- type: grid
cards:
- type: entities
title: R3 · Pis + Jetson
entities:
- sensor.pfv_svrpi_temperature
- sensor.pfv_subopi3_temperature
- sensor.pfv_subopi_dev_3_temperature
- sensor.pfv_subopi_dev_4_temperature
- sensor.pfv_jetson_temperature
- sensor.pfv_jetson_gpu_temperature
- type: grid
cards:
- type: entities
title: R5 · Laptops
entities:
- sensor.pfv_tsys3_cpu_temperature
- sensor.pfv_tsys3_gpu_temperature
- sensor.pfv_tsys3_nvme_temperature
- sensor.pfv_tsys5_nvme_temperature
- type: grid
cards:
- type: entities
title: R5 · Desktops / OptiPlex-class
entities:
- sensor.pfv_tsys1_rack_temperature
- sensor.pfv_tsys4_ambient_temperature
- sensor.pfv_tsys1_cpu_temperature
- sensor.pfv_tsys4_cpu_temperature
- sensor.pfv_tsys9_cpu_temperature
- sensor.pfv_tsys9_pch_temperature
- type: grid
cards:
- type: history-graph
title: CPU temperatures (24h)
entities:
- sensor.pfv_tsys1_cpu_temperature
- sensor.pfv_tsys3_cpu_temperature
- sensor.pfv_tsys6_cpu_temperature
- sensor.pfv_tsys7_cpu_temperature
- sensor.pfv_tsys9_cpu_temperature
hours_to_show: 24
+11 -4
View File
@@ -1,4 +1,4 @@
# Deploy pipeline [#734]: gitea master-HEAD sha watcher. Feeds the # Deploy pipeline [#734]: gitea release-branch sha watcher. Feeds the
# sha-watch deploy automation (reload-only apply). The gitea webhook # sha-watch deploy automation (reload-only apply). The gitea webhook
# (gitea hook #10 -> pfv_deploy_git_webhook automation) is the fast path # (gitea hook #10 -> pfv_deploy_git_webhook automation) is the fast path
# with full restart mapping, but gitea cannot deliver webhooks to # with full restart mapping, but gitea cannot deliver webhooks to
@@ -6,16 +6,23 @@
# (founder action via Cloudron file manager); this watcher is the # (founder action via Cloudron file manager); this watcher is the
# always-works safety net. # always-works safety net.
rest: rest:
- resource: https://git.knownelement.com/api/v1/repos/KNEL/pfv-bms/commits?limit=1 - resource: https://git.knownelement.com/api/v1/repos/KNEL/KNELBMS/commits?sha=release&limit=1
headers: headers:
Authorization: !secret gitea_auth_header Authorization: !secret gitea_auth_header
User-Agent: pfv-bms-deploy-watch User-Agent: pfv-bms-deploy-watch
scan_interval: 300 scan_interval: 300
verify_ssl: true verify_ssl: true
sensor: sensor:
- name: PFV deploy gitea master head - name: PFV deploy gitea release head
unique_id: pfv_deploy_gitea_master_head unique_id: pfv_deploy_gitea_release_head
value_template: "{{ value_json[0].sha if value_json is list and value_json | length > 0 else (value_json | string | truncate(90)) }}" value_template: "{{ value_json[0].sha if value_json is list and value_json | length > 0 else (value_json | string | truncate(90)) }}"
# E2E self-deploy proof commit [#734] - if you can read this in /config, # E2E self-deploy proof commit [#734] - if you can read this in /config,
# the sha-watch deployed it autonomously. # the sha-watch deployed it autonomously.
# Dead-man heartbeat (#792): pushes OK to the out-of-band Uptime Kuma
# (Cloudron, status.knownelement.com) every 5 minutes. If HA stops
# pushing — crash, wedge, power loss, network partition — Kuma pages
# the founder's phone directly. No HA in the alert loop.
shell_command:
kuma_ha_heartbeat: 'curl -sS -m 15 "{{ kuma_push_url }}"'
+11 -1
View File
@@ -7,6 +7,7 @@
# PR gate). The roster is fetched live from the gitea API — NO git # PR gate). The roster is fetched live from the gitea API — NO git
# mechanics, NO submodule (git_pull add-on lacks submodule support), # mechanics, NO submodule (git_pull add-on lacks submodule support),
# NO local copy: access-roster remains the single source of truth. # NO local copy: access-roster remains the single source of truth.
# Reader/door contract (webhook JSON + unlock endpoint): KNEL/doorman README
# #
# UNLOCK PATH (founder-approved 2026-09-03): valid badge + the # UNLOCK PATH (founder-approved 2026-09-03): valid badge + the
# doorman_unlock_enabled input_boolean ON -> rest_command fires the # doorman_unlock_enabled input_boolean ON -> rest_command fires the
@@ -81,6 +82,15 @@ input_boolean:
name: Doorman unlock armed name: Doorman unlock armed
icon: mdi:lock-open-variant icon: mdi:lock-open-variant
shell_command:
# Unlock dispatch via curl — swapped from rest_command 2026-09-04: the
# rest_command integration silently fails to register on this box
# (404 on /api/services/rest_command despite valid config + secret;
# root-cause pending in #345/#741). curl is present in the HA core
# container. Token lives in the URL secret — process-list exposure
# accepted (isolated container).
doorman_unlock: "curl -sS -m 10 '{{ doorman_unlock_url }}'"
rest_command: rest_command:
# Fires the doorctl endpoint on pfvsvrpi (systemd socket-activated, # Fires the doorctl endpoint on pfvsvrpi (systemd socket-activated,
# Tailscale-bound, token+source allowlisted; self-securing relay). # Tailscale-bound, token+source allowlisted; self-securing relay).
@@ -173,7 +183,7 @@ automation:
{{ is_state('input_boolean.doorman_unlock_enabled', 'on') {{ is_state('input_boolean.doorman_unlock_enabled', 'on')
and not (trigger.json.local_unlock | default(false) | bool(false)) }} and not (trigger.json.local_unlock | default(false) | bool(false)) }}
then: then:
- action: rest_command.doorman_unlock - action: shell_command.doorman_unlock
continue_on_error: true continue_on_error: true
- action: logbook.log - action: logbook.log
data: data:
+7
View File
@@ -800,6 +800,13 @@ template:
state: >- state: >-
{% set v = states('sensor.tsys1_ups_battery_runtime') %} {% set v = states('sensor.tsys1_ups_battery_runtime') %}
{{ (v | float / 60) | round(0) if is_number(v) else none }} {{ (v | float / 60) | round(0) if is_number(v) else none }}
- name: "PFV Tripp Lite runtime minutes"
unique_id: pfv_triplite_runtime_minutes
device_class: duration
unit_of_measurement: "min"
state: >-
{% set v = states('sensor.tsys1_triplite_battery_runtime') %}
{{ (v | float / 60) | round(0) if is_number(v) else none }}
- name: "PFV tsys1 drive temperature" - name: "PFV tsys1 drive temperature"
unique_id: pfv_tsys1_drive_temperature unique_id: pfv_tsys1_drive_temperature
device_class: temperature device_class: temperature
+1
View File
@@ -0,0 +1 @@
# intentionally empty — scenes are authored in the HA UI if ever needed
+1
View File
@@ -0,0 +1 @@
# intentionally empty — scripts are authored in the HA UI if ever needed
+51
View File
@@ -0,0 +1,51 @@
#!/bin/bash
# SDLC regression tests (#778) — run in CI (see .gitea/workflows/ci.yml).
# Each test guards a rule that exists because of a real incident or ruling.
set -uo pipefail
cd "$(dirname "$0")/.." || exit 1
FAIL=0
t() {
# $1 = label, $2 = shell command string
if eval "$2"; then
echo " OK $1"
else
echo " FAIL $1"
FAIL=$((FAIL + 1))
fi
}
echo "=== alert twins (every alert ships a recovery twin) ==="
t "utility power restored twin" \
"grep -q 'id: pfv_plant_ups_power_restored' automations.yaml"
t "temperature recovered twin" \
"grep -q 'id: pfv_plant_temperature_recovered' automations.yaml"
echo "=== alert deep links (taps land on dashboards, #344) ==="
ALERTS="POWER OUTAGE|UPS BATTERY LOW|UPS COMMS LOST|UPS RUNTIME LOW|Power restored|HIGH TEMPERATURE|SENSOR WATCHDOG|TEMP RISING FAST|TEMPERATURE RECOVERED"
DL_OK=1
IFS='|' read -ra ALERT_ARR <<< "$ALERTS"
for a in "${ALERT_ARR[@]}"; do
if ! grep -A2 "title: $a$" automations.yaml | grep -q " url: "; then
echo " (missing deep link: $a)"
DL_OK=0
fi
done
t "all alerts deep-linked" "[ \"$DL_OK\" -eq 1 ]"
echo "=== watchdog coverage (#789) ==="
t "watchdog watches the deploy sensor" \
"grep -q 'sensor.pfv_deploy_gitea_release_head, 30' automations.yaml"
t "watchdog automation exists" \
"grep -q 'id: pfv_watchdog_stale_entities' automations.yaml"
echo "=== ghost entities (dead references never return) ==="
t "no tsys9_ups ghost refs" \
"! grep -rq 'tsys9_ups' automations.yaml packages/ dashboards/"
echo "=== secrets ==="
t "no private keys in tree" \
"! grep -rInE 'BEGIN (RSA |OPENSSH |EC |DSA )?PRIVATE KEY' --exclude-dir=.git --exclude-dir=docs ."
echo "=== SDLC SUITE: $FAIL failures ==="
exit "$FAIL"
+2
View File
@@ -0,0 +1,2 @@
# intentional placeholder: themes dir must exist for the
# !include_dir_merge_named in configuration.yaml [#784 gardening]