- cnodes 102/705/603 moved to TS5-SSD storage by founder (etcd apply 4-5s -> <1s)
- wnode-tsys5 slot retired; ultix-streaming joins in its place (5 workers)
- ultix-offstage unreachable; stays excluded until back on tailnet
[#367][#368] https://projects.knownelement.com/issues/368
Live proof: ecobee answers unicast mDNS with PTR only
(_hap._tcp.local -> "Main Floor._hap._tcp.local"), so the old
replace-on-probe wiped learned records every cycle. Merge by
(name, type) instead. CONFIG_PATH now env-overridable for tests.
Details: https://projects.knownelement.com/issues/619#note-5💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Archive the non-ported remainder of the legacy KNELServerBuild repo
under archive/KNELServerBuild/ with its original structure intact,
completing the legacy repo merge for everything except the live
LibreNMS patterns (ported in the previous commit).
Exclusions:
- .git history (superseded; legacy repo remains at its original path)
- ported files (Agents/librenms, Modules/OAM/oam-librenms.sh,
ConfigFiles/SNMP/snmp-sudo.conf)
- vendored KNELShellFramework tree (byte-identical duplicate of the
copy already vendored at vendor/ in this repo)
- SSH authorized-keys files (live access-control material; carrying
them in an archive invites drift — key policy lives elsewhere)
The whole tree is skip-listed in tests/shellcheck.sh (archived legacy
code, not maintained — same standing as vendor/); check-rules.sh
already prunes archive/. Rule 9 (conflict markers) now also excludes
archive/ staged files: preserved-verbatim legacy scripts contain
decorative "====" banners that false-positive as conflict markers
(same archive exclusion precedent as rule 11).
AGENTS.md: note archive/KNELServerBuild and oam/librenms-agent in the
Repository Layout, and fix the stale KNELIAC path to
/home/reachableceo/projects/KNEL/KNELIAC.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Port the live LibreNMS monitoring patterns from the legacy
KNELServerBuild repo into oam/librenms-agent/, joining the existing
OAM tooling (oxidized, unpoller, smokeping, netdisco):
- agent/ — upstream check_mk agent + snmp-extend scripts (dmi, dpkg,
mysql, ntp-client, ntp-server, os-updates, postfix, raspberry, smart,
ss, ups-nut), copied verbatim (md5-verified), never to be edited here
- setup.sh — deploy module ported from ProjectCode/Modules/OAM/
oam-librenms.sh; only the legacy framework bootstrap was replaced
with plain bash (path constants + print_info -> echo)
- snmp-sudo.conf — Debian snmpd sudo rule the extends require
(Debian-snmp NOPASSWD /bin/cat); carried as a file only, sudoers
install is a policy decision per AGENTS.md
Lint gates: extend the existing upstream-skip mechanism for the
verbatim agent scripts (tests/shellcheck.sh + check-rules.sh prune,
same precedent as archive/provisioning/Agents/librenms), and fix the
stale skip path there (provisioning/ moved to archive/provisioning in
6244c1c; the bash extends have been failing the whole-repo shellcheck
gate since).
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
DNS/DHCP/NTP redundancy per founder ruling 2026-08-27:
- DHCP failover converted load-balance -> hot standby (split 255 on
primary; secondary answers only if primary unreachable >3s). DHCP
options now hand out BOTH nodes for DNS and NTP (active/active) —
Tailscale hosts stay on MagicDNS 100.100.100.100.
- netinfra-02 ntpsec deployed (pool.ntp.org, all interfaces); 01
repointed from debian pool zones to pool.ntp.org. NTP now redundant;
GPS stratum-1 on pfvsvrpi tracked as follow-up ticket.
- snmpd on both nodes with scoped ACLs (LibreNMS LAN+TS sources only)
and ntpq/dhcpd-lease extends — prep for DNS/DHCP/NTP graphing.
- Deployed sync-zones.sh (git version, DNS name instead of IP literal)
to netinfra-02; zone sync verified 171/171 zones both nodes.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
The ticket gate required the single .crush/active-ticket file, so two
concurrent sessions overwrote each other's ticket mid-work. The gate now
accepts any non-empty .crush/active-ticket* file; each session keeps its
own (e.g. active-ticket-plant, active-ticket-core). Also prune .crush/
session scratch from both shellcheck scanners so one session's throwaway
probe scripts cannot block the other session's commits. Documented in
AGENTS.md Task Tracking.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
lm-sensors is silent on SBCs (Pis, Jetson-class), so the wrapper now falls
back to /sys/class/thermal zones rendered in lm-sensors format when native
sensors output is empty; suppressed when native output exists to avoid
duplicates. TDD: 3 new unit tests (12 green). Sensor stack deployed to
jetson + pfvsvrpi + 3 subopis (verified from poller), HA pack extended to
16 sensors across 14 hosts, high-temp automation covers the SBC fleet.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
SNMP sensors for all 9 instrumented hosts (temps in °F per founder
ruling, °C converted at display layer), ACPI node-wattage feeds for
tsys6/7, UPS alert automations (on-battery, battery-low, comm-lost,
power-restored, high-temp) targeting Pushover, and Riemann-sum kWh
helpers for the Energy dashboard. PDU and iDRAC blocks left as pinned
placeholders pending PDU community and #462 OMSA work.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
The reservation carried a4:ba:db:0b:df:a0 (core-switch OUI, mis-migrated
from pfv-netboot) so tsys6's iDRAC could never lease its reserved address.
Replaced with the in-band-verified BMC MAC; deployed to both failover
nodes, validated, restarted.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Native lm-sensors output plus optional TEMPer USB probe, unified in
lm-sensors format behind one snmpd extend (lmsensors). Idempotent
deploy binds snmpd to explicit LAN+Tailscale addresses only and
source-scopes the community to the pollers (LibreNMS, Home Assistant
app-connector LAN IP, admin workstation, Cloudron) per the founder's
security ruling. TDD unit suite included; fleet verified live (6 hosts)
and negative-tested (refused source).
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
The live configs were edited in place (canonical renames, stale block
removal, 7 new reservations, 69 total) but the repo copies were never
updated — the exact drift class that bit pi-hole. Repo and live state
now match on both failover nodes.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
EOF
Bring the framework's global baseline (BASELINE-PROMPT.md, PATTERNS.md,
ADOPTING.md) into the repo — AGENTS.md referenced them but the files were
missing. AGENTS.md gains three rules adopted this session: TDD & Linting
(mandatory at the Ansible/IaC transition), IaC codification shadow-tracking
(every manual fleet change same-day ticketed to #454), and Redmine tracker
discipline (Support not Bug — the CLI default created nine misfiled
tickets, now corrected). Q10 records the direct-push vs PR conflict for
the founder to rule on.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
pi-hole on netinfra-01/02 now conditionally forwards 192.168.0.0/16
reverse lookups to this node's Technitium over dnsnet, alongside the
existing knel.net + 100.64/10 pair. Technitium stays the single source
of truth; pollers (NetDisco, phpIPAM, UNPoller, Wazuh soon) resolving
via either node's :53 now get LAN PTRs. Both live revServer arrays and
this compose file are in sync; validated with forward, PTR (192.168 +
100.x), and external lookups against both nodes.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
The full legacy suite mixes repo tests with deployment-target tests
(PAM packages, target resolv.conf, Proxmox repo reachability) that can
only pass on a provisioned server. The workstation gate now runs the
unit suite; validation/security stay invocable for sectestbed runs.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Fill the framework gaps the Makefile already referenced: scripts/test.sh
(wrapper over tests/run-tests.sh, now exercised by the pre-push full
audit) and up.sh/down.sh stubs for this non-compose repo. Extend
kuma-fleet-sync STATIC_GEAR with the fixed network/office devices
(printer, consrv, tsys6/7 OOB, scanners, label printer, r1-tor-top,
DOME) so future runs keep their ICMP monitors in sync; DOME added
paused like the other known-down systems.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Gap-analyzes the tailnet Linux fleet + static network gear (switches,
router, APs, PDU, stor1, Reston VPSes) against Uptime Kuma ping
monitors over the socket.io API, and can create missing monitors with
--add. Websocket transport is forced because the Cloudron proxy drops
engine.io polling pushes. First run closed the last 4 gaps: 84/84.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
pfv-bms and tsys-umbrel were renamed in the Tailscale admin console,
so MagicDNS now serves the canonical names. The audit key switches
from the OS-reported hostname to the admin-assigned DNS name, which
retires the alias table. Reston VPS exclusion keyed to its MagicDNS
name (tsys-cloudron). Every in-scope system now passes: VM name,
DNS name, and Tailscale name all agree.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Names verified matching (netbird TS/DNS) — no fleet access audit
needed, same treatment as the other Reston systems.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Device-side hostname, /etc/hosts, and Tailscale node renamed from
ultixfield to ultix-field to match the DNS canonical name (convention
match with ultix-streaming/ultix-offstage). Verified unprivileged path
is the ultixfield account with passwordless sudo.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
With UCS and Jetson keys in place every auditable system now passes:
68 OK / 2 by-design (Home Assistant, Umbrel). Zero failures.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
3dscanner has been offline ~7 months; sandbox is a disposable
break-fix VM rolled back via Proxmox snapshots, so a failed SSH
probe there is expected noise, not a finding.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
era stlpc uses root, pfvsvrpi is localuser not subodev [#446]
Probed root access on every host where localuser failed: all Proxmox
appliance VMs (PDM/PMG/PBS), stlpc-artroom/garage, pfv-jetson-nano-1,
and ultix-field accept root key auth; pfvsvrpi takes localuser with
passwordless sudo. Audit now models this correctly — 65 of 74 green.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Exclude the Reston prod VPS (tsys-cloudron-new) from all future audits,
mark Home Assistant and Umbrel as by-design no-SSH appliances, and query
Technitium directly (dig @netinfra-01) instead of getent so local
/etc/hosts entries cannot produce false mismatches. Drops aliases made
obsolete by the sectestbed-hfnoc-uisp rename.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Reusable audit over every Linux tailnet peer: knel.net DNS resolution vs
Tailscale IP, Proxmox VM-name match, expected-user SSH login via the
remote.sh/remote-dns.sh chokepoints, and passwordless sudo verification.
First run (2026-08-26): 54/74 peers fully green, report on Discourse #312.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
If a tool has an API, use it. Exhaust all API endpoints before
considering direct DB access. DB schemas change without warning;
APIs are the stable contract. Aligns with upcoming zero trust
proxy migration.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
- UAT rule: user must accept work before done-ratio 100% or close
- check-rules.sh: prune .crush/ from Discourse pointer scan (agent
working space, not documentation)
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
The HUD is an ephemeral status block (active ticket, done items, current
work, queued items, user action items) shown after significant responses.
Written to .crush/hud.md on disk to save context tokens; never committed.
Redmine/Discourse/git remain the durable systems of record.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
mrcharles