fix(netinfra): commit Pi-hole hardening config previously missing from repo

Commit 33b5c76 claimed to harden Pi-hole on both DNS nodes but only
modified markdown — the working docker-compose.yml, gravity-validate.sh,
healthcheck, and autoheal config were never written to the repo, leaving
the DNS hardening unreproducible from version control.

This commits the live, verified-working config from the boxes into
netinfra/pihole/:
- docker-compose.yml (shm_size 1024M root-cause fix, healthcheck, autoheal)
- gravity-validate.sh (pre-start SQLite header check, auto-quarantine corrupt DB)
- .env.example (web UI password templated; real .env gitignored)

Defends against the gravity.db / /dev/shm corruption production outage.
The live password is templated as ${PIHOLE_WEB_PASSWORD} so no secret
enters git.

[#376]

.gitignore

@@ -37,3 +37,6 @@ returned-logs/
 # Kubernetes secrets (kubeconfig contains embedded client certs)
 k8s/kubeconfig.yaml
 k8s/*.token
+
+# Pi-hole web UI password (real value only in on-box .env, never committed)
+netinfra/pihole/.env

AGENTS.md

@@ -1,8 +1,9 @@
 # Agent Guidelines
 
 **Top-level files:** [`README.md`](README.md) (project overview),
-[`STATUS.md`](STATUS.md) (living status, agent-maintained),
-[`docs/docmap.md`](docs/docmap.md) (documentation index). Everything else
+[`STATUS.md`](STATUS.md) (ticket index + infra summary),
+[`docs/docmap.md`](docs/docmap.md) (documentation index). All work is
+tracked in [Redmine](https://projects.knownelement.com). Everything else
 lives in subdirectories.
 
 ## Repository Layout
@@ -54,12 +55,34 @@ vendor/             Vendored KNELShellFramework
    justification. A script that emits any diagnostic is a protocol violation.
    Non-bash scripts (PHP with `.sh` shebang `#!/usr/bin/php`, etc.) are exempt.
 
+## Redmine Tracking Policy
+
+**Redmine is the system of record for all work.** Do not track status,
+checklists, or TODOs in repo files. Use Redmine tickets instead.
+
+- **URL:** https://projects.knownelement.com
+- **Version:** Potential to Kinetic Ready (due 2026-09-30)
+- **Project:** Known Element Enterprises - Technology & Facility Services (id 55)
+
+### Rules
+1. **Every piece of work** (feature, fix, deployment, config change) gets a
+   Redmine ticket. If one doesn't exist, create it.
+2. **Reference tickets in docs and commits** using `[#NNN]` notation.
+   Example: `[#367] Rebuilt k3s control plane after cnode wipe`.
+3. **When work completes**, update the ticket: set done ratio to 100%,
+   add a note describing what was done and where the code lives, then close.
+4. **STATUS.md is a read-only snapshot** — it contains a ticket index and
+   infrastructure summary only. Do not add checklists or status items.
+5. **Link code to tickets** — ticket descriptions and notes should reference
+   the relevant file paths in this repo (e.g., `dcinfra/ups/`).
+
 ## Automatic Gardening Protocol
 
 **Docs and code must be kept in sync.** After any work session, an agent MUST:
 
-1. **Update [`STATUS.md`](STATUS.md)** — reflect completed work, new issues,
-   changed infrastructure state. This file is human read-only; agents own it.
+1. **Update [`STATUS.md`](STATUS.md)** — update the ticket index table if
+   tickets were created or closed. This file is a read-only snapshot;
+   do not add inline status tracking (use Redmine).
 2. **Update [`docs/docmap.md`](docs/docmap.md)** — if a doc was added,
    removed, or substantively changed, update the table and "Last Reviewed"
    date.
@@ -104,5 +127,6 @@ vendor/             Vendored KNELShellFramework
 
 Solo-founder R&D Proxmox cluster in a private residence. Shoestring budget.
 Production lives on a Cloudron VPS in Reston VA. See
-[`STATUS.md`](STATUS.md) for current state and
-[`proxmox/docs/PROJECT.md`](proxmox/docs/PROJECT.md) for the fleet report.
+[`STATUS.md`](STATUS.md) for the ticket index and infrastructure summary.
+All work is tracked in [Redmine](https://projects.knownelement.com)
+(version: Potential to Kinetic Ready, due 2026-09-30).

STATUS.md

@@ -1,227 +1,78 @@
 # Project Status
 
-> **Human read-only. Agents maintain this file automatically after each work
-> session.** Do not edit by hand — the next agent run will overwrite it.
-
-> **Last updated:** 2026-08-05 by Crush (GLM-5.2) — full read-only ground-truth
-> re-orientation (nmap + SNMP + SSH + API queries).
+> **Redmine is the system of record for all work tracking.**
+> https://projects.knownelement.com — Version: **Potential to Kinetic Ready** (due 2026-09-30)
 >
-> **Current authoritative audit:** [`proxmox/docs/AUDIT-2026-08-05.md`](proxmox/docs/AUDIT-2026-08-05.md)
-> — repo restructure, 4 switches up / 1 down, 63 VMs, 167 DNS zones, 753 DHCP
-> leases, k8s cluster DOWN (cnode wiped), netinfra-02 wiped, tsys5 storage
-> configured, tsys4 upgraded, 3-tier Tailscale fleet (80 nodes).
-> Capacity targets are codified in
-> [`proxmox/docs/CAPACITY-GOALS.md`](proxmox/docs/CAPACITY-GOALS.md): 80% RAM
-> steady-state (fleet PASSES, highest tsys1 at 71%).
+> This file is now a **read-only snapshot + ticket index**. Do not add
+> checklists or status items here. All work is tracked as Redmine tickets.
+> When work is done, update the ticket (done ratio, notes, close).
 
-## Current State: MIXED (infra stable; k8s + netinfra-02 need rebuild)
+> **Last updated:** 2026-08-06 — Pi-hole hardening config committed (`netinfra/pihole/`, [#376]).
+
+## Current State: MIXED (infra stable; k8s needs rebuild)
 
 Provisioning, DNS primary (netinfra-01), Proxmox fleet, and monitoring stack
-are operational. **k8s control plane is DOWN** (cnode wiped, no etcd quorum)
-and **netinfra-02 is wiped** (needs rebuild as redundant secondary). Repo was
-restructured into a module/subproject layout (see
-[`proxmox/docs/AUDIT-2026-08-05.md`](proxmox/docs/AUDIT-2026-08-05.md)).
+are operational. **k8s control plane is DOWN** (cnode wiped, no etcd quorum).
 
-## Completed Work
+## Completed Work (tickets closed @ 100%)
 
-### Server Provisioning (validated on sectestbed-sandbox)
-- [x] SetupNewSystem.sh deploys end-to-end to rc=0 (Debian 13 trixie)
-- [x] Security hardening: SSH, SCAP-STIG, Wazuh, 2FA (SSH+Cockpit+Webmin)
-- [x] NTP: redundant pfv-netinfra-01/02 (192.168.3.252/253), synced stratum 3
-- [x] DNS resolv.conf: managed static file pointing at netinfra pair
-- [x] Test suite: 5 tests (framework, safe-download, 2fa, https, system-req)
-- [x] VM validation harness: git-based deploy + auto-rollback + guest-agent access
+| Ticket | Component | Code location |
+|--------|-----------|---------------|
+| [#334](https://projects.knownelement.com/issues/334) | NTP synchronization (redundant stratum 3) | `netinfra/` |
+| [#340](https://projects.knownelement.com/issues/340) | UPS management — NUT on pfv-tsys1 (APC Smart-UPS) | `dcinfra/ups/` |
+| [#357](https://projects.knownelement.com/issues/357) | DNS cluster — Technitium primary/secondary + DHCP failover | `netinfra/dns-cluster-setup/` |
+| [#358](https://projects.knownelement.com/issues/358) | Proxmox perf tuning (5 of 7 hosts) | `proxmox/perf/` |
+| [#359](https://projects.knownelement.com/issues/359) | PDU management — powerman on pfv-tsys1 | `dcinfra/powerman/` |
+| [#360](https://projects.knownelement.com/issues/360) | Console management — ser2net + conman on pfv-tsys4 | `dcinfra/console/` |
+| [#361](https://projects.knownelement.com/issues/361) | AWX deployment — k3s + AWX Operator | `awx/` |
+| [#362](https://projects.knownelement.com/issues/362) | k8s control plane build — k3s HA (initial, now needs rebuild) | `k8s/` |
+| [#363](https://projects.knownelement.com/issues/363) | tsys5 storage configuration (NVMe + SSD) | `proxmox/docs/` |
+| [#364](https://projects.knownelement.com/issues/364) | tsys4 PVE upgrade to 9.2.5 | `proxmox/docs/` |
+| [#365](https://projects.knownelement.com/issues/365) | Repo merge — KNELServerBuild into PFVCluster | repo root |
+| [#366](https://projects.knownelement.com/issues/366) | Server provisioning pipeline (SetupNewSystem.sh → KNELIAC) | `archive/provisioning/`, KNELIAC |
 
-### DNS Cluster (pfv-netinfra-01/02)
-- [x] Production Technitium config replicated from tailscale-router (read-only)
-- [x] 124 zones on both nodes (knel.net + reverse DNS)
-- [x] pfv-netinfra-01 = PRIMARY, pfv-netinfra-02 = SECONDARY
-- [x] Zone replication via rsync systemd timer (every 60s)
-- [x] Credentials + 2FA replicated identically to production
-- [x] Both LAN IPs resolve knel.net device names + recurse externally
-- [x] **DHCP migrated from pfv-netboot → netinfra-01/02** (ISC failover-peer, primary/secondary). Both `normal`, **753 active leases** (was 333). DNS+NTP now handed out as 252/253. Netboot DHCP stopped + disabled. Webmin DHCP module on both nodes.
-- [ ] **netinfra-02 REBUILT 2026-08-05** (from operator's accidental wipe):
-  Docker installed, Technitium (167 zones, zone sync timer active), Pi-hole v6
-  (healthy, gravity.db healthcheck + autoheal), NTP container, DHCP failover
-  secondary configured and serving (99+84 active leases load-balanced). All
-  services operational and redundant.
-- [x] **Technitium now runs as Docker container** (`tsys-dns`) on netinfra-01,
-  not systemd dnsServer. Config at `/home/localuser/services/technitium/config`.
-  **167 zones** (was 124). NTP = `ntpsec` (not chrony).
-- [x] **Pi-hole hardened on BOTH nodes**: shm_size=1024M (operator root-cause fix),
-  custom gravity-validate.sh entrypoint (SQLite header check + auto-recovery),
-  healthcheck validates DNS + gravity.db, autoheal container restarts unhealthy
-  containers. Deployed to netinfra-01 and netinfra-02.
+## Open Work (tickets in progress or queued)
 
-### Proxmox Cluster Ops
-- [x] 5 of 7 hosts fully performance-tuned (tsys1/3/6/7/9)
-- [x] **Full ground-truth audit 2026-07-30** (network + storage + compute + switches)
-- [x] **Cross-rack NFS root cause identified:** core-sw01 LAG hash=layer-2-3 (no L4),
-  g16/g17 cable mismatch (3 of 4 links active), static mode (no LACP)
-- [x] **tsys4/tsys5 storage changes documented:** new NVMe (CT500P1 500GB), relocated
-  SK hynix SSD, D3 changed to USB HDD, /mnt/albert + /mnt/backup added
-- [x] **Console symlink bug fixed:** udev rule singular→plural mismatch broke all 7
-  console ports after reboot
-- [x] **Portable audit tooling built:** probe-storage.sh, probe-network.sh,
-  conman-console.py, snmp-switch-audit.py (AGPLv3-friendly, config-driven)
-- [ ] **Memory goal (≤80% actual): PASS fleet-wide** — highest is tsys9 at 69%
-- [ ] CPU ~50%-idle goal: fleet idles 0-5% (under-utilized; chase via workloads, not hardware)
-- [ ] tsys4: PVE 9.1.5 (behind fleet 9.2.5), needs upgrade
-- [ ] tsys5: NVMe + SK hynix SSD installed but NOT configured
-- [ ] tsys2: pending rebuild from Win10 to Proxmox
+| Ticket | Component | Notes |
+|--------|-----------|-------|
+| [#376](https://projects.knownelement.com/issues/376) | DNS cluster up/down alerts + Pi-hole hardening committed | Config now in `netinfra/pihole/`; flapping source under investigation |
+| [#367](https://projects.knownelement.com/issues/367) | k8s cluster rebuild (cnode wiped, no quorum) | Blocks #368 |
+| [#368](https://projects.knownelement.com/issues/368) | k8s worker nodes join cluster | Blocked by #367 |
+| [#369](https://projects.knownelement.com/issues/369) | Switch config fixes (core-sw01 LAG, LACP, member list) | Onsite Friday |
+| [#370](https://projects.knownelement.com/issues/370) | Vault — migrate all API keys (HIGH priority) | |
+| [#371](https://projects.knownelement.com/issues/371) | AWX HTTPS/TLS configuration | Depends on #342 (PKI/SSL) |
+| [#372](https://projects.knownelement.com/issues/372) | Tripp Lite UPS fix (physical reseat) | Onsite Friday |
+| [#373](https://projects.knownelement.com/issues/373) | pfv-r2-sw console fix (physical, onsite) | Onsite Friday |
+| [#374](https://projects.knownelement.com/issues/374) | PDU outlet renaming | |
+| [#375](https://projects.knownelement.com/issues/375) | Monitoring expansion (unpoller, netdisco, weathermap, smokeping) | Relates to #337, #343 |
+| [#311](https://projects.knownelement.com/issues/311) | OAM: Security Hardening (CMMC/STIG) | KNELIAC roles deployed, expansion ongoing |
+| [#335](https://projects.knownelement.com/issues/335) | OAM: Centralized syslog to Wazuh | Wazuh agents deployed, fleet rollout pending |
+| [#342](https://projects.knownelement.com/issues/342) | OAM: PKI/SSL — tsys-ca VM + HSM CA + ACME | Vault unconfigured, AWX HTTP-only |
+| [#343](https://projects.knownelement.com/issues/343) | OAM: Monitoring coverage matrix | Coverage gaps documented |
 
-### Kubernetes Control Plane (k3s HA — DOWN, needs rebuild)
-- [ ] **Cluster DOWN 2026-08-05**: operator wiped a cnode + shut cnodes down
-  (100% RAM on one). cnode2 has k3s API up, cnode1/cnode3 do not — no etcd
-  quorum. Kubeconfig points to stale IP (100.97.178.106, matches no cnode).
-- [x] Was: 3-node k3s HA control plane (cnode1/2/3, embedded etcd, Tailscale)
-- [x] **All traffic over Tailscale IPs** — no LAN IPs in node status or certs
-- [x] All 3 cnodes tainted `control-plane:NoSchedule` (zero user workloads)
-- [x] 13/13 health checks pass (verify.sh): nodes Ready, etcd quorum,
-  Tailscale IPs, CoreDNS, API server, workload isolation
-- [x] Scripts in `k8s/`: wipe, install-cp, join-servers, post-setup, verify
-- [x] Kubeconfig saved to `~/.kube/config.pfv-k8s` (gitignored, embedded certs)
-- [ ] Workers (wnodes) not yet joined to this cluster
-- [ ] Distro decision: **k3s chosen for regular R&D cluster**. Talos docs
-  preserved in `k8s/docs/` for future ITAR/classified cluster.
+## Known Issues (tracked in tickets)
 
-### PDU Management (powerman on pfv-tsys1 — LIVE)
-- [x] Cyclades AlterPath PM10i (10 outlets) managed via powerman over serial
-- [x] USB-DB9 adapter (Prolific pl2303) with stable udev symlink
-  `/dev/cyclades-pm10`
-- [x] powermand listening on `127.0.0.1:10101` + Tailscale `100.121.189.98:10101`
-- [x] All 10 outlets defined as `outlet-1` through `outlet-10`
-- [x] Validated: outlet 10 cycled off → on (8/8 test checks passed)
-- [ ] Rename outlets to match physical devices (Friday onsite)
-
-### UPS Management (NUT on pfv-tsys1 — LIVE)
-- [x] **APC Smart-UPS C 1500** fully reporting via NUT usbhid-ups + APC HID 0.100
-- [x] upsd listening on Tailscale `100.121.189.98:3493` (port 3493)
-- [x] Local upsmon (master) configured for graceful hypervisor shutdown
-- [x] Home Assistant credentials generated (`homeassistant` user in upsd.users)
-- [x] Scripts in `ups/`: discover, setup (idempotent), status
-- [x] Home Assistant NUT integration **configured via REST API** (`ups/setup-ha-nut.sh`)
-- [x] **Key fix:** upsd now listens on LAN IP (192.168.3.11) in addition to
-  Tailscale — HAOS runs Tailscale as an isolated add-on, so the HA container
-  cannot route to Tailscale IPs. Shared vmbr0 bridge provides instant LAN path.
-- [x] **Key fix:** udev rule covers BOTH `usb` and `hidraw` subsystems (driver
-  opens raw `/dev/bus/usb` after setuid drop, not `/dev/hidraw`)
-- [ ] **Tripp Lite UPS blocked:** driver matches TrippLite HID 0.85 subdriver
-  but fails reading 878-byte HID Report Descriptor (EAGAIN via USB hub).
-  Needs physical reseat to direct motherboard USB port or cable replacement.
-  Driver masked to prevent restart-loop spam.
-- [ ] **Home Assistant NUT integration:** configured and live (3 sensors:
-  battery charge, status, status data). This APC (FW 02.2, mfg 2012) does not
-  report `ups.load` over USB HID — confirmed via both NUT and apcupsd tests.
-  No wattage for Energy Dashboard; smart plug on UPS output is the practical
-  path. apcupsd package conflicts with nut-server (mutually exclusive).
-
-### Console Management (ser2net telnet(rfc2217) + conman on pfv-tsys4 — LIVE)
-- [x] 7 network switch/router consoles managed via ser2net + conman
-- [x] **USB enumeration problem SOLVED:** udev rules pin each adapter by
-  ID_PATH (physical USB port topology) to stable `/dev/consoles/<name>`
-  symlinks that survive reboot regardless of enumeration order
-- [x] ser2net exposes all 7 consoles on telnet(rfc2217) TCP ports (2001-2007)
-  bound to **Tailscale IP only** (`100.70.77.93`)
-- [x] conman connects to TCP ports for logging + multiplexing (7 log
-  files active in `/var/log/conman/`)
-- [x] Both ser2net + conmand enabled via systemd (survive reboot)
-- [x] conmand systemd unit created (Debian package doesn't ship one)
-- [x] conmand remote access enabled (loopback=off, port 7890)
-- [x] Old `/root/conmap` + manual `screen` workflow replaced
-- [x] **Stair-stepping fix:** ser2net accepter changed from raw `tcp` to
-  `telnet(rfc2217)` so conman's telnet NVT negotiates binary mode properly
-- [x] **Validated 6/7 consoles working** via conman from workstation
-  (core-sw01, tor3-mgmt, tor3-stor, rrinfra-rtr, r2-tor-top, subodev-torsw)
-- [ ] **pfv-r2-sw (port 2007): DEAD** — zero bytes at all baud rates
-  (1200-115200), adapter present and mapped correctly (/dev/ttyUSB3).
-  Physical issue: device powered off, console cable disconnected, or
-  console port disabled in device config. Requires onsite investigation.
-
-### Ansible AWX (tsys-awx — LIVE)
-- [x] AWX 24.6.1 deployed via AWX Operator 2.19.1 on k3s single-node
-- [x] Accessible at `http://tsys-awx.knel.net` (port 80, LoadBalancer via k3s ServiceLB)
-- [x] Admin login: `admin` / `REDACTED_PASSWORD`
-- [x] PostgreSQL 15 (bundled, 8 GiB PVC on local-path)
-- [x] VM resized: 32→60 GB disk, swap converted to swapfile
-- [x] Scripts in `awx/`: install-k3s, deploy-awx, verify-awx
-- [ ] HTTPS/TLS not yet configured (currently HTTP on port 80)
-- [ ] Execution environments not yet customized
-
-### Repo Merge
-- [x] KNELServerBuild merged into PFVCluster (history preserved)
-- [x] Directory structure reorganized (provisioning/, tests/, perf/, docs/)
-- [x] All docs gardened: links fixed, stale refs removed, tailscale.md updated
-
-## Known Issues
-
-| Issue | Impact | Status |
-|-------|--------|--------|
-| **core-sw01 LAG hash = layer-2-3** | NFS cross-rack capped at 1 Gbps per host pair | Fix: change to layer-2-3-4 (switch change, pending approval) |
-| **core-sw01 ch1 g16/g17 mismatch** | Only 3 of 4 cross-rack links active | Fix: change member list g(13-15,17)→g(13-16) |
-| **Cross-rack trunk uses `mode on`** | No LACP failure detection | Fix: change to `mode active` (both switches) |
-| **tsys5 bond0 degraded** (1 of 2 slaves) | Single link, no redundancy | Physical: 2nd ethernet cable needed |
-| **tsys5 NVMe + SSD not configured** | Best local storage unused | Config: mount + add to Proxmox storage.cfg |
-| **D2 carries 13 running VMs** | IOPS hotspot persists | Migrate to S2/S3/new disks |
-| **tsys4 PVE 9.1.5** | Behind fleet (9.2.5) | Upgrade needed |
-| **tsys5 SSD 93% full** | Samsung 860 PRO nearly exhausted | Offload to new SK hynix SSD |
-| **Port errors on core-sw01** | g9:2902, g38:590, g26:73 inErr | Investigate CRC/cable |
-| **tsys5 is single-CPU** | CPU2 socket unpopulated | Consider seating spare LGA771 Xeon |
-| **Fleet CPU 0-5% idle** | Misses ~50%-idle utilization goal | Consolidate baseline; grow via workloads |
-
-## Pending (next session priorities)
-
-> **Full audit with findings + recommendations is in**
-> [`proxmox/docs/AUDIT-2026-07-30.md`](proxmox/docs/AUDIT-2026-07-30.md).
-
-**Awaiting operator review of AUDIT-2026-07-30 before executing any fixes.**
-
-**Switch changes (need approval):**
-- Fix core-sw01 ch1 member: g(13-15,17) → g(13-16) [restores 4th link]
-- Fix core-sw01 LAG hash: layer-2-3 → layer-2-3-4 [unlocks multi-link NFS]
-- Convert trunk to LACP: mode on → mode active [failure detection]
-- Remove stale tsys8 config from tor3-stor (ch4/g13/g14)
-
-**Host-side changes (no switch access needed):**
-- Mount + configure tsys5 NVMe (CT500P1 500GB) as LVM-thin
-- Mount + configure tsys5 SK hynix SSD (512GB, relocated from tsys4)
-- Clean stale S3/S4 exports + storage.cfg on tsys5
-- Upgrade tsys4 to PVE 9.2.5 / kernel 7.0.14
-- Apply apply-tunings.sh to tsys4/5 (nconnect, bond hash, rmem/wmem)
-- Audit VM cache modes (writethrough → writeback for NFS)
-
-**Other:**
-- **Schedule:** Operator onsite Monday 0700. Until then: software fixes,
-  documentation, and OAM only (no physical access). Sunday off. Saturday =
-  OAM Day (2026-08-01).
-- **Remote-capable (now–Saturday):** tsys5 NVMe/SSD mount+config, fleet tuning
-  (tsys4/5 nconnect/bond hash/rmem), VM config audit, OAM Day prep, documentation
-- **Onsite-only (Monday+):** switch config changes (core-sw01 ch1 hash+member,
-  tor3-stor LACP conversion), tsys5 2nd ethernet cable, physical hardware work
-- **Saturday OAM Day (2026-08-01):** 100% instrumentation coverage — Beszel
-  (metrics) on all hosts, Uptime Kuma (up/down) → Pushover alerts, LibreNMS
-  (SNMP) for switches, Home Assistant (power/temp) for physical infra,
-  centralized syslog (tool TBD)
-- Investigate core-sw01 port errors (g9/g38/g26)
-- k8s: join wnodes, label by workload class
-- k8s deferred: ETL tooling, HPC scheduler, vcluster policy
+All known infrastructure issues are captured in ticket descriptions and notes.
+See [#369](https://projects.knownelement.com/issues/369) for switch issues,
+[#372](https://projects.knownelement.com/issues/372) for Tripp Lite UPS,
+[#373](https://projects.knownelement.com/issues/373) for dead console port.
 
 ## Infrastructure Summary
 
 | Component | Details |
 |-----------|---------|
 | Proxmox hosts | 7 standalone PVE, managed via PDM (tsys8 retired) |
-| Storage servers | tsys4 (NFS: D1/D2/D3/D5 + /mnt/albert + /mnt/backup), tsys5 (NFS: S1/S2 + local SSD + new NVMe unconfigured) |
-| Switches | core-sw01 (PowerConnect 5448, rack5), tor3-stor/mgmt (PowerConnect 5324, rack3), + 1 unidentified (192.168.0.8) |
+| Storage servers | tsys4 (NFS: D1/D2/D3/D5 + /mnt/albert + /mnt/backup), tsys5 (NFS: S1/S2 + local SSD + NVMe) |
+| Switches | core-sw01 (PowerConnect 5448, rack5), tor3-stor/mgmt (PowerConnect 5324, rack3) |
 | DNS primary | pfv-netinfra-01 (192.168.3.252) — Technitium + Pi-hole |
 | DNS secondary | pfv-netinfra-02 (192.168.3.253) — Technitium + Pi-hole |
-| DNS production | tailscale-router (read-only source of truth) |
-| NTP | pfv-netinfra-01/02 (redundant, LAN IPs, stratum 2/3) |
+| NTP | pfv-netinfra-01/02 (redundant, stratum 3) |
 | Sandbox VM | sectestbed-sandbox (VMID 6000 on pfv-tsys5) |
 | Backup | Proxmox Backup Server (PBS) |
-| **k8s control plane** | **3-node k3s HA (cnode1/2/3), all traffic over Tailscale** |
-| **PDU** | **Cyclades PM10i via powerman on pfv-tsys1 (port 10101)** |
-| **AWX** | **tsys-awx: AWX 24.6.1 on k3s (http://tsys-awx.knel.net:80)** |
-| **Console** | **7 switch consoles via ser2net telnet(rfc2217)+conman on pfv-tsys4 (conmand:7890 on Tailscale)** |
-| **UPS** | **APC Smart-UPS C 1500 via NUT on pfv-tsys1 (upsd:3493 on Tailscale)** |
-| Production | Cloudron VPS, Reston VA (this cluster is R&D only) |
+| k8s control plane | **DOWN** — 3-node k3s HA, needs rebuild ([#367](https://projects.knownelement.com/issues/367)) |
+| PDU | Cyclades PM10i via powerman on pfv-tsys1 (port 10101) |
+| AWX | tsys-awx: AWX 24.6.1 on k3s (http://tsys-awx.knel.net:80) |
+| Console | 7 switch consoles via ser2net + conman on pfv-tsys4 |
+| UPS | APC Smart-UPS C 1500 via NUT on pfv-tsys1 |
+| Production | Cloudron VPS, Reston, VA (this cluster is R&D only) |
+| Config management | KNELIAC (Ansible) — `/home/reachableceo/projects/KNELIAC` |

awx/README.md

@@ -1,5 +1,7 @@
 # AWX — tsys-awx.knel.net
 
+> **Redmine:** [#361](https://projects.knownelement.com/issues/361) (deployment, closed) · [#371](https://projects.knownelement.com/issues/371) (HTTPS/TLS, open)
+
 Ansible AWX deployment on a dedicated k3s single-node VM.
 
 **[→ Deployment Plan](DEPLOYMENT.md)** · **[→ Status](../../STATUS.md)**

dcinfra/console/README.md

@@ -1,130 +1,132 @@
-# Console Management (ser2net + conman)
-
-Network-accessible serial console management for all production network
-switches and routers, running on **pfv-tsys4** (storage server).
-
-## Architecture
-
-```
-USB-DB9 adapters → udev symlinks (/dev/consoles/<name>) → ser2net telnet(rfc2217) TCP → conman (logging + multiplexing)
-```
-
-ser2net owns the physical serial devices and exposes them on TCP ports
-using the **telnet(rfc2217) protocol** bound to the **Tailscale interface
-only** (`100.70.77.93:200X`). conman connects to those TCP ports via
-telnet for session logging, output capture, and multi-user console
-sharing.
-
-**Why telnet(rfc2217)?** The serial devices send `
-␍` (LF+CR) line
+# Console Management (ser2net + conman)
+
+> **Redmine:** [#360](https://projects.knownelement.com/issues/360) (deployment, closed) · [#373](https://projects.knownelement.com/issues/373) (pfv-r2-sw fix, open)
+
+Network-accessible serial console management for all production network
+switches and routers, running on **pfv-tsys4** (storage server).
+
+## Architecture
+
+```
+USB-DB9 adapters → udev symlinks (/dev/consoles/<name>) → ser2net telnet(rfc2217) TCP → conman (logging + multiplexing)
+```
+
+ser2net owns the physical serial devices and exposes them on TCP ports
+using the **telnet(rfc2217) protocol** bound to the **Tailscale interface
+only** (`100.70.77.93:200X`). conman connects to those TCP ports via
+telnet for session logging, output capture, and multi-user console
+sharing.
+
+**Why telnet(rfc2217)?** The serial devices send `
+␍` (LF+CR) line
 endings instead of standard `
-`. Raw TCP transport caused conman's
-telnet NVT to strip bare CR characters, producing stair-stepped output.
-With telnet(rfc2217) on both sides, binary mode is negotiated and CR/LF
-translation is handled correctly by the telnet layer.
-
-**conman and ser2net do NOT share ports** — only one process can open a
-serial device at a time. ser2net owns the physical device; conman connects
-over TCP.
-
-## The USB Enumeration Problem (SOLVED)
-
-The 9 Prolific USB-to-DB9 adapters (`067b:2303`) on pfv-tsys4 have **no
-unique USB serial numbers** and get assigned `/dev/ttyUSB0-8` based on
-enumeration order, which shifts on every boot. This made the old
-`/root/conmap` + manual `screen` workflow break after every reboot.
-
-**Fix:** udev rules pin each adapter by its **ID_PATH** (physical USB port
-topology), which is stable across reboots regardless of enumeration order.
-Each adapter gets a named symlink in `/dev/consoles/` that never changes.
-
-The udev rules are generated from `mapping.txt`, which maps each adapter's
-ID_PATH to a console name and TCP port. To re-map after physically moving
-an adapter, update `mapping.txt` and re-run `setup.sh`.
-
-**Fallback:** if udev trigger doesn't create symlinks for already-discovered
-devices (common on first run), `setup.sh` creates them manually by matching
-ID_PATH. On subsequent boots, udev creates them automatically.
-
-## Port Assignments
-
-| TCP Port | Console Name | ID_PATH | Description |
-|----------|-------------|---------|-------------|
-| 2001 | pfv-core-sw01 | usb-0:1.5.4.4 | Dell PowerConnect 5448 (core switch) |
-| 2002 | pfv-tor3-mgmt | usb-0:1.6.3.1 | Rack 3 management TOR switch |
-| 2003 | pfv-tor3-stor | usb-0:1.6.3.3.2 | Rack 3 storage TOR switch |
-| 2004 | pfv-rrinfra-rtr | usb-0:1.6.3.3.1 | Cisco router (rrinfra) |
-| 2005 | pfv-r2-tor-top | usb-0:1.6.3.3.3 | Rack 2 top-of-rack switch |
-| 2006 | subodev-torsw | usb-0:1.5.4.1 | Suborbital device TOR switch |
-| 2007 | pfv-r2-sw | usb-0:1.6.3.2 | Rack 2 old Dell switch |
-
-All ports listen on the Tailscale IP (`100.70.77.93`) using telnet(rfc2217).
-
-## Scripts
-
-| Script | Purpose |
-|--------|---------|
-| [`mapping.txt`](mapping.txt) | Source of truth: TCP port ↔ ID_PATH ↔ name ↔ baud |
-| [`generate-config.sh`](generate-config.sh) | Generates udev rules, ser2net.yaml, conman.conf from mapping.txt |
-| [`setup.sh`](setup.sh) | Full deploy: generate configs, create symlinks, restart services |
-| [`discover.sh`](discover.sh) | Read-only discovery of USB adapters, existing config, services |
-
-## Usage
-
-### Connect to a console
-
-**Primary method — conman client (with logging + multiplexing):**
-
-```bash
-# From any Tailscale-connected workstation:
-conman -d pfv-tsys4:7890 -f pfv-core-sw01     # connect to console
-conman -d pfv-tsys4:7890 -q                    # list all consoles
-```
-
-Escape sequence: `&.` to disconnect, `&?` for help.
-
-**Direct telnet (emergency only — conflicts with conman):**
-
-```bash
-# Direct telnet to ser2net works ONLY when conmand is stopped, because
-# conmand maintains persistent connections to all 7 TCP ports. Use:
-ssh pfv-tsys4 'systemctl stop conmand'
-telnet pfv-tsys4 2001                          # pfv-core-sw01
-ssh pfv-tsys4 'systemctl start conmand'        # restart when done
-```
-
-**Do NOT use telnet while conmand is running** — conmand will reconnect
-and kick your telnet session immediately ("Connection closed by foreign host").
-The correct workflow is conman client → conmand → ser2net → device.
-
-### Re-deploy after changing mapping.txt
-
-```bash
-PROX_HOST=pfv-tsys4 bash tests/remote.sh prox 'bash /root/console/setup.sh'
-```
-
-### Find the ID_PATH for a new adapter
-
-```bash
-PROX_HOST=pfv-tsys4 bash tests/remote.sh prox-file console/discover.sh
-```
-
-Then match the new adapter's ID_PATH to its physical location and add a line
-to `mapping.txt`.
-
-## Files on pfv-tsys4
-
-| File | Purpose |
-|------|---------|
-| `/etc/udev/rules.d/99-console-ports.rules` | Stable symlinks by ID_PATH |
-| `/etc/ser2net.yaml` | ser2net config (telnet rfc2217 TCP ports → serial symlinks) |
-| `/etc/conman.conf` | conman config (CONSOLE entries between markers) |
-| `/etc/systemd/system/conmand.service` | systemd unit for conmand |
-| `/root/console/mapping.txt` | Copy of the source-of-truth mapping |
-| `/root/console/setup.sh` | Setup script (re-runnable) |
-| `/root/console/generate-config.sh` | Config generator |
-
-## Old workflow (replaced)
-
-The old `/root/conmap` file and manual `screen` sessions are no longer
-needed. The new setup is fully automated and survives reboots.
+`. Raw TCP transport caused conman's
+telnet NVT to strip bare CR characters, producing stair-stepped output.
+With telnet(rfc2217) on both sides, binary mode is negotiated and CR/LF
+translation is handled correctly by the telnet layer.
+
+**conman and ser2net do NOT share ports** — only one process can open a
+serial device at a time. ser2net owns the physical device; conman connects
+over TCP.
+
+## The USB Enumeration Problem (SOLVED)
+
+The 9 Prolific USB-to-DB9 adapters (`067b:2303`) on pfv-tsys4 have **no
+unique USB serial numbers** and get assigned `/dev/ttyUSB0-8` based on
+enumeration order, which shifts on every boot. This made the old
+`/root/conmap` + manual `screen` workflow break after every reboot.
+
+**Fix:** udev rules pin each adapter by its **ID_PATH** (physical USB port
+topology), which is stable across reboots regardless of enumeration order.
+Each adapter gets a named symlink in `/dev/consoles/` that never changes.
+
+The udev rules are generated from `mapping.txt`, which maps each adapter's
+ID_PATH to a console name and TCP port. To re-map after physically moving
+an adapter, update `mapping.txt` and re-run `setup.sh`.
+
+**Fallback:** if udev trigger doesn't create symlinks for already-discovered
+devices (common on first run), `setup.sh` creates them manually by matching
+ID_PATH. On subsequent boots, udev creates them automatically.
+
+## Port Assignments
+
+| TCP Port | Console Name | ID_PATH | Description |
+|----------|-------------|---------|-------------|
+| 2001 | pfv-core-sw01 | usb-0:1.5.4.4 | Dell PowerConnect 5448 (core switch) |
+| 2002 | pfv-tor3-mgmt | usb-0:1.6.3.1 | Rack 3 management TOR switch |
+| 2003 | pfv-tor3-stor | usb-0:1.6.3.3.2 | Rack 3 storage TOR switch |
+| 2004 | pfv-rrinfra-rtr | usb-0:1.6.3.3.1 | Cisco router (rrinfra) |
+| 2005 | pfv-r2-tor-top | usb-0:1.6.3.3.3 | Rack 2 top-of-rack switch |
+| 2006 | subodev-torsw | usb-0:1.5.4.1 | Suborbital device TOR switch |
+| 2007 | pfv-r2-sw | usb-0:1.6.3.2 | Rack 2 old Dell switch |
+
+All ports listen on the Tailscale IP (`100.70.77.93`) using telnet(rfc2217).
+
+## Scripts
+
+| Script | Purpose |
+|--------|---------|
+| [`mapping.txt`](mapping.txt) | Source of truth: TCP port ↔ ID_PATH ↔ name ↔ baud |
+| [`generate-config.sh`](generate-config.sh) | Generates udev rules, ser2net.yaml, conman.conf from mapping.txt |
+| [`setup.sh`](setup.sh) | Full deploy: generate configs, create symlinks, restart services |
+| [`discover.sh`](discover.sh) | Read-only discovery of USB adapters, existing config, services |
+
+## Usage
+
+### Connect to a console
+
+**Primary method — conman client (with logging + multiplexing):**
+
+```bash
+# From any Tailscale-connected workstation:
+conman -d pfv-tsys4:7890 -f pfv-core-sw01     # connect to console
+conman -d pfv-tsys4:7890 -q                    # list all consoles
+```
+
+Escape sequence: `&.` to disconnect, `&?` for help.
+
+**Direct telnet (emergency only — conflicts with conman):**
+
+```bash
+# Direct telnet to ser2net works ONLY when conmand is stopped, because
+# conmand maintains persistent connections to all 7 TCP ports. Use:
+ssh pfv-tsys4 'systemctl stop conmand'
+telnet pfv-tsys4 2001                          # pfv-core-sw01
+ssh pfv-tsys4 'systemctl start conmand'        # restart when done
+```
+
+**Do NOT use telnet while conmand is running** — conmand will reconnect
+and kick your telnet session immediately ("Connection closed by foreign host").
+The correct workflow is conman client → conmand → ser2net → device.
+
+### Re-deploy after changing mapping.txt
+
+```bash
+PROX_HOST=pfv-tsys4 bash tests/remote.sh prox 'bash /root/console/setup.sh'
+```
+
+### Find the ID_PATH for a new adapter
+
+```bash
+PROX_HOST=pfv-tsys4 bash tests/remote.sh prox-file console/discover.sh
+```
+
+Then match the new adapter's ID_PATH to its physical location and add a line
+to `mapping.txt`.
+
+## Files on pfv-tsys4
+
+| File | Purpose |
+|------|---------|
+| `/etc/udev/rules.d/99-console-ports.rules` | Stable symlinks by ID_PATH |
+| `/etc/ser2net.yaml` | ser2net config (telnet rfc2217 TCP ports → serial symlinks) |
+| `/etc/conman.conf` | conman config (CONSOLE entries between markers) |
+| `/etc/systemd/system/conmand.service` | systemd unit for conmand |
+| `/root/console/mapping.txt` | Copy of the source-of-truth mapping |
+| `/root/console/setup.sh` | Setup script (re-runnable) |
+| `/root/console/generate-config.sh` | Config generator |
+
+## Old workflow (replaced)
+
+The old `/root/conmap` file and manual `screen` sessions are no longer
+needed. The new setup is fully automated and survives reboots.

dcinfra/powerman/README.md

@@ -1,5 +1,7 @@
 # Powerman PDU Management
 
+> **Redmine:** [#359](https://projects.knownelement.com/issues/359) (deployment, closed) · [#374](https://projects.knownelement.com/issues/374) (outlet renaming, open)
+
 Centralized power management for the Cyclades AlterPath PM10i PDU via
 [Powerman](https://github.com/chaos/powerman), running on pfv-tsys1.
 

dcinfra/ups/README.md

@@ -1,5 +1,7 @@
 # UPS Management (NUT — Network UPS Tools)
 
+> **Redmine:** [#340](https://projects.knownelement.com/issues/340) (APC, closed) · [#372](https://projects.knownelement.com/issues/372) (Tripp Lite, open)
+
 Centralized UPS monitoring for the server room via
 [NUT](https://networkupstools.org/), running on **pfv-tsys1**. USB HID UPS
 units feed one `upsd` network server; Home Assistant polls it over Tailscale for

docs/docmap.md

@@ -3,7 +3,7 @@
 > **Index of all documentation in this repo.** Agents must update this file
 > whenever a doc is added, removed, or substantively changed.
 
-> **Last updated:** 2026-08-05 (full read-only ground-truth audit: network+SNMP+compute+k8s+netinfra)
+> **Last updated:** 2026-08-06 (Pi-hole hardening config committed to netinfra/pihole/ — [#376])
 
 ## Kubernetes Architecture ([`k8s/`](k8s/))
 
@@ -61,6 +61,7 @@ Server provisioning, security hardening, DNS/NTP configuration.
 | [`../netinfra/dns-cluster-setup/README.md`](../netinfra/dns-cluster-setup/README.md) | Technitium DNS cluster setup: export, deploy, cluster, verify | 2026-07-28 |
 | [`../tests/README.md`](../tests/README.md) | Test suite documentation: unit, security, validation tests | 2026-07-28 |
 | [`../netinfra/pfv-netinfra-setup.md`](../netinfra/pfv-netinfra-setup.md) | pfv-netinfra-01/02 initial setup guide | 2026-07-27 |
+| [`../netinfra/pihole/README.md`](../netinfra/pihole/README.md) | Pi-hole recursive DNS on netinfra-01/02: shm_size fix, gravity-validate.sh, healthcheck, autoheal (defense-in-depth vs gravity.db corruption) | 2026-08-06 |
 | [`../netinfra/pfv-netboot-setup.md`](../netinfra/pfv-netboot-setup.md) | pfv-netboot reference node setup | 2026-07-27 |
 
 ## Archive ([`archive/`](archive/))
@@ -80,12 +81,19 @@ context — do not update; link to active docs instead.
 | [`archive/charles-todo.md`](archive/charles-todo.md) | Charles's early task list (completed) |
 | [`archive/claude-todo.md`](archive/claude-todo.md) | Claude's early task list (completed) |
 
+## External Tracking
+
+| Resource | Description |
+|----------|-------------|
+| [Redmine](https://projects.knownelement.com) | **System of record for all work.** Version: Potential to Kinetic Ready (due 2026-09-30). Project: Technology & Facility Services (id 55) |
+| [Gitea](https://git.knownelement.com/KNEL/PFVCluster) | Source code repository |
+
 ## Top-Level Files
 
 | File | Description |
 |------|-------------|
-| [`../AGENTS.md`](../AGENTS.md) | Agent operating instructions (repo layout, git policy, gardening protocol) |
-| [`../STATUS.md`](../STATUS.md) | Living project status (agent-maintained, human read-only) |
+| [`../AGENTS.md`](../AGENTS.md) | Agent operating instructions (repo layout, git policy, Redmine tracking, gardening protocol) |
+| [`../STATUS.md`](../STATUS.md) | Ticket index + infrastructure summary (read-only snapshot; work tracked in Redmine) |
 | [`../README.md`](../README.md) | Project overview and quick start |
 | [`../LICENSE`](../LICENSE) | License |
 
@@ -95,9 +103,11 @@ context — do not update; link to active docs instead.
 
 When making changes to this repo, agents MUST:
 
-1. **Update [`../STATUS.md`](../STATUS.md)** if the work changes infrastructure
-   state, completes/starts a task, or discovers a new issue.
-2. **Update this file (`docmap.md`)** if a doc is added, removed, or has a
+1. **Update Redmine tickets** — all work status, completion, and issue tracking
+   lives in [Redmine](https://projects.knownelement.com), not in repo files.
+2. **Update [`../STATUS.md`](../STATUS.md)** if the infrastructure summary or
+   ticket index needs refreshing (read-only snapshot only).
+3. **Update this file (`docmap.md`)** if a doc is added, removed, or has a
    substantive content change. Update the "Last Reviewed" date.
 3. **Verify cross-references** — any new `.md` file must be linked from at
    least one existing doc or this map.

k8s/README.md

@@ -1,5 +1,7 @@
 # k8s/ — pfv-k8s Cluster Setup Scripts
 
+> **Redmine:** [#362](https://projects.knownelement.com/issues/362) (initial build, closed) · [#367](https://projects.knownelement.com/issues/367) (rebuild, open) · [#368](https://projects.knownelement.com/issues/368) (worker join, open)
+
 Scripts to bootstrap and manage the k3s control plane on cnode1/2/3.
 All cluster communication goes over Tailscale IPs — no LAN traffic.
 

netinfra/dns-cluster-setup/README.md

@@ -1,5 +1,7 @@
 # Technitium DNS Cluster Setup
 
+> **Redmine:** [#357](https://projects.knownelement.com/issues/357) (deployment, closed)
+
 Replicates the production Technitium DNS Server from `tailscale-router` to the
 `pfv-netinfra-01/02` pair and configures them as a primary/secondary cluster
 with automatic zone transfers.

netinfra/pihole/.env.example

@@ -0,0 +1,3 @@
+# Pi-hole web UI password. NEVER commit the real .env — only this template.
+# Copy to .env and set the value before `docker compose up -d`.
+PIHOLE_WEB_PASSWORD=changeme

netinfra/pihole/README.md

@@ -0,0 +1,43 @@
+# netinfra/pihole/ — Pi-hole recursive DNS (pfv-netinfra-01/02)
+
+> **Redmine:** [#376](https://projects.knownelement.com/issues/376) (up/down alerts + commit hardening) · [#357](https://projects.knownelement.com/issues/357) (cluster build, closed)
+
+Pi-hole v6 runs as the recursive resolver on port 53 of both DNS nodes.
+Technitium (`tsys-dns`) runs as the authoritative server on port 5300; the two
+share the `dnsnet` Docker network so Pi-hole can conditional-forward `knel.net`
+zones to Technitium.
+
+## Hardening (defense-in-depth against gravity.db corruption)
+
+The operator hit a production outage when Pi-hole's `/dev/shm` was too small,
+corrupting `gravity.db`. The live config on both nodes now includes:
+
+- **`shm_size: 1024M`** — fixes the root cause (default 64M was too small).
+- **`gravity-validate.sh`** — entrypoint that checks the SQLite header of
+  `gravity.db` before start; auto-moves a corrupt/empty DB aside so Pi-hole
+  can regenerate it cleanly.
+- **Healthcheck** — `dig +norecurse @127.0.0.1 pi.hole` + gravity.db non-empty;
+  fails the container if DNS or the DB is broken.
+- **`autoheal`** sidecar — restarts any container labeled `autoheal=true` that
+  goes unhealthy.
+
+## Deploy
+
+The compose reads the web UI password from a gitignored `.env`:
+
+```bash
+cd netinfra/pihole/
+cp .env.example .env          # then edit .env and set PIHOLE_WEB_PASSWORD
+docker compose up -d
+```
+
+Files are deployed to `/home/localuser/services/pihole/` on each node. Volumes
+(`./etc-pihole`, `./etc-dnsmasq.d`) hold the persistent state.
+
+## Verify
+
+```bash
+dig @127.0.0.1 +short google.com        # recursive
+dig @127.0.0.1 +short git.knownelement.com   # knel.net via Technitium forward
+docker inspect pihole --format '{{.State.Health.Status}}'
+```

netinfra/pihole/docker-compose.yml

@@ -0,0 +1,49 @@
+services:
+  pihole:
+    container_name: pihole
+    # Root cause of the 2026-08 gravity.db corruption: default /dev/shm (64M)
+    # was too small for FTL's shared-memory metrics. 1024M has been stable.
+    shm_size: '1024M'
+    image: pihole/pihole:latest
+    hostname: pihole
+    entrypoint: ["/usr/local/bin/gravity-validate.sh"]
+    ports:
+      - "53:53/tcp"
+      - "53:53/udp"
+      - "10002:80/tcp"
+      - "10003:443/tcp"
+    environment:
+      TZ: 'America/Chicago'
+      FTLCONF_webserver_api_password: '${PIHOLE_WEB_PASSWORD}'
+      FTLCONF_dns_listeningMode: 'all'
+    volumes:
+      - './etc-pihole:/etc/pihole'
+      - './etc-dnsmasq.d:/etc/dnsmasq.d'
+      - './gravity-validate.sh:/usr/local/bin/gravity-validate.sh:ro'
+    cap_add:
+      - SYS_NICE
+    restart: always
+    healthcheck:
+      test: ["CMD-SHELL", "dig +short +norecurse @127.0.0.1 pi.hole >/dev/null 2>&1 && test -s /etc/pihole/gravity.db || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 60s
+    labels:
+      autoheal: "true"
+    networks:
+      - default
+      - dnsnet
+  autoheal:
+    container_name: autoheal
+    image: willfarrell/autoheal:latest
+    environment:
+      AUTOHEAL_CONTAINER_LABEL: autoheal
+      AUTOHEAL_INTERVAL: 30
+      AUTOHEAL_START_PERIOD: 60
+    volumes:
+      - '/var/run/docker.sock:/var/run/docker.sock:ro'
+    restart: always
+networks:
+  dnsnet:
+    external: true

netinfra/pihole/gravity-validate.sh

@@ -0,0 +1,28 @@
+#!/bin/bash
+# gravity-validate.sh — pre-start integrity check for Pi-hole's gravity.db
+#
+# Runs as the container entrypoint. If gravity.db is empty or has an invalid
+# SQLite header (the symptom of the /dev/shm corruption outage), move it aside
+# so Pi-hole regenerates a clean DB on start instead of crashing.
+set -e
+GRAVITY_DB="/etc/pihole/gravity.db"
+TIMESTAMP=$(date +%Y%m%d%H%M%S)
+
+if [ -f "$GRAVITY_DB" ]; then
+    if [ ! -s "$GRAVITY_DB" ]; then
+        echo "[gravity-validate] gravity.db is empty, moving aside"
+        mv "$GRAVITY_DB" "${GRAVITY_DB}.corrupt.${TIMESTAMP}"
+    else
+        HEADER=$(head -c 15 "$GRAVITY_DB" 2>/dev/null || true)
+        if [ "$HEADER" != "SQLite format 3" ]; then
+            echo "[gravity-validate] gravity.db invalid header, moving aside"
+            mv "$GRAVITY_DB" "${GRAVITY_DB}.corrupt.${TIMESTAMP}"
+        fi
+    fi
+fi
+# Keep only the 3 most recent corrupt backups (names carry a timestamp,
+# so lexical reverse-sort = newest-first).
+find /etc/pihole -maxdepth 1 -name 'gravity.db.corrupt.*' -print 2>/dev/null \
+  | sort -r | tail -n +4 | xargs -r rm -f
+echo "[gravity-validate] OK, starting Pi-hole"
+exec /usr/bin/start.sh "$@"