mrcharles
30ddb37b34
Commit 33b5c76 claimed to harden Pi-hole on both DNS nodes but only
modified markdown — the working docker-compose.yml, gravity-validate.sh,
healthcheck, and autoheal config were never written to the repo, leaving
the DNS hardening unreproducible from version control.
This commits the live, verified-working config from the boxes into
netinfra/pihole/:
- docker-compose.yml (shm_size 1024M root-cause fix, healthcheck, autoheal)
- gravity-validate.sh (pre-start SQLite header check, auto-quarantine corrupt DB)
- .env.example (web UI password templated; real .env gitignored)
Defends against the gravity.db / /dev/shm corruption production outage.
The live password is templated as ${PIHOLE_WEB_PASSWORD} so no secret
enters git.
[#376]
79 lines
5.6 KiB
Markdown
79 lines
5.6 KiB
Markdown
# Project Status
|
|
|
|
> **Redmine is the system of record for all work tracking.**
|
|
> https://projects.knownelement.com — Version: **Potential to Kinetic Ready** (due 2026-09-30)
|
|
>
|
|
> This file is now a **read-only snapshot + ticket index**. Do not add
|
|
> checklists or status items here. All work is tracked as Redmine tickets.
|
|
> When work is done, update the ticket (done ratio, notes, close).
|
|
|
|
> **Last updated:** 2026-08-06 — Pi-hole hardening config committed (`netinfra/pihole/`, [#376]).
|
|
|
|
## Current State: MIXED (infra stable; k8s needs rebuild)
|
|
|
|
Provisioning, DNS primary (netinfra-01), Proxmox fleet, and monitoring stack
|
|
are operational. **k8s control plane is DOWN** (cnode wiped, no etcd quorum).
|
|
|
|
## Completed Work (tickets closed @ 100%)
|
|
|
|
| Ticket | Component | Code location |
|
|
|--------|-----------|---------------|
|
|
| [#334](https://projects.knownelement.com/issues/334) | NTP synchronization (redundant stratum 3) | `netinfra/` |
|
|
| [#340](https://projects.knownelement.com/issues/340) | UPS management — NUT on pfv-tsys1 (APC Smart-UPS) | `dcinfra/ups/` |
|
|
| [#357](https://projects.knownelement.com/issues/357) | DNS cluster — Technitium primary/secondary + DHCP failover | `netinfra/dns-cluster-setup/` |
|
|
| [#358](https://projects.knownelement.com/issues/358) | Proxmox perf tuning (5 of 7 hosts) | `proxmox/perf/` |
|
|
| [#359](https://projects.knownelement.com/issues/359) | PDU management — powerman on pfv-tsys1 | `dcinfra/powerman/` |
|
|
| [#360](https://projects.knownelement.com/issues/360) | Console management — ser2net + conman on pfv-tsys4 | `dcinfra/console/` |
|
|
| [#361](https://projects.knownelement.com/issues/361) | AWX deployment — k3s + AWX Operator | `awx/` |
|
|
| [#362](https://projects.knownelement.com/issues/362) | k8s control plane build — k3s HA (initial, now needs rebuild) | `k8s/` |
|
|
| [#363](https://projects.knownelement.com/issues/363) | tsys5 storage configuration (NVMe + SSD) | `proxmox/docs/` |
|
|
| [#364](https://projects.knownelement.com/issues/364) | tsys4 PVE upgrade to 9.2.5 | `proxmox/docs/` |
|
|
| [#365](https://projects.knownelement.com/issues/365) | Repo merge — KNELServerBuild into PFVCluster | repo root |
|
|
| [#366](https://projects.knownelement.com/issues/366) | Server provisioning pipeline (SetupNewSystem.sh → KNELIAC) | `archive/provisioning/`, KNELIAC |
|
|
|
|
## Open Work (tickets in progress or queued)
|
|
|
|
| Ticket | Component | Notes |
|
|
|--------|-----------|-------|
|
|
| [#376](https://projects.knownelement.com/issues/376) | DNS cluster up/down alerts + Pi-hole hardening committed | Config now in `netinfra/pihole/`; flapping source under investigation |
|
|
| [#367](https://projects.knownelement.com/issues/367) | k8s cluster rebuild (cnode wiped, no quorum) | Blocks #368 |
|
|
| [#368](https://projects.knownelement.com/issues/368) | k8s worker nodes join cluster | Blocked by #367 |
|
|
| [#369](https://projects.knownelement.com/issues/369) | Switch config fixes (core-sw01 LAG, LACP, member list) | Onsite Friday |
|
|
| [#370](https://projects.knownelement.com/issues/370) | Vault — migrate all API keys (HIGH priority) | |
|
|
| [#371](https://projects.knownelement.com/issues/371) | AWX HTTPS/TLS configuration | Depends on #342 (PKI/SSL) |
|
|
| [#372](https://projects.knownelement.com/issues/372) | Tripp Lite UPS fix (physical reseat) | Onsite Friday |
|
|
| [#373](https://projects.knownelement.com/issues/373) | pfv-r2-sw console fix (physical, onsite) | Onsite Friday |
|
|
| [#374](https://projects.knownelement.com/issues/374) | PDU outlet renaming | |
|
|
| [#375](https://projects.knownelement.com/issues/375) | Monitoring expansion (unpoller, netdisco, weathermap, smokeping) | Relates to #337, #343 |
|
|
| [#311](https://projects.knownelement.com/issues/311) | OAM: Security Hardening (CMMC/STIG) | KNELIAC roles deployed, expansion ongoing |
|
|
| [#335](https://projects.knownelement.com/issues/335) | OAM: Centralized syslog to Wazuh | Wazuh agents deployed, fleet rollout pending |
|
|
| [#342](https://projects.knownelement.com/issues/342) | OAM: PKI/SSL — tsys-ca VM + HSM CA + ACME | Vault unconfigured, AWX HTTP-only |
|
|
| [#343](https://projects.knownelement.com/issues/343) | OAM: Monitoring coverage matrix | Coverage gaps documented |
|
|
|
|
## Known Issues (tracked in tickets)
|
|
|
|
All known infrastructure issues are captured in ticket descriptions and notes.
|
|
See [#369](https://projects.knownelement.com/issues/369) for switch issues,
|
|
[#372](https://projects.knownelement.com/issues/372) for Tripp Lite UPS,
|
|
[#373](https://projects.knownelement.com/issues/373) for dead console port.
|
|
|
|
## Infrastructure Summary
|
|
|
|
| Component | Details |
|
|
|-----------|---------|
|
|
| Proxmox hosts | 7 standalone PVE, managed via PDM (tsys8 retired) |
|
|
| Storage servers | tsys4 (NFS: D1/D2/D3/D5 + /mnt/albert + /mnt/backup), tsys5 (NFS: S1/S2 + local SSD + NVMe) |
|
|
| Switches | core-sw01 (PowerConnect 5448, rack5), tor3-stor/mgmt (PowerConnect 5324, rack3) |
|
|
| DNS primary | pfv-netinfra-01 (192.168.3.252) — Technitium + Pi-hole |
|
|
| DNS secondary | pfv-netinfra-02 (192.168.3.253) — Technitium + Pi-hole |
|
|
| NTP | pfv-netinfra-01/02 (redundant, stratum 3) |
|
|
| Sandbox VM | sectestbed-sandbox (VMID 6000 on pfv-tsys5) |
|
|
| Backup | Proxmox Backup Server (PBS) |
|
|
| k8s control plane | **DOWN** — 3-node k3s HA, needs rebuild ([#367](https://projects.knownelement.com/issues/367)) |
|
|
| PDU | Cyclades PM10i via powerman on pfv-tsys1 (port 10101) |
|
|
| AWX | tsys-awx: AWX 24.6.1 on k3s (http://tsys-awx.knel.net:80) |
|
|
| Console | 7 switch consoles via ser2net + conman on pfv-tsys4 |
|
|
| UPS | APC Smart-UPS C 1500 via NUT on pfv-tsys1 |
|
|
| Production | Cloudron VPS, Reston, VA (this cluster is R&D only) |
|
|
| Config management | KNELIAC (Ansible) — `/home/reachableceo/projects/KNELIAC` |
|