Files
PFVCluster/STATUS.md
T
mrcharles 317a98b9b1 docs: migrate work tracking to Redmine system of record
Replace STATUS.md checklists with a Redmine ticket index. All work
status, completion tracking, and issue management now lives in Redmine
(version: Potential to Kinetic Ready, due 2026-09-30). AGENTS.md gains
a Redmine Tracking Policy section with rules for ticket references,
closing procedures, and STATUS.md's new read-only role.
2026-08-06 11:24:48 -05:00

5.4 KiB

Project Status

Redmine is the system of record for all work tracking. https://projects.knownelement.com — Version: Potential to Kinetic Ready (due 2026-09-30)

This file is now a read-only snapshot + ticket index. Do not add checklists or status items here. All work is tracked as Redmine tickets. When work is done, update the ticket (done ratio, notes, close).

Last updated: 2026-08-06 — migrated all tracking to Redmine.

Current State: MIXED (infra stable; k8s needs rebuild)

Provisioning, DNS primary (netinfra-01), Proxmox fleet, and monitoring stack are operational. k8s control plane is DOWN (cnode wiped, no etcd quorum).

Completed Work (tickets closed @ 100%)

Ticket Component Code location
#334 NTP synchronization (redundant stratum 3) netinfra/
#340 UPS management — NUT on pfv-tsys1 (APC Smart-UPS) dcinfra/ups/
#357 DNS cluster — Technitium primary/secondary + DHCP failover netinfra/dns-cluster-setup/
#358 Proxmox perf tuning (5 of 7 hosts) proxmox/perf/
#359 PDU management — powerman on pfv-tsys1 dcinfra/powerman/
#360 Console management — ser2net + conman on pfv-tsys4 dcinfra/console/
#361 AWX deployment — k3s + AWX Operator awx/
#362 k8s control plane build — k3s HA (initial, now needs rebuild) k8s/
#363 tsys5 storage configuration (NVMe + SSD) proxmox/docs/
#364 tsys4 PVE upgrade to 9.2.5 proxmox/docs/
#365 Repo merge — KNELServerBuild into PFVCluster repo root
#366 Server provisioning pipeline (SetupNewSystem.sh → KNELIAC) archive/provisioning/, KNELIAC

Open Work (tickets in progress or queued)

Ticket Component Notes
#367 k8s cluster rebuild (cnode wiped, no quorum) Blocks #368
#368 k8s worker nodes join cluster Blocked by #367
#369 Switch config fixes (core-sw01 LAG, LACP, member list) Onsite Friday
#370 Vault — migrate all API keys (HIGH priority)
#371 AWX HTTPS/TLS configuration Depends on #342 (PKI/SSL)
#372 Tripp Lite UPS fix (physical reseat) Onsite Friday
#373 pfv-r2-sw console fix (physical, onsite) Onsite Friday
#374 PDU outlet renaming
#375 Monitoring expansion (unpoller, netdisco, weathermap, smokeping) Relates to #337, #343
#311 OAM: Security Hardening (CMMC/STIG) KNELIAC roles deployed, expansion ongoing
#335 OAM: Centralized syslog to Wazuh Wazuh agents deployed, fleet rollout pending
#342 OAM: PKI/SSL — tsys-ca VM + HSM CA + ACME Vault unconfigured, AWX HTTP-only
#343 OAM: Monitoring coverage matrix Coverage gaps documented

Known Issues (tracked in tickets)

All known infrastructure issues are captured in ticket descriptions and notes. See #369 for switch issues, #372 for Tripp Lite UPS, #373 for dead console port.

Infrastructure Summary

Component Details
Proxmox hosts 7 standalone PVE, managed via PDM (tsys8 retired)
Storage servers tsys4 (NFS: D1/D2/D3/D5 + /mnt/albert + /mnt/backup), tsys5 (NFS: S1/S2 + local SSD + NVMe)
Switches core-sw01 (PowerConnect 5448, rack5), tor3-stor/mgmt (PowerConnect 5324, rack3)
DNS primary pfv-netinfra-01 (192.168.3.252) — Technitium + Pi-hole
DNS secondary pfv-netinfra-02 (192.168.3.253) — Technitium + Pi-hole
NTP pfv-netinfra-01/02 (redundant, stratum 3)
Sandbox VM sectestbed-sandbox (VMID 6000 on pfv-tsys5)
Backup Proxmox Backup Server (PBS)
k8s control plane DOWN — 3-node k3s HA, needs rebuild (#367)
PDU Cyclades PM10i via powerman on pfv-tsys1 (port 10101)
AWX tsys-awx: AWX 24.6.1 on k3s (http://tsys-awx.knel.net:80)
Console 7 switch consoles via ser2net + conman on pfv-tsys4
UPS APC Smart-UPS C 1500 via NUT on pfv-tsys1
Production Cloudron VPS, Reston, VA (this cluster is R&D only)
Config management KNELIAC (Ansible) — /home/reachableceo/projects/KNELIAC