From 317a98b9b1694f606772bb5554be52852799feb0 Mon Sep 17 00:00:00 2001 From: reachableceo Date: Thu, 6 Aug 2026 11:24:48 -0500 Subject: [PATCH] docs: migrate work tracking to Redmine system of record Replace STATUS.md checklists with a Redmine ticket index. All work status, completion tracking, and issue management now lives in Redmine (version: Potential to Kinetic Ready, due 2026-09-30). AGENTS.md gains a Redmine Tracking Policy section with rules for ticket references, closing procedures, and STATUS.md's new read-only role. --- AGENTS.md | 36 +++++-- STATUS.md | 260 +++++++++++-------------------------------------- docs/docmap.md | 19 +++- 3 files changed, 99 insertions(+), 216 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 87c8513..6eb6522 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,8 +1,9 @@ # Agent Guidelines **Top-level files:** [`README.md`](README.md) (project overview), -[`STATUS.md`](STATUS.md) (living status, agent-maintained), -[`docs/docmap.md`](docs/docmap.md) (documentation index). Everything else +[`STATUS.md`](STATUS.md) (ticket index + infra summary), +[`docs/docmap.md`](docs/docmap.md) (documentation index). All work is +tracked in [Redmine](https://projects.knownelement.com). Everything else lives in subdirectories. ## Repository Layout @@ -54,12 +55,34 @@ vendor/ Vendored KNELShellFramework justification. A script that emits any diagnostic is a protocol violation. Non-bash scripts (PHP with `.sh` shebang `#!/usr/bin/php`, etc.) are exempt. +## Redmine Tracking Policy + +**Redmine is the system of record for all work.** Do not track status, +checklists, or TODOs in repo files. Use Redmine tickets instead. + +- **URL:** https://projects.knownelement.com +- **Version:** Potential to Kinetic Ready (due 2026-09-30) +- **Project:** Known Element Enterprises - Technology & Facility Services (id 55) + +### Rules +1. **Every piece of work** (feature, fix, deployment, config change) gets a + Redmine ticket. If one doesn't exist, create it. +2. **Reference tickets in docs and commits** using `[#NNN]` notation. + Example: `[#367] Rebuilt k3s control plane after cnode wipe`. +3. **When work completes**, update the ticket: set done ratio to 100%, + add a note describing what was done and where the code lives, then close. +4. **STATUS.md is a read-only snapshot** — it contains a ticket index and + infrastructure summary only. Do not add checklists or status items. +5. **Link code to tickets** — ticket descriptions and notes should reference + the relevant file paths in this repo (e.g., `dcinfra/ups/`). + ## Automatic Gardening Protocol **Docs and code must be kept in sync.** After any work session, an agent MUST: -1. **Update [`STATUS.md`](STATUS.md)** — reflect completed work, new issues, - changed infrastructure state. This file is human read-only; agents own it. +1. **Update [`STATUS.md`](STATUS.md)** — update the ticket index table if + tickets were created or closed. This file is a read-only snapshot; + do not add inline status tracking (use Redmine). 2. **Update [`docs/docmap.md`](docs/docmap.md)** — if a doc was added, removed, or substantively changed, update the table and "Last Reviewed" date. @@ -104,5 +127,6 @@ vendor/ Vendored KNELShellFramework Solo-founder R&D Proxmox cluster in a private residence. Shoestring budget. Production lives on a Cloudron VPS in Reston VA. See -[`STATUS.md`](STATUS.md) for current state and -[`proxmox/docs/PROJECT.md`](proxmox/docs/PROJECT.md) for the fleet report. +[`STATUS.md`](STATUS.md) for the ticket index and infrastructure summary. +All work is tracked in [Redmine](https://projects.knownelement.com) +(version: Potential to Kinetic Ready, due 2026-09-30). diff --git a/STATUS.md b/STATUS.md index 3837da4..24d9d60 100644 --- a/STATUS.md +++ b/STATUS.md @@ -1,227 +1,77 @@ # Project Status -> **Human read-only. Agents maintain this file automatically after each work -> session.** Do not edit by hand — the next agent run will overwrite it. - -> **Last updated:** 2026-08-05 by Crush (GLM-5.2) — full read-only ground-truth -> re-orientation (nmap + SNMP + SSH + API queries). +> **Redmine is the system of record for all work tracking.** +> https://projects.knownelement.com — Version: **Potential to Kinetic Ready** (due 2026-09-30) > -> **Current authoritative audit:** [`proxmox/docs/AUDIT-2026-08-05.md`](proxmox/docs/AUDIT-2026-08-05.md) -> — repo restructure, 4 switches up / 1 down, 63 VMs, 167 DNS zones, 753 DHCP -> leases, k8s cluster DOWN (cnode wiped), netinfra-02 wiped, tsys5 storage -> configured, tsys4 upgraded, 3-tier Tailscale fleet (80 nodes). -> Capacity targets are codified in -> [`proxmox/docs/CAPACITY-GOALS.md`](proxmox/docs/CAPACITY-GOALS.md): 80% RAM -> steady-state (fleet PASSES, highest tsys1 at 71%). +> This file is now a **read-only snapshot + ticket index**. Do not add +> checklists or status items here. All work is tracked as Redmine tickets. +> When work is done, update the ticket (done ratio, notes, close). -## Current State: MIXED (infra stable; k8s + netinfra-02 need rebuild) +> **Last updated:** 2026-08-06 — migrated all tracking to Redmine. + +## Current State: MIXED (infra stable; k8s needs rebuild) Provisioning, DNS primary (netinfra-01), Proxmox fleet, and monitoring stack -are operational. **k8s control plane is DOWN** (cnode wiped, no etcd quorum) -and **netinfra-02 is wiped** (needs rebuild as redundant secondary). Repo was -restructured into a module/subproject layout (see -[`proxmox/docs/AUDIT-2026-08-05.md`](proxmox/docs/AUDIT-2026-08-05.md)). +are operational. **k8s control plane is DOWN** (cnode wiped, no etcd quorum). -## Completed Work +## Completed Work (tickets closed @ 100%) -### Server Provisioning (validated on sectestbed-sandbox) -- [x] SetupNewSystem.sh deploys end-to-end to rc=0 (Debian 13 trixie) -- [x] Security hardening: SSH, SCAP-STIG, Wazuh, 2FA (SSH+Cockpit+Webmin) -- [x] NTP: redundant pfv-netinfra-01/02 (192.168.3.252/253), synced stratum 3 -- [x] DNS resolv.conf: managed static file pointing at netinfra pair -- [x] Test suite: 5 tests (framework, safe-download, 2fa, https, system-req) -- [x] VM validation harness: git-based deploy + auto-rollback + guest-agent access +| Ticket | Component | Code location | +|--------|-----------|---------------| +| [#334](https://projects.knownelement.com/issues/334) | NTP synchronization (redundant stratum 3) | `netinfra/` | +| [#340](https://projects.knownelement.com/issues/340) | UPS management — NUT on pfv-tsys1 (APC Smart-UPS) | `dcinfra/ups/` | +| [#357](https://projects.knownelement.com/issues/357) | DNS cluster — Technitium primary/secondary + DHCP failover | `netinfra/dns-cluster-setup/` | +| [#358](https://projects.knownelement.com/issues/358) | Proxmox perf tuning (5 of 7 hosts) | `proxmox/perf/` | +| [#359](https://projects.knownelement.com/issues/359) | PDU management — powerman on pfv-tsys1 | `dcinfra/powerman/` | +| [#360](https://projects.knownelement.com/issues/360) | Console management — ser2net + conman on pfv-tsys4 | `dcinfra/console/` | +| [#361](https://projects.knownelement.com/issues/361) | AWX deployment — k3s + AWX Operator | `awx/` | +| [#362](https://projects.knownelement.com/issues/362) | k8s control plane build — k3s HA (initial, now needs rebuild) | `k8s/` | +| [#363](https://projects.knownelement.com/issues/363) | tsys5 storage configuration (NVMe + SSD) | `proxmox/docs/` | +| [#364](https://projects.knownelement.com/issues/364) | tsys4 PVE upgrade to 9.2.5 | `proxmox/docs/` | +| [#365](https://projects.knownelement.com/issues/365) | Repo merge — KNELServerBuild into PFVCluster | repo root | +| [#366](https://projects.knownelement.com/issues/366) | Server provisioning pipeline (SetupNewSystem.sh → KNELIAC) | `archive/provisioning/`, KNELIAC | -### DNS Cluster (pfv-netinfra-01/02) -- [x] Production Technitium config replicated from tailscale-router (read-only) -- [x] 124 zones on both nodes (knel.net + reverse DNS) -- [x] pfv-netinfra-01 = PRIMARY, pfv-netinfra-02 = SECONDARY -- [x] Zone replication via rsync systemd timer (every 60s) -- [x] Credentials + 2FA replicated identically to production -- [x] Both LAN IPs resolve knel.net device names + recurse externally -- [x] **DHCP migrated from pfv-netboot → netinfra-01/02** (ISC failover-peer, primary/secondary). Both `normal`, **753 active leases** (was 333). DNS+NTP now handed out as 252/253. Netboot DHCP stopped + disabled. Webmin DHCP module on both nodes. -- [ ] **netinfra-02 REBUILT 2026-08-05** (from operator's accidental wipe): - Docker installed, Technitium (167 zones, zone sync timer active), Pi-hole v6 - (healthy, gravity.db healthcheck + autoheal), NTP container, DHCP failover - secondary configured and serving (99+84 active leases load-balanced). All - services operational and redundant. -- [x] **Technitium now runs as Docker container** (`tsys-dns`) on netinfra-01, - not systemd dnsServer. Config at `/home/localuser/services/technitium/config`. - **167 zones** (was 124). NTP = `ntpsec` (not chrony). -- [x] **Pi-hole hardened on BOTH nodes**: shm_size=1024M (operator root-cause fix), - custom gravity-validate.sh entrypoint (SQLite header check + auto-recovery), - healthcheck validates DNS + gravity.db, autoheal container restarts unhealthy - containers. Deployed to netinfra-01 and netinfra-02. +## Open Work (tickets in progress or queued) -### Proxmox Cluster Ops -- [x] 5 of 7 hosts fully performance-tuned (tsys1/3/6/7/9) -- [x] **Full ground-truth audit 2026-07-30** (network + storage + compute + switches) -- [x] **Cross-rack NFS root cause identified:** core-sw01 LAG hash=layer-2-3 (no L4), - g16/g17 cable mismatch (3 of 4 links active), static mode (no LACP) -- [x] **tsys4/tsys5 storage changes documented:** new NVMe (CT500P1 500GB), relocated - SK hynix SSD, D3 changed to USB HDD, /mnt/albert + /mnt/backup added -- [x] **Console symlink bug fixed:** udev rule singular→plural mismatch broke all 7 - console ports after reboot -- [x] **Portable audit tooling built:** probe-storage.sh, probe-network.sh, - conman-console.py, snmp-switch-audit.py (AGPLv3-friendly, config-driven) -- [ ] **Memory goal (≤80% actual): PASS fleet-wide** — highest is tsys9 at 69% -- [ ] CPU ~50%-idle goal: fleet idles 0-5% (under-utilized; chase via workloads, not hardware) -- [ ] tsys4: PVE 9.1.5 (behind fleet 9.2.5), needs upgrade -- [ ] tsys5: NVMe + SK hynix SSD installed but NOT configured -- [ ] tsys2: pending rebuild from Win10 to Proxmox +| Ticket | Component | Notes | +|--------|-----------|-------| +| [#367](https://projects.knownelement.com/issues/367) | k8s cluster rebuild (cnode wiped, no quorum) | Blocks #368 | +| [#368](https://projects.knownelement.com/issues/368) | k8s worker nodes join cluster | Blocked by #367 | +| [#369](https://projects.knownelement.com/issues/369) | Switch config fixes (core-sw01 LAG, LACP, member list) | Onsite Friday | +| [#370](https://projects.knownelement.com/issues/370) | Vault — migrate all API keys (HIGH priority) | | +| [#371](https://projects.knownelement.com/issues/371) | AWX HTTPS/TLS configuration | Depends on #342 (PKI/SSL) | +| [#372](https://projects.knownelement.com/issues/372) | Tripp Lite UPS fix (physical reseat) | Onsite Friday | +| [#373](https://projects.knownelement.com/issues/373) | pfv-r2-sw console fix (physical, onsite) | Onsite Friday | +| [#374](https://projects.knownelement.com/issues/374) | PDU outlet renaming | | +| [#375](https://projects.knownelement.com/issues/375) | Monitoring expansion (unpoller, netdisco, weathermap, smokeping) | Relates to #337, #343 | +| [#311](https://projects.knownelement.com/issues/311) | OAM: Security Hardening (CMMC/STIG) | KNELIAC roles deployed, expansion ongoing | +| [#335](https://projects.knownelement.com/issues/335) | OAM: Centralized syslog to Wazuh | Wazuh agents deployed, fleet rollout pending | +| [#342](https://projects.knownelement.com/issues/342) | OAM: PKI/SSL — tsys-ca VM + HSM CA + ACME | Vault unconfigured, AWX HTTP-only | +| [#343](https://projects.knownelement.com/issues/343) | OAM: Monitoring coverage matrix | Coverage gaps documented | -### Kubernetes Control Plane (k3s HA — DOWN, needs rebuild) -- [ ] **Cluster DOWN 2026-08-05**: operator wiped a cnode + shut cnodes down - (100% RAM on one). cnode2 has k3s API up, cnode1/cnode3 do not — no etcd - quorum. Kubeconfig points to stale IP (100.97.178.106, matches no cnode). -- [x] Was: 3-node k3s HA control plane (cnode1/2/3, embedded etcd, Tailscale) -- [x] **All traffic over Tailscale IPs** — no LAN IPs in node status or certs -- [x] All 3 cnodes tainted `control-plane:NoSchedule` (zero user workloads) -- [x] 13/13 health checks pass (verify.sh): nodes Ready, etcd quorum, - Tailscale IPs, CoreDNS, API server, workload isolation -- [x] Scripts in `k8s/`: wipe, install-cp, join-servers, post-setup, verify -- [x] Kubeconfig saved to `~/.kube/config.pfv-k8s` (gitignored, embedded certs) -- [ ] Workers (wnodes) not yet joined to this cluster -- [ ] Distro decision: **k3s chosen for regular R&D cluster**. Talos docs - preserved in `k8s/docs/` for future ITAR/classified cluster. +## Known Issues (tracked in tickets) -### PDU Management (powerman on pfv-tsys1 — LIVE) -- [x] Cyclades AlterPath PM10i (10 outlets) managed via powerman over serial -- [x] USB-DB9 adapter (Prolific pl2303) with stable udev symlink - `/dev/cyclades-pm10` -- [x] powermand listening on `127.0.0.1:10101` + Tailscale `100.121.189.98:10101` -- [x] All 10 outlets defined as `outlet-1` through `outlet-10` -- [x] Validated: outlet 10 cycled off → on (8/8 test checks passed) -- [ ] Rename outlets to match physical devices (Friday onsite) - -### UPS Management (NUT on pfv-tsys1 — LIVE) -- [x] **APC Smart-UPS C 1500** fully reporting via NUT usbhid-ups + APC HID 0.100 -- [x] upsd listening on Tailscale `100.121.189.98:3493` (port 3493) -- [x] Local upsmon (master) configured for graceful hypervisor shutdown -- [x] Home Assistant credentials generated (`homeassistant` user in upsd.users) -- [x] Scripts in `ups/`: discover, setup (idempotent), status -- [x] Home Assistant NUT integration **configured via REST API** (`ups/setup-ha-nut.sh`) -- [x] **Key fix:** upsd now listens on LAN IP (192.168.3.11) in addition to - Tailscale — HAOS runs Tailscale as an isolated add-on, so the HA container - cannot route to Tailscale IPs. Shared vmbr0 bridge provides instant LAN path. -- [x] **Key fix:** udev rule covers BOTH `usb` and `hidraw` subsystems (driver - opens raw `/dev/bus/usb` after setuid drop, not `/dev/hidraw`) -- [ ] **Tripp Lite UPS blocked:** driver matches TrippLite HID 0.85 subdriver - but fails reading 878-byte HID Report Descriptor (EAGAIN via USB hub). - Needs physical reseat to direct motherboard USB port or cable replacement. - Driver masked to prevent restart-loop spam. -- [ ] **Home Assistant NUT integration:** configured and live (3 sensors: - battery charge, status, status data). This APC (FW 02.2, mfg 2012) does not - report `ups.load` over USB HID — confirmed via both NUT and apcupsd tests. - No wattage for Energy Dashboard; smart plug on UPS output is the practical - path. apcupsd package conflicts with nut-server (mutually exclusive). - -### Console Management (ser2net telnet(rfc2217) + conman on pfv-tsys4 — LIVE) -- [x] 7 network switch/router consoles managed via ser2net + conman -- [x] **USB enumeration problem SOLVED:** udev rules pin each adapter by - ID_PATH (physical USB port topology) to stable `/dev/consoles/` - symlinks that survive reboot regardless of enumeration order -- [x] ser2net exposes all 7 consoles on telnet(rfc2217) TCP ports (2001-2007) - bound to **Tailscale IP only** (`100.70.77.93`) -- [x] conman connects to TCP ports for logging + multiplexing (7 log - files active in `/var/log/conman/`) -- [x] Both ser2net + conmand enabled via systemd (survive reboot) -- [x] conmand systemd unit created (Debian package doesn't ship one) -- [x] conmand remote access enabled (loopback=off, port 7890) -- [x] Old `/root/conmap` + manual `screen` workflow replaced -- [x] **Stair-stepping fix:** ser2net accepter changed from raw `tcp` to - `telnet(rfc2217)` so conman's telnet NVT negotiates binary mode properly -- [x] **Validated 6/7 consoles working** via conman from workstation - (core-sw01, tor3-mgmt, tor3-stor, rrinfra-rtr, r2-tor-top, subodev-torsw) -- [ ] **pfv-r2-sw (port 2007): DEAD** — zero bytes at all baud rates - (1200-115200), adapter present and mapped correctly (/dev/ttyUSB3). - Physical issue: device powered off, console cable disconnected, or - console port disabled in device config. Requires onsite investigation. - -### Ansible AWX (tsys-awx — LIVE) -- [x] AWX 24.6.1 deployed via AWX Operator 2.19.1 on k3s single-node -- [x] Accessible at `http://tsys-awx.knel.net` (port 80, LoadBalancer via k3s ServiceLB) -- [x] Admin login: `admin` / `REDACTED_PASSWORD` -- [x] PostgreSQL 15 (bundled, 8 GiB PVC on local-path) -- [x] VM resized: 32→60 GB disk, swap converted to swapfile -- [x] Scripts in `awx/`: install-k3s, deploy-awx, verify-awx -- [ ] HTTPS/TLS not yet configured (currently HTTP on port 80) -- [ ] Execution environments not yet customized - -### Repo Merge -- [x] KNELServerBuild merged into PFVCluster (history preserved) -- [x] Directory structure reorganized (provisioning/, tests/, perf/, docs/) -- [x] All docs gardened: links fixed, stale refs removed, tailscale.md updated - -## Known Issues - -| Issue | Impact | Status | -|-------|--------|--------| -| **core-sw01 LAG hash = layer-2-3** | NFS cross-rack capped at 1 Gbps per host pair | Fix: change to layer-2-3-4 (switch change, pending approval) | -| **core-sw01 ch1 g16/g17 mismatch** | Only 3 of 4 cross-rack links active | Fix: change member list g(13-15,17)→g(13-16) | -| **Cross-rack trunk uses `mode on`** | No LACP failure detection | Fix: change to `mode active` (both switches) | -| **tsys5 bond0 degraded** (1 of 2 slaves) | Single link, no redundancy | Physical: 2nd ethernet cable needed | -| **tsys5 NVMe + SSD not configured** | Best local storage unused | Config: mount + add to Proxmox storage.cfg | -| **D2 carries 13 running VMs** | IOPS hotspot persists | Migrate to S2/S3/new disks | -| **tsys4 PVE 9.1.5** | Behind fleet (9.2.5) | Upgrade needed | -| **tsys5 SSD 93% full** | Samsung 860 PRO nearly exhausted | Offload to new SK hynix SSD | -| **Port errors on core-sw01** | g9:2902, g38:590, g26:73 inErr | Investigate CRC/cable | -| **tsys5 is single-CPU** | CPU2 socket unpopulated | Consider seating spare LGA771 Xeon | -| **Fleet CPU 0-5% idle** | Misses ~50%-idle utilization goal | Consolidate baseline; grow via workloads | - -## Pending (next session priorities) - -> **Full audit with findings + recommendations is in** -> [`proxmox/docs/AUDIT-2026-07-30.md`](proxmox/docs/AUDIT-2026-07-30.md). - -**Awaiting operator review of AUDIT-2026-07-30 before executing any fixes.** - -**Switch changes (need approval):** -- Fix core-sw01 ch1 member: g(13-15,17) → g(13-16) [restores 4th link] -- Fix core-sw01 LAG hash: layer-2-3 → layer-2-3-4 [unlocks multi-link NFS] -- Convert trunk to LACP: mode on → mode active [failure detection] -- Remove stale tsys8 config from tor3-stor (ch4/g13/g14) - -**Host-side changes (no switch access needed):** -- Mount + configure tsys5 NVMe (CT500P1 500GB) as LVM-thin -- Mount + configure tsys5 SK hynix SSD (512GB, relocated from tsys4) -- Clean stale S3/S4 exports + storage.cfg on tsys5 -- Upgrade tsys4 to PVE 9.2.5 / kernel 7.0.14 -- Apply apply-tunings.sh to tsys4/5 (nconnect, bond hash, rmem/wmem) -- Audit VM cache modes (writethrough → writeback for NFS) - -**Other:** -- **Schedule:** Operator onsite Monday 0700. Until then: software fixes, - documentation, and OAM only (no physical access). Sunday off. Saturday = - OAM Day (2026-08-01). -- **Remote-capable (now–Saturday):** tsys5 NVMe/SSD mount+config, fleet tuning - (tsys4/5 nconnect/bond hash/rmem), VM config audit, OAM Day prep, documentation -- **Onsite-only (Monday+):** switch config changes (core-sw01 ch1 hash+member, - tor3-stor LACP conversion), tsys5 2nd ethernet cable, physical hardware work -- **Saturday OAM Day (2026-08-01):** 100% instrumentation coverage — Beszel - (metrics) on all hosts, Uptime Kuma (up/down) → Pushover alerts, LibreNMS - (SNMP) for switches, Home Assistant (power/temp) for physical infra, - centralized syslog (tool TBD) -- Investigate core-sw01 port errors (g9/g38/g26) -- k8s: join wnodes, label by workload class -- k8s deferred: ETL tooling, HPC scheduler, vcluster policy +All known infrastructure issues are captured in ticket descriptions and notes. +See [#369](https://projects.knownelement.com/issues/369) for switch issues, +[#372](https://projects.knownelement.com/issues/372) for Tripp Lite UPS, +[#373](https://projects.knownelement.com/issues/373) for dead console port. ## Infrastructure Summary | Component | Details | |-----------|---------| | Proxmox hosts | 7 standalone PVE, managed via PDM (tsys8 retired) | -| Storage servers | tsys4 (NFS: D1/D2/D3/D5 + /mnt/albert + /mnt/backup), tsys5 (NFS: S1/S2 + local SSD + new NVMe unconfigured) | -| Switches | core-sw01 (PowerConnect 5448, rack5), tor3-stor/mgmt (PowerConnect 5324, rack3), + 1 unidentified (192.168.0.8) | +| Storage servers | tsys4 (NFS: D1/D2/D3/D5 + /mnt/albert + /mnt/backup), tsys5 (NFS: S1/S2 + local SSD + NVMe) | +| Switches | core-sw01 (PowerConnect 5448, rack5), tor3-stor/mgmt (PowerConnect 5324, rack3) | | DNS primary | pfv-netinfra-01 (192.168.3.252) — Technitium + Pi-hole | | DNS secondary | pfv-netinfra-02 (192.168.3.253) — Technitium + Pi-hole | -| DNS production | tailscale-router (read-only source of truth) | -| NTP | pfv-netinfra-01/02 (redundant, LAN IPs, stratum 2/3) | +| NTP | pfv-netinfra-01/02 (redundant, stratum 3) | | Sandbox VM | sectestbed-sandbox (VMID 6000 on pfv-tsys5) | | Backup | Proxmox Backup Server (PBS) | -| **k8s control plane** | **3-node k3s HA (cnode1/2/3), all traffic over Tailscale** | -| **PDU** | **Cyclades PM10i via powerman on pfv-tsys1 (port 10101)** | -| **AWX** | **tsys-awx: AWX 24.6.1 on k3s (http://tsys-awx.knel.net:80)** | -| **Console** | **7 switch consoles via ser2net telnet(rfc2217)+conman on pfv-tsys4 (conmand:7890 on Tailscale)** | -| **UPS** | **APC Smart-UPS C 1500 via NUT on pfv-tsys1 (upsd:3493 on Tailscale)** | -| Production | Cloudron VPS, Reston VA (this cluster is R&D only) | +| k8s control plane | **DOWN** — 3-node k3s HA, needs rebuild ([#367](https://projects.knownelement.com/issues/367)) | +| PDU | Cyclades PM10i via powerman on pfv-tsys1 (port 10101) | +| AWX | tsys-awx: AWX 24.6.1 on k3s (http://tsys-awx.knel.net:80) | +| Console | 7 switch consoles via ser2net + conman on pfv-tsys4 | +| UPS | APC Smart-UPS C 1500 via NUT on pfv-tsys1 | +| Production | Cloudron VPS, Reston, VA (this cluster is R&D only) | +| Config management | KNELIAC (Ansible) — `/home/reachableceo/projects/KNELIAC` | diff --git a/docs/docmap.md b/docs/docmap.md index d5aa13a..cf9f21b 100644 --- a/docs/docmap.md +++ b/docs/docmap.md @@ -80,12 +80,19 @@ context — do not update; link to active docs instead. | [`archive/charles-todo.md`](archive/charles-todo.md) | Charles's early task list (completed) | | [`archive/claude-todo.md`](archive/claude-todo.md) | Claude's early task list (completed) | +## External Tracking + +| Resource | Description | +|----------|-------------| +| [Redmine](https://projects.knownelement.com) | **System of record for all work.** Version: Potential to Kinetic Ready (due 2026-09-30). Project: Technology & Facility Services (id 55) | +| [Gitea](https://git.knownelement.com/KNEL/PFVCluster) | Source code repository | + ## Top-Level Files | File | Description | |------|-------------| -| [`../AGENTS.md`](../AGENTS.md) | Agent operating instructions (repo layout, git policy, gardening protocol) | -| [`../STATUS.md`](../STATUS.md) | Living project status (agent-maintained, human read-only) | +| [`../AGENTS.md`](../AGENTS.md) | Agent operating instructions (repo layout, git policy, Redmine tracking, gardening protocol) | +| [`../STATUS.md`](../STATUS.md) | Ticket index + infrastructure summary (read-only snapshot; work tracked in Redmine) | | [`../README.md`](../README.md) | Project overview and quick start | | [`../LICENSE`](../LICENSE) | License | @@ -95,9 +102,11 @@ context — do not update; link to active docs instead. When making changes to this repo, agents MUST: -1. **Update [`../STATUS.md`](../STATUS.md)** if the work changes infrastructure - state, completes/starts a task, or discovers a new issue. -2. **Update this file (`docmap.md`)** if a doc is added, removed, or has a +1. **Update Redmine tickets** — all work status, completion, and issue tracking + lives in [Redmine](https://projects.knownelement.com), not in repo files. +2. **Update [`../STATUS.md`](../STATUS.md)** if the infrastructure summary or + ticket index needs refreshing (read-only snapshot only). +3. **Update this file (`docmap.md`)** if a doc is added, removed, or has a substantive content change. Update the "Last Reviewed" date. 3. **Verify cross-references** — any new `.md` file must be linked from at least one existing doc or this map.