From 823eb9b33d7fdb4fee1e5985c2fc1da1af979ad7 Mon Sep 17 00:00:00 2001 From: reachableceo Date: Fri, 4 Sep 2026 05:51:51 -0500 Subject: [PATCH] docs: AGENTS.md repoint to KNEL repos + stale-ref fixes [#769] MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes dba54b2 (which staged deletions only — pathspec'd add missed the content edits). Layout table, Key Scripts, DNS discipline, conman section now point at KNEL/netinfra|facilities|perf|monitoring|inventory; conman example switched from IP literal to pfv-tsys4 (DNS-names rule). https://projects.knownelement.com/issues/769#note-4152 --- AGENTS.md | 82 ++++++++++++++++++++++++------------------ scripts/audit-fleet.sh | 2 +- scripts/check-rules.sh | 2 +- tests/shellcheck.sh | 2 -- 4 files changed, 50 insertions(+), 38 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index c1c7fdf..d47f5db 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -354,27 +354,41 @@ Work tracking stays in [Redmine](https://projects.knownelement.com). ## Repository Layout ``` -dcinfra/ Data-center infra: PDU (powerman), serial console (console), UPS (ups), sensors (temper) -HomeAssistant/ Redirect stub — HA tooling + reference packs moved to KNEL/pfv-bms (tools/, docs/reference-packs/) [#762] -netinfra/ DNS/NTP/DHCP setup + audit; DNS cluster replication (dns-cluster-setup); - switch captures (switches); DHCP config (dhcp) k8s/ k3s cluster setup scripts (HA control plane over Tailscale) + docs/ -oam/ OAM platform tooling (oxidized, unpoller, smokeping, netdisco, weathermap; - librenms-agent: check_mk agent + snmp-extend scripts ported from KNELServerBuild [#474]) -proxmox/ Proxmox fleet docs (hardware audit, capacity, storage) + perf tuning (perf/) +proxmox/ Proxmox fleet docs (hardware audit, capacity, storage) awx/ Ansible AWX deployment (k3s + AWX Operator) -siem/ Wazuh/SIEM deploy tooling (agents on VMs, rsyslog forwarding on PVE) [#335] -ca/ Fleet CA tooling (init/issue/selftest; intermediate on tsys-ca; design: Discourse t/320) [#697] -cmdb/ GLPI/CMDB seed tooling (inventory→CSV converter; design: Discourse t/319, #705) +HomeAssistant/ Redirect stub — HA tooling + reference packs moved to KNEL/pfv-bms (tools/, docs/reference-packs/) [#762] tests/ Test suite + VM validation harness + remote.sh SSH chokepoint scripts/ Framework: git hooks, rule engine (check-rules.sh), shared lib docs/ Server-build docs, docmap index, and archive archive/ Historical/superseded code (provisioning -> replaced by KNELIAC project; - KNELServerBuild -> live patterns ported to oam/librenms-agent/, remainder + KNELServerBuild -> live patterns now in KNEL/monitoring, remainder preserved under archive/KNELServerBuild/ [#474]) vendor/ Vendored KNELShellFramework ``` +- **2026-09-03 repo split (#769)** — domain tooling moved out of this catch-all + into dedicated KNEL repos (working dirs `~/projects/KNEL/`; full git + history retained here): + + | body of work | repo | ticket | + |---|---|---| + | SIEM/syslog (Wazuh) | KNEL/siem | #335 | + | DNS/NTP/DHCP + switches | KNEL/netinfra | #694 | + | Fleet CA | KNEL/ca | #697 | + | Monitoring/OAM | KNEL/monitoring | #343, #773 | + | Inventory/CMDB | KNEL/inventory | #705, #774 | + | Facilities (PDU/console/UPS/sensors) | KNEL/facilities | #458 | + | Performance/tuning | KNEL/perf | #709, #732, #737 | + | Backups (PBS) | KNEL/backups (new stub) | #315, #684 | + | Outbound mail (PMG) | KNEL/mailgateway (new stub) | #696 | + | Secrets mgmt | KNEL/secrets (new stub) | #770 | + | apt-satellite (deb version control) | KNEL/apt-satellite (new stub) | #771 | + | Security Onion (ops side) | KNEL/security-onion (new stub) | #772 | + + Unit tests moved with their code (netinfra, facilities). IaC consumers: + KNEL/KNELIAC. Env map + snapshot lifecycle: Discourse t/331. + - **Server provisioning moved to KNELIAC**: The [`archive/provisioning/`](archive/provisioning/) tree is historical. Active server provisioning lives in the **KNELIAC** project at @@ -384,8 +398,8 @@ vendor/ Vendored KNELShellFramework - **Remote access (NON-NEGOTIABLE):** ALL SSH/SCP to ANY host MUST go through the chokepoint scripts — [`tests/remote.sh`](tests/remote.sh) (Proxmox hosts + all VMs) or - [`netinfra/dns-cluster-setup/remote-dns.sh`](netinfra/dns-cluster-setup/remote-dns.sh) - (DNS infra hosts: netinfra-01/02, netboot). NEVER call + `~/projects/KNEL/netinfra/dns-cluster-setup/remote-dns.sh` + (DNS infra hosts: netinfra-01/02, netboot; lives in KNEL/netinfra [#769]). NEVER call `ssh`/`scp` directly — the harness blocks raw ssh and the command scanner rejects it. There are no exceptions. - **DNS names ONLY (NON-NEGOTIABLE):** NEVER use IP address literals @@ -481,9 +495,9 @@ Founder rule, 2026-09-02 — drift-check exists because git went stale; do not reopen that gap: 1. Make the change (serial + health-gated across the redundant pair). -2. `bash netinfra/dns/technitium/zone-snapshot.sh` — refresh the DZ +2. `cd ~/projects/KNEL/netinfra && bash dns/technitium/zone-snapshot.sh` — refresh the DZ snapshots from the primary. -3. `bash netinfra/dns/drift-check.sh` — must print `ALL IN SYNC`. +3. `cd ~/projects/KNEL/netinfra && bash dns/drift-check.sh` — must print `ALL IN SYNC`. 4. Commit + push the refreshed snapshots in the same session. ## Redmine Tracking Policy @@ -512,7 +526,7 @@ tracker after create (python escape hatch: `tracker_id=3`). [2026-08-27] close — leave that decision to the user. 4. **Operations Status** lives on Discourse ([topic #297](https://community.turnsys.com/t/297)) — update that wiki topic if the infrastructure summary needs refreshing. The `STATUS.md` file in git is now a pointer only. 5. **Link code to tickets** — ticket descriptions and notes should reference - the relevant file paths in this repo (e.g., `dcinfra/ups/`). + the relevant file paths in this repo (e.g., the ups tooling in KNEL/facilities). ### CLI access (read + write) @@ -612,22 +626,22 @@ live in the centralized store at `~/.creds/discourse.env`. | [`scripts/check-rules.sh`](scripts/check-rules.sh) | Rule audit engine (shellcheck, image pinning, Discourse pointers, required files) | | [`scripts/setup-hooks.sh`](scripts/setup-hooks.sh) | Install git hooks (pre-commit, pre-push) | | [`tests/remote.sh`](tests/remote.sh) | **SSH chokepoint** — all Proxmox host + sandbox VM access routes here | -| [`netinfra/dns-cluster-setup/remote-dns.sh`](netinfra/dns-cluster-setup/remote-dns.sh) | SSH chokepoint for DNS infra hosts (netinfra-01/02, netboot) | -| [`netinfra/dns/technitium/zone-snapshot.sh`](netinfra/dns/technitium/zone-snapshot.sh) | Refresh git SoR Technitium zone snapshots from the primary — **required after every DNS change** | -| [`netinfra/dns/drift-check.sh`](netinfra/dns/drift-check.sh) | Live-vs-git drift audit for DHCP/DNS/NTP/zones — must be green before any DNS-adjacent push | -| [`cmdb/seed/from-inventory.sh`](cmdb/seed/from-inventory.sh) | Regenerate the CMDB seed CSV from the Discourse #307 inventory raw (re-run when #307 changes) | +| KNEL/netinfra: `dns-cluster-setup/remote-dns.sh` | SSH chokepoint for DNS infra hosts (netinfra-01/02, netboot) | +| KNEL/netinfra: `dns/technitium/zone-snapshot.sh` | Refresh git SoR Technitium zone snapshots from the primary — **required after every DNS change** | +| KNEL/netinfra: `dns/drift-check.sh` | Live-vs-git drift audit for DHCP/DNS/NTP/zones — must be green before any DNS-adjacent push | +| KNEL/inventory: `seed/from-inventory.sh` | Regenerate the CMDB seed CSV from the Discourse #307 inventory raw (re-run when #307 changes) | | `redmine-cli` container | Redmine CLI (ticket read/write via `docker run`; see `tooling-cli/redmine/`) | | `discourse-cli` container | Discourse CLI (wiki topic read/write via `docker run`; see `tooling-cli/discourse/`) | | `dns-cli` container | Technitium DNS CLI (zones, list, add, delete, search, flush; see `tooling-cli/dns/`) | | [`tests/vm-validation.sh`](tests/vm-validation.sh) | Deploy + validate on sandbox VM | | [`tests/run-tests.sh`](tests/run-tests.sh) | Test suite | -| [`netinfra/dns-cluster-setup/setup.sh`](netinfra/dns-cluster-setup/setup.sh) | DNS cluster replication | +| KNEL/netinfra: `dns-cluster-setup/setup.sh` | DNS cluster replication | | [`k8s/install-cp.sh`](k8s/install-cp.sh) | Bootstrap k3s HA control plane | -| [`dcinfra/powerman/setup.sh`](dcinfra/powerman/setup.sh) | Configure Cyclades PDU via powerman | -| [`dcinfra/console/setup.sh`](dcinfra/console/setup.sh) | Configure serial console access via ser2net + conman | -| [`dcinfra/ups/setup.sh`](dcinfra/ups/setup.sh) | Configure NUT (Network UPS Tools) for UPS monitoring | -| [`proxmox/perf/deploy-tuning.sh`](proxmox/perf/deploy-tuning.sh) | Deploy perf tunings | -| [`proxmox/perf/scripts/`](proxmox/perf/scripts/) | Read-only audit: probe-storage, probe-network, conman-console, snmp-switch-audit, probe-drift, audit-vm-disks, audit-guest-io, deploy-tuned-guests | +| KNEL/facilities: `powerman/setup.sh` | Configure Cyclades PDU via powerman | +| KNEL/facilities: `console/setup.sh` | Configure serial console access via ser2net + conman | +| KNEL/facilities: `ups/setup.sh` | Configure NUT (Network UPS Tools) for UPS monitoring | +| KNEL/perf: `deploy-tuning.sh` | Deploy perf tunings | +| KNEL/perf: `scripts/` | Read-only audit: probe-storage, probe-network, conman-console, snmp-switch-audit, probe-drift, audit-vm-disks, audit-guest-io, deploy-tuned-guests | ## Switch Console Access (conman) @@ -638,16 +652,16 @@ stack on pfv-tsys4. - **ser2net:** pfv-tsys4, TCP ports 2001-2006 (do NOT connect to ser2net directly — it conflicts with conman's persistent sessions; always use the conman client) -- **Script:** [`proxmox/perf/scripts/conman-console.py`](proxmox/perf/scripts/conman-console.py) +- **Script:** KNEL/perf: `scripts/conman-console.py` — drives console sessions read-only via PTY. No expect/tcl required. -- **Command files:** [`netinfra/switches/`](netinfra/switches/) — `.cmds` files +- **Command files:** KNEL/netinfra: `switches/` — `.cmds` files with switch-specific show commands -- **Query available consoles:** `conman -d 100.70.77.93 -q` +- **Query available consoles:** `conman -d pfv-tsys4 -q` ```bash # Pull a switch config (example): -CONMAN_SERVER=100.70.77.93:7890 python3 proxmox/perf/scripts/conman-console.py \ - --console pfv-r5-core-01 --cmds netinfra/switches/pfv-r5-core-01.cmds +CONMAN_SERVER=pfv-tsys4:7890 python3 ~/projects/KNEL/perf/scripts/conman-console.py \ + --console pfv-r5-core-01 --cmds ~/projects/KNEL/netinfra/switches/pfv-r5-core-01.cmds ``` | Console name | TCP port | Device | @@ -712,11 +726,11 @@ PROX_HOST=pfv-tsys5 bash tests/remote.sh prox 'pvesm status' VM_IP=tsys-librenms VM_USER=root bash tests/remote.sh vm 'systemctl status cron' # DNS infra access: -bash netinfra/dns-cluster-setup/remote-dns.sh netinfra01-root 'systemctl status docker' +bash ~/projects/KNEL/netinfra/dns-cluster-setup/remote-dns.sh netinfra01-root 'systemctl status docker' # Switch console (read-only config pull — DNS names only): -CONMAN_SERVER=pfv-tsys4:7890 python3 proxmox/perf/scripts/conman-console.py \ - --console pfv-r5-core-01 --cmds netinfra/switches/pfv-r5-core-01.cmds +CONMAN_SERVER=pfv-tsys4:7890 python3 ~/projects/KNEL/perf/scripts/conman-console.py \ + --console pfv-r5-core-01 --cmds ~/projects/KNEL/netinfra/switches/pfv-r5-core-01.cmds ``` ## Credential Management diff --git a/scripts/audit-fleet.sh b/scripts/audit-fleet.sh index c9574d8..05f01c2 100755 --- a/scripts/audit-fleet.sh +++ b/scripts/audit-fleet.sh @@ -19,7 +19,7 @@ set -uo pipefail REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" REMOTE="$REPO_ROOT/tests/remote.sh" -REMOTE_DNS="$REPO_ROOT/netinfra/dns-cluster-setup/remote-dns.sh" +REMOTE_DNS="$HOME/projects/KNEL/netinfra/dns-cluster-setup/remote-dns.sh" # moved to KNEL/netinfra [#769] DNS_SUFFIX="knel.net" DNS_SERVER="pfv-netinfra-01.knel.net" PROX_NODES=(pfv-tsys1 pfv-tsys3 pfv-tsys4 pfv-tsys5 pfv-tsys6 pfv-tsys7 pfv-tsys9) diff --git a/scripts/check-rules.sh b/scripts/check-rules.sh index c112383..1d0c16d 100755 --- a/scripts/check-rules.sh +++ b/scripts/check-rules.sh @@ -41,7 +41,7 @@ TODAY="$(date +%Y-%m-%d)" # Runs in Docker so the host stays clean (no native shellcheck required). # ---------------------------------------------------------------------------- $RULE_VERBOSE && log_step "Shell scripts (shellcheck)" -mapfile -d '' SH_FILES < <(find . -path ./.git -prune -o -path ./.tmp -prune -o -path ./.crush -prune -o -path ./vendor -prune -o -path ./oam/librenms-agent/agent -prune -o -path ./archive -prune -o -path ./node_modules -prune -o \( -name '*.sh' -o -name '*.bash' \) -print0 2>/dev/null) +mapfile -d '' SH_FILES < <(find . -path ./.git -prune -o -path ./.tmp -prune -o -path ./.crush -prune -o -path ./vendor -prune -o -path ./archive -prune -o -path ./node_modules -prune -o \( -name '*.sh' -o -name '*.bash' \) -print0 2>/dev/null) if [ "${#SH_FILES[@]}" -gt 0 ]; then if have shellcheck; then if shellcheck "${SH_FILES[@]}" >/dev/null 2>&1; then diff --git a/tests/shellcheck.sh b/tests/shellcheck.sh index 8f91779..e8c6037 100644 --- a/tests/shellcheck.sh +++ b/tests/shellcheck.sh @@ -24,7 +24,6 @@ mapfile -t FILES < <( case "$f_abs" in "$ROOT"/vendor/*) ;; # skip vendored "$ROOT"/archive/provisioning/Agents/librenms/*) ;; # upstream librenms-agent scripts - "$ROOT"/oam/librenms-agent/agent/*) ;; # upstream librenms-agent scripts (verbatim port [#474]) "$ROOT"/archive/KNELServerBuild/*) ;; # archived legacy code, not maintained [#474] *) case "$f_abs" in *.sh) echo "$f_abs";; esac ;; esac @@ -35,7 +34,6 @@ mapfile -t FILES < <( case "$f_abs" in "$ROOT"/vendor/*) ;; # skip vendored "$ROOT"/archive/provisioning/Agents/librenms/*) ;; # upstream librenms-agent scripts - "$ROOT"/oam/librenms-agent/agent/*) ;; # upstream librenms-agent scripts (verbatim port [#474]) "$ROOT"/archive/KNELServerBuild/*) ;; # archived legacy code, not maintained [#474] *) echo "$f_abs" ;; esac