From d22db46e9f7ffaa34166d930a64b50cac5a32e03 Mon Sep 17 00:00:00 2001 From: reachableceo Date: Thu, 6 Aug 2026 17:17:37 -0500 Subject: [PATCH] docs(agents): move operational knowledge from harness memory to AGENTS.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Make operational instructions portable across AI harnesses (Crush, Hermes, OpenWebUI). Previously lived in .crush/memory/operational.md which is Crush-specific. New sections added to AGENTS.md: - Switch Console Access (conman): server, script, console map, usage - Mandatory Documentation Protocol: every infra change must be ticketed + logged in Discourse before moving on - Tooling: available tools, shellcheck, HTTP write constraints - Key Commands Quick Reference: common commands for all access patterns - Credential Management: current state + Vault migration goal Updated: - "SSH in Crush" generalized to "SSH access" (harness-agnostic) - Key Scripts table updated with new audit/tuned scripts The .crush/memory/operational.md file is now redundant; all its content lives in AGENTS.md. 💘 Generated with Crush Assisted-by: Crush:glm-5.2 --- AGENTS.md | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 8ba5106..2a7013b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -49,7 +49,7 @@ vendor/ Vendored KNELShellFramework `/home/reachableceo/projects/KNELIAC`. - **Non-bash files**: Some files under `archive/provisioning/Agents/` have `.sh` extension but are PHP (shebang `#!/usr/bin/php`). Skip in syntax checks. -- **SSH in Crush**: Direct ssh/scp is blocked. Use +- **SSH access**: Direct ssh/scp may be blocked by some AI harnesses. Use [`tests/remote.sh`](tests/remote.sh) (Proxmox hosts + sandbox VM) or [`netinfra/dns-cluster-setup/remote-dns.sh`](netinfra/dns-cluster-setup/remote-dns.sh) (DNS infra hosts). @@ -193,7 +193,100 @@ lives (gitignored) at `/home/reachableceo/projects/KNEL-AIMiddleware/discourse-c | [`dcinfra/console/setup.sh`](dcinfra/console/setup.sh) | Configure serial console access via ser2net + conman | | [`dcinfra/ups/setup.sh`](dcinfra/ups/setup.sh) | Configure NUT (Network UPS Tools) for UPS monitoring | | [`proxmox/perf/deploy-tuning.sh`](proxmox/perf/deploy-tuning.sh) | Deploy perf tunings | -| [`proxmox/perf/scripts/`](proxmox/perf/scripts/) | Read-only audit: probe-storage, probe-network, conman-console, snmp-switch-audit, probe-drift | +| [`proxmox/perf/scripts/`](proxmox/perf/scripts/) | Read-only audit: probe-storage, probe-network, conman-console, snmp-switch-audit, probe-drift, audit-vm-disks, audit-guest-io, deploy-tuned-guests | + +## Switch Console Access (conman) + +Switch configs are pulled via serial console through a conman + ser2net +stack on pfv-tsys4. + +- **conmand server:** pfv-tsys4 (Tailscale `100.70.77.93`, port 7890) +- **ser2net:** pfv-tsys4, TCP ports 2001-2007 on Tailscale IP + (do NOT connect to ser2net directly — it conflicts with conman's + persistent sessions; always use the conman client) +- **Script:** [`proxmox/perf/scripts/conman-console.py`](proxmox/perf/scripts/conman-console.py) + — drives console sessions read-only via PTY. No expect/tcl required. +- **Command files:** [`netinfra/switches/`](netinfra/switches/) — `.cmds` files + with switch-specific show commands +- **Query available consoles:** `conman -d 100.70.77.93 -q` + +```bash +# Pull a switch config (example): +CONMAN_SERVER=100.70.77.93:7890 python3 proxmox/perf/scripts/conman-console.py \ + --console pfv-core-sw01 --cmds netinfra/switches/pfv-core-sw01.cmds +``` + +| Console name | TCP port | Device | +|--------------|----------|--------| +| pfv-core-sw01 | 2001 | Dell PowerConnect 5448 (core, rack 5) | +| pfv-tor3-mgmt | 2002 | Dell PowerConnect 5324 (rack 3 mgmt TOR) | +| pfv-tor3-stor | 2003 | Dell PowerConnect 5324 (rack 3 storage TOR) | +| pfv-rrinfra-rtr | 2004 | Cisco router (rrinfra) | +| pfv-r2-tor-top | 2005 | Rack 2 top-of-rack switch | +| subodev-torsw | 2006 | Suborbital device TOR switch | +| pfv-r2-sw | 2007 | Rack 2 old Dell switch | + +## Mandatory: Document ALL Infrastructure Changes + +**This is non-negotiable. Every infrastructure change (VM config, disk +cache, network setting, service config, storage migration) MUST be +documented BEFORE moving to the next task step — not "later" or "at the +end."** + +Required for EVERY infrastructure change: + +1. **Redmine ticket** — create one if none exists. Reference as `[#NNN]`. +2. **Discourse audit log** — reply to topic + [#298](https://community.turnsys.com/t/298) with a dated entry (what + changed, why, where). +3. **Discourse relevant wiki topic** — update the architecture/reference + topic if the change affects documented infrastructure facts (storage + #300, network #299, k8s #305, etc.). +4. **Git commit** — if code/config changed in the repo, commit + push + immediately per the Git Policy above. + +## Tooling + +- `gh`, `docker`, `jq` available on the workstation. +- No native shellcheck — use `bash tests/shellcheck.sh` (Docker wrapper + `koalaman/shellcheck:stable`). ALL scripts must pass including info-level. +- For raw API calls not covered by the Redmine/Discourse CLIs, use + `python3` inside the CLI Docker containers (escape hatch pattern in + `~/daytoday/redmine/AGENTS.md` and `~/daytoday/discourse/AGENTS.md`). +- `curl`/`wget`/`httpie` may be blocked by some harnesses. Use the CLIs or + the python-in-Docker escape hatch for HTTP writes. + +## Key Commands Quick Reference + +```bash +# Tests + validation: +bash tests/run-tests.sh # test suite +bash tests/vm-validation.sh # VM validation +bash tests/shellcheck.sh # shellcheck whole repo +bash tests/shellcheck.sh path/to/*.sh # shellcheck specific files + +# Redmine + Discourse sanity checks (run first in any session): +~/daytoday/redmine/bin/redmine whoami +~/daytoday/discourse/bin/discourse whoami + +# Proxmox host access: +PROX_HOST=pfv-tsys5 bash tests/remote.sh prox 'qm list' +PROX_HOST=pfv-tsys5 bash tests/remote.sh prox 'pvesm status' + +# DNS infra access: +bash netinfra/dns-cluster-setup/remote-dns.sh netinfra01-root 'systemctl status docker' + +# Switch console (read-only config pull): +CONMAN_SERVER=100.70.77.93:7890 python3 proxmox/perf/scripts/conman-console.py \ + --console pfv-core-sw01 --cmds netinfra/switches/pfv-core-sw01.cmds +``` + +## Credential Management + +- API keys currently in `.env` files under KNEL-AIMiddleware (gitignored). +- User goal: migrate all keys to Hashicorp Vault + (`vault.knownelement.com`). No vault token present yet + (`~/.vault-token` missing). Track as high-priority TODO. ## Key Docs