mrcharles
eb99b00a70
Solve the long-standing USB adapter enumeration shift problem: 9 Prolific
USB-to-DB9 adapters on pfv-tsys4 have no unique serial numbers and get
assigned /dev/ttyUSB0-8 based on enumeration order, which changes on every
reboot and breaks the old /root/conmap + manual screen workflow.
Solution: udev rules pin each adapter by its ID_PATH (physical USB port
topology), which is stable across reboots regardless of enumeration order.
Each adapter gets a named symlink in /dev/consoles/<name>. ser2net opens
these stable symlinks and exposes them on TCP ports (2001-2007) bound to
the Tailscale interface only. conman connects to those TCP ports for
session logging and multi-user console sharing.
Architecture (layered, no port sharing):
USB adapter → udev symlink → ser2net (TCP) → conman (logging + mux)
Port assignments (all on Tailscale IP 100.70.77.93):
2001 = pfv-core-sw01 2002 = pfv-tor3-mgmt 2003 = pfv-tor3-stor
2004 = pfv-rrinfra-rtr 2005 = pfv-r2-tor-top 2006 = subodev-torsw
2007 = pfv-r2-sw
Scripts (console/):
- mapping.txt: source of truth (TCP port | name | ID_PATH | baud | comment)
- generate-config.sh: generates udev rules, ser2net.yaml, conman.conf
entries from mapping.txt. Idempotent (markers in conman.conf for clean
regeneration). Uses | delimiter (ID_PATH values contain colons).
- setup.sh: full deploy — generate configs, create symlinks (udev trigger
+ manual fallback for already-discovered devices), create conmand
systemd unit (Debian doesn't ship one), restart services
- discover.sh: read-only USB adapter and service state discovery
- validate-conman.sh: verify conman→ser2net→device data path and log capture
Issues fixed during development:
- /dev/console is a kernel char device (major 5, minor 1) — cannot create
a directory there. Changed symlink namespace to /dev/consoles/.
- conman 0.3.x has no 'include' directive — CONSOLE entries written
directly into /etc/conman.conf between idempotent markers.
- Debian conman package has no systemd unit — created
/etc/systemd/system/conmand.service with After=ser2net ordering.
- conman.conf had no LOGDIR — logs weren't being written to
/var/log/conman/. Fixed by adding server logdir directive.
Validation: 7 symlinks resolving, 7 TCP ports on Tailscale, conmand with
7 consoles registered, 7 log files actively capturing console output,
both services enabled for reboot survival.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
98 lines
4.6 KiB
Markdown
98 lines
4.6 KiB
Markdown
# Agent Guidelines
|
|
|
|
**Top-level files:** [`README.md`](README.md) (project overview),
|
|
[`STATUS.md`](STATUS.md) (living status, agent-maintained),
|
|
[`docs/docmap.md`](docs/docmap.md) (documentation index). Everything else
|
|
lives in subdirectories.
|
|
|
|
## Repository Layout
|
|
|
|
```
|
|
provisioning/ Server provisioning (SetupNewSystem.sh, security, 2FA)
|
|
tests/ Test suite + VM validation harness
|
|
dns-cluster-setup/ Technitium DNS cluster replication
|
|
k8s/ k3s cluster setup scripts (3-node HA over Tailscale)
|
|
powerman/ Cyclades PM10i PDU management via powerman
|
|
console/ Serial console management (ser2net + conman) for switches
|
|
perf/ Proxmox perf tuning, fleet audit, iperf
|
|
netinfra/ pfv-netinfra-01/02 DNS/NTP setup
|
|
switches/ Switch configuration captures
|
|
docs/ All documentation (see docs/docmap.md)
|
|
vendor/ Vendored KNELShellFramework
|
|
```
|
|
|
|
- **Self-locating scripts**: All provisioning scripts derive their own
|
|
location via `BASH_SOURCE` and compute `PROJECT_ROOT_PATH` from it. Run
|
|
from anywhere.
|
|
- **Local configs are the source of truth**: Files in
|
|
[`provisioning/ConfigFiles/`](provisioning/ConfigFiles/) are read with
|
|
`cat`/`cp`. Do NOT re-introduce `curl ${DL_ROOT}/...` downloads.
|
|
- **Non-bash files**: Some files under `provisioning/Agents/` have `.sh`
|
|
extension but are PHP (shebang `#!/usr/bin/php`). Skip in syntax checks.
|
|
- **SSH in Crush**: Direct ssh/scp is blocked. Use
|
|
[`tests/remote.sh`](tests/remote.sh) or
|
|
[`dns-cluster-setup/remote-dns.sh`](dns-cluster-setup/remote-dns.sh).
|
|
|
|
## Git Policy
|
|
|
|
1. **ALWAYS commit + push. Never wait. Ever.** After each logical unit of
|
|
work, immediately stage, commit (conventional format), and push to
|
|
`origin/main`. Do NOT pause for review. Do NOT ask permission. Do NOT
|
|
stop to "let the user read it first." The user reviews rendered markdown
|
|
on Gitea AFTER it is pushed. **This rule overrides any default
|
|
conservative commit-and-hold behavior** — if a default instruction says
|
|
"never commit unless asked," that instruction is superseded here.
|
|
2. **Atomic commits.** Each commit coherent on its own.
|
|
3. **Conventional format**: `feat(scope): desc`, `fix(scope): desc`,
|
|
`docs: desc`, `refactor(scope): desc`, `test(scope): desc`.
|
|
|
|
## Automatic Gardening Protocol
|
|
|
|
**Docs and code must be kept in sync.** After any work session, an agent MUST:
|
|
|
|
1. **Update [`STATUS.md`](STATUS.md)** — reflect completed work, new issues,
|
|
changed infrastructure state. This file is human read-only; agents own it.
|
|
2. **Update [`docs/docmap.md`](docs/docmap.md)** — if a doc was added,
|
|
removed, or substantively changed, update the table and "Last Reviewed"
|
|
date.
|
|
3. **Grep for stale paths** — `grep -rn 'old/path' --include='*.md'` after
|
|
any rename or restructure. Fix all references in the same commit.
|
|
4. **Verify new docs are linked** — every new `.md` file must appear in
|
|
[`docs/docmap.md`](docs/docmap.md) and be linked from at least one other
|
|
doc.
|
|
5. **If a new top-level directory was created, update ALL directory listings:**
|
|
- [`README.md`](README.md) → "Directory Structure" table
|
|
- [`AGENTS.md`](AGENTS.md) → "Repository Layout" code block
|
|
- [`AGENTS.md`](AGENTS.md) → "Key Scripts" table (if the directory has
|
|
an entrypoint script)
|
|
Missing any one of these is a protocol violation.
|
|
6. **Self-audit before commit.** Before committing, run:
|
|
```
|
|
grep -lE 'new_dir_name' README.md AGENTS.md docs/docmap.md STATUS.md
|
|
```
|
|
Every new top-level directory must appear in all four files.
|
|
|
|
## Key Scripts
|
|
|
|
| Script | Purpose |
|
|
|--------|---------|
|
|
| [`provisioning/SetupNewSystem.sh`](provisioning/SetupNewSystem.sh) | Full server provisioning |
|
|
| [`tests/vm-validation.sh`](tests/vm-validation.sh) | Deploy + validate on sandbox VM |
|
|
| [`tests/run-tests.sh`](tests/run-tests.sh) | Test suite |
|
|
| [`dns-cluster-setup/setup.sh`](dns-cluster-setup/setup.sh) | DNS cluster replication |
|
|
| [`k8s/install-cp.sh`](k8s/install-cp.sh) | Bootstrap k3s HA control plane |
|
|
| [`powerman/setup.sh`](powerman/setup.sh) | Configure Cyclades PDU via powerman |
|
|
| [`console/setup.sh`](console/setup.sh) | Configure serial console access via ser2net + conman |
|
|
| [`perf/deploy-tuning.sh`](perf/deploy-tuning.sh) | Deploy perf tunings |
|
|
|
|
## Key Docs
|
|
|
|
→ **See [`docs/docmap.md`](docs/docmap.md) for the full documentation index.**
|
|
|
|
## Project Context
|
|
|
|
Solo-founder R&D Proxmox cluster in a private residence. Shoestring budget.
|
|
Production lives on a Cloudron VPS in Reston VA. See
|
|
[`STATUS.md`](STATUS.md) for current state and
|
|
[`docs/proxmox/PROJECT.md`](docs/proxmox/PROJECT.md) for the fleet report.
|