Commit Graph
3 Commits
Author SHA1 Message Date
mrcharles 16db9b0426 feat(k8s,proxmox): add etcd tuning for spinning-disk storage + VM disk audit script
k8s/install-cp.sh: add --etcd-arg heartbeat-interval=1000 and
election-timeout=5000 to both bootstrap and join commands. Gives etcd 5x
margin to absorb transient fsync stalls on spinning-disk-backed NFS
storage (default 500ms/1500ms too tight for this environment).

proxmox/perf/scripts/audit-vm-disks.sh: new script to audit disk cache
configuration on all VMs across Proxmox hosts. Identifies VMs that would
benefit from cache=writeback (especially etcd/database workloads on
NFS-backed spinning disk).

[#392] [#393]

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
2026-08-06 15:26:58 -05:00
mrcharles 0fa0692c37 chore: enforce shellcheck across the repo
Establish shellcheck as a mandatory pre-commit quality gate and bring all 93
shell scripts to a clean state.

- tests/shellcheck.sh: wrapper that runs koalaman/shellcheck:stable via Docker
  (no native binary needed), skips vendored + upstream librenms-agent scripts.
- .shellcheckrc: documents intentional codebase-wide disables (dynamic source
  paths SC1090/SC1091, client-side ssh expansion SC2029).
- AGENTS.md: new Git Policy rule mandating clean shellcheck for every shell
  script before commit.

Fixes applied (real bugs + quality): missing quote in netinfra/gather-configs.sh
(caused cascading parse errors), unquoted expansions, declare-and-assign masking,
egrep -> grep -E, $FUNCNAME array indexing, unused variable removal, cd || exit.
Intentional patterns (sourced config, sysfs/ps diagnostics, ssh heredocs that
expand local config) get justified targeted disables.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
2026-07-30 08:56:31 -05:00
mrcharles 8f1642bf96 feat(k8s): deploy 3-node k3s HA control plane over Tailscale
Bootstrap a regular (non-ITAR) k3s cluster on cnode1/2/3 with embedded
etcd. All cluster communication — node registration, API server, etcd
peering, flannel VXLAN — runs exclusively over Tailscale IPs. Zero LAN
addresses appear in node status or TLS certificates.

Scripts (k8s/):
- env.sh: shared config (Tailscale IPs, SSH opts, k3s version)
- wipe.sh: remove existing k3s from all cnodes
- install-cp.sh: full bootstrap (cnode1 --cluster-init, then cnode2/3 join)
- join-servers.sh: re-join cnode2/3 only (fixes broken join state)
- post-setup.sh: apply NoSchedule taints, fetch kubeconfig, verify
- verify.sh: 13-point health check (nodes, Tailscale IPs, taints, etcd,
  CoreDNS, API server, workload isolation)
- probe-nodes.sh: SSH + Tailscale reachability check

All 3 cnodes are tainted control-plane:NoSchedule so no user workloads
can schedule on the control plane. 13/13 health checks pass.

Docs updated: k8s README TL;DR reflects k3s (not Talos) as the deployed
choice, with Talos preserved for the future ITAR cluster.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
2026-07-28 12:21:33 -05:00