mrcharles
428e9ecb1d
Docs moved from docs/k8s -> k8s/docs and docs/proxmox -> proxmox/docs to sit
beside their code. Updated all cross-references (docmap, STATUS, README, AGENTS,
k8s README). Fixed SCRIPT_DIR path resolution in 3 perf scripts broken by the
reorg (deploy-check/and-fix/finish-host pointed at root/scripts instead of
perf/scripts). Registered the new top-level proxmox/ dir in all listings.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Kubernetes Architecture & Build Plan
Status: Draft for review. Companion to
../proxmox/K8S.md(which captured the storage/host analysis from the Proxmox audit). This directory takes the next step: which distro, how to build it, how to operate it.
Last updated: 2026-07-28
Documents in this directory
| Document | Purpose |
|---|---|
DISTRO-DECISION.md |
Talos vs k3s analysis. Recommendation: Talos, with rationale grounded in the ITAR/classified requirement. |
ARCHITECTURE.md |
Target architecture: control plane, network, identity, storage, tenant isolation. Mermaid diagrams included. |
TL;DR
| Decision | Recommendation | Why |
|---|---|---|
| Distro | k3s (deployed) / Talos (for future ITAR) | k3s chosen for the regular R&D cluster now live on cnode1/2/3. Talos is the recommendation for when the ITAR/classified cluster comes online. |
| Runtime | containerd | Talos/k3s default. |
| Cnode count | 3 (deployed) | Standard HA. Tolerates 1 failure. |
| Admin access | Tailscale (all nodes joined) | Currently all cnodes are on Tailscale directly. For ITAR cluster, move to subnet-router pattern. |
| Cluster network | Tailscale-only IPs | All node-ip, advertise-address, TLS-SANs are 100.x Tailscale IPs. Zero LAN IPs in cluster state. |
| Identity | OIDC to Keycloak on Cloudron (production) | Future work. |
| Multi-tenancy | vcluster (per tenant) | Future work. |
| Local registry | Harbor on D3 SSD (tsys5, 445 GB free) | Future work. |
| Storage classes | local-fast, nfs-hdd, nfs-ssd |
Per ../proxmox/K8S.md §6. Future work. |
What this directory does NOT cover (deferred)
- ETL tooling choice (GDAL/PostGIS/xarray/Dask) — affects RWX vs RWO design.
- HPC job scheduler (Jobs / Argo Workflows / Volcano) — affects taint/label strategy.
- vcluster per-tenant policy templates.
- Solar-aware scale-out (PowerEdge 19xx/2950 hosts) — capacity planning only.
These are tracked as future session work in ../../STATUS.md.
Open question for the user
- Cnode count: confirm 3 vs 5. Recommendation is 3 (rationale in
ARCHITECTURE.md§2). If your ITAR counsel requires 2-failure tolerance on the control plane, keep 5.