# Kubernetes Architecture & Build Plan

> **Status:** Draft for review. Companion to [`../proxmox/K8S.md`](../proxmox/K8S.md)
> (which captured the storage/host analysis from the Proxmox audit).
> This directory takes the next step: **which distro, how to build it,
> how to operate it.**

**Last updated:** 2026-07-28

## Documents in this directory

| Document | Purpose |
|----------|---------|
| [`DISTRO-DECISION.md`](DISTRO-DECISION.md) | Talos vs k3s analysis. Recommendation: **Talos**, with rationale grounded in the ITAR/classified requirement. |
| [`ARCHITECTURE.md`](ARCHITECTURE.md) | Target architecture: control plane, network, identity, storage, tenant isolation. Mermaid diagrams included. |

## TL;DR

| Decision | Recommendation | Why |
|----------|----------------|-----|
| **Distro** | **Talos Linux** (not k3s) | Immutable, API-only, measured-boot-capable. Intrinsically close to a STIG/CIS posture, where k3s-on-Debian requires constant re-certification. The k3s plan was reasonable before the ITAR/classified requirement was on the table. |
| **Runtime** | **containerd** (Talos default) | Only runtime Talos supports; also the secure default. Non-decision. |
| **Cnode count** | **3** (down from 5) | Standard HA. Tolerates 1 failure. Frees 2 VMs for tenant worker capacity. Re-evaluate if a regulator mandates 5. |
| **Admin access** | **Tailscale subnet router** on existing `tailscale-router` VM | Cluster nodes stay LAN-only (no internet egress = stronger ITAR posture). Operator reaches Talos API from anywhere via Tailscale. |
| **Cluster network** | LAN mTLS only | All cnode/wnode traffic on Proxmox LAN. No Tailscale on cluster nodes themselves. |
| **Identity** | **OIDC to Keycloak** on Cloudron (production) | Already planned. Talos API + Kubernetes API both trust Keycloak tokens. |
| **Multi-tenancy** | **vcluster** (per tenant) inside Talos host cluster | Strong workload isolation. One vcluster per tenant: ITAR, non-ITAR, SLP, RackRental. |
| **Local registry** | **Harbor on D3 SSD** (tsys5, 445 GB free) | Avoids needless round-trips to Docker Hub / Cloudron. Adds image-signing (Cosign) capability for supply-chain integrity. |
| **Storage classes** | `local-fast`, `nfs-hdd`, `nfs-ssd` | Per [`../proxmox/K8S.md`](../proxmox/K8S.md) §6. Unchanged. |

## What this directory does NOT cover (deferred)

- ETL tooling choice (GDAL/PostGIS/xarray/Dask) — affects RWX vs RWO design.
- HPC job scheduler (Jobs / Argo Workflows / Volcano) — affects taint/label strategy.
- vcluster per-tenant policy templates.
- Solar-aware scale-out (PowerEdge 19xx/2950 hosts) — capacity planning only.

These are tracked as future session work in [`../../STATUS.md`](../../STATUS.md).

## Open question for the user

1. **Cnode count: confirm 3 vs 5.** Recommendation is 3 (rationale in
   [`ARCHITECTURE.md`](ARCHITECTURE.md) §2). If your ITAR counsel requires
   2-failure tolerance on the control plane, keep 5.