feat(k8s): deploy 3-node k3s HA control plane over Tailscale

Bootstrap a regular (non-ITAR) k3s cluster on cnode1/2/3 with embedded
etcd. All cluster communication — node registration, API server, etcd
peering, flannel VXLAN — runs exclusively over Tailscale IPs. Zero LAN
addresses appear in node status or TLS certificates.

Scripts (k8s/):
- env.sh: shared config (Tailscale IPs, SSH opts, k3s version)
- wipe.sh: remove existing k3s from all cnodes
- install-cp.sh: full bootstrap (cnode1 --cluster-init, then cnode2/3 join)
- join-servers.sh: re-join cnode2/3 only (fixes broken join state)
- post-setup.sh: apply NoSchedule taints, fetch kubeconfig, verify
- verify.sh: 13-point health check (nodes, Tailscale IPs, taints, etcd,
  CoreDNS, API server, workload isolation)
- probe-nodes.sh: SSH + Tailscale reachability check

All 3 cnodes are tainted control-plane:NoSchedule so no user workloads
can schedule on the control plane. 13/13 health checks pass.

Docs updated: k8s README TL;DR reflects k3s (not Talos) as the deployed
choice, with Talos preserved for the future ITAR cluster.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2

k8s/post-setup.sh

@@ -0,0 +1,104 @@
+#!/usr/bin/bash
+#
+# k8s/post-setup.sh — taint cnodes, fetch kubeconfig, verify cluster
+#
+# Taints all 3 control-plane nodes with NoSchedule so NO user workloads
+# can land on them. Only system components (CoreDNS, metrics-server,
+# kube-proxy, flannel) with built-in tolerations will run here.
+#
+set -uo pipefail
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+# shellcheck source=./env.sh
+source "$SCRIPT_DIR/env.sh"
+
+echo "============================================"
+echo "  Post-setup: taints, kubeconfig, verify"
+echo "============================================"
+
+# -------------------------------------------------------
+# 1. Taint all cnodes NoSchedule (no user workloads on control plane)
+# -------------------------------------------------------
+echo ""
+echo "--- [1/3] Tainting control-plane nodes (NoSchedule) ---"
+for name in "${ALL_CNODE_NAMES[@]}"; do
+  # Set the control-plane role label explicitly
+  cn "$CNODE1_IP" "kubectl label node $name node-role.kubernetes.io/control-plane= --overwrite" 2>/dev/null || true
+  # Apply the taint (idempotent with --overwrite)
+  cn "$CNODE1_IP" "kubectl taint node $name node-role.kubernetes.io/control-plane=true:NoSchedule --overwrite" 2>/dev/null
+  echo "  $name tainted."
+done
+
+# -------------------------------------------------------
+# 2. Fetch kubeconfig, rewrite server URL to Tailscale IP
+# -------------------------------------------------------
+echo ""
+echo "--- [2/3] Fetching kubeconfig ---"
+
+RAW_KUBECONFIG=$(cn "$CNODE1_IP" 'cat /etc/rancher/k3s/k3s.yaml')
+
+# Rewrite 127.0.0.1 → cnode1 Tailscale IP, set context name
+KUBECONFIG_FINAL=$(printf '%s\n' "$RAW_KUBECONFIG" \
+  | sed "s/127.0.0.1/$CNODE1_IP/g" \
+  | sed 's/default/pfv-k8s/g')
+
+KUBECONFIG_DIR="$HOME/.kube"
+KUBECONFIG_FILE="$KUBECONFIG_DIR/config.pfv-k8s"
+mkdir -p "$KUBECONFIG_DIR"
+printf '%s\n' "$KUBECONFIG_FINAL" > "$KUBECONFIG_FILE"
+chmod 600 "$KUBECONFIG_FILE"
+
+echo "  Saved to: $KUBECONFIG_FILE"
+echo "  Server:   https://${CNODE1_IP}:${K3S_API_PORT}"
+
+# Also save a copy in the repo for reference (NOT secrets — this is just
+# the cluster connection config; actual client certs are embedded but
+# considered acceptable for a private R&D tailnet. If ITAR workloads are
+# added later, move to OIDC and remove this file.)
+K8S_DIR="$SCRIPT_DIR"
+printf '%s\n' "$KUBECONFIG_FINAL" > "$K8S_DIR/kubeconfig.yaml"
+chmod 600 "$K8S_DIR/kubeconfig.yaml"
+echo "  Copy saved: $K8S_DIR/kubeconfig.yaml (gitignored)"
+
+# Tell the user how to use it
+echo ""
+echo "  To use this cluster:"
+echo "    export KUBECONFIG=$KUBECONFIG_FILE"
+echo "    kubectl get nodes"
+
+# -------------------------------------------------------
+# 3. Verify cluster health
+# -------------------------------------------------------
+echo ""
+echo "--- [3/3] Verifying cluster health ---"
+
+export KUBECONFIG="$KUBECONFIG_FILE"
+
+echo ""
+echo "=== Nodes ==="
+kubectl get nodes -o wide 2>&1
+
+echo ""
+echo "=== Node taints ==="
+kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints 2>&1
+
+echo ""
+echo "=== Node IPs (should be 100.x Tailscale) ==="
+kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.addresses[?(@.type=="InternalIP")].address}{"\n"}{end}' 2>&1
+
+echo ""
+echo "=== etcd members (should be 3) ==="
+cn "$CNODE1_IP" 'etcdctl --write-out=table endpoint status' 2>&1 \
+  || cn "$CNODE1_IP" 'k3s etcdctl endpoint status --write-out=table' 2>&1 \
+  || echo "(could not query etcd directly)"
+
+echo ""
+echo "=== System pods ==="
+kubectl get pods -A 2>&1
+
+echo ""
+echo "============================================"
+echo "  Cluster is ready."
+echo ""
+echo "  KUBECONFIG: $KUBECONFIG_FILE"
+echo "  Next: bash $SCRIPT_DIR/verify.sh"
+echo "============================================"