Files
PFVCluster/docs/k8s/ARCHITECTURE.md
T
mrcharles 174deea2a9 docs(k8s): add Talos architecture, distro decision, and bootstrap plan
Author the docs/k8s/ directory capturing the pfv-k8s control-plane design:

- README.md: TL;DR of all decisions (distro, runtime, cnode count, admin
  access, identity, tenancy, registry, storage)
- DISTRO-DECISION.md: Talos vs k3s analysis. Recommend Talos because the
  ITAR/classified requirement makes its immutable, API-only, measured-boot
  posture structurally easier to certify than SCAP-hardened Debian. k3s was
  only ever a plan (no cluster deployed), so cutover cost is ~zero.
- ARCHITECTURE.md: target arch with mermaid diagrams covering control
  plane, LAN-only network with Tailscale subnet-router admin, Cilium CNI,
  OIDC to Keycloak, per-tenant vcluster isolation (incl. ITAR tenant),
  Harbor pull-through cache on D3 SSD, bootstrap sequence, and DR.

Gardening: register docs/k8s/ in docmap.md, update STATUS.md with the new
k8s section and the three pending user decisions (cnode count, host spread,
Tailscale pattern).

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
2026-07-28 11:50:56 -05:00

24 KiB
Raw Blame History

pfv-k8s Target Architecture (Talos)

Companion to: DISTRO-DECISION.md (why Talos), ../proxmox/K8S.md (storage/host analysis from the Proxmox audit).

Last updated: 2026-07-28


Table of Contents

  1. High-Level Architecture
  2. Control Plane
  3. Network Topology
  4. CNI: Cilium
  5. Identity and Trust
  6. Tenant Isolation (vcluster)
  7. Storage Integration
  8. Local Image Registry
  9. Bootstrap Procedure
  10. Disaster Recovery
  11. Migration from Current State

1. High-Level Architecture

flowchart TB
    subgraph RESIDENCE["Residence — Proxmox LAN"]
        subgraph CP["Talos Control Plane (3 cnodes)"]
            C1[cnode1<br/>tsys9 · local-SSD]
            C2[cnode2<br/>tsys9 · local-SSD]
            C3[cnode3<br/>tsys1 · local-HDD]
        end
        subgraph WP["Talos Worker Plane"]
            W3[wnode-tsys3<br/>NVMe · 28GB]
            W5[wnode-tsys5<br/>NVMe · 32-64GB]
            W6[wnode-tsys6<br/>NFS-HDD · 64-96GB]
            W7[wnode-tsys7<br/>NFS-HDD · 96-128GB]
            W9[wnode-tsys9<br/>local-SSD · 4-8GB]
        end
        ETCD[(etcd<br/>raft, mTLS)]
        REG[(Harbor registry<br/>on D3 SSD · tsys5)]
        BASTION[tailscale-router VM<br/>subnet router]
    end

    subgraph TAILNET["Tailscale overlay"]
        OP[Operator devices]
    end

    subgraph CLOUDRON["Cloudron production — Reston VA"]
        KC[Keycloak OIDC IdP]
    end

    C1 ---|mTLS LAN| ETCD
    C2 ---|mTLS LAN| ETCD
    C3 ---|mTLS LAN| ETCD
    CP -->|pull images| REG
    WP -->|pull images| REG

    OP -->|Talos API :50000<br/>via subnet route| BASTION
    BASTION -.->|LAN| CP
    CP -->|OIDC| KC
    WP -->|OIDC| KC

    classDef talos fill:#1a1a2e,stroke:#e94560,color:#fff
    classDef infra fill:#0f3460,stroke:#e94560,color:#fff
    classDef external fill:#16213e,stroke:#533483,color:#fff
    class CP,WP,ETCD talos
    class REG,BASTION infra
    class OP,KC,EXTERNAL external

Design principles

  1. LAN-only cluster nodes. Zero internet egress from cnodes/wnodes. Strongest posture for ITAR/classified.
  2. Admin via Tailscale subnet router. Existing tailscale-router VM advertises the cluster LAN subnet. Operator reaches Talos API from anywhere.
  3. Local-first storage. Cnodes boot from local disk (no NFS dependency for etcd). Workers boot from local disk where available; NFS for bulk data only.
  4. Per-tenant vcluster. Workload isolation via virtual clusters on top of the Talos host cluster.
  5. OIDC everywhere. Talos API and Kubernetes API both trust Keycloak tokens. No long-lived static credentials for humans.

2. Control Plane

2.1 Recommendation: 3 cnodes (down from 5)

Option Quorum Failure tolerance etcd write cost Resource cost
3 cnodes (recommended) 2 of 3 Tolerates 1 failure Lower (faster commits) 3 × (2c/4GB/32GB) = 6c / 12GB
5 cnodes (current plan) 3 of 5 Tolerates 2 failures Higher 5 × (2c/4GB/32GB) = 10c / 20GB

For a solo-operated R&D cluster, 3 cnodes is the HA standard. The failure-tolerance jump from 1→2 rarely justifies the doubled etcd write quorum and the extra 4GB/2c per cnode. The 2 freed VM slots (and their host capacity) are better spent on tenant worker allocations.

Caveat: if your ITAR/classified accreditation counsel mandates 2-failure tolerance on the control plane, keep 5. Otherwise 3.

2.2 Cnode placement

Per ../proxmox/K8S.md §4.3, cnodes should use local-lvm boot disks so etcd has no NFS dependency. Concrete placement:

cnode Host Boot disk Type Why
cnode1 tsys9 local-lvm (PNY CS900 SSD) LOCAL-SSD Fastest available for etcd.
cnode2 tsys9 local-lvm (PNY CS900 SSD) LOCAL-SSD Same host, different disk OK (host failure is the failure domain, not disk).
cnode3 tsys1 local-lvm (HDD) LOCAL-HDD Host diversity. Slower than SSD but no NFS hop.

Quorum survival:

Failure cnodes lost Quorum OK?
tsys9 host dies cnode1 + cnode2 NO (1 of 3) — would need 4th cnode elsewhere, or accept this risk.
tsys1 host dies cnode3 YES (2 of 3)
Any storage server dies 0 YES (3 of 3) — local disks unaffected

Refinement: putting both SSD cnodes on tsys9 means tsys9 host failure loses quorum. Alternative: spread cnodes across 3 different hosts. See "open question" at end of this section.

2.3 Machine config strategy

Talos nodes are configured by machine configs (YAML). Two flavors:

  • controlplane.yaml — for cnodes. Enables etcd, scheduler, controller-manager, API server.
  • worker.yaml — for wnodes. Joins cluster, runs kubelet + containerd.

Strategy for this cluster:

  1. One shared talosconfig (client identity) — stored in 1Password and in the Proxmox Backup Server (PBS) encrypted backup target.
  2. Per-node machine config patches — small patches on top of the base controlplane.yaml / worker.yaml for node-specific settings:
    • Hostname
    • Network interface + IP (DHCP or static — recommend static for cnodes)
    • Schematic image digest (pinned Talos version)
    • System extensions (e.g., tailscale — only if running Pattern A instead of recommended Pattern C)
  3. All machine configs in Git under a future k8s/talos-configs/ directory. Secrets are templated in at apply-time from 1Password / sops.
flowchart LR
    BASE[base controlplane.yaml] --> PATCH1[patch: cnode1]
    BASE --> PATCH2[patch: cnode2]
    BASE --> PATCH3[patch: cnode3]
    BASEW[base worker.yaml] --> PATCHW[patch: per-wnode]
    PATCH1 --> APPLY1[talosctl apply]
    PATCH2 --> APPLY2[talosctl apply]
    PATCH3 --> APPLY3[talosctl apply]
    PATCHW --> APPLYW[talosctl apply]

2.4 Open question: cnode host spread

If you accept "tsys9 failure = quorum loss" as a tolerable risk (solo R&D cluster, tsys9 is brand-new hardware, single digit annual failure probability), the layout in §2.2 is fine.

If not, alternative spread across 3 hosts:

cnode Host Boot disk
cnode1 tsys9 local-lvm SSD
cnode2 tsys1 local-lvm HDD
cnode3 tsys3 local-lvm NVMe

tsys3's local-lvm is 349 GB Samsung PM961 NVMe (per ../proxmox/PROJECT.md §3.3) — currently unused, would make an excellent etcd disk.

This 3-host spread survives any single host failure with quorum intact. Recommended.


3. Network Topology

3.1 Zones

flowchart TB
    subgraph INTERNET["Internet"]
        FIBER[Gigabit symmetric fiber]
    end

    subgraph RESLAN["Residence LAN 192.168.x.x/24"]
        subgraph CLUSTERNET["Cluster nodes — LAN only, no egress"]
            CNODES[Cnodes 192.168.3.x]
            WNODES[Wnodes 192.168.3.x]
        end
        BASTION[tailscale-router<br/>192.168.3.x + 100.x.x.x]
        REG[Harbor registry<br/>192.168.3.x]
        STORAGE[NFS servers<br/>tsys4, tsys5]
    end

    subgraph TSNET["Tailscale 100.x.x.x/8"]
        OPS[Operator devices]
        KC[Keycloak<br/>via Cloudron prod]
    end

    FIBER --> BASTION
    BASTION <-. subnet route .-> CLUSTERNET
    OPS -->|TCP 50000 talos API| BASTION
    BASTION -->|LAN forward| CNODES
    CNODES -->|LAN mTLS| WNODES
    CNODES -->|OIDC HTTPS| KC
    WNODES -->|pull images| REG
    WNODES -->|bulk data IO| STORAGE
    CNODES -->|pull images| REG

3.2 Address plan (suggested)

Reserve a small contiguous block in the residence LAN for cluster nodes:

Role Range Count
Cnodes 192.168.3.31-33 3
Wnodes 192.168.3.41-49 up to 9 (1 per Proxmox host + spare)
Bastion existing tailscale-router 1
Registry 192.168.3.50 1 (Harbor)

Static IPs are strongly recommended for cnodes (etcd cluster membership is hostname-based; stable IPs make talosctl targeting simple). Workers can DHCP.

3.3 Firewall posture

Each cnode/wnode has:

  • Ingress from LAN: TCP 50000 (Talos API), TCP 6443 (Kubernetes API on cnodes only), plus CNI ports (varies by CNI — see §4).
  • Ingress from Tailscale: none (cluster nodes are not on Tailscale).
  • Egress: LAN-only. Block all RFC1918-external traffic at the perimeter firewall for these IPs. ITAR workloads must not be able to phone home.

The bastion runs Tailscale and forwards TCP 50000/6443 to cluster nodes via the subnet route.


4. CNI: Cilium

Recommendation: Cilium (eBPF-based CNI).

Property Why it matters here
NetworkPolicy (incl. L7) Per-tenant isolation rules in vclusters.
Node-to-node encryption WireGuard-based IPSec replacement. All inter-node pod traffic is encrypted on the wire. Important for ITAR tenants.
Hubble Observable flows — forensic record of which pod talked to which. Useful for compliance evidence.
No kube-proxy Cilium replaces kube-proxy with eBPF. Smaller attack surface on each node.
Talos integration First-class. Talos docs document the install path.

Cilium is deployed via Helm after cluster bootstrap. Node-to-node encryption enabled. Default-deny NetworkPolicy applied per namespace.


5. Identity and Trust

5.1 Trust flow

sequenceDiagram
    autonumber
    participant Human as Operator
    participant TAIL as Tailscale
    participant BAST as Bastion
    participant TALOS as Talos API :50000
    participant KC as Keycloak (Cloudron)
    participant K8S as Kubernetes API :6443

    Human->>TAIL: Authenticate (device + SSO)
    TAIL-->>Human: Tailnet IP
    Human->>BAST: Reach bastion via tailnet
    BAST->>TALOS: Forward to LAN node :50000
    Human->>KC: OIDC login (browser)
    KC-->>Human: Bearer token (short-lived)
    Human->>TALOS: talosctl (mTLS with client cert)
    Human->>K8S: kubectl --oidc (Keycloak token)
    K8S->>KC: Validate token (introspection)
    KC-->>K8S: Valid + claims
    K8S-->>Human: Authorized response

5.2 Two distinct identity layers

Layer Mechanism Audience
Talos API (node ops) Mutual TLS with client certificate generated from the Talos secrets bundle. Operators (automation + humans).
Kubernetes API (kubectl) OIDC bearer token from Keycloak. RBAC maps group claims → ClusterRole. Humans. Service accounts use projected tokens (no OIDC).

The Talos secrets bundle is the root of trust for the cluster. Lose it and you cannot operate the cluster; an attacker with it owns the cluster. Storage:

  1. Primary: 1Password (or equivalent) — operator-accessible.
  2. Backup: PBS encrypted backup target on tsys4 (existing infra).
  3. NOT in Git. Machine configs go in Git; secrets stay out.

5.3 Keycloak client configuration

On Cloudron-hosted Keycloak, register a client pfv-k8s-talos:

  • Authorization Code + PKCE flow (no implicit, no password).
  • Redirect URIs: http://localhost:8000 (kubectl oidc-login) + Sidero Omni/Rancher URLs if/when those are added.
  • Group claims: k8s-admin, k8s-readonly, k8s-tenant-itar, k8s-tenant-rackrental, etc. These map to Kubernetes RBAC ClusterRoleBinding.

6. Tenant Isolation (vcluster)

6.1 Why vcluster

vcluster runs a virtual Kubernetes control plane (API server, scheduler, controller-manager, etcd) inside a namespace of the host cluster. Tenant workloads run on the host's worker nodes but are isolated by:

  • Separate API server (tenant cannot see host cluster objects).
  • Separate RBAC and admission control.
  • Separate network policies (per-namespace).
  • Separate resource quotas.

This aligns with the user's per-tenant plan from ../proxmox/K8S.md §1.

6.2 Tenant registry

Tenant Compliance Workload example vcluster name
RackRental None (internal R&D) containerlab topology tests vc-rackrental
Suborbital non-ITAR EAR/ITAR-aware but unclassified Payload telemetry processing vc-suborbital-open
Suborbital ITAR ITAR-controlled Firmware build for USML items vc-suborbital-itar
Starting Line Productions Commercial Customer media pipeline vc-slp
flowchart TB
    subgraph HOST["Talos host cluster"]
        CP[Host control plane<br/>3 cnodes · etcd · Keycloak RBAC]
        subgraph NS["Host cluster namespaces"]
            NS_RR[ns: vc-rackrental]
            NS_SO[ns: vc-suborbital-open]
            NS_SI[ns: vc-suborbital-itar]
            NS_SLP[ns: vc-slp]
        end
    end

    subgraph VRR["vcluster: vc-rackrental"]
        API_RR[k8s API + etcd]
    end
    subgraph VSO["vcluster: vc-suborbital-open"]
        API_SO[k8s API + etcd]
    end
    subgraph VSI["vcluster: vc-suborbital-itar"]
        API_SI[k8s API + etcd]
    end
    subgraph VSLP["vcluster: vc-slp"]
        API_SLP[k8s API + etcd]
    end

    CP --> NS_RR & NS_SO & NS_SI & NS_SLP
    NS_RR --> API_RR
    NS_SO --> API_SO
    NS_SI --> API_SI
    NS_SLP --> API_SLP

    classDef itar fill:#3a0000,stroke:#ff0000,color:#fff
    class NS_SI,API_SI itar

6.3 ITAR enforcement at host layer

For the ITAR tenant (vc-suborbital-itar), enforce additional host-layer controls:

  • Node taint workload=itar:NoSchedule on worker nodes dedicated to ITAR workloads (subset of wnodes, marked in node labels).
  • NetworkPolicy default-deny egress for the vc-suborbital-itar namespace. Allow only explicit destinations (registry, NFS for ITAR data tier, Keycloak).
  • Storage isolation: ITAR PVCs target a dedicated NFS export (e.g., D3-itar on tsys5) that no other tenant can mount.
  • Audit: Hubble flows + auditd on the host worker nodes capture all access to ITAR data.

Rancher (or Sidero Omni) sits above this, presenting each tenant's vcluster as a separate "cluster" in its UI, with Keycloak SSO gating access per tenant group claim.


7. Storage Integration

Per ../proxmox/K8S.md §6. Three StorageClasses:

StorageClass Provisioner Backing Speed Use
local-fast local-path wnode local disk (NVMe/SSD/HDD depending on host) 100-3500 MB/s Container runtime, scratch, ephemeral
nfs-hdd nfs.csi.k8s.io tsys4 D2/D5, tsys5 S1-S4 80-120 MB/s Bulk data, weather/GIS datasets
nfs-ssd nfs.csi.k8s.io tsys5 D3, tsys5 T5-SSD 200-400 MB/s Latency-sensitive persistent data

7.1 CSI driver notes

  • NFS CSI: csi-driver-nfs (CNCF sandbox). Deploys via Helm. Each StorageClass points at a specific NFS server + base export path.
  • local-path: Rancher Local Path Provisioner. Single-binary, deploys with one manifest. Uses wnode's kubelet root dir.

7.2 ITAR data isolation

The ITAR tenant should target a dedicated NFS export, not shared nfs-hdd. Recommended:

  • Allocate S4 on tsys5 (currently 99% empty, 435 GB free) as nfs-itar StorageClass. Mountable only from vc-suborbital-itar namespace via RBAC + NetworkPolicy.

8. Local Image Registry

8.1 Recommendation: Harbor on D3 SSD

D3 SSD (tsys5, post-Friday SAS relocation) is 445 GB and 99% empty. Use it for a Harbor instance:

Property Value
Storage D3 SSD on tsys5 (NFS export, fast tier)
VM New VM pfv-registry on tsys5, local-nonprod boot, D3 data
Function (a) Pull-through cache for Docker Hub / Quay / gcr.io
(b) Host private images
(c) Cosign image signing verification
Exposure LAN-only. 192.168.3.50:443. Not exposed to internet.

8.2 Pull-through cache benefit

Cluster nodes have zero internet egress (per §3.3). Without a local cache, image pulls fail. With Harbor as a pull-through cache:

sequenceDiagram
    WNODE->>HARBOR: docker pull nginx:1.25
    alt cache hit
        HARBOR-->>WNODE: layer bytes (LAN-speed)
    else cache miss
        HARBOR->>DOCKERHUB: pull nginx:1.25 (egress)
        DOCKERHUB-->>HARBOR: layer bytes
        HARBOR-->>WNODE: layer bytes (cached for next time)
    end

Cluster nodes pull from Harbor over LAN (gigabit). Harbor is the only machine in the cluster with container-registry internet egress, and that egress can be locked to specific upstreams (docker.io, quay.io, gcr.io, ghcr.io).

8.3 Supply-chain integrity (future)

Harbor + Cosign lets you require that all images deployed to the ITAR tenant are signed by a trusted key. This is a strong ITAR/CISA-attestation control. Implementation deferred to a later session.


9. Bootstrap Procedure

9.1 One-time setup

sequenceDiagram
    autonumber
    participant OP as Operator
    participant GIT as Git repo
    participant ONEPW as 1Password
    participant PBS as PBS (tsys4)
    OP->>GIT: Clone PFVCluster repo
    OP->>ONEPW: Generate Talos secrets bundle (offline)
    ONEPW-->>OP: secrets.yaml
    OP->>PBS: Backup secrets.yaml (encrypted)
    OP->>GIT: Write machine configs (no secrets)

9.2 Provision first cnode (bootstrap)

sequenceDiagram
    autonumber
    participant OP as Operator
    participant PX as Proxmox host
    participant C1 as cnode1
    participant ETCD as etcd (new)

    OP->>PX: qm create VM (Talos QCOW2 disk, local-lvm)
    OP->>PX: qm start VMID
    C1->>C1: Boots Talos (no config yet, "maintenance mode")
    OP->>C1: talosctl apply --patch cnode1.yaml (with secrets)
    C1->>C1: Applies config, restarts services
    OP->>C1: talosctl bootstrap
    C1->>ETCD: Initialize single-node raft
    ETCD-->>C1: ready
    OP->>C1: talosctl kubeconfig (fetch admin kubeconfig)
    OP->>C1: talosctl etcd snapshot (initial backup → PBS)

9.3 Add second and third cnodes

sequenceDiagram
    autonumber
    participant OP as Operator
    participant PX as Proxmox host
    participant C2 as cnode2
    participant C3 as cnode3
    participant C1 as cnode1 (existing)

    OP->>PX: qm create + start cnode2 VM
    C2->>C2: Boots Talos maintenance mode
    OP->>C2: talosctl apply --patch cnode2.yaml
    C2->>C1: Join etcd cluster
    OP->>PX: qm create + start cnode3 VM
    C3->>C3: Boots Talos maintenance mode
    OP->>C3: talosctl apply --patch cnode3.yaml
    C3->>C1: Join etcd cluster
    Note over C1,C3: etcd now has 3/3 members → HA quorum

9.4 Post-bootstrap cluster configuration

Once 3 cnodes are up and joined:

  1. Install Cilium (CNI) via Helm. Enable node-to-node encryption.
  2. Install CSI drivers — nfs-csi + local-path provisioner.
  3. Create StorageClasseslocal-fast, nfs-hdd, nfs-ssd.
  4. Deploy Harbor on the pfv-registry VM, exposed at 192.168.3.50.
  5. Configure Kubernetes API OIDC — Keycloak client (§5.3).
  6. Apply default-deny NetworkPolicy in all namespaces.
  7. Install vcluster CLI + create 4 tenant vclusters (§6).
  8. First etcd snapshot + automated daily snapshot cron → PBS.

9.5 Add workers

Workers are simpler (no etcd):

sequenceDiagram
    OP->>PX: qm create + start wnode-X VM (Talos QCOW2)
    WNODE->>WNODE: Boots maintenance mode
    OP->>WNODE: talosctl apply --patch worker-X.yaml
    WNODE->>C1: Kubelet registers with API server
    C1-->>WNODE: Approved (auto via bootstrap token)
    Note over WNODE: Joins cluster, becomes Ready

10. Disaster Recovery

10.1 Backup strategy

Artifact Frequency Storage Tool
Talos secrets bundle Once (regen only on rotation) 1Password + PBS (encrypted) Manual
Machine configs Continuous (Git) Git remote + PBS Git
etcd snapshot Daily + before each change PBS (tsys4 SMR target, 4.3 TB free) talosctl etcd snapshot
vcluster etcd Daily per vcluster PBS kubectl exec ... etcdctl snapshot
Harbor metadata Daily PBS Harbor built-in backup

10.2 Restore scenarios

Lost 1 cnode (e.g., tsys9 disk failure):

  1. Provision new VM on tsys9 (or other host with local SSD).
  2. Apply cnode2 machine config patch.
  3. New cnode joins etcd, syncs state from survivors.
  4. Quorum was never lost (2 of 3 alive throughout).

Lost 2 cnodes simultaneously (quorum lost):

  1. Use surviving cnode's etcd snapshot.
  2. Provision 3 new cnode VMs.
  3. On first: talosctl bootstrap --recover-from=snapshot.db.
  4. Join other 2 cnodes.
  5. Workers reconnect automatically once API server is back.

Total cluster loss (all 3 cnodes):

  1. Restore from latest PBS etcd snapshot.
  2. Provision new cnode VMs.
  3. talosctl bootstrap --recover-from=snapshot.db.
  4. Re-join workers.
  5. Verify tenant vclusters restored.

10.3 Recovery time objectives

Scenario RTO RPO
Single cnode failure < 30 min 0 (no data loss)
Quorum loss (2 cnodes) < 2 hours ≤ 24 hours (last snapshot)
Total cluster loss < 4 hours ≤ 24 hours

11. Migration from Current State

11.1 Current state

  • 5 cnode VMs exist (Debian stock + Tailscale).
  • No k3s deployed yet. Cluster was never bootstrapped.
  • 6 wnode VMs exist (some stopped).
  • No workloads running in k8s.

11.2 Migration: clean cutover (not a migration)

Since there is no etcd data and no workloads to preserve, the path is a clean rebuild:

Phase Action Risk
0. Prep Generate Talos secrets. Store in 1Password + PBS. Write machine configs to Git. Low.
1. Bootstrap 3 new cnodes Build 3 NEW Talos cnode VMs (not the existing 5). Use local-lvm boot disks (tsys9 × 2, tsys3 × 1 per §2.4 recommended spread). Low. Existing Debian cnodes can keep running idle.
2. Configure cluster Install Cilium, CSI, StorageClasses, OIDC, Harbor. Low.
3. Add workers Re-image existing wnode VMs as Talos, or build new ones. Low. No workloads to drain.
4. Decommission old Debian cnodes Once cluster is stable, shut down + delete the 5 old Debian cnode VMs. Low.
5. Tenant vclusters Stand up per-tenant vclusters. Medium (policy tuning).

11.3 Open dependency: Friday hardware work

Phases 1-2 require:

  • tsys3 local-lvm available. Per ../proxmox/PROJECT.md §3.3, tsys3 has 349 GB free NVMe local-lvm. Currently unused. Ready.
  • tsys9 local-lvm available. 136 GB PNY CS900 SSD. Ready.
  • D3 SSD relocated to tsys5 SAS. Currently USB on tsys4. Per ../proxmox/TODO.md §2, scheduled for Friday. Harbor depends on D3 being available on tsys5.

Bootstrap of the cnodes does NOT depend on Friday hardware work. Only the Harbor registry does.


Appendix: Open questions for next session

  1. Confirm 3 vs 5 cnodes (§2.1). Recommendation: 3.
  2. Confirm cnode host spread (§2.4). Recommendation: 3-host spread (tsys9, tsys1, tsys3).
  3. Static IPs for cnodes (§3.2). Recommendation: yes, 192.168.3.31-33.
  4. Rancher vs Sidero Omni for cluster management UI. Both viable. Defer until cluster is up.
  5. Subnet router ACL approval on Tailscale admin console (§3). Needs approval of 192.168.3.0/24 route advertisement.
  6. ITAR worker node subset (§6.3). Which wnodes are tainted for ITAR? Recommendation: tsys6 + tsys7 (heaviest hosts, NFS-only boot) as general capacity; tsys3 + tsys5 (local fast storage) reserved for non-ITAR HPC.