Add a shared Docker network (dnsnet, 10.53.0.0/24) connecting Pi-hole and
Technitium containers so Pi-hole can conditionally forward knel.net and
Tailscale-reverse queries to the local authoritative Technitium instance
(10.53.0.53) instead of netboot's upstream 192.168.3.16. Also adds
end-to-end documentation for both the reference node (pfv-netboot) and
the replicated nodes (pfv-netinfra-01/02).
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Deployed check.sh to all 7 hosts at 21:50 CDT. Captures the live state
after the user's PDM migrations:
Cnode movements since last audit:
- cnode1: tsys1 -> tsys9
- cnode2: tsys6 -> tsys7
- cnode5: tsys6 -> tsys7, storage D5(tsys4) -> S2(tsys5)
Wnode changes:
- wnode-tsys1 (102): new VM on S2, stopped
- wnode-tsys3: RAM bumped 20 -> 28 GB
- wnode-tsys6: now running (was stopped)
- wnode-tsys9: storage moved S3 -> S2
Storage distribution improved from 90/10 to 73/27 (tsys4/tsys5).
Still need 2 more cnode moves for etcd quorum survival.
Updated executive summary, k8s distribution tables, storage
utilization, and open items with the fresh data. Captured future k8s
requirements: vcluster + Rancher, OIDC to Keycloak, workload isolation
(RackRental/Suborbital ITAR/non-ITAR/SLP), and solar-aware scale-out
with PowerEdge 19xx/2950 systems.
Added tsys9 to deploy-check.sh host list.
Storage philosophy (user directive):
- NVMe/SSD: k8s worker scratch + ultix-streaming (dev workstation
running "cluster of 1" pre-prod jobs before full k8s deployment)
- Spinning rust: all infrastructure VMs (UCS, netinfra, LibreNMS, SIEM)
Clarified that hosts are standalone but managed via Proxmox Datacenter
Manager (PDM), which supports VM migration between nodes through the
UI -- eliminating the need for manual disk copies in the migration plan.
Updated all migration steps to reference PDM storage migrate instead
of manual cp commands.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
UCS (LDAP/AD) does not need SSD. Corrected placement:
- ucs-01 stays on D2 (tsys4 HDD) -- no change needed
- ucs-02 moves from D5 (tsys4) to S2 (tsys5 HDD) for cross-server
redundancy only, not for speed
D3 SSD (moving to tsys5 Friday) is now freed for workloads that
actually benefit: sectestbed k8s nodes, CI/CD artifact cache, or a
container image registry mirror.
Added storage tier characterization per host (tsys4 = bulk/spinning
only, tsys5 = hybrid fast-tier hub, tsys2/3 = SSD/NVMe only). Added
UCS migration TODO item for today.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Verified via lsblk TRAN column and dmesg that tsys6/7 local-lvm is
backed by WD My Passport portable HDDs on USB 2.0 (~30 MB/s), not
internal SATA/SAS as previously assumed. Both R610 and R620 only have
EHCI controllers -- no USB 3.0/xHCI. tsys7 has 5 empty SATA ports but
no drives installed; tsys6's SAS controller is disabled in BIOS.
Corrected all wnode recommendations: tsys6/7 stay on NFS (faster than
their USB 2.0 local-lvm). Only tsys3/5/9/2 have local storage suitable
for VM boot disks.
Also discovered tsys3 boots from Samsung PM961 NVMe 512GB (previously
documented as HDD). Its 349 GB local-lvm is the fastest existing local
storage in the fleet.
Updated speed tier table, wnode placement tables, storage distribution
matrices, migration plan, and caching strategy across PROJECT.md and
K8S.md.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
tsys2 validated via PowerShell on Windows: 2 SSDs (Samsung 960 PRO
NVMe 512GB + Samsung 850 EVO SATA 1TB) -- the best local storage in
the fleet. wnode-tsys2 needs no NFS; boot on NVMe, bulk data on SATA
SSD. Both NICs are USB dongles (ASIX + Realtek) -- unavoidable on
this laptop form factor, no PCIe NIC expansion possible.
Updated storage tiers, wnode placement, and k8s labeling to reflect
that tsys2 is the fastest-storage worker node. Marked TODO.md items
as resolved.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Adds the PowerShell and cmd.exe commands the user needs to run on the
Windows host (pfv-tsys2) to gather disk, network, RAM, GPU, and PCIe
details before rebuilding it as Proxmox. Also documents the Friday
maintenance window steps (tsys4 NIC+RAM, tsys5 cable+NVMe) and open
questions for the next session.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
Add __pycache__/*.pyc to .gitignore and remove the tracked bytecode
file. Add version control instructions to AGENTS.md for future sessions.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
mrcharles