org-buildout/transition-map.md

# COO Handoff & CTO Transition — Master Map

**Status:** DRAFT for review (working session artifact, not yet synthesized to Discourse/Redmine)
**Date:** 2026-08-13
**Deadline:** 2026-09-30 ("Potential to Kinetic Ready" + COO handoff, 48 days / "45 days")
**Author session:** orientation/synthesis pass over Q3 prompts, all project trees, Gitea, Discourse, Redmine

---

## 0. Thesis (the one thing to internalize)

**Revised 2026-08-13:** Timeline split confirmed with user.

| Phase | Window | Focus | Who |
|---|---|---|---|
| **Q3 remainder** | Aug 13 → Sep 30 | TechOps finish (P1-P9) + **stand up AI agent identities** | Charles + AI agents |
| **Q4** | Oct 1 → Dec 31 | Business ops transition + CTO come-up (80% CTO / 20% COO) | Charles + AJ + Patti + Courtney + AI agents |
| **Jan 1 2027** | — | Full COO handoff complete. Charles = 100% CTO. | AJ + COO agent run business ops |

The infrastructure (PFVCluster, KNELIAC, monitoring, k8s) is ~80% there and has a phase plan (P1-P9) with a 9/30 deadline. The "running datacenter" is commodity and will be fully delegated to AI. The **first action is standing up AI agent identities** (Cloudron + Bitwarden + system access) so agents can operate with proper attribution, RBAC, and audit trails. See `agent-identity-bootstrap.md`.

**Business ops transition is deferred to Q4.** The COO Discourse category, Redmine project 53, bizopprodplan handbook refresh, ITSM tool selection, and COO Tier-1 business apps all move to Q4. AJ gets regular briefings through Q3 and takes over business ops in Q4 with Patti and Courtney as additional resources.

---

## 1. Current-State Map

### 1.1 The TSYS Group org structure (as designed in Q3/prompt.md)

```
TSYS Group (Board of Directors)
├── CTO  — Charles (R&D, architecture, tier-4 SME)  [you, transitioning IN fulltime Oct 1]
└── COO  — AJ Lebsch (operations)  [transitioning IN, oversees AI agents]
     ├── SVP KNEL  (Known Element Enterprises — owns ALL IT/business systems)
     │    ├── VP TechOps        ← the ONLY function with real content today
     │    ├── VP SecOps         ← does not exist as a category yet
     │    └── VP TechCompliance ← Discourse cat 75 exists, 0 topics
     └── SVP TCTC  (The Campus Trading Company)
          ├── VP Finance / VP Accounting / VP Investing
          ├── VP Treasury / VP Trading
          └── (RedWFO = RWSCP Family Office, mission-critical, can preempt)
```

**Supporting entities** (each with its own Gitea org + Redmine project + Discourse category, mostly stubs):
Suborbital Systems, HFNOC, HFNFC, RackRental, Starting Line Productions, Rogue Technologies, RedWFO, RWSCP, MeetMorse/MorsePod (FLO entry to CommonsNet), EzEDA, EzPodStack, AFABN, Ap4Ap, MerchantsOfHope, ThePeerNet, sol-calc, TeamRental, SideDoorGroup, YourDreamNameHere.

### 1.2 Systems of Record — actual state vs intended

| System | Role | Actual state |
|---|---|---|
| **Redmine** (projects.knownelement.com) | SoR for ALL work | **55 projects** mirror the org chart. **Only project 55 (TechnicalOperations) is active: 136 tickets, 86 open.** Project 62 (Business Services), 53 (COO), 77 (TSYS Group parent), and all entity projects are **empty/stubs.** |
| **Discourse** (community.turnsys.com) | SoR for ALL docs | **55 categories.** Only **VP TechOps (cat 74, 13 topics)** and **Progress Reports (cat 61, 81 topics)** have real content. COO (6), VP Compliance (75), Board (76), KNEL-Bizops (72) = **0 topics.** Every business-entity category = 0-1 stub topics. |
| **Gitea** (git.knownelement.com) | SoR for executable code | **27 orgs, ~213 repos** (87 are ExternalVendorCode mirrors). Source repos: KNEL (32), reachableceo (23), Suborbital-Systems-Public (18), RWSCP (11), + ~20 entity orgs with 1-3 repos each (mostly `-bizopprodplan` mdBook stubs). |
| **Cloudron** (Reston VPS) | PaaS for ~57 support-stack apps | **10/57 packaged (~17%).** See §1.4. |
| **K8s** (PFVCluster bare metal) | Scalable compute | k3s HA (3 cnode + 6 workers) but **cnodes were wiped and shut down — needs rebuild.** Zero apps deployed. |

### 1.3 The 12 local projects (maturity + COO relevance)

| Project | What | Maturity | Governance | COO relevance |
|---|---|---|---|---|
| **PFVCluster** | Proxmox fleet + OAM + k8s bootstrap | High (most mature infra) | Excellent (full rules engine) | **High** — the compute foundation |
| **football** (KNEL-Football) | Hardened Debian live ISO for tier-0 access | ★★★★★ (788 tests, ISO built, audited) | Exceptional | High — secure access terminal |
| **KNELIAC** | Ansible fleet config-mgmt (replaces bash) | Active (9 roles, AWX wired) | **Weak** (80-line AGENTS.md) | High — fleet baseline |
| **KNEL-AIMiddleware** | MCP/LSP servers for AI agents (42 svcs) | 79% (33/42 prod-ready) | Moderate | **Critical** — the agent tooling backbone |
| **EngStack** | Hardware R&D engineering workstation (45 tools) | Active (8 tools built) | Good | Med (CTO/R&D domain) |
| **TSYS-Cloudron** | 57-app Cloudron packaging | 17% (10/57) | Strong | **High** — COO's business apps |
| **WorkstationStack** | Local dev support stack + SelfStack | Active (7 apps in prod) | Good | Med |
| **hermes-rceo-streaming** | Hermes AI agent deployment | Deployed | Minimal | **High** — agent runtime (security note: full host access) |
| **netbird** | Zero-trust VPN access | Early/stub | Minimal | Med (security-critical, underbuilt) |
| **meta (TSYSGroupAIOS)** | Canonical agent-framework template | **Production-ready, NOT pushed** | Canonical | **Critical** — must propagate to all projects |
| dotfiles / EngineeringWorkstation / TSYS-LocalWorkstation | Personal/scratch | — | — | Low |

### 1.4 Cloudron app fleet — COO-critical gap

**10 packaged:** Webhook, APISIX, Healthchecks, Review Board, WireViz Web, Puter, Corteza, draw.io, Windmill, InvenTree.
**COO Tier-1 MISSING (not started):** Grist (ops spreadsheets), PayrollEngine, KillBill (billing), Rundeck (runbook automation), Comply (compliance tracking).
**Cloudron-blocked (need K8s):** Sentry, SigNoz, DataHub, NetBox, Fleet (all need Redis/Kafka/ClickHouse).

### 1.5 Governance template (TSYSGroupAIOS / meta) — adoption status

`~/daytoday/meta` is the canonical framework: `BASELINE-PROMPT.md` (14 principles) + 10-section `AGENTS.md` template + `scripts/check-rules.sh` (10 checks) + git hooks + `hooks/ticket-gate.sh`. **Self-applying (17 PASS/0 FAIL), ready to push** to `TSYSGroupCorporate/TSYSGroupAIOS` (remote configured, not yet pushed).

| Project | Gap to framework |
|---|---|
| PFVCluster | 🟢 Minor — add BASELINE-PROMPT.md, align scripts |
| football | 🟡 Moderate — replace custom hooks |
| KNEL-AIMiddleware | 🟡 Moderate — drop JOURNAL.md, add pre-commit |
| **KNELIAC** | 🔴 Critical — needs entire enforcement layer |
| **EngStack** | 🔴 Critical — needs entire `scripts/` + governance |

---

## 2. The Central Gap (COO handoff) — what "done" requires on Oct 1

For AJ + the COO AI agent to actually run operations, these must be true. Today **none of them are.**

### 2.1 Org / role
- [ ] COO AI agent stood up with Cloudron account, logged into all role-appropriate systems (Redmine, Discourse, Gitea, Cloudron, monitoring). **Repo exists: `KNEL/TSG-COOAndBoard-AIAgents-Public` — not yet built out.**
- [ ] SVP KNEL agent + SVP TCTC agent (synthesize policy/strategy → COO).
- [ ] VP TechOps / VP SecOps / VP TechCompliance agents scoped to their Redmine projects.
- [ ] AJ's access formally documented (he "has full access" per prompt — needs an access matrix in Discourse).

### 2.2 Documentation (Discourse categories to populate)
- [ ] **COO (cat 6):** operations manual, decision authority, escalation paths, daily/weekly/monthly cadence.
- [ ] **VP SecOps:** does not exist — create (needs admin key; current API user is trust-4, **cannot create categories** — blocker).
- [ ] **VP Compliance (75):** 0 topics — CMMC L3, STIG, ITAR program docs.
- [ ] **KNEL-Bizops (72):** 0 topics — the business-services knowledge base.
- [ ] **Board (76):** 0 topics.
- [ ] Each business entity category: operations content (currently 0-1 stubs).

### 2.3 Work tracking (Redmine)
- [ ] **Project 53 (COO) / 62 (Business Services):** define versions/milestones + seed tickets. Currently empty.
- [ ] COO workstream tickets mirroring the operational layer (not just TechOps P1-P9).
- [ ] AI-staff scoping: each Hermes agent scoped to its Redmine project only.

### 2.4 Systems / tools the COO depends on
- [ ] **ITSM/workflow engine** (prompt.md line 101: "We really need an ITSM tool — Discourse? Windmill? Nextcloud?"). **Unresolved decision.** Windmill is packaged (✅) — strongest candidate.
- [ ] COO Tier-1 apps deployed: Grist, PayrollEngine, KillBill, Rundeck, Comply (all not-started).
- [ ] Credential migration to Vault (P5, critical path) — unblocks agent secret access.

### 2.5 Culture / process (the "correct" part)
- [ ] TSYSGroupAIOS framework **pushed and adopted** into all 5 gap projects.
- [ ] Full SDLC enforced everywhere (red/green TDD, linters, CI/CD lockstep local+hosted).
- [ ] `sectestbed-` / `preprod-` VM workflow operational for IaC testing.
- [ ] "Gardening" loop running to prevent doc sprawl.

---

## 3. CTO Transition — what changes for Charles

| Stop doing (COO/founder work) | Start/continue doing (CTO work) |
|---|---|
| Day-to-day ops decisions | R&D architecture (EngStack, Suborbital, k8s platform) |
| Physical infra firefighting (delegate to VP TechOps agent) | Tier-4 escalation only |
| Direct fleet config (→ KNELIAC/AWX) | SDLC/governance ownership (the framework) |
| Ticket-level execution on P1-P9 | Mentor/oversee AI agents; review their work |
| Building business-ops docs | Document TSYS Group component architecture (Oct onboarding task per prompt.md) |

**The CTO transition is gated on the COO handoff:** you cannot stop doing COO work until AJ + the COO agent can absorb it.

---

## 4. Roadmap — Revised (Q3/Q4 split)

### Q3: Aug 13 → Sep 30 — TechOps + Agent Identities

**Workstream A — Finish Infrastructure (P1-P9, Redmine project 55, 86 open tickets)**
*Keep executing the existing phase plan.* Critical path: **P5 Vault → P6 IaC → P9 Compliance → P7 k8s → P8 Apps.**
- Aug 13-17: Friday onsite physical batch. Finish P1/P2/P3.
- Aug 18-31: P5 Vault (CRITICAL), P4 monitoring UAT, P6 IaC basics, P9 STIG/CMMC seed, P7 k8s cnode rebuild.
- Sep 1-30: P8 Cloudron app fleet (TechOps-relevant), P7 k8s apps (cluster only), compliance hardening.

**Workstream B — Stand Up AI Agent Identities (NEW, first priority)**
*Identity-first. See `agent-identity-bootstrap.md` for full plan.*
- Week 1: User provides prerequisites (Linux `coo` account, Bitwarden account, Cloudron invites). Agent enrolls 3 Q3 identities (vp-techops, vp-secops, vp-techcompliance) via Playwright.
- Week 2: Deploy per-agent SSH keys, set up `coo` Linux environment, smoke test (agent creates ticket, edits Discourse, opens PR from own identity).
- Week 3+: Agents begin operating from own identities. Enroll Q4 identities (Cloudron only, no system access yet).

**Workstream C — Governance & Culture (cross-cutting)**
- Week 1-2: Push TSYSGroupAIOS template; adopt into KNELIAC (critical) + EngStack (critical).
- Week 2-4: Adopt into KNEL-AIMiddleware (drop JOURNAL.md), football, PFVCluster (minor).
- Ongoing: `sectestbed-`/`preprod-` VM workflow, CI/CD lockstep, gardening loop.

### Q4: Oct 1 → Dec 31 — Business Ops Transition + CTO Come-Up

*Charles at 80% CTO / 20% COO. Working alongside AJ (business ops lead), Patti, Courtney.*

- **October (Charles's stated focus):** Figure out the K8S workload. What apps run on K8S vs Cloudron. Create the K8S repo + Redmine project.
- **October–November:** Build the COO business-ops layer: Discourse cat 6 content, Redmine project 53 tickets, ITSM tool selection (Windmill), COO Tier-1 apps (Grist, PayrollEngine, KillBill, Rundeck, Comply), bizopprodplan handbook refresh.
- **November–December:** Activate Q4 AI agents (COO, SVP KNEL, SVP TCTC, financial VPs). AJ UAT on COO agent + business systems. Dry-run operational week.
- **Jan 1 2027:** Full COO handoff. Charles = 100% CTO.

---

## 5. Open Decisions — Updated

### Resolved by user (2026-08-13)
- ~~Business ops timeline~~ → **Q4 (Oct–Dec), full handoff Jan 1 2027.** Q3 = TechOps only.
- ~~Agent identity approach~~ → **Identity-first.** Stand up Cloudron + Bitwarden accounts as the very first action.
- ~~COO/CTO split in Q4~~ → **80% CTO / 20% COO.** AJ leads business ops with Patti + Courtney.

### Still open

1. **Cloudron SSO** — Are Gitea/Discourse/Redmine Cloudron-managed (auto-SSO)? Or standalone? **This determines provisioning complexity.** *(blocking agent bootstrap)*
2. **Discourse admin key** — API user is trust-4, **cannot create categories** (VP SecOps etc.). Provide admin key or create via web UI. *(blocking VP SecOps setup)*
3. **Linux account model** — Single `coo` account (recommended Q3) vs per-agent accounts (stronger audit). *(see bootstrap spec D1)*
4. **Agent runtime** — Crush per-agent config dirs (recommended Q3) vs Hermes vs OpenWebUI. *(see bootstrap spec D3)*
5. **K8S scope in P7** — Is P7 (due Aug 31) just cluster bootstrap, with app deployment deferred to October (your stated K8S focus)? *Rec: yes — P7 = cluster ready, apps = October.*
6. **ITSM tool** — Windmill (packaged) vs Discourse-only vs Nextcloud. *(deferred to Q4 but worth deciding early)*
7. **bizopprodplan repos** — Legacy mdBook stubs (KNEL one still says "CIO Documentation"). Refresh in place during Q4 or treat as superseded by Discourse?

---

## 5.5 SecOps & Compliance Context (from user's VP SecOps notes)

This is the security architecture that the TechOps agents operate within. Drives P5 (PKI/Vault) and P9 (Security/Compliance).

### Compliance targets
- **CMMC Level 3** is the goal (not L1 or L2)
- **Full STIG compliance** — highest level (mission critical classified)
- **CUI minimum everywhere** — all systems, no exceptions
- **ITAR** — governs all technical operations
- **Multi-tenant** — isolation between business entities (RackRental franchisees, Suborbital, TCTC, etc.)
- **Eventually productized** as a Your Dream Name Here (YDN) offering — the compliance stack itself becomes a product

### Zero trust access model
- **NetBird** (primary zero-trust mesh) + **Tailscale** (existing, nested solution)
- **Apple account** referenced (likely for iPad-based access)
- All access through zero-trust — no flat network trust
- **Remote access SCIF** capability needed
- **Keycloak** deployed — initial setup done for NetBird integration
- Cloudron IdP has "simple groups, lacks granular permission levels at least via GUI" — Keycloak fills the RBAC gap

### FOCI concern (Foreign Ownership Control Influence)
- **Netcup** (Cloudron VPS host) is a **German company**, even though hosting in Reston VA
- Question: can a VPS attest back to PFV (the on-prem cluster)?
- This affects what data/workloads can live on Cloudron vs must stay on-prem

### CA / PKI (drives P5)
- **Nitrokey HSM** as the root CA hardware
- SSH certificates (not just keys) for day-to-day operations via Ansible
- Custom **Ubuntu 24.04 ISO** needed as the Cloudron base image

### SSH key migration (directly impacts agent identity bootstrap)

| Current state | Target state |
|---|---|
| **Ultix-highside** (Win11 Surface, no local admin, Zoc terminal) — on-disk OpenSSH key present on every system except recent deploys | Single Bitwarden SSH key + BW agent (not yet working on Windows) |
| **This VM's key** — on a subset of machines, being expanded | Replaced by Bitwarden key/agent |
| **iPad secure enclave key** — used via Blink, public key on all accessible systems | **Keep** — one of two authorized broad-access keys |
| | **iPad enclave key + Bitwarden key only.** No key material on disk. |

**Agent identity impact:** Agent SSH keys (Phase 3 of bootstrap) must use the **SSH certificate** model, not raw key deployment. The CA (Nitrokey HSM) signs agent certificates. This is stronger than key-based auth and aligns with the "day-to-day operations via Ansible and SSH certificates" target.

### Bitwarden structure
- **Three BW accounts** currently exist (RCEO owns all creds/orgs/collections)
- **Envwarden** in use
- A **fourth BW account** (COO/AI agents) will be created per the bootstrap plan
- AJ has **broad Bitwarden access** (business continuity)

### Business continuity
- **Patti** — iPad
- **Remy** — iPhone
- **Albert (AJ)** — broad Bitwarden access

### Identity/IAM gaps to resolve
1. **Cloudron IdP lacks granular RBAC** — Keycloak must fill this gap for agent scoping
2. **SSH certificate infrastructure** — Nitrokey HSM → SSH CA → certificate signing for agents + humans
3. **Custom Ubuntu 24.04 Cloudron ISO** — needed before app fleet deployment
4. **NetBird/Tailscale nesting** — agents must operate within the zero-trust mesh, not bypass it

---

## 5.6 OAM — Environmental Monitoring (from user's notes)

This is the physical/environmental monitoring layer that feeds into P4 (Monitoring & Instrumentation).

| Sensor/System | Method | Purpose |
|---|---|---|
| **DRAC** (Dell Remote Access Controller) | IPMI/Redfish API | Out-of-band management of Dell hosts (console, power, hardware status) |
| **SNMPd on non-PowerEdge** | SNMP polling | Hardware health on non-Dell systems (Pi, custom builds) |
| **Sensors** (lm-sensors / TEMPer USB) | Direct probe | CPU temp, ambient temp, fan speed |
| **Home Assistant** | HA integrations | All site environmental monitoring (temp, humidity, presence, power events) |
| **Beszel** | Agent-based | RAM / CPU / disk metrics on all hosts |

**Integration with agent identity:** Environmental monitoring systems (Home Assistant, Beszel, DRAC interfaces) are Cloudron-managed or on-prem. Agent identities need scoped access to these for the vp-techops agent (alerting, dashboard) and vp-secops agent (security event correlation).

**Current state:**
- TEMPer USB probes — Redmine #341 (Friday onsite batch)
- Tripp Lite UPS integration — #372, #439
- UNPoller (UniFi monitoring) — deployed, placeholder creds
- Beszel — agent installed on hosts, dashboard accessible
- Home Assistant — planned, not yet deployed
- SNMPd — deployed via KNELIAC `system_config` role
- LibreNMS — the poller/correlation layer (tsys-librenms)

---

## 6. Risk Register

| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| **Vault (P5) slips** → agents can't access secrets | High | High | P5 is critical path; prioritize immediately |
| **Discourse admin blocker** → can't create VP SecOps category | High | Med | User creates categories via web UI (confirmed will do) |
| **k8s cnodes not rebuilt** → P7 slips → Oct K8s focus starts late | Med | High | Rebuild cnodes in Aug onsite window |
| **SSH cert infrastructure not ready** → agents can't use cert-based SSH | Med | High | Nitrokey HSM SSH CA setup as part of P5 |
| **FOCI concern unresolved** → uncertain what data can live on Cloudron | Med | High | Decide data classification boundaries early in Q4 |
| **Cloudron RBAC limits** → can't scope agent access granularly | High | Med | Keycloak fills the gap; wire before agent activation |
| **Custom Ubuntu ISO not built** → blocks Cloudron app fleet | Med | Med | Prioritize after P1-P3 physical work |
| **Scope creep** — P1-P9 alone is 86 open tickets | High | Med | Keep Q3 TechOps / Q4 Business split strict |
| **Hermes full-host-access security** → COO agent blast radius | Med | Critical | Define trust model before Q4 COO agent activation |

---

## 7. Immediate Next Actions

### Confirmed and ready to execute
1. **Agent identity bootstrap** (see `agent-identity-bootstrap.md`) — *awaiting user prerequisites (coo account, BW account, Cloudron invites)*
2. **Push TSYSGroupAIOS** to Gitea + create as template repo — *ready now*
3. **Continue P1-P9 execution** — Friday onsite batch, then P5 Vault critical path

### Needs user input first
4. **Discourse VP SecOps category** — user creates via web UI (API user can't)
5. **Cloudron invites** for the 3 Q3 agents — user generates
6. **`coo` Linux account + Bitwarden account** — user creates

### Q4 prep (not yet started)
7. **K8S workload planning** — user's stated October focus. Need: K8S repo, Redmine project, app triage (Cloudron vs K8S)
8. **COO business-ops layer** — Discourse cat 6 content, Redmine project 53 tickets, ITSM tool decision
9. **Custom Ubuntu 24.04 Cloudron ISO** — dependency for app fleet
10. **SSH certificate infrastructure** — Nitrokey HSM → SSH CA → cert signing