Planning documents for TSYS Group's COO→CTO handoff and AI agent
identity architecture. Shared publicly as a bootstrapping reference.
Includes: org prompts, transition map, agent identity bootstrap plan,
TechOps/K8s/SecOps context notes.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
20 KiB
COO Handoff & CTO Transition — Master Map
Status: DRAFT for review (working session artifact, not yet synthesized to Discourse/Redmine) Date: 2026-08-13 Deadline: 2026-09-30 ("Potential to Kinetic Ready" + COO handoff, 48 days / "45 days") Author session: orientation/synthesis pass over Q3 prompts, all project trees, Gitea, Discourse, Redmine
0. Thesis (the one thing to internalize)
Revised 2026-08-13: Timeline split confirmed with user.
| Phase | Window | Focus | Who |
|---|---|---|---|
| Q3 remainder | Aug 13 → Sep 30 | TechOps finish (P1-P9) + stand up AI agent identities | Charles + AI agents |
| Q4 | Oct 1 → Dec 31 | Business ops transition + CTO come-up (80% CTO / 20% COO) | Charles + AJ + Patti + Courtney + AI agents |
| Jan 1 2027 | — | Full COO handoff complete. Charles = 100% CTO. | AJ + COO agent run business ops |
The infrastructure (PFVCluster, KNELIAC, monitoring, k8s) is ~80% there and has a phase plan (P1-P9) with a 9/30 deadline. The "running datacenter" is commodity and will be fully delegated to AI. The first action is standing up AI agent identities (Cloudron + Bitwarden + system access) so agents can operate with proper attribution, RBAC, and audit trails. See agent-identity-bootstrap.md.
Business ops transition is deferred to Q4. The COO Discourse category, Redmine project 53, bizopprodplan handbook refresh, ITSM tool selection, and COO Tier-1 business apps all move to Q4. AJ gets regular briefings through Q3 and takes over business ops in Q4 with Patti and Courtney as additional resources.
1. Current-State Map
1.1 The TSYS Group org structure (as designed in Q3/prompt.md)
TSYS Group (Board of Directors)
├── CTO — Charles (R&D, architecture, tier-4 SME) [you, transitioning IN fulltime Oct 1]
└── COO — AJ Lebsch (operations) [transitioning IN, oversees AI agents]
├── SVP KNEL (Known Element Enterprises — owns ALL IT/business systems)
│ ├── VP TechOps ← the ONLY function with real content today
│ ├── VP SecOps ← does not exist as a category yet
│ └── VP TechCompliance ← Discourse cat 75 exists, 0 topics
└── SVP TCTC (The Campus Trading Company)
├── VP Finance / VP Accounting / VP Investing
├── VP Treasury / VP Trading
└── (RedWFO = RWSCP Family Office, mission-critical, can preempt)
Supporting entities (each with its own Gitea org + Redmine project + Discourse category, mostly stubs): Suborbital Systems, HFNOC, HFNFC, RackRental, Starting Line Productions, Rogue Technologies, RedWFO, RWSCP, MeetMorse/MorsePod (FLO entry to CommonsNet), EzEDA, EzPodStack, AFABN, Ap4Ap, MerchantsOfHope, ThePeerNet, sol-calc, TeamRental, SideDoorGroup, YourDreamNameHere.
1.2 Systems of Record — actual state vs intended
| System | Role | Actual state |
|---|---|---|
| Redmine (projects.knownelement.com) | SoR for ALL work | 55 projects mirror the org chart. Only project 55 (TechnicalOperations) is active: 136 tickets, 86 open. Project 62 (Business Services), 53 (COO), 77 (TSYS Group parent), and all entity projects are empty/stubs. |
| Discourse (community.turnsys.com) | SoR for ALL docs | 55 categories. Only VP TechOps (cat 74, 13 topics) and Progress Reports (cat 61, 81 topics) have real content. COO (6), VP Compliance (75), Board (76), KNEL-Bizops (72) = 0 topics. Every business-entity category = 0-1 stub topics. |
| Gitea (git.knownelement.com) | SoR for executable code | 27 orgs, ~213 repos (87 are ExternalVendorCode mirrors). Source repos: KNEL (32), reachableceo (23), Suborbital-Systems-Public (18), RWSCP (11), + ~20 entity orgs with 1-3 repos each (mostly -bizopprodplan mdBook stubs). |
| Cloudron (Reston VPS) | PaaS for ~57 support-stack apps | 10/57 packaged (~17%). See §1.4. |
| K8s (PFVCluster bare metal) | Scalable compute | k3s HA (3 cnode + 6 workers) but cnodes were wiped and shut down — needs rebuild. Zero apps deployed. |
1.3 The 12 local projects (maturity + COO relevance)
| Project | What | Maturity | Governance | COO relevance |
|---|---|---|---|---|
| PFVCluster | Proxmox fleet + OAM + k8s bootstrap | High (most mature infra) | Excellent (full rules engine) | High — the compute foundation |
| football (KNEL-Football) | Hardened Debian live ISO for tier-0 access | ★★★★★ (788 tests, ISO built, audited) | Exceptional | High — secure access terminal |
| KNELIAC | Ansible fleet config-mgmt (replaces bash) | Active (9 roles, AWX wired) | Weak (80-line AGENTS.md) | High — fleet baseline |
| KNEL-AIMiddleware | MCP/LSP servers for AI agents (42 svcs) | 79% (33/42 prod-ready) | Moderate | Critical — the agent tooling backbone |
| EngStack | Hardware R&D engineering workstation (45 tools) | Active (8 tools built) | Good | Med (CTO/R&D domain) |
| TSYS-Cloudron | 57-app Cloudron packaging | 17% (10/57) | Strong | High — COO's business apps |
| WorkstationStack | Local dev support stack + SelfStack | Active (7 apps in prod) | Good | Med |
| hermes-rceo-streaming | Hermes AI agent deployment | Deployed | Minimal | High — agent runtime (security note: full host access) |
| netbird | Zero-trust VPN access | Early/stub | Minimal | Med (security-critical, underbuilt) |
| meta (TSYSGroupAIOS) | Canonical agent-framework template | Production-ready, NOT pushed | Canonical | Critical — must propagate to all projects |
| dotfiles / EngineeringWorkstation / TSYS-LocalWorkstation | Personal/scratch | — | — | Low |
1.4 Cloudron app fleet — COO-critical gap
10 packaged: Webhook, APISIX, Healthchecks, Review Board, WireViz Web, Puter, Corteza, draw.io, Windmill, InvenTree. COO Tier-1 MISSING (not started): Grist (ops spreadsheets), PayrollEngine, KillBill (billing), Rundeck (runbook automation), Comply (compliance tracking). Cloudron-blocked (need K8s): Sentry, SigNoz, DataHub, NetBox, Fleet (all need Redis/Kafka/ClickHouse).
1.5 Governance template (TSYSGroupAIOS / meta) — adoption status
~/daytoday/meta is the canonical framework: BASELINE-PROMPT.md (14 principles) + 10-section AGENTS.md template + scripts/check-rules.sh (10 checks) + git hooks + hooks/ticket-gate.sh. Self-applying (17 PASS/0 FAIL), ready to push to TSYSGroupCorporate/TSYSGroupAIOS (remote configured, not yet pushed).
| Project | Gap to framework |
|---|---|
| PFVCluster | 🟢 Minor — add BASELINE-PROMPT.md, align scripts |
| football | 🟡 Moderate — replace custom hooks |
| KNEL-AIMiddleware | 🟡 Moderate — drop JOURNAL.md, add pre-commit |
| KNELIAC | 🔴 Critical — needs entire enforcement layer |
| EngStack | 🔴 Critical — needs entire scripts/ + governance |
2. The Central Gap (COO handoff) — what "done" requires on Oct 1
For AJ + the COO AI agent to actually run operations, these must be true. Today none of them are.
2.1 Org / role
- COO AI agent stood up with Cloudron account, logged into all role-appropriate systems (Redmine, Discourse, Gitea, Cloudron, monitoring). Repo exists:
KNEL/TSG-COOAndBoard-AIAgents-Public— not yet built out. - SVP KNEL agent + SVP TCTC agent (synthesize policy/strategy → COO).
- VP TechOps / VP SecOps / VP TechCompliance agents scoped to their Redmine projects.
- AJ's access formally documented (he "has full access" per prompt — needs an access matrix in Discourse).
2.2 Documentation (Discourse categories to populate)
- COO (cat 6): operations manual, decision authority, escalation paths, daily/weekly/monthly cadence.
- VP SecOps: does not exist — create (needs admin key; current API user is trust-4, cannot create categories — blocker).
- VP Compliance (75): 0 topics — CMMC L3, STIG, ITAR program docs.
- KNEL-Bizops (72): 0 topics — the business-services knowledge base.
- Board (76): 0 topics.
- Each business entity category: operations content (currently 0-1 stubs).
2.3 Work tracking (Redmine)
- Project 53 (COO) / 62 (Business Services): define versions/milestones + seed tickets. Currently empty.
- COO workstream tickets mirroring the operational layer (not just TechOps P1-P9).
- AI-staff scoping: each Hermes agent scoped to its Redmine project only.
2.4 Systems / tools the COO depends on
- ITSM/workflow engine (prompt.md line 101: "We really need an ITSM tool — Discourse? Windmill? Nextcloud?"). Unresolved decision. Windmill is packaged (✅) — strongest candidate.
- COO Tier-1 apps deployed: Grist, PayrollEngine, KillBill, Rundeck, Comply (all not-started).
- Credential migration to Vault (P5, critical path) — unblocks agent secret access.
2.5 Culture / process (the "correct" part)
- TSYSGroupAIOS framework pushed and adopted into all 5 gap projects.
- Full SDLC enforced everywhere (red/green TDD, linters, CI/CD lockstep local+hosted).
sectestbed-/preprod-VM workflow operational for IaC testing.- "Gardening" loop running to prevent doc sprawl.
3. CTO Transition — what changes for Charles
| Stop doing (COO/founder work) | Start/continue doing (CTO work) |
|---|---|
| Day-to-day ops decisions | R&D architecture (EngStack, Suborbital, k8s platform) |
| Physical infra firefighting (delegate to VP TechOps agent) | Tier-4 escalation only |
| Direct fleet config (→ KNELIAC/AWX) | SDLC/governance ownership (the framework) |
| Ticket-level execution on P1-P9 | Mentor/oversee AI agents; review their work |
| Building business-ops docs | Document TSYS Group component architecture (Oct onboarding task per prompt.md) |
The CTO transition is gated on the COO handoff: you cannot stop doing COO work until AJ + the COO agent can absorb it.
4. Roadmap — Revised (Q3/Q4 split)
Q3: Aug 13 → Sep 30 — TechOps + Agent Identities
Workstream A — Finish Infrastructure (P1-P9, Redmine project 55, 86 open tickets) Keep executing the existing phase plan. Critical path: P5 Vault → P6 IaC → P9 Compliance → P7 k8s → P8 Apps.
- Aug 13-17: Friday onsite physical batch. Finish P1/P2/P3.
- Aug 18-31: P5 Vault (CRITICAL), P4 monitoring UAT, P6 IaC basics, P9 STIG/CMMC seed, P7 k8s cnode rebuild.
- Sep 1-30: P8 Cloudron app fleet (TechOps-relevant), P7 k8s apps (cluster only), compliance hardening.
Workstream B — Stand Up AI Agent Identities (NEW, first priority)
Identity-first. See agent-identity-bootstrap.md for full plan.
- Week 1: User provides prerequisites (Linux
cooaccount, Bitwarden account, Cloudron invites). Agent enrolls 3 Q3 identities (vp-techops, vp-secops, vp-techcompliance) via Playwright. - Week 2: Deploy per-agent SSH keys, set up
cooLinux environment, smoke test (agent creates ticket, edits Discourse, opens PR from own identity). - Week 3+: Agents begin operating from own identities. Enroll Q4 identities (Cloudron only, no system access yet).
Workstream C — Governance & Culture (cross-cutting)
- Week 1-2: Push TSYSGroupAIOS template; adopt into KNELIAC (critical) + EngStack (critical).
- Week 2-4: Adopt into KNEL-AIMiddleware (drop JOURNAL.md), football, PFVCluster (minor).
- Ongoing:
sectestbed-/preprod-VM workflow, CI/CD lockstep, gardening loop.
Q4: Oct 1 → Dec 31 — Business Ops Transition + CTO Come-Up
Charles at 80% CTO / 20% COO. Working alongside AJ (business ops lead), Patti, Courtney.
- October (Charles's stated focus): Figure out the K8S workload. What apps run on K8S vs Cloudron. Create the K8S repo + Redmine project.
- October–November: Build the COO business-ops layer: Discourse cat 6 content, Redmine project 53 tickets, ITSM tool selection (Windmill), COO Tier-1 apps (Grist, PayrollEngine, KillBill, Rundeck, Comply), bizopprodplan handbook refresh.
- November–December: Activate Q4 AI agents (COO, SVP KNEL, SVP TCTC, financial VPs). AJ UAT on COO agent + business systems. Dry-run operational week.
- Jan 1 2027: Full COO handoff. Charles = 100% CTO.
5. Open Decisions — Updated
Resolved by user (2026-08-13)
Business ops timeline→ Q4 (Oct–Dec), full handoff Jan 1 2027. Q3 = TechOps only.Agent identity approach→ Identity-first. Stand up Cloudron + Bitwarden accounts as the very first action.COO/CTO split in Q4→ 80% CTO / 20% COO. AJ leads business ops with Patti + Courtney.
Still open
- Cloudron SSO — Are Gitea/Discourse/Redmine Cloudron-managed (auto-SSO)? Or standalone? This determines provisioning complexity. (blocking agent bootstrap)
- Discourse admin key — API user is trust-4, cannot create categories (VP SecOps etc.). Provide admin key or create via web UI. (blocking VP SecOps setup)
- Linux account model — Single
cooaccount (recommended Q3) vs per-agent accounts (stronger audit). (see bootstrap spec D1) - Agent runtime — Crush per-agent config dirs (recommended Q3) vs Hermes vs OpenWebUI. (see bootstrap spec D3)
- K8S scope in P7 — Is P7 (due Aug 31) just cluster bootstrap, with app deployment deferred to October (your stated K8S focus)? Rec: yes — P7 = cluster ready, apps = October.
- ITSM tool — Windmill (packaged) vs Discourse-only vs Nextcloud. (deferred to Q4 but worth deciding early)
- bizopprodplan repos — Legacy mdBook stubs (KNEL one still says "CIO Documentation"). Refresh in place during Q4 or treat as superseded by Discourse?
5.5 SecOps & Compliance Context (from user's VP SecOps notes)
This is the security architecture that the TechOps agents operate within. Drives P5 (PKI/Vault) and P9 (Security/Compliance).
Compliance targets
- CMMC Level 3 is the goal (not L1 or L2)
- Full STIG compliance — highest level (mission critical classified)
- CUI minimum everywhere — all systems, no exceptions
- ITAR — governs all technical operations
- Multi-tenant — isolation between business entities (RackRental franchisees, Suborbital, TCTC, etc.)
- Eventually productized as a Your Dream Name Here (YDN) offering — the compliance stack itself becomes a product
Zero trust access model
- NetBird (primary zero-trust mesh) + Tailscale (existing, nested solution)
- Apple account referenced (likely for iPad-based access)
- All access through zero-trust — no flat network trust
- Remote access SCIF capability needed
- Keycloak deployed — initial setup done for NetBird integration
- Cloudron IdP has "simple groups, lacks granular permission levels at least via GUI" — Keycloak fills the RBAC gap
FOCI concern (Foreign Ownership Control Influence)
- Netcup (Cloudron VPS host) is a German company, even though hosting in Reston VA
- Question: can a VPS attest back to PFV (the on-prem cluster)?
- This affects what data/workloads can live on Cloudron vs must stay on-prem
CA / PKI (drives P5)
- Nitrokey HSM as the root CA hardware
- SSH certificates (not just keys) for day-to-day operations via Ansible
- Custom Ubuntu 24.04 ISO needed as the Cloudron base image
SSH key migration (directly impacts agent identity bootstrap)
| Current state | Target state |
|---|---|
| Ultix-highside (Win11 Surface, no local admin, Zoc terminal) — on-disk OpenSSH key present on every system except recent deploys | Single Bitwarden SSH key + BW agent (not yet working on Windows) |
| This VM's key — on a subset of machines, being expanded | Replaced by Bitwarden key/agent |
| iPad secure enclave key — used via Blink, public key on all accessible systems | Keep — one of two authorized broad-access keys |
| iPad enclave key + Bitwarden key only. No key material on disk. |
Agent identity impact: Agent SSH keys (Phase 3 of bootstrap) must use the SSH certificate model, not raw key deployment. The CA (Nitrokey HSM) signs agent certificates. This is stronger than key-based auth and aligns with the "day-to-day operations via Ansible and SSH certificates" target.
Bitwarden structure
- Three BW accounts currently exist (RCEO owns all creds/orgs/collections)
- Envwarden in use
- A fourth BW account (COO/AI agents) will be created per the bootstrap plan
- AJ has broad Bitwarden access (business continuity)
Business continuity
- Patti — iPad
- Remy — iPhone
- Albert (AJ) — broad Bitwarden access
Identity/IAM gaps to resolve
- Cloudron IdP lacks granular RBAC — Keycloak must fill this gap for agent scoping
- SSH certificate infrastructure — Nitrokey HSM → SSH CA → certificate signing for agents + humans
- Custom Ubuntu 24.04 Cloudron ISO — needed before app fleet deployment
- NetBird/Tailscale nesting — agents must operate within the zero-trust mesh, not bypass it
5.6 OAM — Environmental Monitoring (from user's notes)
This is the physical/environmental monitoring layer that feeds into P4 (Monitoring & Instrumentation).
| Sensor/System | Method | Purpose |
|---|---|---|
| DRAC (Dell Remote Access Controller) | IPMI/Redfish API | Out-of-band management of Dell hosts (console, power, hardware status) |
| SNMPd on non-PowerEdge | SNMP polling | Hardware health on non-Dell systems (Pi, custom builds) |
| Sensors (lm-sensors / TEMPer USB) | Direct probe | CPU temp, ambient temp, fan speed |
| Home Assistant | HA integrations | All site environmental monitoring (temp, humidity, presence, power events) |
| Beszel | Agent-based | RAM / CPU / disk metrics on all hosts |
Integration with agent identity: Environmental monitoring systems (Home Assistant, Beszel, DRAC interfaces) are Cloudron-managed or on-prem. Agent identities need scoped access to these for the vp-techops agent (alerting, dashboard) and vp-secops agent (security event correlation).
Current state:
- TEMPer USB probes — Redmine #341 (Friday onsite batch)
- Tripp Lite UPS integration — #372, #439
- UNPoller (UniFi monitoring) — deployed, placeholder creds
- Beszel — agent installed on hosts, dashboard accessible
- Home Assistant — planned, not yet deployed
- SNMPd — deployed via KNELIAC
system_configrole - LibreNMS — the poller/correlation layer (tsys-librenms)
6. Risk Register
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| Vault (P5) slips → agents can't access secrets | High | High | P5 is critical path; prioritize immediately |
| Discourse admin blocker → can't create VP SecOps category | High | Med | User creates categories via web UI (confirmed will do) |
| k8s cnodes not rebuilt → P7 slips → Oct K8s focus starts late | Med | High | Rebuild cnodes in Aug onsite window |
| SSH cert infrastructure not ready → agents can't use cert-based SSH | Med | High | Nitrokey HSM SSH CA setup as part of P5 |
| FOCI concern unresolved → uncertain what data can live on Cloudron | Med | High | Decide data classification boundaries early in Q4 |
| Cloudron RBAC limits → can't scope agent access granularly | High | Med | Keycloak fills the gap; wire before agent activation |
| Custom Ubuntu ISO not built → blocks Cloudron app fleet | Med | Med | Prioritize after P1-P3 physical work |
| Scope creep — P1-P9 alone is 86 open tickets | High | Med | Keep Q3 TechOps / Q4 Business split strict |
| Hermes full-host-access security → COO agent blast radius | Med | Critical | Define trust model before Q4 COO agent activation |
7. Immediate Next Actions
Confirmed and ready to execute
- Agent identity bootstrap (see
agent-identity-bootstrap.md) — awaiting user prerequisites (coo account, BW account, Cloudron invites) - Push TSYSGroupAIOS to Gitea + create as template repo — ready now
- Continue P1-P9 execution — Friday onsite batch, then P5 Vault critical path
Needs user input first
- Discourse VP SecOps category — user creates via web UI (API user can't)
- Cloudron invites for the 3 Q3 agents — user generates
cooLinux account + Bitwarden account — user creates
Q4 prep (not yet started)
- K8S workload planning — user's stated October focus. Need: K8S repo, Redmine project, app triage (Cloudron vs K8S)
- COO business-ops layer — Discourse cat 6 content, Redmine project 53 tickets, ITSM tool decision
- Custom Ubuntu 24.04 Cloudron ISO — dependency for app fleet
- SSH certificate infrastructure — Nitrokey HSM → SSH CA → cert signing