org-buildout/agent-identity-bootstrap.md

# AI Agent Identity Bootstrapping — Provisioning Plan

**Status:** DRAFT — architecture confirmed, awaiting user prerequisites
**Date:** 2026-08-13
**Goal:** Stand up dedicated AI agent identities with proper attribution, RBAC, and audit trails. Agents own their own credentials in Bitwarden (no shared `~/.creds/` env files).

---

## 1. Confirmed Architecture

```
┌─────────────────────────────────────────────────────────┐
│  CLOUDRON (tsys-cloudron.knel.net)                      │
│  Identity root — IdP for ALL apps except Uptime Kuma     │
│                                                          │
│  Agent identities (each a separate Cloudron user):       │
│    vp-techops, vp-secops, vp-techcompliance (Q3)         │
│    coo, svp-knel, svp-tctc, vp-finance, ... (Q4)        │
└────────────┬────────────────────────────────────────────┘
             │ Cloudron SSO (auto-provisions identity)
             ▼
┌─────────────────────────────────────────────────────────┐
│  CLOUDRON-MANAGED APPS                                  │
│  Gitea · Discourse · Redmine · + all other apps         │
│                                                          │
│  Each agent SSOs in → generates its OWN API keys →      │
│  stores in Bitwarden. Agents are fully independent.     │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│  BITWARDEN (dedicated "COO" account)                    │
│  Credential vault — RCEO owns all creds/orgs/collections │
│                                                          │
│  Collections: vp-techops/, vp-secops/, shared/, etc.    │
│  Each item: service URL, username, password, API key,   │
│             TOTP secret, SSH private key                 │
│                                                          │
│  bw CLI authenticates via client_id/client_secret        │
│  (machine-to-machine, non-interactive)                  │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│  COO LINUX ACCOUNT (orchestration layer)                │
│  Currently: reachableceo's workstation                   │
│  Soon: dedicated hardened VM on PFVCluster              │
│                                                          │
│  This is where Charles/AJ interact with agents.          │
│  Agents are "personalities" (Crush/Hermes sessions)     │
│  that run here, each sourcing its own creds from BW.    │
│                                                          │
│  NO key material on disk. SSH via BW SSH agent.         │
└─────────────────────────────────────────────────────────┘
```

### Key principles
1. **Cloudron is the identity root.** One invite = identity everywhere (SSO auto-provisions).
2. **Bitwarden is the ONLY credential store.** The only secret that lives on disk is the BW access info itself (client_id, client_secret, master password). Everything else — Discourse API keys, Redmine keys, Gitea tokens, SSH private keys, TOTP secrets, app credentials — lives in Bitwarden exclusively. No `~/.creds/*.env` files. No key material on disk.
3. **Agents own their credentials.** Each agent generates its own API keys after SSO login, stores them in Bitwarden. Agents are fully independent — each is a separate Cloudron user with its own SSO sessions, API keys, and Bitwarden collection.
4. **The `coo` Linux account is the orchestration layer** — where humans (Charles/AJ) launch and interact with agent sessions. Not where agents "live" — agents live as Cloudron identities.
5. **SSH via BW SSH agent.** No private keys on disk. Aligns with SecOps target: only iPad enclave key + Bitwarden key.
6. **reachableceo's personal Crush session stays as-is** — Charles's personal assistant for reviewing tickets, personal projects, and transitioning work to the new agents on his behalf. The TSGCOO account + new agents are an additional layer, not a replacement.

---

## 2. The Credential Migration (`~/.creds/` → Bitwarden)

### Current state (14 credential files)

```
~/.creds/
├── beszel.env          ├── librenms.env       ├── redmine.env
├── discourse.env       ├── oxidized.env       ├── switch-creds.env
├── grafana.env         ├── phpipam.env        ├── technitium.env
├── pushover.env        ├── prometheus.env     ├── unifi-creds.env
└── uptime-kuma.env
```

All CLIs invoked as: `docker run --rm --env-file ~/.creds/<service>.env <cli-image> ...`
All using `reachableceo`'s API keys. No per-agent credential separation.

### Target state

```
Bitwarden org: "TSYS Group AI Agents"
├── vp-techops/          (collection)
│   ├── Cloudron         (login + TOTP)
│   ├── Gitea            (API token)
│   ├── Discourse        (API key)
│   ├── Redmine          (API key)
│   ├── SSH key          (Ed25519 private key)
│   └── <infra creds>    (LibreNMS, Grafana, etc. — scoped to TechOps)
├── vp-secops/
│   ├── Cloudron / Gitea / Discourse / Redmine / SSH
│   └── <security creds>
├── vp-techcompliance/
│   └── ...
├── shared/              (fleet-wide creds all agents need)
│   ├── Technitium DNS
│   ├── Pushover
│   ├── Switch credentials
│   └── Beszel
└── coo/ (Q4)
    └── ...
```

### Migration path (2 stages)

**Stage 1 — Agent provisioning.** New agent identities are created in Cloudron, generate their own API keys via SSO, and store everything in Bitwarden. They source credentials exclusively via `bw-run.sh`. The `~/.creds/` files are NOT used by agents — only by `reachableceo`'s personal session.

**Stage 2 — Full migration (Redmine #440, Aug 19).** `reachableceo`'s `~/.creds/` files are migrated into Bitwarden too. After this, the ONLY secret on disk is the BW access info (client_id/client_secret/master password). All CLI invocations use `bw-run.sh`. The `~/.creds/` directory is deleted.

**Principle: the only thing outside Bitwarden is Bitwarden's own access information.** Everything else — API keys, passwords, SSH keys, TOTP secrets — lives in BW exclusively.

### The credential sourcing layer

A wrapper script that replaces the `--env-file ~/.creds/` pattern:

```bash
#!/usr/bin/env bash
# bw-run.sh — run a CLI command with credentials sourced from Bitwarden
# Usage: bw-run.sh <agent> <service> <image> [args...]
# Example: bw-run.sh vp-techops redmine redmine-cli:latest list --assigned-to-me -p 55

set -euo pipefail

AGENT="$1"; SERVICE="$2"; IMAGE="$3"; shift 3

# Ensure BW session is active
BW_SESSION="${BW_SESSION:-$(bw unlock --raw 2>/dev/null)}" || true
export BW_SESSION

# Fetch credential item from Bitwarden
ITEM_NAME="${AGENT} ${SERVICE}"
ENV_JSON=$(bw get item "$ITEM_NAME" | jq -r '
  .login |
  "URL=\(.uris[0].uri // "")\n" +
  "USERNAME=\(.username // "")\n" +
  "PASSWORD=\(.password // "")\n" +
  (.fields[]? | "\(.name)=\(.value)\n")
')

# Write to temp env file (cleaned up on exit)
ENVFILE=$(mktemp /tmp/bw-${AGENT}-${SERVICE}-XXXXXX.env)
trap 'rm -f "$ENVFILE"' EXIT
echo -e "$ENV_JSON" > "$ENVFILE"

# Run the CLI with BW-sourced credentials
docker run --rm --env-file "$ENVFILE" "$IMAGE" "$@"
```

This preserves the existing CLI invocation pattern (`docker run --env-file ... <image>`) while sourcing credentials from Bitwarden instead of static files. Agents call `bw-run.sh vp-techops redmine ...` instead of `docker run --env-file ~/.creds/redmine.env ...`.

### What needs to change in each project

| Project | Change | Effort |
|---|---|---|
| **TSYSGroupAIOS** (meta) | Add `bw-run.sh` to `scripts/`. Update `BASELINE-PROMPT.md` credential references. Document the BW credential model. | Med |
| **PFVCluster** | Update `AGENTS.md` CLI examples to use `bw-run.sh`. Add agent-profile system. Update `scripts/check-rules.sh` to enforce no `~/.creds/` refs in new code. | Med |
| **KNEL-AIMiddleware** | Update `tooling-cli/*/AGENTS.md` to document BW sourcing. Update `KNELCredsManager` to be the BW integration point. | Med |
| **All CLI containers** | No image changes needed — they still read env vars. The sourcing layer (`bw-run.sh`) is outside the container. | None |

---

## 3. Identity Roster

### Q3 — TechOps Agents (stand up NOW)

| Agent | Cloudron user | Redmine scope | Gitea scope | Discourse scope |
|---|---|---|---|---|
| **vp-techops** | vp-techops@<cloudron-domain> | 55, 59 | KNEL, TechnicalOperations | 74, 20 |
| **vp-secops** | vp-secops@<cloudron-domain> | 55 (security) | KNEL | 74 + new VP SecOps cat |
| **vp-techcompliance** | vp-techcompliance@<cloudron-domain> | 55 (compliance) | KNEL | 75 |

### Q4 — Business Agents (enroll in Cloudron now, activate in Q4)

| Agent | Redmine scope |
|---|---|
| coo | 53, 62, 77 |
| svp-knel | 62, 55 (read) |
| svp-tctc | 15 (TCTC), 31 (RedWFO) |
| vp-finance, vp-accounting, vp-investing, vp-treasury, vp-trading | 15 (TCTC) |

**Recommendation:** Enroll ALL identities in Cloudron during Q3 (cheap — just invite acceptance). Only provision API keys + system access for the 3 Q3 agents. Q4 agents get activated when business ops work begins.

---

## 4. Provisioning Pipeline

### Phase 0 — Prerequisites (USER provides)

| # | Item | Detail |
|---|---|---|
| 1 | **Linux `coo` account** | `useradd -m -s /bin/bash coo`; add to `docker` group; `mkdir ~/.ssh ~/.creds ~/projects` |
| 2 | **Bitwarden "COO" account** | Dedicated BW account. RCEO owns the org. Create collections per agent. |
| 3 | **BW API credentials** | `client_id` + `client_secret` for non-interactive `bw login --apikey`. Place in a file the agent can read. |
| 4 | **Cloudron invite manifest** | Text file, one line per identity: `agent-name | display-name | invite-link | priority` |
| 5 | **Discourse admin key** *(for VP SecOps category)* | Current API user (trust-4) cannot create categories. User creates VP SecOps category via web UI OR provides admin key. |

### Phase 1 — Cloudron Identity Enrollment (Playwright automation)

For each identity in the manifest:

```
1. Launch Playwright Docker (mcr.microsoft.com/playwright:v1.52.0-noble)
2. Navigate to Cloudron invite link
3. Generate strong password (bw generate -uluns --length 32)
4. Fill password fields, submit → identity created
5. Enable 2FA:
   a. Navigate to Cloudron account security settings
   b. Initiate TOTP enrollment
   c. Extract TOTP secret from QR code
   d. Store TOTP secret in Bitwarden
   e. Confirm with current TOTP code (bw code from stored secret)
6. Store Cloudron credential in Bitwarden:
   - Collection: <agent-name>
   - Name: "<agent-name> Cloudron"
   - URL: https://tsys-cloudron.knel.net
   - Username: <agent-name>@<domain>
   - Password: <generated>
   - TOTP: <secret>
```

**Result:** Agent identity exists in Cloudron. SSO works for all managed apps. No per-system account creation needed — SSO handles it.

### Phase 2 — API Key Generation (agent logs in via SSO, creates keys)

For each agent, for each system (Gitea, Discourse, Redmine):

```
1. Launch Playwright
2. Navigate to the app URL → Cloudron SSO redirect
3. Complete SSO login (use agent's Cloudron creds from BW + TOTP from BW)
4. First-time SSO → account auto-provisioned in the app
5. Navigate to API key / access token settings:
   - Gitea: Settings → Applications → Generate New Token (scopes: api, repo, read:org)
   - Discourse: Preferences → API Keys → (may need admin to create user API key)
   - Redmine: My Account → API access key → Show
6. Copy the generated key
7. Store in Bitwarden:
   - Collection: <agent-name>
   - Name: "<agent-name> <SystemName>"
   - URL: <system URL>
   - Password field: <API key>
8. Verify: use the key to call the system's API (curl or CLI)
```

**After Phase 2, each agent has:**
- Cloudron identity (login + TOTP) in BW
- Gitea API token in BW
- Discourse API key in BW
- Redmine API key in BW
- Ability to operate independently in all three systems

### Phase 3 — SSH Key Provisioning

```
1. Generate Ed25519 keypair per agent: ssh-keygen -t ed25519 -f /tmp/<agent>-ed25519 -N ""
2. Store PRIVATE key in Bitwarden (as "Secure Note" or BW SSH Agent item)
3. Deploy PUBLIC key to fleet via KNELIAC:
   - Add to inventory/group_vars/all.yml managed_users
   - Run: ansible-playbook playbooks/setup_new_system.yml -t security_ssh --limit <targets>
4. Delete private key from /tmp (lives only in BW)
5. Configure BW SSH Agent on coo account: export SSH_AUTH_SOCK=... (bw agent)
```

### Phase 4 — Agent Profile System (on `coo` Linux account)

Each agent has a profile that sets its identity context:

```bash
# ~/agents/vp-techops/profile.sh
export AGENT_NAME="vp-techops"
export AGENT_DISPLAY="VP TechOps"
export AGENT_EMAIL="vp-techops@turnsys.com"
export AGENT_BW_COLLECTION="vp-techops"
export GIT_AUTHOR_NAME="VP TechOps"
export GIT_COMMITTER_NAME="VP TechOps"
export GIT_AUTHOR_EMAIL="$AGENT_EMAIL"
export GIT_COMMITTER_EMAIL="$AGENT_EMAIL"
export REDMINE_PROJECT_SCOPE="55,59"
export GITEA_ORG_SCOPE="KNEL,TechnicalOperations"
export DISCOURSE_CAT_SCOPE="74,20"
# SSH: BW agent provides the key for this agent
```

Switching agent context:
```bash
source ~/agents/vp-techops/profile.sh   # become vp-techops
source ~/agents/vp-secops/profile.sh    # become vp-secops
```

### Phase 5 — Operational Environment

```
1. Clone repos into /home/coo/projects/ (or the future hardened VM)
2. Copy/adopt TSYSGroupAIOS framework (check-rules.sh, hooks, etc.)
3. Install bw-run.sh credential sourcing layer
4. Set up per-agent Crush config directories:
   ~/.config/crush/vp-techops/
   ~/.config/crush/vp-secops/
5. Configure tea CLI per agent (separate logins = separate tokens)
6. Install git hooks (ticket-gate, pre-commit, pre-push)
7. Smoke test per agent:
   - Create a Redmine ticket (appears as agent identity)
   - Edit a Discourse topic (appears as agent identity)
   - Open a Gitea PR (appears as agent identity)
   - SSH to a host (appears in auth log as agent's key)
```

---

## 5. Playwright Automation Design

The Playwright script handles the browser-driven steps (Phases 1-2). Containerized — no browser on host.

```
provision-agent/
├── Dockerfile              # Based on mcr.microsoft.com/playwright:v1.52.0-noble
├── provision-agent.py      # Main script
├── bw-helper.py            # Bitwarden CLI wrapper (gen password, store item, get TOTP)
└── agents.yaml             # The manifest (invite links + agent config)
```

**Flow:**
```python
for agent in manifest:
    # Phase 1: Cloudron enrollment
    browser → invite_link
    fill password (bw generate)
    enable 2FA → extract TOTP secret → store in BW
    
    # Phase 2: API keys (per system)
    for system in [gitea, discourse, redmine]:
        browser → system_url → SSO redirect → login with BW creds + TOTP
        navigate to API key page → generate → copy
        store in BW
    
    # Verify
    for system in [gitea, discourse, redmine]:
        api_call(system, key_from_bw) → assert success
```

**Security:** Playwright runs in Docker, no persistent browser state. Credentials are generated/stored via `bw` CLI (mounted into container). No credentials in browser memory after the script exits.

---

## 6. Security & Compliance Posture

| Control | How this design satisfies it |
|---|---|
| **CMMC IA-2(1)** (MFA for all accounts) | Every agent identity has TOTP via Bitwarden |
| **CMMC AU-2/AU-12** (audit events) | Every action attributable to specific agent identity (Redmine, Discourse, Gitea, SSH auth logs) |
| **CMMC AC-2/AC-3** (least privilege) | Each agent scoped to its Redmine projects, Gitea orgs, Discourse categories, and fleet SSH access |
| **CMMC IA-5(1)** (authenticator management) | No key material on disk. All in Bitwarden. SSH via BW agent. |
| **Shared credential elimination** | Each agent has its own API keys and SSH keys — no more single key on "every single system" |
| **ITAR/SCIF alignment** | Per-identity access control + audit trail for classified-adjacent systems |

---

## 7. Implementation Timeline

| Week | What | Who |
|---|---|---|
| **Week 1** | User provides Phase 0 prerequisites. Agent builds Playwright automation + bw-run.sh. Executes Phase 1-2 for 3 Q3 agents. | User + Agent |
| **Week 2** | Phase 3 (SSH keys via KNELIAC). Phase 4-5 (agent profiles, coo env, smoke tests). Enroll Q4 identities in Cloudron (Phase 1 only). | Agent |
| **Week 3+** | Agents begin operating from own identities on P1-P9 work. Charles reviews as himself. Agents @mention Charles in Redmine/Discourse. Agents open PRs for review. | Agents + Charles |

---

## 8. What I Need From You to Start

### Done from this session (reachableceo — Charles's personal assistant)
- [x] Playwright automation repo built and pushed (`TSYSGroupCorporate/agent-identity-provisioning`)
- [x] `bw-run.sh` credential sourcing layer built and pushed (`TSYSGroupAIOS/scripts/bw-run.sh`)
- [x] Redmine tickets filed (#440 BW migration, #441 taxonomy audit, #442 provisioning)
- [ ] **Ticket transitions** — once agents are provisioned, I (reachableceo session) will transition tickets to the new agent identities on Charles's behalf

### To be done from the TSGCOO session (the new operational layer)
1. **TSGCOO Linux account** is created. Set up the Crush session there.
2. **BW access info** — place ONLY the BW client_id/client_secret/master password in the TSGCOO account (e.g., `~/.config/bw/env`). This is the ONLY secret on disk. Everything else comes from BW at runtime.
3. **Discourse admin access** — set `DISCOURSE_API_USERNAME=system` in the TSGCOO session's environment (sourced from BW, not a static file). The `system` user has admin privileges for category creation.
4. **Cloudron invites** — generate invite links for vp-techops, vp-secops, vp-techcompliance. Put them in `agents.yaml`.
5. **Run the provisioning** — `docker compose up --build` from the `agent-identity-provisioning` repo.
6. **VP SecOps category** — the vp-techops agent (or the TSGCOO session) creates the Discourse VP SecOps category using the `system` admin credentials.

### reachableceo session role (unchanged)
The reachableceo Crush session continues as Charles's personal assistant — reviewing tickets, working personal projects, and handling the transition of work to the new agents. It is not replaced by the TSGCOO layer; it operates alongside it.