# Open Terminal fleet — ops note (#610, deployed 2026-08-31)

OpenWebUI "Open Terminal" for every account on ultix-streaming, in computer
mode (bare metal, real shell). Wired to start on reboot. UAT passed end to
end the same night (see 9-uat-openwebui.out).

## Layout

| account | port | bind | cwd | unit |
|---|---|---|---|---|
| reachableceo | 30000 | 100.101.187.119 (tailscale ONLY) | ~/projects | open-terminal@reachableceo |
| TSGCOO | 30001 | 〃 | ~/projects | open-terminal@TSGCOO |
| TSGCCO | 30002 | 〃 | ~ (no ~/projects yet) | open-terminal@TSGCCO |
| TSGCTO | 30003 | 〃 | ~/projects | open-terminal@TSGCTO |
| TSGBOD | 30004 | 〃 | ~ | open-terminal@TSGBOD |
| COS-RCEO | 30005 | 〃 | ~ | open-terminal@COS-RCEO |
| COS-WFO | 30006 | 〃 | ~ | open-terminal@COS-WFO |
| COS-TSG | 30007 | 〃 | ~ | open-terminal@COS-TSG |
| reachableceo-offstage | 30008 | 〃 | ~/projects | open-terminal@reachableceo-offstage |

- Binary: /usr/local/bin/open-terminal (open-terminal==0.11.34 via uv tool,
  managed python at /opt/uv-python, venvs at /opt/uv-tools; world-readable
  on purpose — non-root units exec it). Host-install exception authorized by
  human 2026-08-31 ("pip/uv directly on my host... mandatory").
- Template unit: /etc/systemd/system/open-terminal@.service (After
  tailscaled; Restart=always). Per-user TOML at
  /etc/ukrrs/open-terminal/<user>.toml (key lives THERE, not on any command
  line — nothing leaks via ps). Per-user cwd env alongside it.
- All 8 non-founder accounts were also added to the docker group (human
  ruling 2026-08-31).
- Bind is 100.101.187.119 ONLY (tailscale interface; human ruling
  2026-08-31 night: no LAN, no 0.0.0.0). Containers reach it fine (bridge →
  host IP routing); so does anything over tailscale.

## API keys (NEVER in this repo)

~/.creds/open-terminal.env (reachableceo, 0600): URL + key per account in
`<ACCOUNT>_URL` / `<ACCOUNT>_API_KEY` form. Rotate a key with:
  sudo sed -i "s/^api_key = .*/api_key = \"$(openssl rand -hex 32)\"/" \
    /etc/ukrrs/open-terminal/<user>.toml && sudo systemctl restart open-terminal@<user>
then update ~/.creds/open-terminal.env to match.

## Wiring a production OpenWebUI (admin, server-side key)

Your Cloudron prod runs Open WebUI 0.11.1 — new enough. Admin UI:
Settings → Admin → Integrations → Open Terminal → "+":
  URL: http://100.101.187.119:3000X      (per-account port from the table)
  API key: from ~/.creds/open-terminal.env
  Auth: Bearer. Save → green "Connected".
API-only equivalent (what the UAT does):
  POST /api/v1/configs/terminal_servers/verify   {"url":…,"key":…}
  POST /api/v1/configs/terminal_servers          {"TERMINAL_SERVER_CONNECTIONS":[{"id":…,"url":…,"key":…,"auth_type":"bearer","enabled":true}]}
Scope per-user access via the connection's access control (each user gets
THEIR OWN port; do not share one terminal across accounts — one shell each).
In chat: pick the terminal (cloud icon) once per conversation; native
function calling is already the default in 0.11.x.

## UAT record (2026-08-31 22:48, all passed)

Throwaway openwebui:v0.11.1 (matched to Cloudron prod) on 127.0.0.1:8081,
LLM via the beta gateway lane:
- P3 connection verify+list OK; P4 direct exec runs as uid 1001 with real
  ~/projects output; P5 model emitted run_command, WE executed it through
  OpenWebUI's terminal proxy (server holds the key), fed the result back,
  and the model answered with 9/9 REAL directory names (no hallucination —
  earlier runs without the tool attached fabricated names, proving the test
  catches it). Full log: 9-uat-openwebui.out.

Gotcha learned: for raw API callers OpenWebUI returns tool_calls for the
CALLER to execute (the server-side loop needs a socket session — that's the
UI path). UI chats run the loop automatically.

## Hygiene notes

- open-terminal keeps per-process logs in
  ~/.local/state/open-terminal/logs/processes/ per user (unbounded; watch
  when agents go wild).
- 0.0.0.0 was rejected by ruling; if tailscale is down at boot the units
  retry (Restart=always) until the IP exists.
- Not in any ukrrs slice yet; fold into per-account slices when mkacct
  lands (#607).