21 lines
1.8 KiB
Markdown
21 lines
1.8 KiB
Markdown
# TASK: Deploy LiteLLM proxy + wire RCEO pilot stack
|
|
|
|
## Objective
|
|
One LiteLLM container fronting the shared z.ai key on this host; virtual keys issued; RCEO crush pointed at it; usage API verified. Implements DESIGN-24x7-execution.md §v2.2.
|
|
|
|
## Context
|
|
- z.ai provider + key currently live in /home/reachableceo/.local/share/crush/providers.json (NEVER print key values in REPORT/logs).
|
|
- reachableceo is in the docker group — containers allowed, no sudo needed.
|
|
- This host already runs docker (25 containers). Offstage (other machine) must reach the proxy later, so bind to the LAN interface, not just loopback.
|
|
|
|
## Deliverables
|
|
1. Config: /home/_crossfeed/tooling/litellm/config.yaml — z.ai models via OpenAI-compatible endpoint (api.z.ai), master_key from credentials/, embedded SQLite. Secrets file mounted 600.
|
|
2. Container: name `litellm`, restart unless-stopped, port 4000 on LAN interface (4001+ if busy, note it).
|
|
3. Virtual keys (8): rceo, bod, cco, cto, coo, offstage-1, offstage-2, offstage-3. Names + budgets documented; key VALUES only in /home/_crossfeed/tooling/credentials/litellm-virtual-keys.md (mode 600).
|
|
4. RCEO pilot wiring: backup providers.json -> providers.json.pre-litellm.bak; point base_url at http://<lan-ip>:4000 with the rceo virtual key.
|
|
5. Verify: one headless crush call through the proxy succeeds; hit the LiteLLM usage/key-info endpoint and capture (redacted) output.
|
|
6. REPORT to inbox-pmo: container name/health, port, virtual key NAMES (not values), backup path, redacted usage sample, emergency fallback = restore providers.json.pre-litellm.bak.
|
|
|
|
## Constraints
|
|
- No sudo. No secrets in any REPORT/log. Do not touch TSG account homes. Do not stop or restart any existing container. If any step fails, BLOCKED report, do not improvise credentials handling.
|