MOPAC/docs/PORTING-NOTES-crush.md

Porting notes: crush (reference/crush @ internal/)

Scope for MOPAC harness: MCP client, sessions/resume, provider routing, headless loop; OAuth deferred (LiteLLM key suffices). Paths relative to reference/crush/; line refs from study 2026-08-28, drift possible.

Status: 2026-08-28 — complete; reference study, line refs may drift with upstream.

1. MCP plumbing

Code in internal/agent/tools/mcp/ (NOT internal/client/ = daemon RPC). Uses official SDK modelcontextprotocol/go-sdk/mcp. Config: crush.json top-level "mcp" map name->MCPConfig (config/config.go:185-252, :383/:723):

{ "mcp": {
  "local":  { "type": "stdio", "command": "npx", "args": ["-y","pkg"], "env": {"K":"V"} },
  "remote": { "type": "http", "url": "http://h/mcp", "headers": {"Authorization":"Bearer $TOKEN"} },
  "legacy": { "type": "sse", "url": "https://h/sse" } } }

• type: stdio|sse|http (http=streamable). Extras: disabled, enabled_tools/disabled_tools, timeout secs (dflt 10s, 30s OAuth), sessionless, oauth*. $VAR/$(cmd) expansion: config.go:438-528. PORT FORMAT VERBATIM for KNEL-AIMiddleware compat. Transports (mcp/init.go:1035-1195 createTransport): stdio -> mcp.CommandTransport; env=os.Environ()+resolved; child in own process group, group SIGKILL + 5s WaitDelay (process_unix.go:22-40). PORT: process-group kill or zombies leak. http -> mcp.StreamableClientTransport; static headers via headerRoundTripper (init.go:1197-1206). sse -> SSEClientTransport (legacy, keep for compat). Health/restart: getOrRenewClient (init.go:659-771) — Ping fast path; on failure teardown->reconnect->re-register tools. Per-server renewal mutex + generation counters discard stale connects. Config-change reconcile single-flighted (mcp/lifecycle.go:45-125). Init: per-server goroutines + WaitGroup; WaitForInit gates NON-interactive runs only (headless waits, TUI proceeds lazily). Tool registration: discovery via ListTools on connect + list-changed subscription (mcp/tools.go:147-169). Naming mcp_<server>_<tool> SINGLE underscores (agent/tools/mcp-tools.go:58-60). Adapter maps JSON-Schema properties/required -> ToolInfo (mcp-tools.go:70-97); results map text/image/audio, base64-normalized (tools.go:54-108). Routing: RunTool -> client lookup -> CallTool; auto-renews first. Aggregation: per-run palette = built-ins + GetMCPTools, filtered per-agent by AllowedMCP map (agent/coordinator.go:768-790). Go port: adopt go-sdk/mcp; copy config schema, naming convention, renewal-with-generations, WaitForInit gate. Skip: Claude "channel" push hack (mcp/channel.go), docker-mcp gateway (config/docker_mcp.go).

2. Session/persistence

Single SQLite DB {dataDir}/crush.db (default .crush/ per project, discovered cwd-up-to-git-root, config/load.go:574-582). Sibling crush.lock flock + owner JSON stops concurrent processes (db/datadirlock.go). Schema (db/migrations/, goose, sqlc queries in db/sql/): sessions(uuid, parent_id, title, message_count, tokens, cost, todos JSON, summary_message_id), messages(id, session_id, role, parts JSON, model, provider, finished_at), files (snapshots), read_files (tracker). Parts = {"type","data"} array (message/content.go:56-143). Pragmas WAL+busy_timeout=30s; SetMaxOpenConns(1) avoids SQLITE_NOTADB (db/connect.go:18-41,137-142). Short user ID = XXH3 hex of UUID (session/session.go:28-32). -C/--continue vs -s/--session (mutually exclusive, cmd/root.go:63): Continue = GetLast ORDER BY updated_at DESC skipping agent-tool/child sessions (app/app.go:238-263). ID resolve: exact UUID, else XXH3 prefix scan; 0 hits=not found, >1=ambiguous (root.go:941-967). Resume restores messages+parts, file snapshots, read-files, model from last assistant msg; NOT restored: permissions, LSP/MCP (re-init). Crash safety: 33ms debounced writes, sync flush on terminal events (message/message.go:218-302); read-time repair in agent/agent.go:1527-1708 — drops orphaned tool results, SYNTHESIZES results for orphaned tool_calls (interrupted sessions still pass API validation), skips empty cancelled msgs. PORT THIS — it is what makes resume robust. Harness v1 (bounded turns): JSONL per session is acceptable; what matters is the repair pass and finished_at-only-on-Finish-part semantics.

3. Provider layer

No in-house LLM client: calls delegate to external charm.land/fantasy; catalog from charm.land/catwalk. Neither vendored here — harness owns this layer against LiteLLM (openai-compat): port the CONFIG CONTRACT, not code.

• Catalog: embedded + remote fetch (catwalk.charm.land, ETag) w/ fallback remote->cache->embedded (config/catwalk.go:37-90, provider.go:37-90; cache ~/.local/share/crush/providers.json, atomic write). Harness: static embedded catalog, no remote sync.
• Layering low->high (config/load.go:926-965): /etc/crush -> ~/.config/crush -> ~/.local/share/crush -> project walk (.crushrc|crushrc| .crush.json|crush.json, cwd up to worktree root) -> workspace dataDir copy LAST. Deep-merge via go-jsons.
• Catalog x user (load.go:213-365): providers.<id> overrides base_url/api_key/extra_headers; user models[] PREPENDED, wins by ID; disable_default_providers nukes catalog. Routing: parseModelStr (app/provider.go:15-27) — first /-segment vs provider IDs, else whole string = model ID; ambiguous -> demand provider/model. Defaults from catalog DefaultLarge/SmallModelID (load.go:716-789); per-model overrides (max_tokens/temperature/reasoning) load.go:826-899; provider switch on Type agent/coordinator.go:806-1161.
• Retries: INSIDE fantasy, not this repo. Crush only observes: OnRetry resets streamed content so retry doesn't concatenate (agent/agent.go:931-941); OnAuthRefresh = transparent 401 retry (:127). PORT: harness-layer retry (429/5xx/transport backoff) + reset partial assistant buffer first. Stream-truncation surfacing (z.ai "Message content shorter than read bytes"): stream errors classified at agent/agent.go:1067-1189; IsTransportError -> FinishReasonError w/ wrapped err (:1177) — unexpected EOF lands here. In headless, the z.ai-class error is DETECTED at app/app.go:414-416 (reconciles streamed bytes vs final msg, fails the run, exit 1). Zero-usage fallback: agent/usage_fallback.go. Harness v1: on EOF/short-read, retry request (per DESIGN retry/backoff) and resume turn from persisted messages; reuse delta-cursor print (app.go:406-431).

4. Headless mode

No -p/--print exists. Headless = crush run|r [prompt...] (cmd/run.go:31-169). Prompt = args joined + stdin (pipe/file, not TTY) PREPENDED (root.go:918-935); empty -> error. Output: plain markdown to stdout; spinner only if stderr is TTY. Flags: -q, -v, -m provider/model, --small-model, -s, -C. Two paths: in-process (app.RunNonInteractive, app/app.go:267-438) or client/server SSE (run.go:334-455). Port:

• Delta printing: track messageReadBytes per msg id, print only content[readBytes:] (app.go:406-431). Server path keys completion on RunComplete event by runID, NOT message finish (tool_use mid-turn regression, run.go:385-397).
• Permissions headless: session auto-approved wholesale (AutoApproveSession app.go:349; permission.go:280-284) — never blocks; interactive-only tools excluded (initCoderAgent(false) app.go:674-703). Harness: invert — per-vertical allowlist, deny-default, no auto-approve.
• Exit codes: 0/1 only. 1 = RunE error (no prompt, no providers, agent error, run failed); CANCEL returns 0 (app.go:398-401); tool failures do NOT affect exit. Harness: distinct codes gate/budget/agent/tool.

5. OAuth (deferred v1 — code map)

• Hyper device-code flow: oauth/hyper/device.go (:40 initiate, :87 poll, :153 refresh-exchange). Copilot device flow + GH->Copilot token exchange: oauth/copilot/oauth.go (:36,:69,:149), disk import copilot/disk.go, spoof headers copilot/http.go. Login CLI: cmd/login.go, cmd/logout.go.
• MCP OAuth 2.1 auth-code+PKCE, dynamic client registration (RFC 7591), PR metadata (8414/8707): oauth/mcp/handler.go:110-219. Loopback /callback, ports 40704-40713 first-free (pin via oauth_callback_port); listener bound only during in-flight auth (:487); concurrent 401s deduped to one browser tab (authFlight :420). Tokens -> config key mcp.<name>.oauth_token; save-on-change wrapper oauth/mcp/savingtokensource.go. Browser suppressed non-interactive (ErrInteractiveAuthRequired handler.go:32). If harness needs OAuth, port THIS flow (MCP); provider OAuth moot behind LiteLLM. Token plumbing: access token stored AS providers.<id>.api_key (config/store.go:564-595); refresh singleflight + cross-process flock <datadir>/locks/<provider>.refresh.lock, adopts peer-rotated token (store.go:658-903). Token model oauth/token.go. Anthropic Claude- subscription OAuth: REMOVED (config/load.go:306-314).

6. Reading list + gotchas

Top 10 to read in detail during build:

1. internal/agent/tools/mcp/init.go (1332 ln) — sessions, transports, renewal; MCP port blueprint.
2. internal/agent/tools/mcp-tools.go + mcp/tools.go — tool adapter, naming, filtering, result mapping.
3. internal/agent/agent.go (2288 ln) — stream callbacks, error classification, prompt repair (:1527-1708).
4. internal/app/app.go — RunNonInteractive, resolveSession, delta print.
5. internal/cmd/run.go + cmd/root.go — headless flags, stdin handling, config discovery walk.
6. internal/config/load.go — config layering/merge, model resolution.
7. internal/message/message.go + content.go — debounced persistence, part model.
8. internal/session/session.go + db/sql/ — session service + queries.
9. internal/permission/permission.go — precedence chain (skip -> allowed_tools -> hook -> auto-approve -> interactive).
10. internal/agent/coordinator.go — tool palette assembly, provider construction, AllowedMCP filtering. Gotchas:
11. internal/client/ is daemon RPC, not LLM/MCP — don't port by name. LLM lives in external charm.land/fantasy (not vendored here).
12. MCP tool names use SINGLE underscores (mcp_server_tool); double- underscore assumption breaks KNEL compat.
13. stdio MCP children need own process group + group kill; plain Kill orphans grandchildren (process_unix.go).
14. Retries concatenate partial stream output unless the assistant buffer is reset first (agent.go:931-941 OnRetry pattern).
15. Resume correctness lives in read-time repair (orphaned tool_call/result synthesis), not write-time transactions — skipping it makes every interrupted session permanently API-invalid.