mrcharles
c6da05fac5
README: verified quickstart (Docker builder, dry-run, live demo), mermaid
architecture, CLI/config/routing reference tables, status table from
REPORT.md, correct ukrrs repo URLs. docs/: Status front-matter lines on
all PORTING-NOTES (secrets notes now tracked). DESIGN: table of contents
with anchor links; sections untouched.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
183 lines
10 KiB
Markdown
183 lines
10 KiB
Markdown
# Porting notes: crush (reference/crush @ internal/)
|
|
|
|
Scope for MOPAC harness: MCP client, sessions/resume, provider routing,
|
|
headless loop; OAuth deferred (LiteLLM key suffices). Paths relative to
|
|
`reference/crush/`; line refs from study 2026-08-28, drift possible.
|
|
|
|
Status: 2026-08-28 — complete; reference study, line refs may drift with upstream.
|
|
|
|
## 1. MCP plumbing
|
|
|
|
Code in `internal/agent/tools/mcp/` (NOT `internal/client/` = daemon RPC).
|
|
Uses official SDK `modelcontextprotocol/go-sdk/mcp`.
|
|
Config: `crush.json` top-level `"mcp"` map name->MCPConfig
|
|
(`config/config.go:185-252`, :383/:723):
|
|
```json
|
|
{ "mcp": {
|
|
"local": { "type": "stdio", "command": "npx", "args": ["-y","pkg"], "env": {"K":"V"} },
|
|
"remote": { "type": "http", "url": "http://h/mcp", "headers": {"Authorization":"Bearer $TOKEN"} },
|
|
"legacy": { "type": "sse", "url": "https://h/sse" } } }
|
|
```
|
|
- `type`: `stdio`|`sse`|`http` (http=streamable). Extras: `disabled`,
|
|
`enabled_tools`/`disabled_tools`, `timeout` secs (dflt 10s, 30s OAuth),
|
|
`sessionless`, `oauth*`. `$VAR`/`$(cmd)` expansion: config.go:438-528.
|
|
PORT FORMAT VERBATIM for KNEL-AIMiddleware compat.
|
|
Transports (`mcp/init.go:1035-1195` createTransport): stdio ->
|
|
`mcp.CommandTransport`; env=os.Environ()+resolved; child in own process
|
|
group, group SIGKILL + 5s WaitDelay (`process_unix.go:22-40`). PORT:
|
|
process-group kill or zombies leak. http -> `mcp.StreamableClientTransport`;
|
|
static headers via `headerRoundTripper` (init.go:1197-1206). sse ->
|
|
`SSEClientTransport` (legacy, keep for compat).
|
|
Health/restart: `getOrRenewClient` (init.go:659-771) — Ping fast path; on
|
|
failure teardown->reconnect->re-register tools. Per-server renewal mutex +
|
|
generation counters discard stale connects. Config-change reconcile
|
|
single-flighted (`mcp/lifecycle.go:45-125`). Init: per-server goroutines +
|
|
WaitGroup; `WaitForInit` gates NON-interactive runs only (headless waits,
|
|
TUI proceeds lazily).
|
|
Tool registration: discovery via `ListTools` on connect + list-changed
|
|
subscription (`mcp/tools.go:147-169`). Naming `mcp_<server>_<tool>` SINGLE
|
|
underscores (`agent/tools/mcp-tools.go:58-60`). Adapter maps JSON-Schema
|
|
properties/required -> ToolInfo (mcp-tools.go:70-97); results map
|
|
text/image/audio, base64-normalized (`tools.go:54-108`). Routing: RunTool ->
|
|
client lookup -> CallTool; auto-renews first. Aggregation: per-run palette =
|
|
built-ins + `GetMCPTools`, filtered per-agent by AllowedMCP map
|
|
(`agent/coordinator.go:768-790`).
|
|
Go port: adopt `go-sdk/mcp`; copy config schema, naming convention,
|
|
renewal-with-generations, WaitForInit gate. Skip: Claude "channel" push
|
|
hack (`mcp/channel.go`), docker-mcp gateway (`config/docker_mcp.go`).
|
|
|
|
## 2. Session/persistence
|
|
|
|
Single SQLite DB `{dataDir}/crush.db` (default `.crush/` per project,
|
|
discovered cwd-up-to-git-root, `config/load.go:574-582`). Sibling
|
|
`crush.lock` flock + owner JSON stops concurrent processes
|
|
(`db/datadirlock.go`). Schema (`db/migrations/`, goose, sqlc queries in
|
|
`db/sql/`): `sessions(uuid, parent_id, title, message_count, tokens, cost,
|
|
todos JSON, summary_message_id)`, `messages(id, session_id, role, parts
|
|
JSON, model, provider, finished_at)`, `files` (snapshots), `read_files`
|
|
(tracker). Parts = `{"type","data"}` array (`message/content.go:56-143`).
|
|
Pragmas WAL+busy_timeout=30s; `SetMaxOpenConns(1)` avoids SQLITE_NOTADB
|
|
(`db/connect.go:18-41,137-142`). Short user ID = XXH3 hex of UUID
|
|
(`session/session.go:28-32`).
|
|
`-C/--continue` vs `-s/--session` (mutually exclusive, `cmd/root.go:63`):
|
|
Continue = GetLast `ORDER BY updated_at DESC` skipping agent-tool/child
|
|
sessions (`app/app.go:238-263`). ID resolve: exact UUID, else XXH3 prefix
|
|
scan; 0 hits=not found, >1=ambiguous (`root.go:941-967`). Resume restores
|
|
messages+parts, file snapshots, read-files, model from last assistant msg;
|
|
NOT restored: permissions, LSP/MCP (re-init).
|
|
Crash safety: 33ms debounced writes, sync flush on terminal events
|
|
(`message/message.go:218-302`); read-time repair in `agent/agent.go:1527-1708`
|
|
— drops orphaned tool results, SYNTHESIZES results for orphaned tool_calls
|
|
(interrupted sessions still pass API validation), skips empty cancelled msgs.
|
|
PORT THIS — it is what makes resume robust. Harness v1 (bounded turns):
|
|
JSONL per session is acceptable; what matters is the repair pass and
|
|
finished_at-only-on-Finish-part semantics.
|
|
|
|
## 3. Provider layer
|
|
|
|
No in-house LLM client: calls delegate to external `charm.land/fantasy`;
|
|
catalog from `charm.land/catwalk`. Neither vendored here — harness owns
|
|
this layer against LiteLLM (openai-compat): port the CONFIG CONTRACT, not code.
|
|
- Catalog: embedded + remote fetch (catwalk.charm.land, ETag) w/ fallback
|
|
remote->cache->embedded (`config/catwalk.go:37-90`, `provider.go:37-90`;
|
|
cache `~/.local/share/crush/providers.json`, atomic write). Harness:
|
|
static embedded catalog, no remote sync.
|
|
- Layering low->high (`config/load.go:926-965`): /etc/crush ->
|
|
~/.config/crush -> ~/.local/share/crush -> project walk (.crushrc|crushrc|
|
|
.crush.json|crush.json, cwd up to worktree root) -> workspace dataDir copy
|
|
LAST. Deep-merge via go-jsons.
|
|
- Catalog x user (`load.go:213-365`): `providers.<id>` overrides
|
|
base_url/api_key/extra_headers; user `models[]` PREPENDED, wins by ID;
|
|
`disable_default_providers` nukes catalog. Routing: `parseModelStr`
|
|
(`app/provider.go:15-27`) — first `/`-segment vs provider IDs, else
|
|
whole string = model ID; ambiguous -> demand `provider/model`.
|
|
Defaults from catalog DefaultLarge/SmallModelID (`load.go:716-789`);
|
|
per-model overrides (max_tokens/temperature/reasoning) `load.go:826-899`;
|
|
provider switch on Type `agent/coordinator.go:806-1161`.
|
|
- Retries: INSIDE fantasy, not this repo. Crush only observes: `OnRetry`
|
|
resets streamed content so retry doesn't concatenate (`agent/agent.go:931-941`);
|
|
`OnAuthRefresh` = transparent 401 retry (:127). PORT: harness-layer retry
|
|
(429/5xx/transport backoff) + reset partial assistant buffer first.
|
|
Stream-truncation surfacing (z.ai "Message content shorter than read
|
|
bytes"): stream errors classified at `agent/agent.go:1067-1189`;
|
|
`IsTransportError` -> FinishReasonError w/ wrapped err (:1177) — unexpected
|
|
EOF lands here. In headless, the z.ai-class error is DETECTED at
|
|
`app/app.go:414-416` (reconciles streamed bytes vs final msg, fails the
|
|
run, exit 1). Zero-usage fallback: `agent/usage_fallback.go`. Harness v1:
|
|
on EOF/short-read, retry request (per DESIGN retry/backoff) and resume
|
|
turn from persisted messages; reuse delta-cursor print (app.go:406-431).
|
|
|
|
## 4. Headless mode
|
|
|
|
No `-p`/`--print` exists. Headless = `crush run|r [prompt...]`
|
|
(`cmd/run.go:31-169`). Prompt = args joined + stdin (pipe/file, not TTY)
|
|
PREPENDED (`root.go:918-935`); empty -> error. Output: plain markdown to
|
|
stdout; spinner only if stderr is TTY. Flags: -q, -v, -m provider/model,
|
|
--small-model, -s, -C. Two paths: in-process (`app.RunNonInteractive`,
|
|
`app/app.go:267-438`) or client/server SSE (`run.go:334-455`). Port:
|
|
- Delta printing: track messageReadBytes per msg id, print only
|
|
`content[readBytes:]` (app.go:406-431). Server path keys completion on
|
|
RunComplete event by runID, NOT message finish (tool_use mid-turn
|
|
regression, run.go:385-397).
|
|
- Permissions headless: session auto-approved wholesale
|
|
(`AutoApproveSession` app.go:349; permission.go:280-284) — never blocks;
|
|
interactive-only tools excluded (`initCoderAgent(false)` app.go:674-703).
|
|
Harness: invert — per-vertical allowlist, deny-default, no auto-approve.
|
|
- Exit codes: 0/1 only. 1 = RunE error (no prompt, no providers, agent
|
|
error, run failed); CANCEL returns 0 (app.go:398-401); tool failures do
|
|
NOT affect exit. Harness: distinct codes gate/budget/agent/tool.
|
|
|
|
## 5. OAuth (deferred v1 — code map)
|
|
|
|
- Hyper device-code flow: `oauth/hyper/device.go` (:40 initiate, :87 poll,
|
|
:153 refresh-exchange). Copilot device flow + GH->Copilot token exchange:
|
|
`oauth/copilot/oauth.go` (:36,:69,:149), disk import `copilot/disk.go`,
|
|
spoof headers `copilot/http.go`. Login CLI: `cmd/login.go`, `cmd/logout.go`.
|
|
- MCP OAuth 2.1 auth-code+PKCE, dynamic client registration (RFC 7591),
|
|
PR metadata (8414/8707): `oauth/mcp/handler.go:110-219`. Loopback
|
|
`/callback`, ports 40704-40713 first-free (pin via `oauth_callback_port`);
|
|
listener bound only during in-flight auth (:487); concurrent 401s deduped
|
|
to one browser tab (`authFlight` :420). Tokens -> config key
|
|
`mcp.<name>.oauth_token`; save-on-change wrapper
|
|
`oauth/mcp/savingtokensource.go`. Browser suppressed non-interactive
|
|
(`ErrInteractiveAuthRequired` handler.go:32). If harness needs OAuth,
|
|
port THIS flow (MCP); provider OAuth moot behind LiteLLM.
|
|
Token plumbing: access token stored AS `providers.<id>.api_key`
|
|
(`config/store.go:564-595`); refresh singleflight + cross-process flock
|
|
`<datadir>/locks/<provider>.refresh.lock`, adopts peer-rotated token
|
|
(store.go:658-903). Token model `oauth/token.go`. Anthropic Claude-
|
|
subscription OAuth: REMOVED (`config/load.go:306-314`).
|
|
|
|
## 6. Reading list + gotchas
|
|
|
|
Top 10 to read in detail during build:
|
|
1. `internal/agent/tools/mcp/init.go` (1332 ln) — sessions, transports,
|
|
renewal; MCP port blueprint.
|
|
2. `internal/agent/tools/mcp-tools.go` + `mcp/tools.go` — tool adapter,
|
|
naming, filtering, result mapping.
|
|
3. `internal/agent/agent.go` (2288 ln) — stream callbacks, error
|
|
classification, prompt repair (:1527-1708).
|
|
4. `internal/app/app.go` — RunNonInteractive, resolveSession, delta print.
|
|
5. `internal/cmd/run.go` + `cmd/root.go` — headless flags, stdin handling,
|
|
config discovery walk.
|
|
6. `internal/config/load.go` — config layering/merge, model resolution.
|
|
7. `internal/message/message.go` + `content.go` — debounced persistence,
|
|
part model.
|
|
8. `internal/session/session.go` + `db/sql/` — session service + queries.
|
|
9. `internal/permission/permission.go` — precedence chain (skip ->
|
|
allowed_tools -> hook -> auto-approve -> interactive).
|
|
10. `internal/agent/coordinator.go` — tool palette assembly, provider
|
|
construction, AllowedMCP filtering.
|
|
Gotchas:
|
|
1. `internal/client/` is daemon RPC, not LLM/MCP — don't port by name.
|
|
LLM lives in external `charm.land/fantasy` (not vendored here).
|
|
2. MCP tool names use SINGLE underscores (`mcp_server_tool`); double-
|
|
underscore assumption breaks KNEL compat.
|
|
3. stdio MCP children need own process group + group kill; plain Kill
|
|
orphans grandchildren (`process_unix.go`).
|
|
4. Retries concatenate partial stream output unless the assistant buffer
|
|
is reset first (`agent.go:931-941` OnRetry pattern).
|
|
5. Resume correctness lives in read-time repair (orphaned tool_call/result
|
|
synthesis), not write-time transactions — skipping it makes every
|
|
interrupted session permanently API-invalid.
|