Add verbatim AGPLv3 LICENSE + README
This commit is contained in:
@@ -0,0 +1,181 @@
|
||||
# Porting notes: crush (reference/crush @ internal/)
|
||||
|
||||
Scope for MOPAC harness: MCP client, sessions/resume, provider routing,
|
||||
headless loop; OAuth deferred (LiteLLM key suffices). Paths relative to
|
||||
`reference/crush/`; line refs from study 2026-08-28, drift possible.
|
||||
|
||||
## 1. MCP plumbing
|
||||
|
||||
Code in `internal/agent/tools/mcp/` (NOT `internal/client/` = daemon RPC).
|
||||
Uses official SDK `modelcontextprotocol/go-sdk/mcp`.
|
||||
Config: `crush.json` top-level `"mcp"` map name->MCPConfig
|
||||
(`config/config.go:185-252`, :383/:723):
|
||||
```json
|
||||
{ "mcp": {
|
||||
"local": { "type": "stdio", "command": "npx", "args": ["-y","pkg"], "env": {"K":"V"} },
|
||||
"remote": { "type": "http", "url": "http://h/mcp", "headers": {"Authorization":"Bearer $TOKEN"} },
|
||||
"legacy": { "type": "sse", "url": "https://h/sse" } } }
|
||||
```
|
||||
- `type`: `stdio`|`sse`|`http` (http=streamable). Extras: `disabled`,
|
||||
`enabled_tools`/`disabled_tools`, `timeout` secs (dflt 10s, 30s OAuth),
|
||||
`sessionless`, `oauth*`. `$VAR`/`$(cmd)` expansion: config.go:438-528.
|
||||
PORT FORMAT VERBATIM for KNEL-AIMiddleware compat.
|
||||
Transports (`mcp/init.go:1035-1195` createTransport): stdio ->
|
||||
`mcp.CommandTransport`; env=os.Environ()+resolved; child in own process
|
||||
group, group SIGKILL + 5s WaitDelay (`process_unix.go:22-40`). PORT:
|
||||
process-group kill or zombies leak. http -> `mcp.StreamableClientTransport`;
|
||||
static headers via `headerRoundTripper` (init.go:1197-1206). sse ->
|
||||
`SSEClientTransport` (legacy, keep for compat).
|
||||
Health/restart: `getOrRenewClient` (init.go:659-771) — Ping fast path; on
|
||||
failure teardown->reconnect->re-register tools. Per-server renewal mutex +
|
||||
generation counters discard stale connects. Config-change reconcile
|
||||
single-flighted (`mcp/lifecycle.go:45-125`). Init: per-server goroutines +
|
||||
WaitGroup; `WaitForInit` gates NON-interactive runs only (headless waits,
|
||||
TUI proceeds lazily).
|
||||
Tool registration: discovery via `ListTools` on connect + list-changed
|
||||
subscription (`mcp/tools.go:147-169`). Naming `mcp_<server>_<tool>` SINGLE
|
||||
underscores (`agent/tools/mcp-tools.go:58-60`). Adapter maps JSON-Schema
|
||||
properties/required -> ToolInfo (mcp-tools.go:70-97); results map
|
||||
text/image/audio, base64-normalized (`tools.go:54-108`). Routing: RunTool ->
|
||||
client lookup -> CallTool; auto-renews first. Aggregation: per-run palette =
|
||||
built-ins + `GetMCPTools`, filtered per-agent by AllowedMCP map
|
||||
(`agent/coordinator.go:768-790`).
|
||||
Go port: adopt `go-sdk/mcp`; copy config schema, naming convention,
|
||||
renewal-with-generations, WaitForInit gate. Skip: Claude "channel" push
|
||||
hack (`mcp/channel.go`), docker-mcp gateway (`config/docker_mcp.go`).
|
||||
|
||||
## 2. Session/persistence
|
||||
|
||||
Single SQLite DB `{dataDir}/crush.db` (default `.crush/` per project,
|
||||
discovered cwd-up-to-git-root, `config/load.go:574-582`). Sibling
|
||||
`crush.lock` flock + owner JSON stops concurrent processes
|
||||
(`db/datadirlock.go`). Schema (`db/migrations/`, goose, sqlc queries in
|
||||
`db/sql/`): `sessions(uuid, parent_id, title, message_count, tokens, cost,
|
||||
todos JSON, summary_message_id)`, `messages(id, session_id, role, parts
|
||||
JSON, model, provider, finished_at)`, `files` (snapshots), `read_files`
|
||||
(tracker). Parts = `{"type","data"}` array (`message/content.go:56-143`).
|
||||
Pragmas WAL+busy_timeout=30s; `SetMaxOpenConns(1)` avoids SQLITE_NOTADB
|
||||
(`db/connect.go:18-41,137-142`). Short user ID = XXH3 hex of UUID
|
||||
(`session/session.go:28-32`).
|
||||
`-C/--continue` vs `-s/--session` (mutually exclusive, `cmd/root.go:63`):
|
||||
Continue = GetLast `ORDER BY updated_at DESC` skipping agent-tool/child
|
||||
sessions (`app/app.go:238-263`). ID resolve: exact UUID, else XXH3 prefix
|
||||
scan; 0 hits=not found, >1=ambiguous (`root.go:941-967`). Resume restores
|
||||
messages+parts, file snapshots, read-files, model from last assistant msg;
|
||||
NOT restored: permissions, LSP/MCP (re-init).
|
||||
Crash safety: 33ms debounced writes, sync flush on terminal events
|
||||
(`message/message.go:218-302`); read-time repair in `agent/agent.go:1527-1708`
|
||||
— drops orphaned tool results, SYNTHESIZES results for orphaned tool_calls
|
||||
(interrupted sessions still pass API validation), skips empty cancelled msgs.
|
||||
PORT THIS — it is what makes resume robust. Harness v1 (bounded turns):
|
||||
JSONL per session is acceptable; what matters is the repair pass and
|
||||
finished_at-only-on-Finish-part semantics.
|
||||
|
||||
## 3. Provider layer
|
||||
|
||||
No in-house LLM client: calls delegate to external `charm.land/fantasy`;
|
||||
catalog from `charm.land/catwalk`. Neither vendored here — harness owns
|
||||
this layer against LiteLLM (openai-compat): port the CONFIG CONTRACT, not code.
|
||||
- Catalog: embedded + remote fetch (catwalk.charm.land, ETag) w/ fallback
|
||||
remote->cache->embedded (`config/catwalk.go:37-90`, `provider.go:37-90`;
|
||||
cache `~/.local/share/crush/providers.json`, atomic write). Harness:
|
||||
static embedded catalog, no remote sync.
|
||||
- Layering low->high (`config/load.go:926-965`): /etc/crush ->
|
||||
~/.config/crush -> ~/.local/share/crush -> project walk (.crushrc|crushrc|
|
||||
.crush.json|crush.json, cwd up to worktree root) -> workspace dataDir copy
|
||||
LAST. Deep-merge via go-jsons.
|
||||
- Catalog x user (`load.go:213-365`): `providers.<id>` overrides
|
||||
base_url/api_key/extra_headers; user `models[]` PREPENDED, wins by ID;
|
||||
`disable_default_providers` nukes catalog.
|
||||
- Routing: `parseModelStr` (`app/provider.go:15-27`) — first `/`-segment
|
||||
vs provider IDs, else whole string is model ID; multi-provider hit ->
|
||||
error demanding `provider/model`. Defaults from catalog
|
||||
DefaultLarge/SmallModelID (`load.go:716-789`); per-model overrides
|
||||
(max_tokens, temperature, reasoning) `load.go:826-899`. Provider
|
||||
construction switch on Type `agent/coordinator.go:806-1161`.
|
||||
- Retries: INSIDE fantasy, not this repo. Crush only observes: `OnRetry`
|
||||
resets streamed content so retry doesn't concatenate (`agent/agent.go:931-941`);
|
||||
`OnAuthRefresh` = transparent 401 retry (:127). PORT: harness-layer retry
|
||||
(429/5xx/transport backoff) + reset partial assistant buffer first.
|
||||
Stream-truncation surfacing (z.ai "Message content shorter than read
|
||||
bytes"): stream errors classified at `agent/agent.go:1067-1189`;
|
||||
`IsTransportError` -> FinishReasonError w/ wrapped err (:1177) — unexpected
|
||||
EOF lands here. In headless, the z.ai-class error is DETECTED at
|
||||
`app/app.go:414-416` (reconciles streamed bytes vs final msg, fails the
|
||||
run, exit 1). Zero-usage fallback: `agent/usage_fallback.go`. Harness v1:
|
||||
on EOF/short-read, retry request (per DESIGN retry/backoff) and resume
|
||||
turn from persisted messages; reuse delta-cursor print (app.go:406-431).
|
||||
|
||||
## 4. Headless mode
|
||||
|
||||
No `-p`/`--print` exists. Headless = `crush run|r [prompt...]`
|
||||
(`cmd/run.go:31-169`). Prompt = args joined + stdin (pipe/file, not TTY)
|
||||
PREPENDED (`root.go:918-935`); empty -> error. Output: plain markdown to
|
||||
stdout; spinner only if stderr is TTY. Flags: -q, -v, -m provider/model,
|
||||
--small-model, -s, -C. Two paths: in-process (`app.RunNonInteractive`,
|
||||
`app/app.go:267-438`) or client/server SSE (`run.go:334-455`). Port:
|
||||
- Delta printing: track messageReadBytes per msg id, print only
|
||||
`content[readBytes:]` (app.go:406-431). Server path keys completion on
|
||||
RunComplete event by runID, NOT message finish (tool_use mid-turn
|
||||
regression, run.go:385-397).
|
||||
- Permissions headless: session auto-approved wholesale
|
||||
(`AutoApproveSession` app.go:349; permission.go:280-284) — never blocks;
|
||||
interactive-only tools excluded (`initCoderAgent(false)` app.go:674-703).
|
||||
Harness: invert — per-vertical allowlist, deny-default, no auto-approve.
|
||||
- Exit codes: 0/1 only. 1 = RunE error (no prompt, no providers, agent
|
||||
error, run failed); CANCEL returns 0 (app.go:398-401); tool failures do
|
||||
NOT affect exit. Harness: distinct codes gate/budget/agent/tool.
|
||||
|
||||
## 5. OAuth (deferred v1 — code map)
|
||||
|
||||
- Hyper device-code flow: `oauth/hyper/device.go` (:40 initiate, :87 poll,
|
||||
:153 refresh-exchange). Copilot device flow + GH->Copilot token exchange:
|
||||
`oauth/copilot/oauth.go` (:36,:69,:149), disk import `copilot/disk.go`,
|
||||
spoof headers `copilot/http.go`. Login CLI: `cmd/login.go`, `cmd/logout.go`.
|
||||
- MCP OAuth 2.1 auth-code+PKCE, dynamic client registration (RFC 7591),
|
||||
PR metadata (8414/8707): `oauth/mcp/handler.go:110-219`. Loopback
|
||||
`/callback`, ports 40704-40713 first-free (pin via `oauth_callback_port`);
|
||||
listener bound only during in-flight auth (:487); concurrent 401s deduped
|
||||
to one browser tab (`authFlight` :420). Tokens -> config key
|
||||
`mcp.<name>.oauth_token`; save-on-change wrapper
|
||||
`oauth/mcp/savingtokensource.go`. Browser suppressed non-interactive
|
||||
(`ErrInteractiveAuthRequired` handler.go:32). If harness needs OAuth,
|
||||
port THIS flow (MCP); provider OAuth moot behind LiteLLM.
|
||||
Token plumbing: access token stored AS `providers.<id>.api_key`
|
||||
(`config/store.go:564-595`); refresh singleflight + cross-process flock
|
||||
`<datadir>/locks/<provider>.refresh.lock`, adopts peer-rotated token
|
||||
(store.go:658-903). Token model `oauth/token.go`. Anthropic Claude-
|
||||
subscription OAuth: REMOVED (`config/load.go:306-314`).
|
||||
|
||||
## 6. Reading list + gotchas
|
||||
|
||||
Top 10 to read in detail during build:
|
||||
1. `internal/agent/tools/mcp/init.go` (1332 ln) — sessions, transports,
|
||||
renewal; MCP port blueprint.
|
||||
2. `internal/agent/tools/mcp-tools.go` + `mcp/tools.go` — tool adapter,
|
||||
naming, filtering, result mapping.
|
||||
3. `internal/agent/agent.go` (2288 ln) — stream callbacks, error
|
||||
classification, prompt repair (:1527-1708).
|
||||
4. `internal/app/app.go` — RunNonInteractive, resolveSession, delta print.
|
||||
5. `internal/cmd/run.go` + `cmd/root.go` — headless flags, stdin handling,
|
||||
config discovery walk.
|
||||
6. `internal/config/load.go` — config layering/merge, model resolution.
|
||||
7. `internal/message/message.go` + `content.go` — debounced persistence,
|
||||
part model.
|
||||
8. `internal/session/session.go` + `db/sql/` — session service + queries.
|
||||
9. `internal/permission/permission.go` — precedence chain (skip ->
|
||||
allowed_tools -> hook -> auto-approve -> interactive).
|
||||
10. `internal/agent/coordinator.go` — tool palette assembly, provider
|
||||
construction, AllowedMCP filtering.
|
||||
Gotchas:
|
||||
1. `internal/client/` is daemon RPC, not LLM/MCP — don't port by name.
|
||||
LLM lives in external `charm.land/fantasy` (not vendored here).
|
||||
2. MCP tool names use SINGLE underscores (`mcp_server_tool`); double-
|
||||
underscore assumption breaks KNEL compat.
|
||||
3. stdio MCP children need own process group + group kill; plain Kill
|
||||
orphans grandchildren (`process_unix.go`).
|
||||
4. Retries concatenate partial stream output unless the assistant buffer
|
||||
is reset first (`agent.go:931-941` OnRetry pattern).
|
||||
5. Resume correctness lives in read-time repair (orphaned tool_call/result
|
||||
synthesis), not write-time transactions — skipping it makes every
|
||||
interrupted session permanently API-invalid.
|
||||
Reference in New Issue
Block a user