Porting notes: maki (bash permission scopes, token reduction, subagent tiers)

Maps the maki reference to the harness build: tree-sitter scope extraction
and the 4-way rule matcher for the permission gate, index/tool_search token
reduction, weak/medium/strong subagent tiers with model clamping, and the
headless spawn model that the conductor loop follows. Gotchas preserved:
headless prompt==deny for complex bash, matcher suffix order, deny-exact vs
allow-broad asymmetry, symlink-aware path checks.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2

docs/PORTING-NOTES-maki.md

@@ -0,0 +1,180 @@
+# Porting notes: maki (reference/maki @ plugins/ + maki-agent/)
+
+Scope for MOPAC harness: tree-sitter bash permission parsing, token-reduction
+(index / MCP tool_search), subagent tiers, headless loop. Paths relative to
+`reference/maki/`; line refs from study 2026-08-28, drift possible. Rust core
++ Lua tool plugins; Lua API mirrors tree-sitter via maki-lua.
+
+## 1. Tree-sitter bash permission parsing
+
+Two halves: Lua plugin extracts "scopes" from command text; Rust
+PermissionManager matches them against allow/deny rules.
+
+**Scope extraction** — `plugins/bash/init.lua:275-296` (`permission_scopes`):
+- Parse `command` with tree-sitter-bash. Parse error (`root:has_error()`)
+  OR any node (recursive) of type `command_substitution |
+  process_substitution | subshell | arithmetic_expansion` (is_complex,
+  :176-193) -> `{scopes=[whole command], force_prompt=true}`. Never
+  decompose `$(...)`; force prompt instead. Else `collect_commands`
+  (:209-235): `program`/`list` -> recurse (flattens `&&`/`||`/`;`);
+  `pipeline` -> one scope per named child; leaf types `command,
+  redirected_statement, negated_command, subshell, compound_statement,
+  if/while/for/case, function_definition, c_style_for_statement` ->
+  trimmed node text. `git diff && rm -rf /` -> ["git diff","rm -rf /"];
+  `a | b` -> [a, b]. `cd X && cmd` -> workdir+cmd hint (:37-49); the
+  `workdir` input param is preferred (description enforces).
+
+**Rule engine** — `maki-agent/src/permissions.rs`:
+- `check_inner` (:308-417): per scope, iterate rules session -> config ->
+  builtin -> plugin. ANY matching deny => whole call Denied immediately.
+  Matching allow => covered. Uncovered => NeedsPrompt (or per-tool/global
+  default Deny|Allow|Prompt). `force_prompt` => prompt all scopes despite
+  allows. Yolo => Allowed but only AFTER deny scan (:356): deny beats yolo.
+- `scope_matches` (:680-706), order critical: `pfx/**` => normalized path
+  prefix (absolutize + `incremental_canonicalize`, symlink-aware);
+  `cmd *` => equals `cmd` or starts `"cmd "` (word boundary: `pwd *`
+  matches `pwd`/`pwd -L`, not `pwdx`); `pfx*` => raw prefix; else exact.
+  Universal `*`,`/**`,`/*` short-circuit every scope.
+- Builtin rules (:32-48): file-write tools allow `cwd/**`; `task` `*`;
+  nothing for bash. Answers (:113-183): allow once/session/always_local/
+  always_global, deny(+guidance)/always. `apply_decision` (:495-545)
+  GENERALIZES before persisting: bash -> first token + `" *"`
+  (`cargo test -p x` -> `cargo *`, :737-740); file writes -> parent
+  `/**`; MCP -> `*`. Deny persists EXACT scope (tight deny, broad
+  allow — deliberate).
+- `enforce` (:548-633): NeedsPrompt => PermissionRequest event + await on
+  response channel; NO channel (headless) => deny (user_abort). Headless
+  needs rules/yolo covering everything it will do.
+
+**Go port**: `github.com/tree-sitter/tree-sitter-bash`; node kinds are
+identical — port COMPLEX_TYPES/LEAF_COMMAND_TYPES verbatim. Rules as
+`{tool, scopePattern, effect}` + the 4-way suffix matcher. Keep
+scopes=segments: it's what makes `allow bash cargo *` safe under
+chaining. Skip: plugin rule store, OTel, plan-path auto-allow.
+
+## 2. Token reduction
+
+Rationale: `site/docs/content/token-economy/_index.md` — every result is
+re-paid each turn; attack result size AND round-trip count.
+
+**index tool** — `plugins/index/{init,indexer}.lua`, 30+ extractors in
+`plugins/index/lang/*.lua`: sections are `imports: [lo-hi]` with paths as
+a trie tree; module/type names as wrapped CSV; signatures one per line
+suffixed ` [start-end]`; test lines separate; module doc range. ~70-90%
+smaller than full read. Contract: index first, then `read offset/limit`
+the ranges (hints init.lua:144-152 + description). 2MB cap. PORT: yes,
+high value, pure tree-sitter. Start: go, rust, python, ts, markdown,
+json/yaml fallback. Keep the `text [lo-hi]` suffix format.
+
+**MCP tool_search** — `maki-agent/src/mcp/mod.rs`:
+- Defer when server's non-`always_load` tools > `defer_tools` (dflt 10;
+  below that defs cost less than a catalog). Hidden tools become ONE
+  synthetic `tool_search` whose description embeds a name-only catalog
+  per server (:1199-1241). 117-tool server -> 1 def. `search_tools`
+  (:400-485): name hit=2, description/schema-prop hit=1; exact name
+  always wins; top 5; model-origin search inserts matches into session
+  `loaded` set -> full defs on NEXT request; overflow <=20 names.
+  Calling a deferred tool by name also loads it (:506).
+- Session-private `loaded` (:306-344); resume seeds from history tool_use
+  names containing `__`. `extend_tools` recomputed EVERY request
+  (`agent/run.rs:287-299`) — caching freezes the catalog. PORT: yes, with
+  MCP client (crush notes §1); per-turn recompute, per-session loads,
+  resume-seeding are all cheap and load-bearing.
+
+**code_execution** — `plugins/code_execution/init.lua` + `maki-interpreter`
+(pydantic `monty` embedded Rust Python): tools as async fns, chained/
+filtered calls stay in sandbox, only prints enter context; custom
+`gather()` keeps sibling results when one call fails (:34-50); 30s budget
+(tool-call await excluded), 50MB, no fs/network, fresh sandbox; audience-
+gated so read-only subs can't reach edit/write through it (:171-184).
+SKIP: violates 100%-Go HARD RULE. Substitute now = `batch` tool
+(`plugins/batch/init.lua`: N independent calls, one turn/request); Go
+filter DSL later; keep the "only compact output enters context" rule.
+
+**Also portable cheaply**: global truncation (max lines/bytes per tool;
+overlong grep lines skipped); partial-output preservation on Esc/deadline
+(bash keeps streamed lines, task keeps half transcript) — for us: on turn
+timeout return what the tool produced so far, tagged partial.
+## 3. Subagent model -> TASK/dispatch
+
+`plugins/task/init.lua` + audiences `maki-agent/src/tools/registry.rs:21-48`:- Types: `research` (default, read-only) vs `general` (full). Enforcement
+  = audience bitflags (MAIN, RESEARCH_SUB, GENERAL_SUB, INTERPRETER,
+  WORKFLOW). edit/write declare `{main, general_sub, interpreter}`
+  (edit:254+, write:33) so research lacks them; `research.md` prompt
+  doubles down. bash/read/grep default ALL — research CAN bash (git log)
+  but shares parent PermissionManager (Arc clone, api/agent.rs:589), so
+  write-ish bash still denies/prompts. Advertised == callable via one
+  shared filter fn. Tiers: weak|medium|strong (Haiku/Sonnet/Opus, strong
+  ~5x medium). `resolve_model_from_ctx` (api/agent.rs:44-63) CLAMPS
+  `requested.min(ctx.tier)` — subagents only go cheaper, never escalate.
+  Tier->model: registry overrides -> discovered -> positional
+  (Strong=models[0], Medium=[1], Weak=[2]) (`maki-providers/src/
+  model_registry.rs:171-238`); pricing per provider in providers/*.
+  Concurrency: process-wide semaphore dflt 8; pcall so error can't leak
+  permit (:178-239).
+- Structured output: optional JSON Schema -> session-local
+  `structured_output` tool; invalid input = inline error model fixes
+  same-run; <=2 nudges, <=3 schema errors. Isolation: fresh session,
+  prompt inlined, single result string (or validated JSON) back;
+  description demands "concise summaries with file:line refs".
+
+**Map to MOPAC**: grinder dispatch = weak read-only research; P1/feature
+= medium general; founder gate stays human. structured_output schema ==
+REPORT file contract. Semaphore == class-aware LLM slot GATE. Skip the
+Lua layer: task tool = Go fn spawning a nested bounded agent run with
+filtered toolset + clamped model alias.
+
+## 4. Headless / --print + session parallelism
+- `src/print.rs:140-362`: prompt arg or stdin; output text|json|stream-
+  json. JSON = Claude-Code-style events: `system/init` (cwd, session_id,
+  tools, model), `assistant` per turn (content, usage, model,
+  parent_tool_use_id for subagent turns), `user` tool results,
+  `system/api_retry`, final `result` {subtype, duration_ms, num_turns,
+  result, stop_reason, session_id, total_cost_usd, usage}. Cost summed as
+  turns land (rates move mid-run).- `maki-agent/src/headless.rs`: `spawn` (:164-282) one-shot agent on smol
+  task: own PermissionManager, fresh FileReadTracker, event channel,
+  session persisted per turn. `question` excluded (:182). NO response
+  channel => uncovered scopes deny (§1). `spawn_interactive` (:318+)
+  adds input/answer/cancel/model channels (ACP/SDK, `src/sdk_mode.rs`).
+  SessionMailbox drained before each user turn (run.rs:266). Each spawn
+  self-contained => N concurrent sessions trivial (SDK mode does it);
+  subagent semaphore is the only global cap. Resumable by id from state dir.
+
+**Port**: this IS our core loop — bounded headless run, JSON events to
+log, session file per turn for resume/audit. Copy the result envelope
+(cost/usage/stop_reason) so drip-chaining can budget-gate.
+## 5. Re-read during build (top 10)
+
+1. `plugins/bash/init.lua` — scopes, exec, cd-hint, partials.
+2. `maki-agent/src/permissions.rs` — engine+matcher+generalization; tests :776+ are the spec.
+3. `maki-agent/src/mcp/mod.rs:358-510,1199+` — tool_search catalog.
+4. `plugins/index/indexer.lua` + `lang/go.lua` — skeleton format.
+5. `plugins/task/init.lua` — subagent lifecycle, nudges, semaphore.
+6. `maki-lua/src/api/agent.rs` — model clamp, session, tools fn.
+7. `maki-agent/src/headless.rs` — spawn/spawn_interactive wiring.
+8. `src/print.rs` — event wire format + result envelope.
+9. `maki-agent/src/tools/registry.rs` — audiences + definitions filter.
+10. `maki-agent/src/agent/run.rs` — loop, per-turn request_tools, mailbox.
+
+## 6. Gotchas (top 5)
+
+1. Headless prompt==deny: parse-error/complex bash forces prompt, and
+   headless has no channel => deny. Rules must be prefix scopes (`cargo
+   *`, `git *`) or commands stay trivial; test compound commands against
+   the decomposer, not just single words.
+2. Matcher order: try `/**` and `" *"` suffixes BEFORE bare `*` or a
+   plain prefix swallows them; `*`/`/**`/`/*` are universal (deny `*`
+   blocks every tool). `pwd *` must not match `pwdx`.
+3. Never cache tool definitions: MCP catalog + loads recomputed every
+   request; advertise==callable must hold for audiences too, or a
+   read-only sub gets edit schemas it can't call — or calls them.
+4. Generalization asymmetry: allow-always broadens (bash first word,
+   writes to parent dir, MCP `*`), deny stays exact. Copy deliberately —
+   broad denies brick sessions, exact allows spam prompts.
+5. Path scopes need symlink-aware canonicalization incrementally
+   left-to-right BEFORE `..` applies (`physical_boundary_check`,
+   permissions.rs:720-735); lexical prefix matching is smugglable via
+   `symlink/..`. Same care for "writes confined to declared roots".
+Bonus: tier clamp silently downgrades (medium parent + strong -> medium)
+— that's the cost control, don't "fix" it. tree-sitter-bash node names
+shift across grammar majors (e.g. `command_name`); pin your test version.