mrcharles
9042f99ea6
Records what the skeleton does (config, routing, single-shot loop, intake,
gated bash, REPORT writeback, test coverage), the one missing input for the
live MVP demo (HARNESS_LITELLM_KEY), the stubbed items (redmine notes,
budget gate, bw refs, streaming/resume, session persistence), and the
ordered phase 3 work.
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
84 lines
4.6 KiB
Markdown
84 lines
4.6 KiB
Markdown
# REPORT — harness skeleton (build phase 2) — 2026-08-28
|
|
|
|
## Works
|
|
|
|
- **Config**: `harness.toml` via a stdlib-only TOML-subset parser (tables,
|
|
bare keys incl. hyphens, strings/ints/bools, single- and multi-line
|
|
arrays; trailing commas tolerated; anything richer fails loudly with
|
|
line numbers). Defaults + validation; secrets are refs only
|
|
(`env:NAME` / `file:PATH` / `literal:VALUE`; `bw:` reserved) and redacted
|
|
from every error path. `harness.toml.example` is tracked and load-tested
|
|
so it can never rot; real configs are gitignored.
|
|
- **Model routing v0**: `[models]` tier map + `[models.classes]` class map;
|
|
requests go out with the CONCRETE model resolved from the map
|
|
(mopac-study→glm-4.7-flash, mopac-code→glm-5.2, mopac-review→glm-5-turbo,
|
|
mopac-primary→glm-5.3, + mopac-vision→glm-4.6v). No heuristic code;
|
|
unknown class = hard error naming the config section.
|
|
- **Conductor single-shot**: `harness once` = intake → routing → bounded
|
|
turn → REPORT, then exit 0. Chainable, no daemon. `--dry-run` = intake +
|
|
plan (resolved model, tool bounds) with ZERO LLM calls (test-asserted).
|
|
`--task-id` filter for chaining. Exit codes: 0 ok/no-tasks, 1
|
|
config/usage, 2 intake, 4 llm/turn.
|
|
- **Intake**: Redmine `/issues.json` scope query (raw filter params or
|
|
saved query id), task class from a configurable custom field with
|
|
default-class fallback; `--demo` builds the issue from `[demo]` (no
|
|
Redmine needed).
|
|
- **Bounded turn**: OpenAI-compatible chat via LiteLLM (stdlib http;
|
|
base_url ± `/v1` normalized; Bearer auth; retry/backoff on 429/5xx/
|
|
transport; usage accounting), tool-calling loop capped at `max_rounds`;
|
|
gate denials feed back to the model as tool results and are counted, not
|
|
fatal. On mid-turn LLM failure after content exists, a partial REPORT is
|
|
still written with the error as stop reason.
|
|
- **Exec tool**: allow-listed bash. maki-derived scope semantics without
|
|
tree-sitter: compound commands split segment-by-segment (`&&`/`||`/`;`/
|
|
`|`, quote-aware), `cmd *` word-boundary, `pfx*` raw prefix, `pfx/**`
|
|
path prefix, `*` universal; deny beats allow; `$()`/backticks/subshells
|
|
always denied (headless has no prompt channel). Per-command timeout with
|
|
process-group cleanup on unix; output truncation.
|
|
- **Writeback**: `REPORT-<vertical>-<task>-<ts>.md` + `REPORT-latest.md`
|
|
(atomic tmp+rename) with model/tier/class/tokens/rounds/denied/stop
|
|
telemetry per the DESIGN auditability bar.
|
|
- **Tests**: table-driven, stdlib `testing` only — TOML subset (valid doc
|
|
+ 9 error cases), config defaults/validation/key refs (incl. leak check),
|
|
routing decisions, scope matcher + gate (14 cases) + exec (timeout,
|
|
truncation, deny), writeback, Redmine intake (httptest), LLM client
|
|
(auth/model/retry/4xx/empty-choices), and loop end-to-end against a
|
|
scripted fake OpenAI server (demo turn, dry-run zero-call, tool
|
|
round-trip with message-shape assertions, denial counting, round limit,
|
|
error-class mapping). `go build ./...`, `go vet ./...`, `go test ./...`
|
|
all clean on go1.26.7; every intermediate commit builds standalone.
|
|
|
|
## MVP demo bar status
|
|
|
|
`harness once --demo` is wired end-to-end: prompt "tell me about yourself"
|
|
→ LiteLLM (glm-5.3 via mopac-primary) → GLM self-description → REPORT.
|
|
Proven against a scripted fake in tests. The live proxy
|
|
(http://192.168.3.78:4001) is reachable from this host and answers 401
|
|
without a key, so the live run needs exactly one thing:
|
|
|
|
export HARNESS_LITELLM_KEY=<vertical virtual key>
|
|
./bin/harness once --demo
|
|
|
|
## Stubbed / known gaps
|
|
|
|
- Redmine issue-note writeback (SoR note after the REPORT) — file only.
|
|
- Budget/semaphore GATE (LiteLLM spend APIs, class-aware slots) and cost
|
|
in REPORT (tokens only today).
|
|
- `bw:` key refs error until the bitwarden wrapper (phase 3).
|
|
- Streaming + turn resume on truncation — retry is request-level today.
|
|
- Session persistence + read-time repair (crush notes §2) — not started.
|
|
- Write confinement to declared roots + symlink-aware canonicalization;
|
|
redirects currently match by segment text (phase 3 permission layer).
|
|
- Task selection is first-in-scope; no P1-first ordering or status
|
|
transitions, so chained `once` re-picks the same issue (use --task-id).
|
|
- Local inbox intake (DESIGN core-loop step 1, second half) — not started.
|
|
|
|
## Next chunk (phase 3)
|
|
|
|
1. bitwarden-go wrapper + `bw:` refs (unblocks secret posture).
|
|
2. Full permission layer: tree-sitter bash scopes, write roots, symlink
|
|
checks, per-vertical allow/deny presets.
|
|
3. Redmine note writeback + status transition so chaining advances scope.
|
|
4. Budget gate via LiteLLM spend APIs; cost line in REPORT.
|
|
5. Streaming with truncation retry + turn resume.
|