MOPAC/REPORT.md

# REPORT — harness skeleton (build phase 2) — 2026-08-28

## Works

- **Config**: `harness.toml` via a stdlib-only TOML-subset parser (tables,
  bare keys incl. hyphens, strings/ints/bools, single- and multi-line
  arrays; trailing commas tolerated; anything richer fails loudly with
  line numbers). Defaults + validation; secrets are refs only
  (`env:NAME` / `file:PATH` / `literal:VALUE`; `bw:` reserved) and redacted
  from every error path. `harness.toml.example` is tracked and load-tested
  so it can never rot; real configs are gitignored.
- **Model routing v0**: `[models]` tier map + `[models.classes]` class map;
  requests go out with the CONCRETE model resolved from the map
  (mopac-study→glm-4.7-flash, mopac-code→glm-5.2, mopac-review→glm-5-turbo,
  mopac-primary→glm-5.3, + mopac-vision→glm-4.6v). No heuristic code;
  unknown class = hard error naming the config section.
- **Conductor single-shot**: `harness once` = intake → routing → bounded
  turn → REPORT, then exit 0. Chainable, no daemon. `--dry-run` = intake +
  plan (resolved model, tool bounds) with ZERO LLM calls (test-asserted).
  `--task-id` filter for chaining. Exit codes: 0 ok/no-tasks, 1
  config/usage, 2 intake, 4 llm/turn.
- **Intake**: Redmine `/issues.json` scope query (raw filter params or
  saved query id), task class from a configurable custom field with
  default-class fallback; `--demo` builds the issue from `[demo]` (no
  Redmine needed).
- **Bounded turn**: OpenAI-compatible chat via LiteLLM (stdlib http;
  base_url ± `/v1` normalized; Bearer auth; retry/backoff on 429/5xx/
  transport; usage accounting), tool-calling loop capped at `max_rounds`;
  gate denials feed back to the model as tool results and are counted, not
  fatal. On mid-turn LLM failure after content exists, a partial REPORT is
  still written with the error as stop reason.
- **Exec tool**: allow-listed bash. maki-derived scope semantics without
  tree-sitter: compound commands split segment-by-segment (`&&`/`||`/`;`/
  `|`, quote-aware), `cmd *` word-boundary, `pfx*` raw prefix, `pfx/**`
  path prefix, `*` universal; deny beats allow; `$()`/backticks/subshells
  always denied (headless has no prompt channel). Per-command timeout with
  process-group cleanup on unix; output truncation.
- **Writeback**: `REPORT-<vertical>-<task>-<ts>.md` + `REPORT-latest.md`
  (atomic tmp+rename) with model/tier/class/tokens/rounds/denied/stop
  telemetry per the DESIGN auditability bar.
- **Tests**: table-driven, stdlib `testing` only — TOML subset (valid doc
  + 9 error cases), config defaults/validation/key refs (incl. leak check),
  routing decisions, scope matcher + gate (14 cases) + exec (timeout,
  truncation, deny), writeback, Redmine intake (httptest), LLM client
  (auth/model/retry/4xx/empty-choices), and loop end-to-end against a
  scripted fake OpenAI server (demo turn, dry-run zero-call, tool
  round-trip with message-shape assertions, denial counting, round limit,
  error-class mapping). `go build ./...`, `go vet ./...`, `go test ./...`
  all clean on go1.26.7; every intermediate commit builds standalone.

## MVP demo bar status

`harness once --demo` is wired end-to-end: prompt "tell me about yourself"
→ LiteLLM (glm-5.3 via mopac-primary) → GLM self-description → REPORT.
Proven against a scripted fake in tests. The live proxy
(http://192.168.3.78:4001) is reachable from this host and answers 401
without a key, so the live run needs exactly one thing:

    export HARNESS_LITELLM_KEY=<vertical virtual key>
    ./bin/harness once --demo

## Stubbed / known gaps

- Event → turn dispatch: `harness events` stores + maps events and hands
  them to `Conductor.DispatchEvent`, which is a printing stub (phase 3).
- Budget/semaphore GATE (LiteLLM spend APIs, class-aware slots) and cost
  in REPORT (tokens only today).
- `bw:` key refs error until the bitwarden wrapper (phase 3).
- Streaming + turn resume on truncation — retry is request-level today.
- Session persistence + read-time repair (crush notes §2) — not started.
- Write confinement to declared roots + symlink-aware canonicalization;
  redirects currently match by segment text (phase 3 permission layer).
- `once` task selection is first-in-scope (`--task-id` to chain); the
  loop's dedup + status map supersede this for self-hosted runs.
- Local inbox intake (DESIGN core-loop step 1, second half) — not started.

## Build phase 2b — events receiver (`mopac events`) — 2026-08-28

Works:

- **`harness events` CLI**: stdlib net/http receiver (no frameworks), runs
  until SIGINT/SIGTERM, graceful shutdown, `-config` / `-listen` flags.
  Routes `POST /hooks/{redmine,discourse,gitea}` + `GET /healthz`; GET on a
  hook = 405, unknown path = 404, oversized body (> 1 MiB) = 413.
- **Verification, deny-first**: gitea = hex HMAC-SHA256 of the raw body in
  `X-Gitea-Signature` (constant-time `hmac.Equal`); redmine/discourse =
  shared-secret header (constant-time compare; header names configurable,
  defaults `X-Redmine-Webhook-Secret` / `X-Discourse-Webhook-Secret`).
  Unsigned/unverified = 401 with a single generic error body; rejection
  logs name the failure class only, never header values or secrets.
- **Normalization**: one internal Event (source, kind, actor, canonical
  subject id `redmine:issue:42` / `discourse:topic:7` /
  `gitea:pr:ukrrs/MOPAC#5`, title, repo, provider event id, payload
  sha256 digest, received-at) — tolerant extraction across known payload
  variants (redmine_webhooks + flat shapes, Discourse headers + payload,
  Gitea action/pull_request/issue shapes incl. closed+merged → pr_merged).
- **Action mapping (DESIGN)**: redmine issue update/note/journal →
  `dispatch_turn`; discourse post reply → `respond_turn`; gitea PR
  approved/merged → `pipeline_step`; everything else stored-but-ignored.
- **Persistence**: append-only `state/events/events.jsonl` (0600, dir
  0700), dedup by provider event id (`X-Gitea-Delivery`,
  `X-Discourse-Event-Id`, `X-Redmine-Delivery` when present; payload
  digest fallback). Dedup index rebuilt from the file at startup (torn
  tail line skipped), so replays across restarts still dedup —
  at-least-once delivery, exactly-once reaction.
- **Config**: `[events]` listen/state_dir + per-source secret refs
  (`env:`/`file:`/`literal:`, resolved at startup; server refuses to start
  with zero secrets). Full validation, ref values never echoed.
- **dev.sh**: every build/vet/test/run path routes through the
  digest-pinned builder `golang@sha256:e8c859f...` (= golang:1.26-bookworm;
  alpine has no bash for the exec tool's tests).
- **Tests**: table-driven — signature verification (7 HMAC + 5 shared
  secret cases), normalization (13 payload/shape cases + action map),
  store dedup/restart/torn-tail/permissions, end-to-end httptest (401/400/
  200 paths, replay dedup, ignore-not-dispatched, oversized body, log/JSONL
  secret-leak assertions). Docker `build/vet/test` clean.
- **Smoke (live, LAN port 4100)**: containerized receiver driven by host
  python3 urllib (curl banned on host): unsigned → 401, bad HMAC → 401,
  wrong shared secret → 401, valid → 200 `stored` + JSONL line, replay →
  200 `duplicate`, per-provider actions correct, stub dispatch fired once
  per stored actionable event. `./dev.sh smoke` reproduces it end-to-end.

Stubbed:

- Event → turn dispatch (`Conductor.DispatchEvent` prints the would-be
  action; real wiring lands in phase 3 per plan).
- Cross-host/event forwarding (T1 replication later; JSONL is local-only).

## Build phase 3a — self-host core (`harness loop`) — 2026-08-28 ~22:00

Charles (21:30): "get to be self hosting asap — smallest core that can
bootstrap itself (redmine/gitea interaction fine, discourse later)". The
harness now drives ITSELF: Redmine is the SoR, the loop is the worker, no
bash middle layer.

Works:

- **`harness loop` CLI**: the self-host daemon. Polls the `/issues.json`
  intake every `[loop] poll_interval_secs` (default 120; `-interval`
  override; `--once` = single scan for cron-style setups; `--dry-run` =
  scan + print, zero LLM calls, zero state writes — test-asserted).
- **Dispatch discipline**: one bounded turn per issue not yet processed at
  its current `updated_on`, strictly sequential (v0 — the concurrency
  slot-file cap of the old stack is replaced by "one at a time"; a knob
  can come later). The dedup marker is written BEFORE the turn, so a
  failed turn is recorded and never retried (update the issue to
  re-release it) — a down proxy cannot hot-loop the poll.
- **State**: append-only `state/loop/loop.jsonl` (0600, dir 0700), one
  JSON line per action (dispatch / report / note / status / commit /
  refresh / error), index rebuilt at startup (torn tail skipped). Restart
  keeps exactly-once-reaction semantics — test-asserted across a new
  Conductor instance.
- **Redmine note writeback** (was the phase-2 stub): after the turn, the
  REPORT body is POSTed onto the issue as a journal note
  (PUT `/issues/{id}.json`), so the SoR carries the result, not just the
  filesystem.
- **Status transitions**: `[redmine.status_map]` (quoted TOML keys, e.g.
  `"In Progress" = "Done"`); names resolve to ids via
  `/issue_statuses.json` (fetched once, cached, case-insensitive);
  unmapped statuses are left alone.
- **Self-retrigger prevention**: the loop's own note/status writes bump
  `updated_on`; after writeback the loop re-fetches the issue and
  advances its dedup marker to the post-writeback `updated_on`
  (`refresh` event). If that refresh call fails, the JSONL records the
  error and one redundant re-dispatch may follow (visible, bounded).
- **`mpk:` key refs** (`[keyproxy]`): resolved through the
  ukrrs/mopac-keyproxy hop — POST `/v1/resolve`, bearer token from a
  LOCAL `token_ref` (no recursion), 60s in-memory cache, transport
  errors query-string-stripped so a misconfigured hop cannot echo the
  token. `env:`/`file:`/`literal:` refs untouched: the loop runs with or
  without keyproxy up. All hosts in harness.toml, none in code.
- **Gitea REPORT commit** (optional, off by default): `[gitea]
  commit_reports = true` commits each REPORT file right after it lands
  (contents API: GET for sha → POST create / PUT update, `token` auth,
  branch from config).
- **Module rename**: `git.knownelement.com/reachableceo/MOPAC/harness` →
  `ukrrs.com/mopac/harness` (the doc-rot finding folded in; build + vet
  + test verified in the Docker builder).
- **Tests**: loop e2e against a fake Redmine (mutable state: notes bump
  updated_on like the real SoR) + fake OpenAI server — dispatch/note/
  status happy path, dedup on rescan, re-dispatch on PMO issue update,
  failed turn recorded without hot-loop, dry-run zero-trace, restart
  dedup survival, status-map miss, gitea create/update/auth shapes,
  keyproxy resolver (cache hit, auto-prefix, 401/404, local refs
  untouched), writeback unit tests for PUT body shapes.

Replaced by this turn (die list for the PMO report; deletion pending
Charles' sign-off): `/home/_crossfeed/tooling/agent-stack/semaphore.sh` +
slot-file cap; `~/.coordinate/scripts/queue-next.sh`, `queue-after.sh`,
`pmo-heartbeat.sh`; the dispatch-turn.sh screen doorbell (already dead —
it targets screen `reachableceo-PMO`, the live screen is `RCEO-PMO`, so
the message was silently dropped).

## Build phase 3b — OWUI front door (`harness serve`) — 2026-08-29

Charles' TASK (2026-08-29 00:00): OpenAI-compatible front door, OWUI is the
client; DESIGN "OWUI front door" (hermes killed). OWUI chat and webhooks are
two doors into the same conductor loop.

Works:

- **`harness serve` CLI**: stdlib net/http receiver in the events style,
  SIGINT/SIGTERM graceful shutdown, `-config` / `-listen` flags, own port
  (`[serve] listen`, default `:8090`) so it coexists with `harness events`.
  Routes: `POST /v1/chat/completions`, `GET /v1/models`, `GET /healthz`.
- **Catalog = the class -> tier map**: every `[models.classes]` class is
  served as model `mopac-<class>` (mopac-study, mopac-code, mopac-primary,
  ...); the request's model routes through the SAME tier table `once` uses
  to a concrete proxy model. Unknown model = 400 naming the valid ones.
  `[serve] enabled_models` optionally narrows the catalog (validated at
  load).
- **Stateless bounded turns**: OWUI sends the full history each call; the
  server runs ONE conductor turn over it (shared `runTurn` after a clean
  refactor — `turn()` and `ServeTurn` both delegate; no copy-paste) and
  returns the final text as a single assistant message + usage (summed over
  rounds). No session storage. Client system message preserved verbatim; a
  minimal vertical-identity system prompt is prepended only when the
  history has none.
- **Auth**: Bearer vkey (`[serve] vkey_ref`, refs only, resolved at
  startup), SHA-256 digest compare in constant time; missing/malformed/
  wrong keys get one byte-identical generic 401; vkey never logged, never
  in any response.
- **v0 knobs**: `temperature` + `max_tokens` forwarded upstream; `stream:
  true` = explicit 400 (OWUI tolerates non-streaming providers); tools
  OFF — a hallucinated tool call is refused as a tool result and counted,
  never executed; upstream failures = terse 502 (bodies never forwarded).
- **dev.sh serve**: digest-pinned builder, host port 8090 published,
  `HARNESS_SERVE_VKEY` / `HARNESS_LITELLM_KEY` passed through.
- **Tests**: scripted fake OpenAI upstream against the REAL server
  (httptest): auth matrix (401 paths + no-leak), catalog + subset,
  end-to-end chat (concrete model out, no tools on the wire, OpenAI
  response shape, usage), history assembly both ways, multi-round usage
  sums with refused tool-call feedback, knob forwarding, 400/502 table,
  fail-fast construction. Loop-package tests cover ServeTurn prepend/
  preserve/tools-off/knobs.
- **Live LAN proof (2026-08-29)**: port 8090, python urllib driver —
  healthz 200, wrong key 401, catalog 200 (9 models), unknown model 400
  naming them, stream 400, real turns through LiteLLM: `mopac-primary` ->
  glm-5.3 (255 tokens) and `mopac-study` -> glm-4.7-flash (166 tokens),
  audit lines clean.

Stubbed / Next:

- Streaming (SSE) on the serve door; tool use inside serve turns (gated
  palette per vertical); per-connection vkeys via LiteLLM/keyproxy minting
  (today one vkey per harness instance).

## Next chunk (phase 3)

1. bitwarden-go wrapper + `bw:` refs (unblocks secret posture).
2. Full permission layer: tree-sitter bash scopes, write roots, symlink
   checks, per-vertical allow/deny presets.
3. Event → turn dispatch wiring: stored actionable events actually chain
   conductor iterations for the affected stack.
4. Budget gate via LiteLLM spend APIs; cost line in REPORT.
5. Streaming with truncation retry + turn resume (serve door SSE included).
6. Loop concurrency knob (in-process limit) once load justifies it.
7. Tool use inside serve turns (per-vertical gated palette) + per-connection
   vkey minting.