# MOPAC harness

The MOPAC harness is a headless agent conductor written in Go: it pulls a task
from Redmine (or a demo issue), routes it to the right model through LiteLLM,
runs ONE bounded turn with an allow-listed bash tool, and writes the result as
a REPORT file — with `harness loop` it also drives ITSELF: Redmine is the SoR,
the loop is the worker, no bash middle layer. There is no TUI — humans and
other agents interact with a stack only through Redmine (SoR), Discourse
(docs) and Gitea (code), never by attaching to the loop.

Status: 2026-08-29 — skeleton + event receiver + self-host loop + **OWUI
front door live** + **quota/resource gates** (Redmine 490+491): `harness
loop` consults the z.ai credit buckets (5h + weekly, polled or locally
estimated), a TZ-aware peak window (default 01:00-05:00 CST weekdays) and
host load/mem/disk/IO before every dispatch — gated work DEFERS with a
logged reason and is reconsidered next scan, never hard-failed; per-class
token+credit accounting lands in the loop JSONL (`harness quota status`).
`harness serve` (OWUI front door, LAN 8090), `harness loop` (fake-Redmine
e2e test-asserted), the MVP demo path and `harness events` (port 4100)
all live. Multi-account deploy packaging (Redmine 494): `make release` +
`deploy/` — 9 accounts / 2 hosts, per-account ports + state, no root.

## Quickstart

All dev work happens inside a Docker builder (host stays toolchain-free);
`docker pull` of the builder is pre-authorized. Commands below were verified
on 2026-08-28 from a fresh clone.

### Build and test

`dev.sh` routes every compile/vet/test path through the digest-pinned
builder container (the host stays toolchain-free):

```sh
./dev.sh check   # = go build + go vet + go test, all inside the builder
```

The equivalent raw command:

```sh
docker run --rm -v "$PWD:/h" -w /h \
  -u "$(id -u):$(id -g)" -e HOME=/tmp \
  golang@sha256:e8c859f5632dcfde7b32d2012b4351728f6437930887c2f6a91ea242459e5514 \
  sh -c 'go build -o bin/harness ./cmd/harness && go vet ./... && go test ./...'
```

(that digest = `golang:1.26-bookworm`; alpine lacks bash, which the exec
tool's tests need.)

`make release` builds the deployable static binary
(`bin/harness-linux-amd64`, CGO off) through the same pinned builder;
`make deploy-test` tests the multi-account packaging — see
"Deploy: 9 accounts / 2 hosts" below.

Expected output (tail):

```text
ok  	ukrrs.com/mopac/harness/internal/config
ok  	ukrrs.com/mopac/harness/internal/events
ok  	ukrrs.com/mopac/harness/internal/intake
ok  	ukrrs.com/mopac/harness/internal/llm
ok  	ukrrs.com/mopac/harness/internal/loop
ok  	ukrrs.com/mopac/harness/internal/models
ok  	ukrrs.com/mopac/harness/internal/serve
ok  	ukrrs.com/mopac/harness/internal/tools
ok  	ukrrs.com/mopac/harness/internal/writeback
```

(The module path is `ukrrs.com/mopac/harness`; the repo lives at
[git.knownelement.com/ukrrs/MOPAC](https://git.knownelement.com/ukrrs/MOPAC).)

### Configure

```sh
cp harness.toml.example harness.toml
```

`harness.toml` is gitignored. It holds no secrets — only key refs
(`env:NAME`, `file:PATH`, `literal:VALUE`; `bw:` is reserved for the
bitwarden wrapper, build phase 3) that are resolved at runtime and redacted
from every error path.

### Dry-run (no LLM call, no REPORT)

```sh
./bin/harness once --dry-run --demo
```

Expected output:

```text
harness: intake: demo issue from harness.toml [demo]
harness: task demo-1 (demo): "MVP demo: GLM self-description" class=primary -> mopac-primary -> glm-5.3
PLAN (dry-run)
  vertical: demo
  task: [demo-1] MVP demo: GLM self-description (source demo)
  routing: class "primary" -> tier mopac-primary -> model glm-5.3
  tools: bash (allow=18 deny=7 default=deny timeout=60s)
  bound: max 8 rounds
harness: dry-run complete, no LLM call made
```

### Live demo turn (the MVP bar)

```sh
HARNESS_LITELLM_KEY=<vertical virtual key> ./bin/harness once --demo
```

The model's reply lands verbatim in `reports/REPORT-latest.md` with model /
tier / class / tokens / rounds telemetry. Without the key the run fails fast
and clean (verified):

```text
harness: litellm key: environment variable HARNESS_LITELLM_KEY is not set
```

exit code 1 (config error). The with-key path was proven live on 2026-08-28:
the `[demo]` prompt "tell me about yourself" routed to `glm-5.3` via tier
`mopac-primary` and the GLM self-description landed as the REPORT.

### Self-host loop (`harness loop`)

The loop makes the harness its own worker: Redmine is the SoR, the loop is
the conductor. It polls the `/issues.json` intake every `[loop]
poll_interval_secs` (default 120; `-interval` overrides, `--once` = single
scan for cron-style setups) and, for every issue not yet processed at its
current `updated_on`:

1. runs ONE bounded turn (the same conductor `once` uses; sequential, v0 —
   one turn at a time, no concurrency knob yet);
2. writes the REPORT file as always;
3. POSTs the REPORT body back onto the issue as a Redmine journal note
   (PUT `/issues/{id}.json`);
4. transitions status per `[redmine.status_map]` (e.g. `"In Progress" =
   "Done"`; names resolved via `/issue_statuses.json`, cached);
5. optionally commits the REPORT to Gitea (`[gitea] commit_reports`, off by
   default);
6. refreshes its dedup marker to the issue's post-writeback `updated_on` so
   its own note never re-triggers it.

State is an append-only `state/loop/loop.jsonl` (`[loop] state_dir`): one JSON
line per action (dispatch / report / note / status / commit / refresh /
error), rebuilt into the dedup index at startup — restarts keep
exactly-once-reaction semantics. A failed turn is recorded and NOT retried
(update the issue to re-release it), so a down proxy cannot hot-loop the
poll. Every action logs one line to stdout and one to the JSONL.

```sh
./bin/harness loop                # daemon: poll, dispatch, write back; SIGINT stops
./bin/harness loop --dry-run      # scan + print what would dispatch; no turns, no state
./bin/harness loop --once         # single scan (cron-able)
```

Secrets may come from the keyproxy hop: `key_ref = "mpk:mpk-redmine"` in
`harness.toml` plus a `[keyproxy]` section resolves through
[mopac-keyproxy](https://git.knownelement.com/ukrrs/mopac-keyproxy)'s
`POST /v1/resolve` (short in-memory cache). `env:` / `file:` / `literal:`
refs keep working untouched, so the loop runs with or without keyproxy up.

### Webhook receiver (`harness events`)

Redmine / Discourse / Gitea webhooks punch the harness through the event
receiver. Configure `[events]` in `harness.toml` (secrets as env refs — see
`harness.toml.example`), then:

```sh
HARNESS_REDMINE_WEBHOOK_SECRET=... HARNESS_DISCOURSE_WEBHOOK_SECRET=... \
HARNESS_GITEA_WEBHOOK_SECRET=... ./dev.sh events
```

publishes port 4100 on the host LAN (override with `-listen`). Routes:

| Route | Verification | Maps to |
|---|---|---|
| `POST /hooks/redmine` | shared-secret header (default `X-Redmine-Webhook-Secret`) | `dispatch_turn` on issue update/note |
| `POST /hooks/discourse` | shared-secret header (default `X-Discourse-Webhook-Secret`) | `respond_turn` on post reply |
| `POST /hooks/gitea` | HMAC-SHA256 `X-Gitea-Signature` over the raw body | `pipeline_step` on PR approved/merged |
| `GET /healthz` | none | liveness |

Unsigned or unverified deliveries get 401; verified ones are normalized to
one internal event record, appended to `state/events/events.jsonl` (0600,
deduped by provider event id: `X-Gitea-Delivery` / `X-Discourse-Event-Id`,
payload-digest fallback), and handed to the conductor (turn dispatch is a
stub until phase 3 wiring). End-to-end smoke, including the 401/200 paths
and the JSONL record, driven with python urllib (curl is banned on host):

```sh
./dev.sh smoke
```

### OpenAI-compatible front door (`harness serve`)

OpenWebUI (Cloudron + SSO) is the interactive surface; MOPAC is just another
OpenAI connection to it. Configure `[serve]` in `harness.toml`, then:

```sh
HARNESS_SERVE_VKEY=<vkey> HARNESS_LITELLM_KEY=<vertical key> ./dev.sh serve
```

publishes port 8090 on the host LAN (override with `-listen`). OWUI
connection settings: base URL `http://<host>:8090/v1`, API key = the vkey.
Routes:

| Route | Auth | Behavior |
|---|---|---|
| `POST /v1/chat/completions` | Bearer vkey | one bounded stateless conductor turn over the sent history; reply = single assistant message + usage |
| `GET /v1/models` | Bearer vkey | the servable catalog: one model per `[models.classes]` class, named `mopac-<class>` |
| `GET /healthz` | none | liveness |

Semantics (v0):

- **Stateless**: OWUI sends the full conversation history each call; the
  harness runs ONE bounded turn over it (same `runTurn` machinery `once`
  uses) and returns the final text. No session storage.
- **Model routing**: `request.model` (e.g. `mopac-primary`) maps through the
  existing class -> tier -> concrete-model tables; unknown model = 400
  naming the valid ones. `[serve] enabled_models` optionally narrows the
  catalog.
- **History assembly**: the client's leading system message is preserved
  verbatim (OWUI personas win); only when none is sent does the harness
  prepend a minimal identity prompt for the vertical.
- **Auth**: the Bearer vkey is compared as SHA-256 digests in constant time;
  missing/wrong keys get one byte-identical generic 401. The vkey is a ref
  (`vkey_ref`), resolved at startup, never logged.
- **Non-streaming**: `stream:true` gets an explicit 400 (OWUI tolerates
  non-streaming providers); streaming is Next.
- **Tools OFF**: serve turns are pure chat — any tool call a model produces
  anyway is refused as a tool result, never executed. Tool use inside serve
  turns is Next.
- `temperature` / `max_tokens` are forwarded upstream; usage is summed over
  rounds and returned with the reply; upstream failures surface as a terse
  502 (upstream bodies never reach the client).

Live on the LAN 2026-08-29: catalog of 9 models, 401/400/200 paths, and real
turns through LiteLLM (`mopac-primary` -> glm-5.3, `mopac-study` ->
glm-4.7-flash) with usage accounting, driven by python urllib.

### Quota + resource gates (`[quota]` / `[resources]`, Redmine 490+491)

The 2026-08-28 ~19:00 quota wall killed dispatched turns mid-flight; the
gates turn that failure mode into a logged throttle. Before EVERY dispatch
the loop consults, in order:

1. **Host resources** (`[resources]`, read-only `/proc` + statfs): loadavg,
   mem available, work-root disk free, IO pressure (`/proc/pressure/io`,
   skipped when PSI is absent). Any violation defers with all reasons
   surfaced.
2. **Quota buckets** (`[quota]`): the z.ai coding plan's 5-hour and weekly
   credit windows. `>= block_at_pct` (default 95%) defers EVERYTHING with
   the bucket/ratio in the reason — the wall, caught early.
3. **Peak window**: z.ai peak hours (documented Mon-Fri 14:00-18:00
   Singapore == 01:00-05:00 CST in winter) charge full rate; inside the
   window only `peak_classes` (the flash/LLM-lite tier) dispatch, heavy
   classes defer to off-peak (50% credit cost).
4. **Soft quota**: `>= defer_at_pct` (default 85%) defers heavy classes
   while LLM-lite continues.

A deferred task is NOT consumed: no turn, no note, no dedup marker — the
next scan reconsiders it (the 19:00-wall scenario is replayed as a test:
wall up -> defer + loop clean -> quota recovers -> dispatch). Defer events
land in `loop.jsonl` (`"type":"defer"` + reason), deduped per task+reason.

**Quota state** comes from two sources: the provider endpoint (`usage_url`,
bearer `key_ref`, parsed into buckets with reset times) and, when that is
unconfigured/unreachable, locally ESTIMATED consumption — per-turn credits
computed from the documented z.ai formula (input x 6.9 + cached x 1.7 +
output x 24, per 10k tokens; flash 2.3/0.56/8; off-peak 50% off) against
the configured plan limits. z.ai documents the buckets but publishes no
usage REST route today (probed 2026-08-29 — see the REPORT); the parser
targets the documented shape and is fake-server-tested, so flipping
`usage_url` on when z.ai ships it is a config edit. LIVE VERIFICATION open.

**Shared state**: with `redis_url` set, the latest snapshot and the credit
estimates live in one redis container so all harness instances of an
account (9 accounts across 2 hosts) coordinate — see the runbook below.
Redis down = this instance's local estimate; the loop never stops for it.

**Usage accounting**: every dispatched turn appends class + tokens +
estimated credits to its `report` event in `loop.jsonl`; `harness quota
status` renders the per-class table (the feed for the per-instance
Discourse usage reports).

```sh
./bin/harness quota status   # snapshot + peak window + resources + usage table
./bin/harness quota gate     # the allow/defer verdict per class, right now
./bin/harness quota probe    # one usage_url poll; parsed buckets or the error
```

### Help

```sh
./bin/harness help
```

Prints the full usage block reproduced under [CLI reference](#cli-reference).

## Architecture

```mermaid
flowchart LR
    P(["harness loop<br/>poll every poll_interval_secs"]) --> I["INTAKE<br/>/issues.json scope query"]
    I -->|"id + updated_on not in loop.jsonl"| T["BOUNDED TURN<br/>per issue, sequential (v0)"]
    T --> R["REPORT<br/>reports/ file (atomic)"]
    R --> N["REDMINE WRITEBACK<br/>journal note (PUT /issues/{id}.json)<br/>+ status per [redmine.status_map]"]
    G["GITEA COMMIT<br/>optional, [gitea] commit_reports"] -.-> N
    R -.-> G
    N --> S[("state/loop/loop.jsonl<br/>append-only, dedup index<br/>refreshed to post-writeback updated_on")]
    S -->|"next scan skips processed"| I
```

The loop replaces the crossfeed bash stack (semaphore slot files, queue
scripts, screen doorbells): concurrency is simply "one turn at a time"
for v0, wake discipline is the poll interval, and the only cross-process
state is the append-only JSONL.

One conductor iteration per process (`harness once`) still works exactly
as before for manual/chained runs:

```mermaid
flowchart TD
    S(["harness once"]) --> I["INTAKE<br/>Redmine /issues.json scope query<br/>or the [demo] issue"]
    I -->|"task + class"| RT["ROUTING<br/>class -&gt; tier -&gt; concrete model<br/>[models] + [models.classes]"]
    RT --> T["BOUNDED TURN<br/>OpenAI-compatible chat via LiteLLM<br/>gated bash tool, max_rounds cap"]
    T --> W["REPORT<br/>reports/REPORT-&lt;vertical&gt;-&lt;task&gt;-&lt;ts&gt;.md<br/>+ REPORT-latest.md, atomic write"]
    W --> E(["exit 0"])
    E -.->|"re-invoke to chain the next turn"| S
```

One conductor iteration per process: INTAKE resolves the released scope (a
Redmine query, or the `[demo]` issue with `--demo`) into a task with a class;
ROUTING maps the class through the config-only tier table to a concrete proxy
model; the bounded TURN drives an OpenAI-compatible chat loop through LiteLLM
(retry/backoff on 429/5xx/transport, usage accounting) with a tool-calling
loop capped at `max_rounds`, where the only tool today is an allow-listed
bash exec whose gate denials feed back to the model as tool results instead
of failing the turn; WRITEBACK persists the final assistant reply as a REPORT
file plus `REPORT-latest.md` (atomic tmp+rename). On a mid-turn LLM failure
after content exists, a partial REPORT is still written with the error as the
stop reason. Chaining is by re-invocation; the loop is the daemon form of
the same iteration, run per new/updated issue with the SoR writebacks on
top.

The event path is a second door into the same loop:

```mermaid
flowchart LR
    RM["Redmine webhook"] --> V["VERIFY<br/>shared-secret / HMAC headers"]
    DC["Discourse webhook"] --> V
    GT["Gitea webhook"] --> V
    V -->|401 unverified| X(["rejected"])
    V --> N["NORMALIZE<br/>one Event record: source, kind, actor,<br/>subject id, digest, received-at"]
    N --> S["STORE<br/>state/events/events.jsonl, append-only,<br/>dedup by provider event id"]
    S -->|"dispatch_turn / respond_turn / pipeline_step"| C(["conductor (dispatch stub)"])
```

Verification rejects unsigned and unverified deliveries with 401 (no detail
beyond "unverified"); secret material and header values never reach logs or
the event log — only normalized fields and a payload digest do.

OWUI chat is a second door into the same loop (the bounded-turn core is
shared, tools off):

```mermaid
flowchart LR
    O(["OpenWebUI (Cloudron + SSO)"]) -->|"Bearer vkey"| A["AUTH<br/>constant-time vkey compare<br/>401 generic on miss"]
    A -->|"POST /v1/chat/completions"| C["CATALOG ROUTING<br/>model mopac-<class> -> tier -> concrete model<br/>"]
    C --> T["BOUNDED TURN (stateless)<br/>full client history per call<br/>tools OFF v0"]
    T -->|"assistant message + usage"| O
    A -->|"GET /v1/models"| M(["catalog: one model per class"])
```

## CLI reference

Subcommands (from `harness help`):

| Subcommand | Purpose |
|---|---|
| `harness help` | print usage (also `-h`, `--help`) |
| `harness once` | run ONE conductor iteration, then exit |
| `harness loop` | run the self-host daemon until SIGINT (poll -> gate -> turn -> note/status writeback) |
| `harness events` | run the webhook receiver until SIGINT/SIGTERM |
| `harness serve` | run the OpenAI-compatible front door until SIGINT/SIGTERM (the OWUI connection) |
| `harness quota` | gate surface: `status` (snapshot + usage accounting), `gate` (per-class verdicts), `probe` (one usage poll) |

Flags for `once`:

| Flag | Meaning |
|---|---|
| `-config PATH` | config file (default `$HARNESS_CONFIG`, then `./harness.toml`) |
| `-dry-run` | intake + plan only; no LLM call, no REPORT |
| `-demo` | run the `[demo]` issue instead of Redmine intake |
| `-task-id ID` | run only the task/issue with this id |

Flags for `loop`:

| Flag | Meaning |
|---|---|
| `-config PATH` | config file (default `$HARNESS_CONFIG`, then `./harness.toml`) |
| `-interval DUR` | poll interval (overrides `[loop]` poll_interval_secs) |
| `-once` | single scan then exit (cron-able) |
| `-dry-run` | scan + print what would dispatch; no turns, no state writes |

Flags for `events`:

| Flag | Meaning |
|---|---|
| `-config PATH` | config file (default `$HARNESS_CONFIG`, then `./harness.toml`) |
| `-listen ADDR` | bind address (overrides `[events]` listen) |

Flags for `serve`:

| Flag | Meaning |
|---|---|
| `-config PATH` | config file (default `$HARNESS_CONFIG`, then `./harness.toml`) |
| `-listen ADDR` | bind address (overrides `[serve]` listen) |

Flags for `quota` (`harness quota <status|probe|gate> [flags]`):

| Flag | Meaning |
|---|---|
| `-config PATH` | config file (default `$HARNESS_CONFIG`, then `./harness.toml`) |

Exit codes:

| Code | Meaning |
|---|---|
| 0 | ok (including "no tasks in scope"; loop: clean SIGINT stop) |
| 1 | usage / config / routing / writeback error |
| 2 | intake error |
| 4 | llm / turn error |

## Configuration

Every `harness.toml` section (see `harness.toml.example`, load-tested so it
cannot rot):

| Section | Keys | Meaning |
|---|---|---|
| top level | `vertical` | vertical/stack identity for this harness instance |
| | `work_root` | where the bash tool executes (default `.`) |
| | `report_dir` | where REPORT files land (default `reports`) |
| `[loop]` | `max_rounds` | max LLM round trips per turn, tool calls included (default 8) |
| | `poll_interval_secs` | `harness loop` scan interval (default 120) |
| | `state_dir` | loop state dir (default `state/loop`); `loop.jsonl` created 0600, one JSON line per action |
| `[redmine]` | `url`, `key_ref` | SoR issues endpoint + auth key ref |
| | `scope_query` / `scope_query_id` | released-scope filter: raw `/issues.json` params, or a saved query id (`scope_query` wins when both are set) |
| | `class_field`, `default_class` | custom field carrying the task class; fallback for issues without it (default `Class` / `primary`) |
| | `limit` | max issues fetched per intake (default 50) |
| `[redmine.status_map]` | `"In Progress" = "Done"` style entries | loop: after a REPORT is noted, an issue whose current status matches a key moves to the value (names resolved via `/issue_statuses.json`); empty map = no transitions. Quoted keys allow spaces |
| `[litellm]` | `base_url`, `key_ref` | OpenAI-compatible proxy endpoint + key ref |
| | `timeout_secs`, `max_retries` | per-request timeout (default 120) and retry count (default 2) |
| `[keyproxy]` | `url`, `token_ref` | optional mopac-keyproxy resolve hop for `mpk:` refs (bearer token is itself a LOCAL ref); url + token_ref are all-or-nothing |
| | `cache_ttl_secs` | in-memory `mpk:` resolution cache TTL (default 60) |
| `[gitea]` | `url`, `key_ref`, `owner`, `repo` | optional REPORT-commit step; all required when `commit_reports = true` |
| | `branch`, `commit_reports` | target branch (empty = repo default); the whole step is off by default |
| `[models]` | tier keys, `default_tier` | tier alias → concrete proxy model (see below) |
| `[models.classes]` | class keys | task class → tier alias (see below) |
| `[tools.bash]` | `enabled`, `timeout_secs`, `max_output_bytes` | bash gate on/off, per-command timeout (default 60s), output truncation (default 100000) |
| | `default` | verdict for commands matching no rule: `allow` or `deny` (org preset: deny) |
| | `allow`, `deny` | rule lists; deny beats allow; `cmd *` word-boundary, `pfx*` raw prefix, `pfx/**` path prefix, `*` universal; compound commands checked segment by segment; command substitution and subshells always denied |
| `[demo]` | `id`, `subject`, `prompt`, `class` | the issue `--demo` runs instead of Redmine intake |
| `[events]` | `listen` | receiver bind address (default `:4100`; publish via `docker -p`) |
| | `state_dir` | event log dir (default `state/events`); `events.jsonl` created 0600 |
| `[events.redmine]` | `secret_ref`, `secret_header` | shared-secret ref + header name (default `X-Redmine-Webhook-Secret`) |
| `[events.discourse]` | `secret_ref`, `secret_header` | shared-secret ref + header name (default `X-Discourse-Webhook-Secret`) |
| `[events.gitea]` | `secret_ref` | HMAC secret ref; verification is always `X-Gitea-Signature` (hex HMAC-SHA256 of the raw body) |
| `[serve]` | `listen` | `harness serve` bind address (default `:8090`; own port, coexists with `[events]`) |
| | `vkey_ref` | bearer vkey ref for the OpenAI-compatible front door (the key the OWUI connection presents) |
| | `enabled_models` | optional subset of the catalog (`mopac-<class>` names); empty = every `[models.classes]` class

Key refs accepted anywhere a `*_ref` appears: `env:NAME`, `file:PATH`,
`literal:VALUE` (last resort), `mpk:PLACEHOLDER` (resolved through the
optional `[keyproxy]` hop — see [mopac-keyproxy](https://git.knownelement.com/ukrrs/mopac-keyproxy);
`mpk:redmine` and `mpk:mpk-redmine` are equivalent), and `bw:REF`
(reserved; errors until the bitwarden wrapper lands in build phase 3).

## Model routing

Static, config-only, no heuristics: a task's class maps to a tier alias, the
tier alias maps to the concrete model actually sent to the proxy. Unknown
class = hard error naming the config section.

Tiers (`[models]`):

| Tier | Concrete model | Role |
|---|---|---|
| `mopac-study` | `glm-4.7-flash` | flash tier |
| `mopac-code` | `glm-5.2` | flagship |
| `mopac-review` | `glm-5-turbo` | mid |
| `mopac-primary` | `glm-5.3` | default / flagship+ |
| `mopac-vision` | `glm-4.6v` | vision when needed |

Classes (`[models.classes]`):

| Task class | Tier | Concrete model |
|---|---|---|
| `study` | `mopac-study` | `glm-4.7-flash` |
| `read` | `mopac-study` | `glm-4.7-flash` |
| `code` | `mopac-code` | `glm-5.2` |
| `architecture` | `mopac-code` | `glm-5.2` |
| `review` | `mopac-review` | `glm-5-turbo` |
| `summarize` | `mopac-review` | `glm-5-turbo` |
| `writeback` | `mopac-review` | `glm-5-turbo` |
| `vision` | `mopac-vision` | `glm-4.6v` |
| `primary` | `mopac-primary` | `glm-5.3` |
| (no class) | `default_tier` = `mopac-primary` | `glm-5.3` |

## Status

Sourced from [REPORT.md](REPORT.md) — keep both in sync.

| Area | State | Detail |
|---|---|---|
| Config | Works | TOML-subset parser, defaults + validation, secret refs only, redacted errors |
| Model routing v0 | Works | tier + class maps; concrete model resolved before the request leaves |
| Conductor single-shot | Works | `harness once` chainable; `--dry-run` makes zero LLM calls (test-asserted) |
| Redmine intake | Works | `/issues.json` scope query, class custom field, `--demo` fallback |
| Bounded turn | Works | LiteLLM chat, retry/backoff, tool loop capped at `max_rounds`, partial REPORT on mid-turn failure |
| Exec tool | Works | allow-listed bash, segment-wise compound checks, timeout + truncation |
| REPORT writeback | Works | timestamped file + `REPORT-latest.md`, atomic, full telemetry |
| Event receiver | Works | `harness events`: verify (HMAC/shared-secret) → normalize → append-only JSONL with provider-id dedup → action mapping; smoke-proven on LAN port 4100 |
| OpenAI front door | Works | `harness serve`: `/v1/models` catalog (one model per class, `mopac-<class>`), stateless bounded `/v1/chat/completions` turns, constant-time vkey auth, tier-map routing, usage accounting; live-proven on LAN port 8090 via LiteLLM (glm-5.3 + glm-4.7-flash) |
| Self-host loop | Works | `harness loop`: poll intake → sequential bounded turns → REPORT note writeback → status map → dedup by id + updated_on (append-only loop.jsonl); `--once`, `--dry-run`, `-interval`; fake-Redmine e2e test-asserted |
| Redmine note writeback | Works | loop path POSTs the REPORT body as a journal note (PUT `/issues/{id}.json`); `once` stays file-only |
| Status transitions | Works | `[redmine.status_map]` names → ids via `/issue_statuses.json`; refresh advances the dedup marker past its own writes |
| `mpk:` key refs | Works | `[keyproxy]` hop (POST `/v1/resolve`, bearer, cached); local refs unaffected |
| Gitea REPORT commit | Works (off) | `[gitea] commit_reports`: contents-API create-or-update right after the REPORT lands |
| Quota gate (490) | Works (off) | `[quota]`: 5h/weekly credit buckets (poll or estimate), block/defer/peak back-pressure, defer-not-fail; usage accounting per class in loop.jsonl; z.ai usage endpoint LIVE VERIFICATION open |
| Shared quota state | Works (off) | `redis_url`: one redis container, all instances of an account share snapshot + estimates; stdlib RESP2 mini-client, fail-soft |
| Resource gate (491) | Works (off) | `[resources]`: loadavg/mem/disk/IO-PSI thresholds, read-only, defer-not-fail |
| Event → turn dispatch | Stubbed | conductor `DispatchEvent` prints what it would do; wiring is phase 3 |
| Tests | Works | table-driven, stdlib only; build/vet/test clean on go1.26 |
| Budget/semaphore gate | Partial | credit-bucket back-pressure is live (rows above); LiteLLM-$-spend budget keys remain open |
| `bw:` key refs | Stubbed | error until the bitwarden wrapper (phase 3) |
| Loop concurrency | Stubbed | v0 = one turn at a time; in-process concurrency knob later |
| Streaming + turn resume | Stubbed | retry is request-level today; serve door is non-streaming by design v0 |
| Serve tool use | Stubbed | serve turns are pure chat (tools off); wiring the gated tool palette into serve turns is Next |
| Session persistence/repair | Stubbed | not started |
| Write confinement | Stubbed | declared roots + symlink checks land with the permission layer |
| Local inbox intake | Stubbed | not started |
| Next (phase 3) | Next | bitwarden wrapper, full permission layer, event → turn wiring, budget gate via LiteLLM spend APIs, serve streaming + serve tools, loop concurrency knob |

## Runbook: shared quota state + resource control (deploy-time)

**Redis container** (the shared-state hop for `[quota] redis_url`; one per
host-pair, container only, no host packages):

```sh
docker run -d --name mopac-quota-redis --restart unless-stopped \
  -p 192.168.3.78:6390:6379 \
  -v /srv/mopac-quota-redis:/data \
  redis:7-alpine --appendonly yes
```

Every harness instance of the same z.ai account then sets the same
`account` + `redis_url` in `[quota]`; keys are namespaced
`mopac:quota:<account>:{snapshot,est:*}`. Redis unreachable = local
estimates only (fail-soft, logged). The harness speaks RESP2 directly —
no client library, no host redis-cli needed.

**cgroup enforcement** (ticket 491, deploy-time): the in-harness gate is
read-only and advisory — it defers dispatch when the HOST is busy. To keep
builds/turns from making the host busy in the first place, run each loop
container under cgroup limits at deploy:

```sh
docker run ... --memory 4g --cpus 2 --pids-limit 512 \
  --io-max bandwidth=/data:100mb ... # device-specific; see docker run(1)
```

or a systemd slice for non-container deploys (`CPUQuota=200%`,
`MemoryMax=4G`, `IOWeight`). The gate catches what the limits don't.

## Deploy: 9 accounts / 2 hosts (Redmine 494)

The full multi-account packaging lives in [`deploy/`](deploy/) — one
harness instance per Linux account (5 on ultix-streaming, 4 on
ultix-offstage), each entirely under that account's `~/.mopac/` (binary,
config, 0600 env secrets, state, reports). No root, no systemd, no host
packages: `sudo` is only used to switch identity when installing into the
other accounts.

```sh
make release      # static linux/amd64 binary: bin/harness-linux-amd64
                  # (docker builder, CGO_ENABLED=0 — runs anywhere)
make deploy-test  # packaging tests: port scheme, template substitution,
                  # installer idempotence in a fake HOME
```

- [`deploy/accounts.tsv`](deploy/accounts.tsv) — the fleet authority:
  account, host, vertical, Redmine project, quota group, and the port
  scheme `events = 4100 + index` / `serve = 8090 + index` (index 0-8, so
  the daemons of any two accounts on one host never collide; loop state
  is per-account under `~/.mopac/state/`).
- [`deploy/install-account.sh <account>`](deploy/install-account.sh) —
  idempotent installer, run AS the target user: creates
  `~/.mopac/{bin,state,reports,work}`, installs the static binary,
  renders `harness.toml` from [`deploy/harness.toml.in`](deploy/harness.toml.in)
  + the TSV row, writes the 0600 `~/.mopac/env` secrets template and
  `mopac-start`/`mopac-stop` helpers. Existing `harness.toml`/`env` are
  NEVER overwritten (re-running = the upgrade path).
- [`deploy/runbook.md`](deploy/runbook.md) — the exact Charles sequence:
  build once, stage per host, install per account, secrets bootstrap,
  verify (`once --dry-run --demo`, healthz, first `loop --once --dry-run`),
  start (nohup helpers + optional cron `@reboot`), rollback (stop +
  `rm -rf ~/.mopac`), per-host time estimates.

Generated configs keep `[quota]`/`[resources]` commented with values
pre-filled per account — flipping them on is a per-account one-liner
once the shared redis container (above) is up.

## Docs and links

- [DESIGN.md](DESIGN.md) — design spec (hard rules, build order, org model)
- [REPORT.md](REPORT.md) — current build status
- [docs/SPEC-20260829-charles-brief.md](docs/SPEC-20260829-charles-brief.md) — Charles's end-to-end design brief (spec of record)
- [docs/PORTING-NOTES-crush.md](docs/PORTING-NOTES-crush.md) — sessions/MCP/provider study of the crush agent
- [docs/PORTING-NOTES-maki.md](docs/PORTING-NOTES-maki.md) — permission parsing and token-reduction study of maki
- [docs/PORTING-NOTES-secrets.md](docs/PORTING-NOTES-secrets.md) — Bitwarden secrets study; feeds the bitwarden-go tool
- Repository: <https://git.knownelement.com/ukrrs/MOPAC>
- Sibling tool repos (spec seeds): [mopac-keyproxy](https://git.knownelement.com/ukrrs/mopac-keyproxy), [mopac-bitwarden-go](https://git.knownelement.com/ukrrs/mopac-bitwarden-go)
- SoR: Redmine project MOPAC; docs: Discourse

## License

AGPLv3 — see [LICENSE](LICENSE).