docs: rewrite README to the docs standard; polish docs/

README: verified quickstart (Docker builder, dry-run, live demo), mermaid
architecture, CLI/config/routing reference tables, status table from
REPORT.md, correct ukrrs repo URLs. docs/: Status front-matter lines on
all PORTING-NOTES (secrets notes now tracked). DESIGN: table of contents
with anchor links; sections untouched.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
2026-08-28 21:10:04 -05:00
parent ab29d8a948
commit c6da05fac5
5 changed files with 389 additions and 11 deletions
+22
View File
@@ -5,6 +5,28 @@ inner loop. All patterns established 2026-08-28 STAY; only the substrate
changes. References: ../reference/{crush,maki}; siblings to port: changes. References: ../reference/{crush,maki}; siblings to port:
../siblings/{KNEL-AIMiddleware, KNELSecretsManager}. ../siblings/{KNEL-AIMiddleware, KNELSecretsManager}.
## Contents
- [What it is NOT](#what-it-is-not)
- [Core loop (from DESIGN-24x7-execution.md, carried over)](#core-loop-from-design-24x7-executionmd-carried-over)
- [Tools (port from crush/KNEL, Go)](#tools-port-from-crushknel-go)
- [Providers](#providers)
- [Cross-host (5 onstage + 3 offstage)](#cross-host-5-onstage--3-offstage)
- [Build order (go slow, grinder TASKs, quota-aware)](#build-order-go-slow-grinder-tasks-quota-aware)
- [Constraints](#constraints)
- [Parity target (Charles, 2026-08-28 ~18:45)](#parity-target-charles-2026-08-28-1845)
- [Org model + interaction layer (Charles, 2026-08-28 evening)](#org-model--interaction-layer-charles-2026-08-28-evening)
- [Toolchain policy (Charles, 2026-08-28 evening) — HARD RULE](#toolchain-policy-charles-2026-08-28-evening--hard-rule)
- [Tooling architecture (open question -> PMO recommendation)](#tooling-architecture-open-question---pmo-recommendation)
- [MVP demo bar (Charles, 2026-08-28 ~19:00)](#mvp-demo-bar-charles-2026-08-28-1900)
- [Model selection layer (Charles, 2026-08-28 ~19:00)](#model-selection-layer-charles-2026-08-28-1900)
- [Key proxy: placeholders only, material never leaves the vault (Charles, 2026-08-28 evening)](#key-proxy-placeholders-only-material-never-leaves-the-vault-charles-2026-08-28-evening)
- [Tooling = standalone public FLOSS repos, loosely coupled (Charles, 2026-08-28 evening)](#tooling--standalone-public-floss-repos-loosely-coupled-charles-2026-08-28-evening)
- [ALL dev work in Docker (Charles, 2026-08-28 evening) — BIG RULE](#all-dev-work-in-docker-charles-2026-08-28-evening--big-rule)
- [Events are V1 scope: webhook receiver now (Charles, 2026-08-28 evening)](#events-are-v1-scope-webhook-receiver-now-charles-2026-08-28-evening)
- [Hermes killed; OpenWebUI is the interactive front door (Charles, 2026-08-28 evening)](#hermes-killed-openwebui-is-the-interactive-front-door-charles-2026-08-28-evening)
- [Budget + quota reality (Charles, 2026-08-28 evening)](#budget--quota-reality-charles-2026-08-28-evening)
## What it is NOT ## What it is NOT
- Not a TUI. Inner loop = pull Redmine -> run turn -> write Discourse/Redmine - Not a TUI. Inner loop = pull Redmine -> run turn -> write Discourse/Redmine
back -> chain next turn. Humans observe via files/Redmine/Discourse, not by back -> chain next turn. Humans observe via files/Redmine/Discourse, not by
+243 -11
View File
@@ -1,16 +1,248 @@
# MOPAC harness # MOPAC harness
Headless multi-vertical agent harness in Go. AGPLv3. SoR: Redmine project MOPAC; docs: Discourse; code: Gitea reachableceo/MOPAC. The MOPAC harness is a headless agent conductor written in Go: it pulls a task
Spec: DESIGN.md. Status: see Redmine MOPAC project issues and REPORT.md. from Redmine (or a demo issue), routes it to the right model through LiteLLM,
runs ONE bounded turn with an allow-listed bash tool, and writes the result as
a REPORT file. There is no daemon and no TUI — callers chain turns by
re-invoking `harness once`. It is the execution core for the org's vertical
stacks: humans and other agents interact with a stack only through Redmine
(SoR), Discourse (docs) and Gitea (code), never by attaching to the loop.
## Build & run (v0 skeleton) Status: 2026-08-28 — skeleton; MVP demo path live (single GLM turn through
LiteLLM to REPORT, proven live 2026-08-28).
go build ./... && go vet ./... && go test ./... ## Quickstart
cp harness.toml.example harness.toml # then set the key refs
./bin/harness once --dry-run --demo # intake + plan, no LLM call
HARNESS_LITELLM_KEY=sk-... ./bin/harness once --demo # MVP demo turn
`harness once` runs ONE conductor iteration (intake -> model routing -> All dev work happens inside a Docker builder (host stays toolchain-free);
bounded turn via LiteLLM -> REPORT file) and exits; chain iterations by `docker pull` of the builder is pre-authorized. Commands below were verified
re-invoking. There is no daemon. Exit codes: 0 ok, 1 config/usage, on 2026-08-28 from a fresh clone.
2 intake, 4 llm/turn. See `harness help`.
### Build and test
```sh
docker run --rm -v "$PWD:/h" -w /h \
-u "$(id -u):$(id -g)" -e HOME=/tmp \
golang:1.26-bookworm \
sh -c 'go build -o bin/harness ./cmd/harness && go vet ./... && go test ./...'
```
Expected output (tail):
```text
ok git.knownelement.com/reachableceo/MOPAC/harness/internal/config
ok git.knownelement.com/reachableceo/MOPAC/harness/internal/intake
ok git.knownelement.com/reachableceo/MOPAC/harness/internal/llm
ok git.knownelement.com/reachableceo/MOPAC/harness/internal/loop
ok git.knownelement.com/reachableceo/MOPAC/harness/internal/models
ok git.knownelement.com/reachableceo/MOPAC/harness/internal/tools
ok git.knownelement.com/reachableceo/MOPAC/harness/internal/writeback
```
(The module path still carries the old `reachableceo` org prefix; the repo
lives at [git.knownelement.com/ukrrs/MOPAC](https://git.knownelement.com/ukrrs/MOPAC).)
### Configure
```sh
cp harness.toml.example harness.toml
```
`harness.toml` is gitignored. It holds no secrets — only key refs
(`env:NAME`, `file:PATH`, `literal:VALUE`; `bw:` is reserved for the
bitwarden wrapper, build phase 3) that are resolved at runtime and redacted
from every error path.
### Dry-run (no LLM call, no REPORT)
```sh
./bin/harness once --dry-run --demo
```
Expected output:
```text
harness: intake: demo issue from harness.toml [demo]
harness: task demo-1 (demo): "MVP demo: GLM self-description" class=primary -> mopac-primary -> glm-5.3
PLAN (dry-run)
vertical: demo
task: [demo-1] MVP demo: GLM self-description (source demo)
routing: class "primary" -> tier mopac-primary -> model glm-5.3
tools: bash (allow=18 deny=7 default=deny timeout=60s)
bound: max 8 rounds
harness: dry-run complete, no LLM call made
```
### Live demo turn (the MVP bar)
```sh
HARNESS_LITELLM_KEY=<vertical virtual key> ./bin/harness once --demo
```
The model's reply lands verbatim in `reports/REPORT-latest.md` with model /
tier / class / tokens / rounds telemetry. Without the key the run fails fast
and clean (verified):
```text
harness: litellm key: environment variable HARNESS_LITELLM_KEY is not set
```
exit code 1 (config error). The with-key path was proven live on 2026-08-28:
the `[demo]` prompt "tell me about yourself" routed to `glm-5.3` via tier
`mopac-primary` and the GLM self-description landed as the REPORT.
### Help
```sh
./bin/harness help
```
Prints the full usage block reproduced under [CLI reference](#cli-reference).
## Architecture
```mermaid
flowchart TD
S(["harness once"]) --> I["INTAKE<br/>Redmine /issues.json scope query<br/>or the [demo] issue"]
I -->|"task + class"| RT["ROUTING<br/>class -&gt; tier -&gt; concrete model<br/>[models] + [models.classes]"]
RT --> T["BOUNDED TURN<br/>OpenAI-compatible chat via LiteLLM<br/>gated bash tool, max_rounds cap"]
T --> W["REPORT<br/>reports/REPORT-&lt;vertical&gt;-&lt;task&gt;-&lt;ts&gt;.md<br/>+ REPORT-latest.md, atomic write"]
W --> E(["exit 0"])
E -.->|"re-invoke to chain the next turn"| S
```
One conductor iteration per process: INTAKE resolves the released scope (a
Redmine query, or the `[demo]` issue with `--demo`) into a task with a class;
ROUTING maps the class through the config-only tier table to a concrete proxy
model; the bounded TURN drives an OpenAI-compatible chat loop through LiteLLM
(retry/backoff on 429/5xx/transport, usage accounting) with a tool-calling
loop capped at `max_rounds`, where the only tool today is an allow-listed
bash exec whose gate denials feed back to the model as tool results instead
of failing the turn; WRITEBACK persists the final assistant reply as a REPORT
file plus `REPORT-latest.md` (atomic tmp+rename). On a mid-turn LLM failure
after content exists, a partial REPORT is still written with the error as the
stop reason. Chaining is by re-invocation — there is no daemon.
## CLI reference
Subcommands (from `harness help`):
| Subcommand | Purpose |
|---|---|
| `harness help` | print usage (also `-h`, `--help`) |
| `harness once` | run ONE conductor iteration, then exit |
Flags for `once`:
| Flag | Meaning |
|---|---|
| `-config PATH` | config file (default `$HARNESS_CONFIG`, then `./harness.toml`) |
| `-dry-run` | intake + plan only; no LLM call, no REPORT |
| `-demo` | run the `[demo]` issue instead of Redmine intake |
| `-task-id ID` | run only the task/issue with this id |
Exit codes:
| Code | Meaning |
|---|---|
| 0 | ok (including "no tasks in scope") |
| 1 | usage / config / routing / writeback error |
| 2 | intake error |
| 4 | llm / turn error |
## Configuration
Every `harness.toml` section (see `harness.toml.example`, load-tested so it
cannot rot):
| Section | Keys | Meaning |
|---|---|---|
| top level | `vertical` | vertical/stack identity for this harness instance |
| | `work_root` | where the bash tool executes (default `.`) |
| | `report_dir` | where REPORT files land (default `reports`) |
| `[loop]` | `max_rounds` | max LLM round trips per turn, tool calls included (default 8) |
| `[redmine]` | `url`, `key_ref` | SoR issues endpoint + auth key ref |
| | `scope_query` / `scope_query_id` | released-scope filter: raw `/issues.json` params, or a saved query id (`scope_query` wins when both are set) |
| | `class_field`, `default_class` | custom field carrying the task class; fallback for issues without it (default `Class` / `primary`) |
| | `limit` | max issues fetched per intake (default 50) |
| `[litellm]` | `base_url`, `key_ref` | OpenAI-compatible proxy endpoint + key ref |
| | `timeout_secs`, `max_retries` | per-request timeout (default 120) and retry count (default 2) |
| `[models]` | tier keys, `default_tier` | tier alias → concrete proxy model (see below) |
| `[models.classes]` | class keys | task class → tier alias (see below) |
| `[tools.bash]` | `enabled`, `timeout_secs`, `max_output_bytes` | bash gate on/off, per-command timeout (default 60s), output truncation (default 100000) |
| | `default` | verdict for commands matching no rule: `allow` or `deny` (org preset: deny) |
| | `allow`, `deny` | rule lists; deny beats allow; `cmd *` word-boundary, `pfx*` raw prefix, `pfx/**` path prefix, `*` universal; compound commands checked segment by segment; command substitution and subshells always denied |
| `[demo]` | `id`, `subject`, `prompt`, `class` | the issue `--demo` runs instead of Redmine intake |
Key refs accepted anywhere a `*_ref` appears: `env:NAME`, `file:PATH`,
`literal:VALUE` (last resort), and `bw:REF` (reserved; errors until the
bitwarden wrapper lands in build phase 3).
## Model routing
Static, config-only, no heuristics: a task's class maps to a tier alias, the
tier alias maps to the concrete model actually sent to the proxy. Unknown
class = hard error naming the config section.
Tiers (`[models]`):
| Tier | Concrete model | Role |
|---|---|---|
| `mopac-study` | `glm-4.7-flash` | flash tier |
| `mopac-code` | `glm-5.2` | flagship |
| `mopac-review` | `glm-5-turbo` | mid |
| `mopac-primary` | `glm-5.3` | default / flagship+ |
| `mopac-vision` | `glm-4.6v` | vision when needed |
Classes (`[models.classes]`):
| Task class | Tier | Concrete model |
|---|---|---|
| `study` | `mopac-study` | `glm-4.7-flash` |
| `read` | `mopac-study` | `glm-4.7-flash` |
| `code` | `mopac-code` | `glm-5.2` |
| `architecture` | `mopac-code` | `glm-5.2` |
| `review` | `mopac-review` | `glm-5-turbo` |
| `summarize` | `mopac-review` | `glm-5-turbo` |
| `writeback` | `mopac-review` | `glm-5-turbo` |
| `vision` | `mopac-vision` | `glm-4.6v` |
| `primary` | `mopac-primary` | `glm-5.3` |
| (no class) | `default_tier` = `mopac-primary` | `glm-5.3` |
## Status
Sourced from [REPORT.md](REPORT.md) — keep both in sync.
| Area | State | Detail |
|---|---|---|
| Config | Works | TOML-subset parser, defaults + validation, secret refs only, redacted errors |
| Model routing v0 | Works | tier + class maps; concrete model resolved before the request leaves |
| Conductor single-shot | Works | `harness once` chainable; `--dry-run` makes zero LLM calls (test-asserted) |
| Redmine intake | Works | `/issues.json` scope query, class custom field, `--demo` fallback |
| Bounded turn | Works | LiteLLM chat, retry/backoff, tool loop capped at `max_rounds`, partial REPORT on mid-turn failure |
| Exec tool | Works | allow-listed bash, segment-wise compound checks, timeout + truncation |
| REPORT writeback | Works | timestamped file + `REPORT-latest.md`, atomic, full telemetry |
| Tests | Works | table-driven, stdlib only; build/vet/test clean on go1.26 |
| Redmine note writeback | Stubbed | REPORT is file-only today |
| Budget/semaphore gate | Stubbed | tokens in REPORT, no cost/spend enforcement yet |
| `bw:` key refs | Stubbed | error until the bitwarden wrapper (phase 3) |
| Streaming + turn resume | Stubbed | retry is request-level today |
| Session persistence/repair | Stubbed | not started |
| Write confinement | Stubbed | declared roots + symlink checks land with the permission layer |
| Task ordering | Stubbed | first-in-scope; chain with `--task-id` until status transitions land |
| Local inbox intake | Stubbed | not started |
| Next (phase 3) | Next | bitwarden wrapper, full permission layer, Redmine note writeback + status transitions, budget gate via LiteLLM spend APIs, streaming with resume |
## Docs and links
- [DESIGN.md](DESIGN.md) — design spec (hard rules, build order, org model)
- [REPORT.md](REPORT.md) — current build status
- [docs/PORTING-NOTES-crush.md](docs/PORTING-NOTES-crush.md) — sessions/MCP/provider study of the crush agent
- [docs/PORTING-NOTES-maki.md](docs/PORTING-NOTES-maki.md) — permission parsing and token-reduction study of maki
- [docs/PORTING-NOTES-secrets.md](docs/PORTING-NOTES-secrets.md) — Bitwarden secrets study; feeds the bitwarden-go tool
- Repository: <https://git.knownelement.com/ukrrs/MOPAC>
- Sibling tool repos (spec seeds): [mopac-keyproxy](https://git.knownelement.com/ukrrs/mopac-keyproxy), [mopac-bitwarden-go](https://git.knownelement.com/ukrrs/mopac-bitwarden-go)
- SoR: Redmine project MOPAC; docs: Discourse
## License
AGPLv3 — see [LICENSE](LICENSE).
+2
View File
@@ -4,6 +4,8 @@ Scope for MOPAC harness: MCP client, sessions/resume, provider routing,
headless loop; OAuth deferred (LiteLLM key suffices). Paths relative to headless loop; OAuth deferred (LiteLLM key suffices). Paths relative to
`reference/crush/`; line refs from study 2026-08-28, drift possible. `reference/crush/`; line refs from study 2026-08-28, drift possible.
Status: 2026-08-28 — complete; reference study, line refs may drift with upstream.
## 1. MCP plumbing ## 1. MCP plumbing
Code in `internal/agent/tools/mcp/` (NOT `internal/client/` = daemon RPC). Code in `internal/agent/tools/mcp/` (NOT `internal/client/` = daemon RPC).
+2
View File
@@ -5,6 +5,8 @@ Scope for MOPAC harness: tree-sitter bash permission parsing, token-reduction
`reference/maki/`; line refs from study 2026-08-28, drift possible. Rust core `reference/maki/`; line refs from study 2026-08-28, drift possible. Rust core
+ Lua tool plugins; Lua API mirrors tree-sitter via maki-lua. + Lua tool plugins; Lua API mirrors tree-sitter via maki-lua.
Status: 2026-08-28 — complete; reference study, line refs may drift with upstream.
## 1. Tree-sitter bash permission parsing ## 1. Tree-sitter bash permission parsing
Two halves: Lua plugin extracts "scopes" from command text; Rust Two halves: Lua plugin extracts "scopes" from command text; Rust
+120
View File
@@ -0,0 +1,120 @@
# Porting notes: KNELSecretsManager (reference/KNELSecretsManager)
Scope for MOPAC harness: Bitwarden secret retrieval for `bw:` key refs
(reserved in `internal/config/keys.go:34-35`, errors until "phase 3").
DESIGN.md: Go `bitwarden-go` wrapper replaces KNELSecretsManager; secrets
never in logs. Line refs from study 2026-08-28, drift possible.
Status: 2026-08-28 — complete; open questions for Charles pending (section 3), feeds ukrrs/mopac-bitwarden-go.
## 1. Current surface
Two generations ship in the repo; ADR-002 (`docs/ADR-002-ContainerBW.md`)
supersedes the standalone script with a containerized `bw` wrapper.
**Legacy `secrets-manager.sh` (v2.0)** — single bash script, CLI only:
`install | get <name> | list | test`, `-c/--config FILE`. Every invocation
redoes the full lifecycle: source plaintext config (BW_SERVER_URL,
BW_CLIENTID, BW_CLIENTSECRET, BW_PASSWORD) -> `bw config server` ->
`bw logout` -> `bw login --apikey` -> `bw unlock --raw` -> `bw get password
<name>` (secrets-manager.sh:119-159). Exit codes 10/20/30/40/50/60/70
(config/bw-missing/install/server/session/secret/login). Logs to
`/tmp/secrets-manager.sh.log` (names, not values). Designed to be vendored
into shell frameworks; Makefile lint/test/vendor-test targets.
**Current container wrapper (production since 2026-08-13, ADR-002)**:
- `bin/bw-install.sh` — downloads pre-compiled bw `2026.7.0` (inside an
alpine container; no host Node/wget), builds `reachableceo-bw-native:
2026.7.0` (debian:bookworm-slim + ca-certificates, `docker/bw-native/
Dockerfile`), installs wrapper to `~/.local/bin/bw` + entrypoint to
`~/.local/share/bw/entrypoint.sh`.
- `bin/bw-cli.sh` (host) — loads `~/.config/bw/env` (single-quoted values,
mode 600; the only secrets-on-disk allowed org-wide), then `docker run`
with named volume `tsys-bw-cli-state` mounted at `/root/.config/Bitwarden
CLI` so login state persists across calls.
- `bin/bw-entrypoint.sh` (container) — idempotent: `bw config server` ->
`bw login --apikey` (no TOTP; API key minted from an authenticated web
session) -> `bw unlock --passwordfile` (password briefly in /tmp) ->
`bw sync` -> `exec bw "$@"` with BW_SESSION.
- Interface is transparent `bw`: `status | list items/collections | get
password|totp|item <name> | generate`. Lookup is by item **name**; the
default secret is the item's password field (e.g. `APIKEY-pushover`).
- Backend: Vaultwarden at `https://pwvault.turnsys.com`, machine account
`coo@turnsys.com`, API-key auth + master-password unlock.
**Consumers**: nothing in the MOPAC harness consumes it yet — sole planned
integration is the `bw:` key-ref prefix (`internal/config/keys.go`,
asserted by `config_test.go:126,130`; `harness.toml.example:5`). First
real secrets needed: Redmine API key + LiteLLM key (`[redmine] key_ref`,
`[litellm] key_ref`). Siblings still on Bitwarden: KNELCredsManager
(`~/.creds/*.env` feeder) and `mcp-bitwarden-wrapper.sh` in
KNEL-AIMiddleware.
**Weaknesses to fix in the port**: plaintext master password + API secret
on disk; full login/unlock/sync per call (slow, racy logout in legacy);
secret values echoed to stdout and captured by callers into env; docker
required per host; "native" bw binary is secretly a Node SEA (ADR-002
caveat); legacy script `bw logout`s any concurrent session.
## 2. Replacement design sketch (Go)
**Shape**: `internal/secrets` package + thin `MOPAC/tools/bitwarden-go`
CLI (per DESIGN.md:88) so both harness and humans share one resolver.
Config gains `ResolveKeyRef` case for `bw:REF` (keys.go already reserves
it; keep `redact()` semantics).
**bw execution**: exec `bw` from PATH (the ADR-002 wrapper makes it
transparent whether bw is native or containerized). Treat as opaque
subprocess: `bw get password <name> --session <tok> --nointeraction`,
JSON mode where useful (`bw get item <name>`), 30s timeout, capture
stdout into a `[]byte` not a logged string.
**Machine credentials** (bootstrapping only): prefer env
`BW_CLIENTID/BW_CLIENTSECRET/BW_PASSWORD`; fallback `~/.config/bw/env`
parsed by hand (single-quoted values, require mode 0600). Never in
harness.toml, never in logs. Server URL from env `BW_SERVER` with
Vaultwarden default.
**Unlock strategy**: once per process — config server (idempotent),
`bw login --apikey` (skip if `bw status` says logged in), `bw unlock
--passwordenv BW_PASSWORD --raw` -> session key held **in memory only**;
single `bw sync` after login. On session-expired error mid-run: re-unlock
once, then fail. Never `bw logout` (breaks concurrent clients).
**Lookup by key**: `bw:<item-name>` -> password field (matches current
usage). Optional `bw:<name>#<field>` and `bw:<name>#totp` if non-password
fields are needed. Not-found => typed `ErrSecretNotFound`; auth failures
=> `ErrUnlockFailed` (mirrors exit codes 60/50/70 for callers).
**Caching**: per-run in-memory `map[ref]string` + mutex (multi-vertical
workers may resolve concurrently); negative results cached too. Session
token cached for process lifetime, zeroed on exit. No disk cache of
values; optional disk persistence of BW_SESSION is an open question.
**Never-log rules** (extends keys.go:70-78): log refs only as
`bw:<name>****`; error strings never embed values; subprocess stderr from
bw is scrubbed/dropped (bw can echo fragments); secrets never written to
REPORT.md, spans, or crash dumps; `go test` uses fake bw stub, never the
real vault.
## 3. Open questions for Charles
1. **Substrate**: keep shelling out to `bw` (containerized per ADR-002,
Docker on every harness host) or implement the Vaultwarden REST API
directly in Go (truly zero Node/bw, but reimplements crypto + sync)?
DESIGN.md:84 says "replaces Node bw CLI dependency" — which side of
that line do you want?
2. **Machine account**: dedicated service account for the harness instead
of `coo@turnsys.com`, with a vault collection scoped to only the
secrets the harness needs?
3. **Ref syntax**: is `bw:<item-name>` (password field) sufficient for
v1, or do you want `#field` / `#totp` / collection-scoped refs now?
4. **Session lifetime**: unlock per `harness once` run only, or persist
BW_SESSION (0600, TTL) across runs to avoid unlock cost each start?
5. **Secret inventory + names**: confirm exact vault item names for
Redmine and LiteLLM keys so `key_ref` values can be pinned in
harness.toml.example.
6. **Scope of port**: absorb KNELCredsManager's `~/.creds/*.env`
contract too, or strictly the `bw:` refs used by the harness?
7. **Subprocess injection**: any consumers needing secrets as env vars
for child processes (LiteLLM?), vs HTTP-header-only as today?