ops(archive): quota, deploy, pdf TASK/REPORT pairs verified+archived
This commit is contained in:
@@ -0,0 +1,159 @@
|
||||
# REPORT — Quota monitoring + back-pressure + usage mgmt + resource gate (Redmine 490+491)
|
||||
|
||||
- **When**: 2026-08-29 ~05:00-05:50 CST
|
||||
- **Repo**: `projects/meta/MOPAC/harness` @ `fc518c4` (pushed to origin main)
|
||||
- **Spec**: `docs/SPEC-20260829-charles-brief.md` — "It's a marathon" + Roadmap 1
|
||||
- **Gate**: `./dev.sh check` (build + vet + test) clean, all packages `ok`
|
||||
(config, quota, loop, serve, events, intake, llm, models, tools, writeback)
|
||||
|
||||
## 1. Decision: Redis container (not LiteLLM-native)
|
||||
|
||||
**Chosen: one Redis container, spoken to by a stdlib RESP2 mini-client.**
|
||||
|
||||
Rationale:
|
||||
|
||||
1. **LiteLLM tracks dollars, z.ai meters credits.** The thing we must
|
||||
share is the coding-plan 5-hour + weekly CREDIT buckets with peak
|
||||
multipliers (input x6.9 + cached x1.7 + output x24 per 10k tokens;
|
||||
flash 2.3/0.56/8; off-peak 50% off). LiteLLM's budget system is
|
||||
per-virtual-key $ spend against budgets it enforces itself — a lossy
|
||||
double-accounting that breaks on every plan change and cannot express
|
||||
"two harness accounts share one z.ai key".
|
||||
2. **We own the write path anyway.** With no z.ai usage endpoint (see
|
||||
§2), consumption is OUR estimate; a store only we write to is the
|
||||
natural fit. Doing it "in LiteLLM" would mean SQL against another
|
||||
project's schema — an upgrade hazard, not an integration.
|
||||
3. **9 instances, 2 hosts, one account each**: TTL'd rolling-window
|
||||
counters (`INCRBYFLOAT` + `EXPIRE`), snapshot publish/read, atomic
|
||||
cross-instance — this is exactly Redis. Postgres via LiteLLM gives
|
||||
us none of those primitives for free.
|
||||
4. **Charles's constraints honored**: docker container only
|
||||
(`redis:7-alpine`, runbook in README; bind to 192.168.3.78:6390),
|
||||
config in `harness.toml` (`redis_url`), zero host packages, zero Go
|
||||
dependencies (the repo remains stdlib-only; the RESP2 client is
|
||||
~150 lines and fake-server tested).
|
||||
5. **Fail-soft**: redis down = this instance's local estimate, loop
|
||||
keeps running. LiteLLM-native would couple the loop's health to the
|
||||
proxy's DB.
|
||||
|
||||
Keys: `mopac:quota:<account>:{snapshot, est:5h:<window>, est:weekly:<week>}`.
|
||||
LiteLLM $-spend budget keys remain a separate, still-open item (README
|
||||
status table says so honestly).
|
||||
|
||||
## 2. z.ai usage endpoint findings (LIVE VERIFICATION: OPEN)
|
||||
|
||||
Probed 2026-08-29 ~05:05 with the real plan key (from the litellm
|
||||
container env; never logged, never written to disk):
|
||||
|
||||
- `GET /api/coding/paas/v4/{usage,limits,info,quota,credit/usage,...}`
|
||||
-> **404** (JSON 404 from the gateway, so the route does not exist)
|
||||
- `/api/coding/paas/v4` (the live chat endpoint LiteLLM uses) is real
|
||||
and serving — the 404s are route-level, not auth or reachability.
|
||||
- `/api/coding/{user/info,usage}` -> 200 with `{"code":500,"msg":"404
|
||||
NOT_FOUND"}` — gateway routes, inner services absent.
|
||||
- Docs sweep (`docs.z.ai` llms.txt + devpack pages): the plan DOCS the
|
||||
buckets in detail (5h + weekly credits per tier, multipliers,
|
||||
off-peak discount, reset rules) and points humans at the billing web
|
||||
page for consumption — **no public usage REST route is documented.**
|
||||
|
||||
Per the task's fallback: the parser targets the documented bucket model
|
||||
(strict decode; unknown shapes error so an HTML error page can never
|
||||
read as "quota fine"; alias-tolerant field names), is table-driven
|
||||
tested against a fake server (bearer header asserted, key-leak
|
||||
asserted), and `usage_url` in `[quota]` flips it on the day z.ai ships
|
||||
one. Until then the gate runs on locally estimated consumption from the
|
||||
documented credit formula — which is exactly the number the loop
|
||||
controls anyway.
|
||||
|
||||
## 3. Peak window — Charles's guess verified
|
||||
|
||||
z.ai docs: **peak = Mon-Fri 14:00-18:00 Singapore (UTC+8), off-peak
|
||||
50% off.** 14:00 SGT = 01:00 CST (winter, UTC-6) / 00:00 CDT (summer,
|
||||
UTC-5). So "0100 to 0500 CST" is correct for winter and one hour early
|
||||
during US DST — config carries the window + `timezone` + `peak_weekdays_only`
|
||||
(defaults `01:00`-`05:00` America/Chicago, weekdays only), and the
|
||||
report notes the March/November drift. Schedule logic is table-tested
|
||||
across the edges: window boundaries, weekend exclusion, overnight-wrap
|
||||
windows (weekday check on the window's start day), and zone-vs-instant
|
||||
semantics.
|
||||
|
||||
## 4. Config surface (see `harness.toml.example` + README table)
|
||||
|
||||
- `[quota]`: `enabled`, `account`, `plan_5h_credits` (28000),
|
||||
`plan_weekly_credits` (140000), `usage_url`+`key_ref`+`poll_interval_secs`,
|
||||
`defer_at_pct` (85) / `block_at_pct` (95), `peak_start`/`peak_end`/
|
||||
`timezone`/`peak_weekdays_only`, `peak_classes` (study, read),
|
||||
`redis_url`.
|
||||
- `[resources]`: `enabled`, `max_load_avg` (6), `min_mem_available_mb`
|
||||
(2048), `min_disk_free_mb` (5120), `max_io_delay_pct` (90), test seams
|
||||
`proc_root`/`sys_root`.
|
||||
- Both **off by default** (existing configs unchanged — test-asserted).
|
||||
- CLI: `harness quota status | probe | gate`.
|
||||
|
||||
## 5. How the loop behaves at quota exhaustion (the 19:00 wall, replayed)
|
||||
|
||||
`TestLoopQuotaWallDefersAndRecovers` (fake Redmine + fake LLM + fake
|
||||
usage endpoint at 97% weekly):
|
||||
|
||||
1. Scan sees the issue -> gate decides DEFER -> **zero LLM calls**, one
|
||||
stdout line + one `"type":"defer"` JSONL event with the reason
|
||||
("quota: weekly bucket at 97% (>= block 95%): all classes deferred
|
||||
until reset").
|
||||
2. The task is **not consumed** (no dispatch marker, no note) — defer is
|
||||
a throttle, so quota recovery needs no human touch.
|
||||
3. Endpoint flips healthy -> next scan dispatches normally, notes the
|
||||
REPORT, status transitions. The 19:00-class failure no longer
|
||||
exists: pre-wall, the loop runs LLM-lite classes only past
|
||||
`defer_at`; at `block_at` everything pauses, surfaced, until reset.
|
||||
|
||||
Decision order: resources -> block wall -> peak-window class
|
||||
restriction -> soft-quota heavy deferral. Never a hard fail; unknown
|
||||
quota state is permissive.
|
||||
|
||||
## 6. Usage accounting (feeds the Discourse reports)
|
||||
|
||||
Every dispatched turn appends `class`, `prompt/completion/total_tokens`,
|
||||
and estimated `credits` to its `report` event in `loop.jsonl`;
|
||||
`loopState` rebuilds per-class totals from the JSONL at startup, and
|
||||
`harness quota status` renders the table (turns/tokens/credits per
|
||||
class + TOTAL). Also verified live in-container: real `/proc` reads
|
||||
(load 13.43, PSI io 0.3%, disk 44.9GB) and correct TZ evaluation.
|
||||
|
||||
## 7. Test results (TDD, table-driven, fake clock/servers)
|
||||
|
||||
- `internal/quota`: parser (canonical + aliases + hostile bodies),
|
||||
schedule edges (TZ, wrap, weekdays), credit math (peak/off-peak,
|
||||
flash/flagship, unknown-model conservatism), decision levels (healthy
|
||||
/ defer / block / peak interactions), estimate snapshots (redis +
|
||||
local), fake-redis RESP2 server round-trips, dead/nil-state fail-soft,
|
||||
resource fixtures (PSI present/absent, unreadable).
|
||||
- `internal/loop`: wall defer+recover, peak heavy-defer + flash-run +
|
||||
off-peak release, resource-busy defer, usage accounting JSONL +
|
||||
table, gate-off-by-default regression.
|
||||
- `internal/config`: [quota]/[resources] load + 7 validation-error
|
||||
cases; TOML float support.
|
||||
- Docker dev discipline throughout (`./dev.sh` only); host untouched.
|
||||
|
||||
## 8. Resource gate (491) + cgroup runbook
|
||||
|
||||
Read-only monitor (loadavg, MemAvailable, statfs disk free,
|
||||
`/proc/pressure/io` some-avg60 — skipped when PSI absent, read errors
|
||||
never defer). cgroup enforcement documented as deploy-time in the README
|
||||
runbook (`--memory/--cpus/--pids-limit` / systemd slice), per Charles's
|
||||
"docker + cgroups for all work" rule.
|
||||
|
||||
## 9. Open items
|
||||
|
||||
- **LIVE VERIFICATION open**: z.ai usage endpoint (flip `usage_url`
|
||||
when it ships; parser + probe command ready).
|
||||
- Weekly bucket reset is approximated to Monday 00:00 plan-TZ until
|
||||
`reset_at` arrives from a real endpoint.
|
||||
- Turn `cached_tokens` not yet captured from LiteLLM responses (LLM
|
||||
client returns prompt/completion only) — credits currently
|
||||
conservative (cache discount not credited).
|
||||
- Redis container not yet deployed to 192.168.3.78 (runbook written;
|
||||
`enabled = false` defaults mean nothing regresses until then).
|
||||
- LiteLLM $-spend budget keys (separate from credit buckets) still
|
||||
phase 3.
|
||||
|
||||
— PMO worker, MOPAC harness self-host loop
|
||||
@@ -0,0 +1,118 @@
|
||||
# REPORT — Multi-account deploy packaging + runbook (Redmine 494)
|
||||
|
||||
- **When**: 2026-08-29 ~05:45-06:00 CST
|
||||
- **Repo**: `projects/meta/MOPAC/harness` @ `b7799ea` (pushed to origin main)
|
||||
- **Spec**: `docs/SPEC-20260829-charles-brief.md` — account list (9 Linux
|
||||
accounts / 2 hosts), "no root on target accounts" PMO constraint
|
||||
- **Gates**: `./dev.sh check` clean (all 11 Go packages ok, untouched);
|
||||
`make deploy-test` **13/13 ok**; `make release` verified static +
|
||||
end-to-end smoke (install COSWFO in a fake HOME -> `once --dry-run
|
||||
--demo` exit 0 via the actual release binary)
|
||||
|
||||
## 1. What Charles executes (the whole deployment)
|
||||
|
||||
Authority: `deploy/runbook.md` + `deploy/accounts.tsv` in the repo.
|
||||
|
||||
1. **Build once** (workstation, ~2-3 min): `make release` ->
|
||||
`bin/harness-linux-amd64` (digest-pinned docker builder, CGO off,
|
||||
linux/amd64, static + stripped — `file` says "statically linked").
|
||||
2. **Stage per host** (~1 min each): `tar czf /tmp/mopac-deploy.tgz
|
||||
deploy bin/harness-linux-amd64` + one `scp` per host.
|
||||
3. **Install per account** (~30s each, idempotent): ssh in, then either
|
||||
direct (`install-account.sh reachableceo`) or via identity switch
|
||||
(`sudo -u TSGBOD -H sh -c '... && install-account.sh TSGBOD'` — sudo
|
||||
is identity-switch only, nothing system-wide). The installer creates
|
||||
`~/.mopac/{bin,state/loop,state/events,reports,work}`, installs the
|
||||
binary, renders `harness.toml` from the template + TSV row, writes the
|
||||
0600 `~/.mopac/env` secrets template, generates `mopac-start`/`mopac-stop`.
|
||||
4. **Secrets bootstrap** (~1 min/account): fill `~/.mopac/env` from
|
||||
Bitwarden (Redmine key, LiteLLM vkey, 3 webhook secrets, serve vkey).
|
||||
Configs stay secret-free (env: refs only).
|
||||
5. **Verify then start** (~25s/account): see §3.
|
||||
6. **Reboot persistence**: cron `@reboot` line (documented verbatim) or
|
||||
manual re-run of the idempotent `mopac-start`. **No systemd** — that
|
||||
needs root, which the accounts don't have; both options documented.
|
||||
|
||||
## 2. Concurrency guard (why 9 daemons per host can't collide)
|
||||
|
||||
Port scheme from `accounts.tsv` (index is global 0-8, listed order):
|
||||
`events = 4100 + index`, `serve = 8090 + index`.
|
||||
|
||||
| account | host | events | serve | | account | host | events | serve |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| reachableceo | streaming | 4100 | 8090 | | reachableceo-offstage | offstage | 4105 | 8095 |
|
||||
| TSGBOD | streaming | 4101 | 8091 | | COSRCEO-Personal | offstage | 4106 | 8096 |
|
||||
| TSGCOO | streaming | 4102 | 8092 | | COSRCEO-Biz | offstage | 4107 | 8097 |
|
||||
| TSGCTO | streaming | 4103 | 8093 | | COSWFO | offstage | 4108 | 8098 |
|
||||
| TSGCCO | streaming | 4104 | 8094 | | | | | |
|
||||
|
||||
The loop daemon has no port; its state (`state/loop`, `state/events`) is
|
||||
per-account under `~/.mopac/`. The installer hard-asserts the scheme per
|
||||
row (dies if `events != 4100+index`), and `deploy/tests.sh` re-asserts
|
||||
fleet-wide (host,port) uniqueness.
|
||||
|
||||
## 3. Verification steps (per account, runbook step 5+6)
|
||||
|
||||
1. `~/.mopac/bin/harness once --dry-run --demo -config ~/.mopac/harness.toml`
|
||||
— no secrets, no LLM call; must exit 0 printing the PLAN with the
|
||||
account's vertical (this exact path is what the release smoke ran).
|
||||
2. `. ~/.mopac/env && ~/.mopac/bin/harness loop --once --dry-run -config ...`
|
||||
— first real Redmine scan of the account's scope (needs the Redmine
|
||||
key): prints what would dispatch, writes nothing.
|
||||
3. `mopac-start`, then healthz on the account's two ports
|
||||
(`curl http://127.0.0.1:<events>/healthz`, `<serve>/healthz`) and an
|
||||
authenticated `GET /v1/models` through the serve vkey; `tail
|
||||
~/.mopac/state/loop.log` for scan lines.
|
||||
|
||||
## 4. Rollback (per account, ~30s)
|
||||
|
||||
`~/.mopac/bin/mopac-stop` (SIGTERM, clean) -> `rm -rf ~/.mopac` (or `mv`
|
||||
aside to keep evidence) -> drop the cron `@reboot` line if used. Every
|
||||
trace of an instance lives under `~/.mopac/`; Redmine/Gitea data is
|
||||
untouched by removal.
|
||||
|
||||
## 5. Time estimate (the Charles window)
|
||||
|
||||
| | streaming (5 accts) | offstage (4 accts) |
|
||||
|---|---|---|
|
||||
| stage + install | ~3 min | ~2.5 min |
|
||||
| secrets bootstrap | ~5 min (unless prefilled) | ~4 min |
|
||||
| verify + start + healthz | ~2 min | ~1.5 min |
|
||||
| **total** | **~10 min** (mechanical only: ~5) | **~8 min** (mechanical: ~4) |
|
||||
|
||||
Build + Redmine project bootstrap happen before the window.
|
||||
|
||||
## 6. Tests (deploy/tests.sh, 13 assertions, all green)
|
||||
|
||||
TSV: 9 rows, spec-exact account/host sets, unique accounts + (host,port)
|
||||
pairs, `events=4100+idx`/`serve=8090+idx` on every row. Installer: render
|
||||
all 9 accounts into fake HOMEs (no leftover `@PLACEHOLDER@`s, correct
|
||||
vertical/ports/project/absolute paths, env mode 0600, helpers executable
|
||||
with the right ports); idempotent re-run (exit 0, config+env
|
||||
byte-identical); hand-edited `harness.toml` survives re-runs; unknown
|
||||
account exits non-zero; staged-bundle binary lookup works; rendered TOML
|
||||
actually loads (`once --dry-run --demo`, exit 0). Plus the live
|
||||
release-binary smoke described above.
|
||||
|
||||
## 7. Assumptions flagged in the runbook (one-line fixes, no redeploy)
|
||||
|
||||
- **Redmine project identifiers** `mopac-<vertical>` x9 must exist (or
|
||||
edit the TSV column before staging / the `scope_query` after install —
|
||||
installer never overwrites an existing config).
|
||||
- **Quota grouping** assumes one z.ai Max plan per host
|
||||
(`zai-max-1`/`zai-max-2`); `[quota]` ships commented with values
|
||||
pre-filled — confirm grouping, then flip `enabled = true` per account.
|
||||
- `[redmine.status_map]` ships empty (workflow names are per-project;
|
||||
the generated file documents the Released->Done pair to set).
|
||||
- Host short names assumed `ultix-streaming`/`ultix-offstage`; the
|
||||
installer prints an advisory on mismatch, never blocks.
|
||||
|
||||
## 8. Open items
|
||||
|
||||
- Actual ssh/scp execution is Charles's window (PMO runtime cannot
|
||||
ssh/sudo) — everything is scripted, tested locally, and pushed.
|
||||
- Webhook registration in Redmine/Discourse/Gitea per account (URLs +
|
||||
secrets) is post-install config, not packaging; noted in runbook step 0.
|
||||
- LiteLLM virtual keys per account assumed to exist on 192.168.3.78:4001.
|
||||
|
||||
— PMO worker, MOPAC harness self-host loop
|
||||
@@ -0,0 +1,79 @@
|
||||
# REPORT-20260829-1300-pdf — mopac-pdf v0 (Redmine 499)
|
||||
|
||||
Status: DONE. ukrrs/mopac-pdf created (Gitea API/tea), seeded AGPLv3,
|
||||
implemented, tested, smoked, pushed to `main`
|
||||
(https://git.knownelement.com/ukrrs/mopac-pdf). Clone at
|
||||
`~/projects/meta/MOPAC/pdf`.
|
||||
|
||||
## Engine decision: typst (digest-pinned), pandoc+LaTeX rejected
|
||||
|
||||
Full rationale + measurements committed as `docs/ENGINE-DECISION.md` in
|
||||
the repo. Summary, measured 2026-08-29 on identical content (TOC + table +
|
||||
chart figure):
|
||||
|
||||
| | typst 0.15.1 | pandoc 3.5 + xelatex |
|
||||
|---|---|---|
|
||||
| compile | 0.94 s | 7.66 s (~8x slower) |
|
||||
| image | ~200 MB | 758 MB |
|
||||
| templates | typed functions, `#show: report.with(...)` | LaTeX preamble/class surgery, multi-pass TOC |
|
||||
| output | modern typography out of the box | classic LaTeX look |
|
||||
|
||||
typst also natively streams `compile doc.typ -` to stdout, which keeps our
|
||||
engine layer pure bytes-in/bytes-out (no root-owned files, no temp
|
||||
collisions). Supply chain per Charles' clarification: prebuilt Rust binary
|
||||
in a digest-pinned image is tooling; OUR code is Go only.
|
||||
Pin: `ghcr.io/typst/typst@sha256:032e292...9c4c422f` (= 0.15.1,
|
||||
cross-checked), run `--network none` with a private `/work` root.
|
||||
|
||||
## What shipped (v0)
|
||||
|
||||
- CLI `mopac-pdf`: markdown + front-matter (title/subtitle/author/date/
|
||||
classification/template) -> PDF to stdout or `-o`; `-t`/`-T` template
|
||||
pick/override (templates embedded in the binary AND loadable from a
|
||||
dir); `-pages` helper; exit codes 0 ok / 1 usage-input / 2 engine.
|
||||
- Templates: `report` (title page, TOC, numbered headings, running header,
|
||||
page X/Y + classification footer) and `brief` (dense 1-3 pager, compact
|
||||
title block, classification badge, small tables).
|
||||
- Markdown subset the whole stack actually emits: headings, paragraphs,
|
||||
inline bold/italic/code/links, ul/ol, blockquotes, hr, GFM tables with
|
||||
alignment, fenced code, block images.
|
||||
- Charts: fenced ```chart data blocks -> bar charts rendered PURE GO
|
||||
(vendored go-chart, MIT) -> PNG figures. One type proven end to end,
|
||||
per v0 scope.
|
||||
- Dev in docker only: `dev.sh`/`make` route through the family builder
|
||||
digest; deps vendored (hermetic); smoke drives the REAL typst container
|
||||
from the host (11 checks: both templates, stdin/stdout, page counts,
|
||||
exit codes, custom template dir — all green).
|
||||
- Tests: 7 packages ok — front-matter table tests, parser tests, golden
|
||||
.md -> .typ fixtures (drift-catching), chart PNG decode, tiny
|
||||
/Pages /Count parser tests, engine argv contract via stub docker, CLI
|
||||
flag/exit-code matrix.
|
||||
|
||||
## Sample outputs (local, regenerable via ./dev.sh smoke)
|
||||
|
||||
- ~/projects/meta/MOPAC/pdf/out/sample-report.pdf — 4 pages (title, TOC, body + table + chart)
|
||||
- ~/projects/meta/MOPAC/pdf/out/sample-brief.pdf — 1 page (dense exec brief)
|
||||
- ~/projects/meta/MOPAC/pdf/out/stdin-report.pdf — stdin->stdout path
|
||||
- fixtures live in-repo: testdata/sample-{report,brief}.md (+ .typ.golden)
|
||||
|
||||
## How the briefing pipeline will call it
|
||||
|
||||
The COS briefing/harness side already speaks markdown; the contract here
|
||||
is a single exec: `mopac-pdf -o <path>.pdf <brief.md>` (or pipe bytes:
|
||||
`mopac-pdf < brief.md > brief.pdf`). Front-matter carries the identity
|
||||
(title/date/classification/template=brief for the 0630 morning brief,
|
||||
report for long-form), so callers never touch typst. Failure semantics
|
||||
are harness-shaped: 0/1/2, engine stderr surfaced verbatim, no partial
|
||||
output files on failure. Engine image + docker binary overridable by env
|
||||
for canary/migration. No daemon, no state — a pure function, safe to call
|
||||
from any loop turn.
|
||||
|
||||
## Next (proposals, not started)
|
||||
|
||||
- More chart types (line, donut) — renderer is a one-function extension
|
||||
point; needs a data-block convention for series.
|
||||
- More templates: letterhead, invoice, slide-deck-ish landscape.
|
||||
- Nested lists >1 level; footnotes; typst-side escape hardening fuzz.
|
||||
- Optional direct embedding of briefing charts without PNG round-trip
|
||||
(emit typst vector drawing for crisp print) once typst scripting
|
||||
surface stabilizes.
|
||||
@@ -80,3 +80,5 @@
|
||||
2026-08-29T05:20:40-05:00 | PMO | language policy tiered: rust ok / python avoid-but-ok in supply chain / node last-resort + consult Charles (workers must stop-and-ask)
|
||||
2026-08-29T05:24:29-05:00 | PMO | tickets 490-499 fleshed (objective/background/scope/acceptance/status structure); mopac-pmo repo live on Gitea, 7 atomic commits pushed; commit+push-as-we-go now standing practice
|
||||
2026-08-29T05:26:51-05:00 | PMO | Redmine full-power: 4 versions, 9 categories, 10 majors updated, 15 fleet children (500-514), 515/516 added, relations set; roadmap-buildout.py committed
|
||||
2026-08-29T05:29:53-05:00 | PMO | trackers corrected (Bug=defects only), bodies -> CommonMark; committed+pushed
|
||||
2026-08-29T05:45:03-05:00 | Work | ukrrs/mopac-pdf v0 done+pushed: Go md->PDF via digest-pinned typst 0.15.1, templates report/brief, pure-Go bar charts (go-chart), docker-only dev, tests+smoke green; REPORT-20260829-1300-pdf.md filed; task archived
|
||||
|
||||
@@ -14,3 +14,4 @@
|
||||
2026-08-29T01:56:30-05:00 chain drained — heartbeat exiting
|
||||
2026-08-29T04:55:19-05:00 wake sent (pending=4 active=1)
|
||||
2026-08-29T05:02:15-05:00 wake sent (pending=4 active=1)
|
||||
2026-08-29T06:02:15-05:00 wake sent (pending=4 active=1)
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
Context gathered. Setting up the task list, then finishing recon (Redmine 495, deploy template, remotes).Redmine 495 confirms scope and repo name `ukrrs/mopac-discourse-go`. Creating the Gitea repo now.Repo created (id 428). Cloning and checking the push credential pattern.Recon complete. Building the client repo now.Now the typed surfaces: categories, topics, posts.Now the fake-server test suite:Now the CLI:Now the fake server for the smoke test, plus dev.sh/Makefile/env.example:The smoke.sh has ordering bugs — rewriting it cleanly:
|
||||
@@ -0,0 +1,12 @@
|
||||
Now I have full context. Setting up the work plan:Host confirmed as ultix-streaming itself. Quick check of loop dedup/status semantics so the runbook is accurate:Dedup is refresh-safe (evRefresh). Writing the deliverables now:Now the installer:Now the runbook:Now the Makefile and the test suite:All 13 packaging tests pass. Building the release binary and running the full Go suite:While that runs, the README deploy section:Full Go suite green. Live smoke of the actual release binary against a rendered config:All green. Committing:Pushed as b7799ea. Writing the PMO report:Done — commit `b7799ea` pushed to ukrrs/MOPAC main; report at `~/.coordinate/inbox-pmo/REPORT-20260829-0900-deploy.md`.
|
||||
|
||||
**Delivered**
|
||||
- `deploy/accounts.tsv` — fleet authority: 9 accounts × (host, vertical, redmine project, quota group); ports `events=4100+idx` / `serve=8090+idx`, global index 0-8 so daemons never collide on a host
|
||||
- `deploy/install-account.sh <account>` — idempotent, runs as the target user: builds `~/.mopac/{bin,state,reports,work}`, installs the static binary, renders `harness.toml` from `deploy/harness.toml.in`, writes 0600 `~/.mopac/env` (never overwrites config/secrets; re-run = upgrade), generates `mopac-start`/`mopac-stop` (nohup, no systemd)
|
||||
- `deploy/runbook.md` — exact Charles sequence: build → stage → install (incl. `sudo -u` fan-out per host) → secrets bootstrap → verify → start → rollback, with the account-port table, cron `@reboot` option, and per-host time estimates (~10 min streaming / ~8 min offstage incl. secrets)
|
||||
- `Makefile` — `release` (digest-pinned docker builder, CGO=0, static+stripped `bin/harness-linux-amd64`) plus `check`/`deploy-test`
|
||||
- README deploy section + docs links
|
||||
|
||||
**Verified**: `./dev.sh check` all green; `make deploy-test` 13/13 (TSV scheme, all-9-account render, idempotence, refuse-to-overwrite, binary lookup); `make release` confirmed statically linked; end-to-end smoke installed COSWFO in a fake HOME and ran the release binary's `once --dry-run --demo` (exit 0).
|
||||
|
||||
**Flagged assumptions** (runbook §assumptions): the 9 `mopac-*` Redmine project identifiers must exist (or edit TSV/scope), quota grouping zai-max-1/2 per host is a guess pending key mapping, status_map ships empty by design.
|
||||
+6
-1
@@ -4,4 +4,9 @@
|
||||
Failed to override models: large model: model "glm-5.2" found in multiple providers: litellm and zai. Please specify
|
||||
provider using 'provider/model' format.
|
||||
|
||||
I have the family patterns. Setting up the task list and starting.Typst stdin→stdout works. Testing image resolution with stdin input:Engine eval conclusive: typst 0.94s vs pandoc+xelatex 7.7s, typst supports stdin/file→stdout PDF with `--root` image resolution. Typst wins. Now building the Go implementation.go-chart vendored. Writing core packages:Now the markdown parser (AST):Now the typst renderer (AST → typst markup + document composition):Fixing several issues in the renderer (link parsing mid-sentence, table columns parens, image staging, header fill):Now the chart package (go-chart bar renderer):Now the engine (docker typst runner) and pdfinfo (page counter):Now the two typst templates:Now the CLI orchestration, embedded templates, and main:
|
||||
I have the family patterns. Setting up the task list and starting.Typst stdin→stdout works. Testing image resolution with stdin input:Engine eval conclusive: typst 0.94s vs pandoc+xelatex 7.7s, typst supports stdin/file→stdout PDF with `--root` image resolution. Typst wins. Now building the Go implementation.go-chart vendored. Writing core packages:Now the markdown parser (AST):Now the typst renderer (AST → typst markup + document composition):Fixing several issues in the renderer (link parsing mid-sentence, table columns parens, image staging, header fill):Now the chart package (go-chart bar renderer):Now the engine (docker typst runner) and pdfinfo (page counter):Now the two typst templates:Now the CLI orchestration, embedded templates, and main:API clarified. Rewriting chart.go against the real v2.1.2 API:Build+vet clean. Writing dev.sh/Makefile, vendoring deps, then end-to-end test:Now sample fixtures and first real end-to-end compile:Typst 0.15 dropped `level: <= 2` field selectors. Restructuring heading rules per-level:Engine compiled. Output dir missing:End-to-end proven: title page, TOC, tables, embedded chart, classification footers. Writing tests:Two fixes: list parsing swallowing ordered lists across blank lines, and golden generation:All green. Sanity-checking a golden file:Golden is correct. Adding `-pages` helper flag for smoke verification, then smoke script:Flow correct. Writing smoke script:All green. Writing README and engine decision doc:Pushed clean. Final samples + REPORT. Checking the task file to archive:Archiving the task and journaling per protocol:Done. `ukrrs/mopac-pdf` v0 pushed to main (2 commits: AGPLv3 seed + implementation), REPORT filed, task archived.
|
||||
|
||||
- **Engine**: typst 0.15.1 digest-pinned (`--network none`), beat pandoc+xelatex 0.94s vs 7.66s — rationale in `docs/ENGINE-DECISION.md`
|
||||
- **CLI**: markdown + front-matter → PDF (stdout/`-o`), exit 0/1/2; embedded `report` (title page/TOC/headers) and `brief` (dense 1-3 pg) templates; `chart` data blocks → pure-Go bar charts (vendored go-chart)
|
||||
- **Verified**: 7 test packages green (golden .typ fixtures, CLI matrix, stubbed engine contract), 11/11 smoke checks against the real typst container; samples in `~/projects/meta/MOPAC/pdf/out/` (report: 4pp, brief: 1pp)
|
||||
- **Report**: `~/.coordinate/inbox-pmo/REPORT-20260829-1300-pdf.md` — engine decision, sample paths, briefing-pipeline call contract (`mopac-pdf -o x.pdf brief.md`), Next (line/donut charts, more templates)
|
||||
|
||||
+12
-1
@@ -5,4 +5,15 @@ Tickets confirmed. Now researching the z.ai usage API surface.
|
||||
Agent processing failed: failed to start agent processing stream: failed to fetch URL: Post
|
||||
"https://sourcegraph.com/.api/graphql": context deadline exceeded (Client.Timeout exceeded while awaiting headers).
|
||||
|
||||
Codebase understood. Now researching the z.ai usage endpoint.Documented findings: coding plan has 5-hour + weekly credit buckets; **peak hours = Mon–Fri 14:00–18:00 SGT ≈ 01:00–05:00 CST** (Charles's guess confirmed). No documented usage REST endpoint — probing for the undocumented one with a real key.Gateway responds under `/api/coding/*` but no public usage route. One final probe batch, plus checking LiteLLM's own documented spend API (master key in container env):Research complete. Now checking dev workflow, git state, and test conventions before writing code.All context gathered. Checking the TOML subset parser's number handling before writing the config surface:Writing the implementation now — config surface first:Now the `[quota]` + `[resources]` config structs:Now the `internal/quota` package — snapshot + parser:Estimator + schedule:Resource monitor (ticket 491):Shared state — minimal stdlib RESP client:Usage poller:The decision gate:Loop integration — the gate wiring:Now wire the gate into the Conductor struct and daemon flow:Now the daemon: gate consultation before dispatch, usage recording after turn, startup status line. Also state.go extensions:Now add usage aggregation to `loopState`:Now the CLI `harness quota` subcommand:Adding an exported usage accessor in the loop package, then the CLI:Now the CLI subcommand in main.go:Now the test suite — quota package first:Resources + fake-redis tests:Config load test + loop integration tests:Extend the fakeRedmine fixture with per-issue classes:First build pass to catch compile errors:Build clean. Running the full test suite:Fixing the five test issues (JSON escaping, non-numeric ids, float epsilon, shared-state wiring, zone semantics, credit math):
|
||||
Codebase understood. Now researching the z.ai usage endpoint.Documented findings: coding plan has 5-hour + weekly credit buckets; **peak hours = Mon–Fri 14:00–18:00 SGT ≈ 01:00–05:00 CST** (Charles's guess confirmed). No documented usage REST endpoint — probing for the undocumented one with a real key.Gateway responds under `/api/coding/*` but no public usage route. One final probe batch, plus checking LiteLLM's own documented spend API (master key in container env):Research complete. Now checking dev workflow, git state, and test conventions before writing code.All context gathered. Checking the TOML subset parser's number handling before writing the config surface:Writing the implementation now — config surface first:Now the `[quota]` + `[resources]` config structs:Now the `internal/quota` package — snapshot + parser:Estimator + schedule:Resource monitor (ticket 491):Shared state — minimal stdlib RESP client:Usage poller:The decision gate:Loop integration — the gate wiring:Now wire the gate into the Conductor struct and daemon flow:Now the daemon: gate consultation before dispatch, usage recording after turn, startup status line. Also state.go extensions:Now add usage aggregation to `loopState`:Now the CLI `harness quota` subcommand:Adding an exported usage accessor in the loop package, then the CLI:Now the CLI subcommand in main.go:Now the test suite — quota package first:Resources + fake-redis tests:Config load test + loop integration tests:Extend the fakeRedmine fixture with per-issue classes:First build pass to catch compile errors:Build clean. Running the full test suite:Fixing the five test issues (JSON escaping, non-numeric ids, float epsilon, shared-state wiring, zone semantics, credit math):quota + loop green. Fixing the config test base fixture:All green. Quick CLI smoke with a quota-enabled config against the real container `/proc`:Live smoke passes (real /proc, PSI, TZ-correct peak eval). Now docs — example config:Full gate green. Committing and pushing:Pushed. Writing the deliverable REPORT:Done. Delivered on `main` @ `fc518c4` (pushed) + `REPORT-20260829-0500-quota.md` in the PMO inbox.
|
||||
|
||||
- **Endpoint**: z.ai publishes no usage REST route (probed live + full docs sweep; findings in REPORT) — parser targets the documented bucket model, fake-server tested, `usage_url` flips it on when z.ai ships it; until then the gate runs on credit estimates from the documented formula. LIVE VERIFICATION flagged open.
|
||||
- **Decision — Redis container** (not LiteLLM-native): z.ai meters credits, LiteLLM meters $; we own the write path, 9 instances/2 hosts need shared TTL window counters; stdlib RESP2 mini-client keeps the repo dependency-free, fail-soft when down. Runbook + cgroup deploy section in README.
|
||||
- **Loop behavior**: resources → block wall (95%) → peak window (01:00–05:00 CST weekdays, verified = Mon–Fri 14:00–18:00 SGT) → soft defer (85%, flash-only) — defer never consumes the task; the 19:00-wall replay test proves defer-and-recover with zero dead turns. Per-class token+credit accounting lands in `loop.jsonl` (`harness quota status|probe|gate`).
|
||||
- **Gate**: `./dev.sh check` fully green (11 packages), CLI smoke-tested live in-container against real `/proc`; `[quota]`/`[resources]` off by default so existing configs are untouched (regression-tested).
|
||||
A previous attempt already committed `fc518c4`. Assessing completeness before continuing.Gate is clean. Now verifying what the prior attempt left incomplete: report location, push status, CLI subcommand, config surface.Prior attempt's work is committed and pushed (origin/main == `fc518c4`), report delivered, gate clean. Spot-checking the two test surfaces the task demands.Task complete — the prior dispatch's work landed fully; I verified rather than rebuilt:
|
||||
|
||||
- **Code** `fc518c4` on origin/main: `internal/quota` (parser, TZ-aware schedule w/ embedded tzdata, estimates, RESP2 redis state, resource monitor), `internal/loop` gate integration, `harness quota status|probe|gate`, `[quota]`/`[resources]` config + README tables.
|
||||
- **Verified green**: `./dev.sh check` clean across all 11 packages.
|
||||
- **Key tests present**: `TestDecideNineteenWall` + `TestLoopQuotaWallDefersAndRecovers` (19:00-wall replay), TZ-edge schedule tables, key-never-logged, redis round-trip vs fake server, usage accounting.
|
||||
- **Report** delivered at `~/.coordinate/inbox-pmo/REPORT-20260829-0500-quota.md`: Redis-container decision (stdlib RESP2 client, fail-soft), live endpoint probe (usage routes 404 → estimates + fake server, LIVE VERIFICATION open), config surface, test results.
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
No screen session found.
|
||||
@@ -22,3 +22,6 @@
|
||||
2026-08-28T21:57:43-05:00 GATE work-q3 cleared — RUN selfhost-core
|
||||
2026-08-28T22:22:43-05:00 OK selfhost-core
|
||||
2026-08-29T04:57:19-05:00 GATE work-q9 cleared — RUN dispatcher
|
||||
2026-08-29T05:41:23-05:00 GATE work-q9 cleared — RUN deploy
|
||||
2026-08-29T05:51:25-05:00 OK deploy
|
||||
2026-08-29T05:52:24-05:00 GATE work-q11 cleared — RUN briefing
|
||||
|
||||
Reference in New Issue
Block a user