ops(archive): TASK prompts and REPORT verifications, studies through V1
This commit is contained in:
@@ -0,0 +1,117 @@
|
||||
# REPORT: mopac events — webhook receiver (build phase 2b, V1 scope)
|
||||
|
||||
Date: 2026-08-28
|
||||
Operator: phase-2b events turn (DESIGN "Events are V1 scope" section)
|
||||
Status: COMPLETE — 5 commits on main, pushed to ukrrs/MOPAC; build/vet/test
|
||||
clean in the digest-pinned Docker builder; live smoke proven on LAN port
|
||||
4100 (401 without secret, 200 + JSONL record with).
|
||||
|
||||
## What shipped
|
||||
|
||||
`harness events` — Go stdlib net/http receiver (no frameworks, zero new
|
||||
deps) that lets Redmine/Discourse/Gitea punch the harness:
|
||||
|
||||
- Routes: `POST /hooks/{redmine,discourse,gitea}`, `GET /healthz`; GET on a
|
||||
hook = 405, unknown path = 404, body > 1 MiB = 413.
|
||||
- Verification, deny-first: gitea = hex HMAC-SHA256 of the raw body in
|
||||
`X-Gitea-Signature` (constant-time hmac.Equal); redmine/discourse =
|
||||
shared-secret header, constant-time compare, header names configurable
|
||||
(defaults `X-Redmine-Webhook-Secret` / `X-Discourse-Webhook-Secret`).
|
||||
Unsigned/unverified = 401 with ONE generic error body; rejection logs
|
||||
name the failure class only.
|
||||
- Normalization: every provider payload becomes one internal Event
|
||||
(source, kind, actor, canonical subject id — `redmine:issue:42`,
|
||||
`discourse:topic:7`, `gitea:pr:ukrrs/MOPAC#5` — title, repo, provider
|
||||
event id, sha256 payload digest, received-at). Tolerant across known
|
||||
payload variants (redmine_webhooks + flat shapes, Discourse headers or
|
||||
payload, Gitea action/pull_request/issue incl. closed+merged →
|
||||
pr_merged).
|
||||
- Action mapping per DESIGN: redmine issue update/note/journal →
|
||||
dispatch_turn; discourse post reply → respond_turn; gitea PR
|
||||
approved/merged → pipeline_step; rest stored-but-ignored.
|
||||
- Persistence: append-only `state/events/events.jsonl` (file 0600, dir
|
||||
0700), dedup by provider event id (`X-Gitea-Delivery`,
|
||||
`X-Discourse-Event-Id`, `X-Redmine-Delivery`; payload-digest fallback);
|
||||
dedup index rebuilt from the file at startup (torn tail skipped) so
|
||||
replays across restarts still dedup — at-least-once delivery,
|
||||
exactly-once reaction.
|
||||
- Config: `[events]` listen (default `:4100`), state_dir, per-source
|
||||
secret refs (`env:NAME` / `file:` / `literal:`), resolved at startup;
|
||||
refuses to start with zero secrets; ref values never echoed.
|
||||
- Secrets posture: never logged, never persisted; audit lines carry
|
||||
normalized fields + truncated digest only. Test-asserted (log + JSONL
|
||||
scanned for secret material).
|
||||
|
||||
## What is stubbed
|
||||
|
||||
- Event → turn dispatch: stored actionable events reach
|
||||
`Conductor.DispatchEvent` (internal/loop/loop.go), which today prints
|
||||
the would-be action. Real wiring (dispatch/redirect turn for Redmine,
|
||||
context update + response for Discourse, pipeline step for Gitea) is
|
||||
the next chunk, per plan.
|
||||
- Cross-host forwarding/replication of the event log (T1 later); JSONL
|
||||
is local-only today.
|
||||
- Discourse HMAC signature mode (their `X-Discourse-Event-Signature`) is
|
||||
not implemented; the spec asked for the secret-header mode, which is
|
||||
what shipped.
|
||||
|
||||
## Dev-in-docker compliance
|
||||
|
||||
Every compile/vet/test path routes through `dev.sh`, which pins the
|
||||
builder by digest: `golang@sha256:e8c859f5632...` (= golang:1.26-bookworm).
|
||||
Note: golang:1.26-alpine was tried first per the task text but its image
|
||||
has no bash, and the phase-2 exec tool's tests exec bash — bookworm keeps
|
||||
the pin and the suite green. Host ran only docker + python3 stdlib urllib
|
||||
(curl is banned on host). Exact commands used:
|
||||
|
||||
- Gate: `./dev.sh check` →
|
||||
`docker run --rm -v "$PWD:/h" -w /h -u "$(id -u):$(id -g)" -e HOME=/tmp
|
||||
-e GOFLAGS=-buildvcs=false golang@sha256:e8c859f... sh -c 'go build -o
|
||||
bin/harness ./cmd/harness && go vet ./... && go test ./...'`
|
||||
- Targeted runs: `./dev.sh test -run TestNormalize -v` (same container).
|
||||
- Smoke: `./dev.sh smoke` → builds via the same builder, then
|
||||
`docker run -d --name mopac-events-smoke -p 4100:4100 -v "$PWD:/h" -w /h
|
||||
-u "$(id -u):$(id -g)" -e HOME=/tmp golang@sha256:e8c859f...
|
||||
/h/bin/harness events -config /h/.smoke/harness.toml -listen ":4100"`,
|
||||
probed from the host with `BASE_URL=http://127.0.0.1:4100 python3
|
||||
smoke/probe.py`, torn down with `docker rm -f`.
|
||||
|
||||
## Smoke results (port 4100, all PASS)
|
||||
|
||||
- healthz answers; GET hook = 405.
|
||||
- gitea unsigned → 401 `{"error":"unverified webhook"}`; bad signature →
|
||||
401; valid HMAC → 200
|
||||
`{"action":"pipeline_step","id":"gitea:smoke-delivery-1","status":"stored"}`;
|
||||
replay → 200 `status:duplicate`.
|
||||
- redmine shared secret → 200 stored, `dispatch_turn`; wrong secret → 401.
|
||||
- discourse shared secret → 200 stored, `respond_turn`.
|
||||
- JSONL: 3 lines (gitea/redmine/discourse), correct fields, no secret
|
||||
material; container log shows one audit line per delivery + the stub
|
||||
dispatch firing exactly once per stored actionable event.
|
||||
|
||||
## Tests
|
||||
|
||||
Table-driven, stdlib only: verification (7 HMAC + 5 shared-secret cases),
|
||||
normalization (13 shape cases + 12-row action map + digest stability),
|
||||
store (dedup, restart persistence, torn tail, 0600 perms), end-to-end
|
||||
httptest server (401/400/200/405/413 paths, replay dedup,
|
||||
ignore-not-dispatched, secret-leak scans of responses/log/JSONL), config
|
||||
(events defaults, parse, bad-ref validation). `go build ./...`, `go vet
|
||||
./...`, `go test ./...` all clean; every commit builds standalone.
|
||||
|
||||
## Commits (main, pushed c6da05f..f89b694)
|
||||
|
||||
1. `164b145` events: config surface for the webhook receiver
|
||||
2. `05ec1a4` events: verify, normalize, append-only store with
|
||||
provider-id dedup
|
||||
3. `043e03b` events: HTTP receiver, CLI wiring, conductor dispatch stub
|
||||
4. `88e7b7b` dev.sh + smoke: docker-routed dev loop and webhook smoke test
|
||||
5. `f89b694` docs: events receiver in README, phase 2b status in REPORT
|
||||
|
||||
## Open for Charles
|
||||
|
||||
- Confirm the Cloudron apps can be configured with the three secret
|
||||
headers (Redmine needs a webhook plugin — redmine_webhooks works with
|
||||
its secret setting; header name is configurable if the plugin differs).
|
||||
- Port choice 4100 is a default, not a decision — override via
|
||||
`[events] listen` or `-listen` / `docker -p` when the real stacks land.
|
||||
Reference in New Issue
Block a user