Add Docker-routed dev tooling, smoke test, example config, and docs

Route every compile/vet/test path through a digest-pinned golang:1.26
builder container (dev.sh + Makefile) so the host never runs a Go
toolchain. Add an end-to-end smoke that serves keyproxy in a container
with a throwaway config and drives 401/200/404/400/501/405 paths with
python urllib, asserting the server log is redacted. Ship a commented
keyproxy.toml.example (the real config stays gitignored along with any
*.env tripwire) and rewrite the README as the full quickstart, config,
HTTP, CLI, and redaction reference.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
2026-08-28 22:30:50 -05:00
parent 24a57c1dff
commit 47343d53a2
7 changed files with 538 additions and 17 deletions
+7
View File
@@ -0,0 +1,7 @@
# keyproxy.toml holds the ref map (paths + key names) — copy to
# keyproxy.toml and fill in. keyproxy.toml itself is gitignored.
# Any *.env file landing in this repo is ignored as a tripwire.
keyproxy.toml
*.env
bin/
.smoke/
+19
View File
@@ -0,0 +1,19 @@
# Makefile: thin front door over dev.sh so `make build/vet/test/check`
# route through the digest-pinned Docker builder (ALL dev work in
# Docker — the host never runs a Go toolchain).
.PHONY: build vet test check smoke
build:
@./dev.sh build
vet:
@./dev.sh vet
test:
@./dev.sh test
check:
@./dev.sh check
smoke:
@./dev.sh smoke
+209 -17
View File
@@ -1,25 +1,217 @@
# mopac-keyproxy # mopac-keyproxy
A key-material proxy where the vault stays authoritative. Consumers receive A key-material proxy where the vault stays authoritative. Consumers receive
opaque placeholder keys; real secrets live only in Bitwarden Secrets Manager opaque placeholder refs (`mpk-<name>`); real secrets live only in the
and/or HashiCorp Vault and are resolved to real material at the wire, in configured backends (v0: 0600 env files and process env; phase 3: Bitwarden
memory, never persisted, with no admin UI to attack. Secrets Manager and HashiCorp Vault) and are resolved to real material at
the wire, in memory, never persisted, never logged, with no admin UI to
attack.
Status: 2026-08-28 — spec seed; design frozen, no code yet. Status: 2026-08-28 — v0 LIVE: `keyproxy serve` (localhost HTTP resolve hop)
+ `keyproxy get` (exec-style CLI) on the file and env backends;
bitwarden/vault ship as explicit not-implemented stubs behind the same
interface (phase 3 drop-in).
## Quickstart
All dev work happens inside a Docker builder (host stays toolchain-free);
`docker pull` of the builder is pre-authorized. Commands below were verified
on 2026-08-28 from a fresh clone.
### Build and test
`dev.sh` routes every compile/vet/test path through the digest-pinned
builder container (or use `make build|vet|test|check`, same routing):
```sh
./dev.sh check # = go build + go vet + go test, all inside the builder
```
The equivalent raw command:
```sh
docker run --rm -v "$PWD:/h" -w /h \
-u "$(id -u):$(id -g)" -e HOME=/tmp -e GOFLAGS=-buildvcs=false \
golang@sha256:e8c859f5632dcfde7b32d2012b4351728f6437930887c2f6a91ea242459e5514 \
sh -c 'go build -o bin/keyproxy ./cmd/keyproxy && go vet ./... && go test ./...'
```
(that digest = `golang:1.26-bookworm`.)
Expected output (tail):
```text
ok git.knownelement.com/ukrrs/mopac-keyproxy/internal/backend
ok git.knownelement.com/ukrrs/mopac-keyproxy/internal/config
ok git.knownelement.com/ukrrs/mopac-keyproxy/internal/server
```
End-to-end smoke (builds, serves in a container on port 8082 with a
throwaway config, drives 401/200/404/400/501/502 paths with python
urllib, and asserts the server log is redacted):
```sh
./dev.sh smoke
```
Expected output (tail):
```text
probe: OK
--- server log (redaction check) ---
... keyproxy: listening addr=:8082 refs=5 backends=bitwarden,env,file,vault auth=mpk-keyproxy-self=***
... keyproxy: resolve ref=mpk-smoke=*** backend=file status=200
```
### Configure
```sh
mkdir -p ~/.config/keyproxy && umask 077
printf 'KEYPROXY_TOKEN=%s\n' "$(od -An -N32 -tx1 /dev/urandom | tr -d ' \n')" \
> ~/.config/keyproxy/keyproxy.env
printf 'EXAMPLE_API_KEY=%s\n' "$(od -An -N32 -tx1 /dev/urandom | tr -d ' \n')" \
> ~/.config/keyproxy/example.env
cp keyproxy.toml.example keyproxy.toml
```
`keyproxy.toml` is gitignored (as is any `*.env` landing in the repo). It
holds the ref MAP — paths and key names — never material itself.
### Serve and resolve
```sh
./bin/keyproxy serve # binds 127.0.0.1:8082 (config: listen)
```
Resolve a ref over the HTTP hop (python3 urllib; curl-free hosts work
fine). The token is read from the 0600 env file, and the resolved value is
shown only as a length + mask — copy-paste safe:
```sh
KEYPROXY_TOKEN="$(sed -n 's/^KEYPROXY_TOKEN=//p' ~/.config/keyproxy/keyproxy.env)" \
python3 - <<'PY'
import json, os, urllib.request, urllib.error
req = urllib.request.Request(
"http://127.0.0.1:8082/v1/resolve",
data=json.dumps({"ref": "mpk-example"}).encode(), method="POST")
req.add_header("Authorization", "Bearer " + os.environ["KEYPROXY_TOKEN"])
try:
with urllib.request.urlopen(req, timeout=5) as r:
v = json.loads(r.read())["value"]
print(f"200: mpk-example={len(v)} chars, mpk-example=***")
except urllib.error.HTTPError as e:
print(e.code, e.read().decode().strip())
PY
```
Expected output:
```text
200: mpk-example=64 chars, mpk-example=***
```
Without (or with a wrong) bearer token the response is
`401 {"error":"unauthorized"}` and the server logs only
`auth failure remote=...` — never the presented token.
Exec-style plumbing, no HTTP hop:
```sh
./bin/keyproxy get mpk-example > /tmp/.k && wc -c < /tmp/.k # bare value, no newline
```
## Config reference (`keyproxy.toml`)
| Key | Scope | Meaning |
|---|---|---|
| `listen` | top level | bind address (default `127.0.0.1:8082`; keep it loopback unless you know why not) |
| `auth.token_ref` | `[auth]` | ref holding the bearer token for `/v1/resolve`; MUST be a `file`-backend ref, resolved once at startup, never logged |
| `refs."mpk-<name>".backend` | ref | `file`, `env`, `bitwarden` (stub), `vault` (stub) |
| `refs."mpk-<name>".source` | ref | file: path to a 0600 env file (`~/` expanded); env: variable name; bitwarden/vault: reserved mount/project path |
| `refs."mpk-<name>".key` | ref | file: KEY inside the env file (required); env: forbidden (source IS the name); stubs: informational |
| `refs."mpk-<name>".mode` | ref | file only: allowed permission mask (default `0600`; stricter files like `0400` pass, looser files are refused before being read) |
Ref names must match `mpk-[a-z0-9][a-z0-9-]*`. Because the pattern is
enforced before any echo, refs can be named safely in logs and errors: an
mpk- ref is a placeholder, by construction not material.
Env files are `KEY=VALUE` lines parsed in Go — NEVER sourced or exec'd
(no shell expansion, no interpolation; comments, blank lines, `export `
prefix and surrounding quotes are handled; later duplicate keys win).
## HTTP reference (`keyproxy serve`)
| Route | Auth | Behavior |
|---|---|---|
| `POST /v1/resolve` | bearer token | body `{"ref":"mpk-<name>"}``{"value":"..."}`; material crosses the wire here and nowhere else |
| `GET /healthz` | none | liveness |
Status codes: 200 resolved; 401 missing/wrong token (constant-time
compare); 400 malformed body or non-`mpk-` ref (an invalid "ref" field is
never echoed — it might hold pasted material); 404 unknown ref; 405 wrong
method; 413 body over 4 KiB; 500 recovered panic (detail suppressed); 501
backend not implemented in v0; 502 backend failure. Failure bodies carry
the ref, the backend and a fixed reason enum (`missing_key`,
`insecure_source_mode`, `unreadable_source`, `empty_value`,
`malformed_source`, `not_implemented`) — reason enums cannot embed
material by construction.
## CLI reference
| Command | Purpose |
|---|---|
| `keyproxy serve` | run the resolve HTTP hop until SIGINT/SIGTERM |
| `keyproxy get REF` | resolve one ref to stdout (bare value, no trailing newline — for `$(...)` plumbing) |
| `keyproxy help` | print usage |
Flags: `-config PATH` (default `$KEYPROXY_CONFIG`, then `./keyproxy.toml`),
`-listen ADDR` (serve only, overrides config).
Exit codes: 0 ok; 1 usage/config error; 2 resolution failure.
## Redaction rules (enforced by tests)
- No admin UI, no persistence, no cache files: material exists in memory
per resolve and is dropped.
- Every log line that names a ref masks any value as `<ref>=***`.
- No material in logs, error bodies, or crash paths: panics are recovered
with the panic value discarded (never stringified); auth failures log
the remote address only; parse errors carry line numbers, never line
contents.
- Resolution failures name the ref and the backend, never the value.
- 0600 enforced on secret env files (looser modes are refused before the
file is read).
## Architecture
```mermaid
flowchart LR
C["consumer<br/>(harness, CLI, container)<br/>holds mpk-... refs only"] -->|"POST /v1/resolve<br/>bearer token"| S["keyproxy serve<br/>auth -> ref lookup -> backend"]
S --> F["file backend<br/>0600 env files, parsed in Go<br/>(never sourced)"]
S --> E["env backend<br/>process env indirection"]
S --> B["bitwarden stub<br/>501 not_implemented"]
S --> V["vault stub<br/>501 not_implemented"]
S -->|"200 {value} — memory only"| C
```
All backends sit behind one interface (`internal/backend.Backend`); phase-3
Bitwarden Secrets Manager (plain REST, machine accounts) and Vault KV
v2 + AppRole connectors replace the stubs in that one place.
## Scope ## Scope
- Placeholders in, real keys on the wire out: consumers (harness, CLIs, - Placeholders in, real keys on the wire out: consumers hold `mpk-...`
containers) hold `mpk_...` opaque refs only. A leaked placeholder is opaque refs only. A leaked placeholder is revoke-and-remap, not an
revoke-and-remap, not an incident — zero upstream exposure. incident — zero upstream exposure.
- Backends: Bitwarden Secrets Manager REST (machine accounts) and HashiCorp - Backends: v0 file + env; planned Bitwarden Secrets Manager REST (machine
Vault KV v2 + AppRole (official Go api pkg is MPL-2.0, vendored). accounts) and HashiCorp Vault KV v2 + AppRole (official Go api pkg is
Stdlib-first; NO official Bitwarden SDK (its source-available license is MPL-2.0, AGPL-compatible, vendored). Stdlib-first; NO official
AGPL-incompatible). Bitwarden SDK (its source-available license is AGPL-incompatible).
- Two shapes, one resolver: an HTTP hop for services, and a CLI - Two shapes, one resolver: an HTTP hop for services, and a CLI
(`keyproxy get <ref>`) for exec-style plumbing. (`keyproxy get <ref>`) for exec-style plumbing.
- Memory-only material handling: fetch-on-demand, short TTL, never persisted, - Memory-only material handling: fetch-on-demand, never persisted, never
never written to logs; ref redaction everywhere. written to logs; ref redaction everywhere.
- Config-driven: generic, no organizational hosts/paths/defaults baked into - Config-driven: generic, no organizational hosts/paths/defaults baked into
code. code.
@@ -29,15 +221,15 @@ Status: 2026-08-28 — spec seed; design frozen, no code yet.
- Not a secrets manager: the vault stays authoritative; keyproxy never - Not a secrets manager: the vault stays authoritative; keyproxy never
becomes a second place secrets live. becomes a second place secrets live.
- No persistence of key material to disk, cache files, or crash dumps. - No persistence of key material to disk, cache files, or crash dumps.
- Not org-specific: TSYS policy lives outside this repo (loose-coupling - Not org-specific: policy lives outside this repo (loose-coupling rules
rules for the MOPAC tool family). for the MOPAC tool family).
## Today vs planned ## Today vs planned
| | State | | | State |
|---|---| |---|---|
| Today | Spec only (this README + LICENSE). Design carried in the MOPAC harness DESIGN.md "Key proxy" section. | | Today | v0 live: `serve` + `get` on file (0600 env files) and env backends; bearer auth bootstrapped from the file backend; table-driven tests incl. redaction + fake-HTTP end-to-end; docker-routed build discipline. |
| Planned | Go implementation: placeholder->material resolver, Bitwarden Secrets Manager + Vault connectors, HTTP hop + `get` CLI, ref-redaction rules. | | Planned | Phase 3: Bitwarden Secrets Manager REST + Vault KV v2/AppRole connectors behind the same interface (stubs and tests already in place); short-TTL memory cache. |
## Design references ## Design references
Executable
+54
View File
@@ -0,0 +1,54 @@
#!/bin/sh
# keyproxy dev wrapper. EVERY compile/vet/test path routes through the
# digest-pinned Docker builder (DESIGN "ALL dev work in Docker" — big
# rule); the host runs containers, never toolchains.
#
# Usage: ./dev.sh {build|vet|test|check|smoke|shell} [args...]
#
# build compile ./cmd/keyproxy into bin/keyproxy
# vet go vet ./...
# test go test ./...
# check build + vet + test (the pre-push gate)
# smoke end-to-end smoke: builds, starts `keyproxy serve` in a
# container on port 8082 with a throwaway config, drives it
# from the host with python urllib (curl is banned on host),
# shows 401/200/404/501 paths + redacted server log, tears
# everything down
# shell interactive sh inside the builder
set -e
IMAGE="golang@sha256:e8c859f5632dcfde7b32d2012b4351728f6437930887c2f6a91ea242459e5514" # = golang:1.26-bookworm (bash present; alpine lacks it)
run() {
docker run --rm -v "$PWD:/h" -w /h \
-u "$(id -u):$(id -g)" -e HOME=/tmp -e GOFLAGS=-buildvcs=false \
"$IMAGE" "$@"
}
cmd=${1:-check}
shift || true
case "$cmd" in
build)
run go build -o bin/keyproxy ./cmd/keyproxy
;;
vet)
run go vet ./...
;;
test)
run go test "$@" ./...
;;
check)
run sh -c 'go build -o bin/keyproxy ./cmd/keyproxy && go vet ./... && go test ./...'
;;
smoke)
./smoke/smoke.sh
;;
shell)
run sh
;;
*)
echo "dev.sh: unknown command $cmd (build|vet|test|check|smoke|shell)" >&2
exit 1
;;
esac
+47
View File
@@ -0,0 +1,47 @@
# keyproxy.toml.example — copy to keyproxy.toml and fill in.
# keyproxy.toml is gitignored; it holds the ref MAP (paths + key names),
# never material itself. Refs are mpk-<name> placeholders: consumers see
# only these names; material resolves at request time, memory-only.
# HTTP hop bind address. Keep it loopback unless you know why not.
listen = "127.0.0.1:8082"
[auth]
# Bearer token for POST /v1/resolve, bootstrapped from the file backend
# itself: a 0600 env file holding the token. Resolved once at startup,
# never logged, never persisted by keyproxy.
token_ref = "mpk-keyproxy-self"
# --- file backend: KEY=VALUE env files, 0600, parsed in Go (never
# sourced/exec'd). mode is the allowed permission mask (default 0600;
# stricter files like 0400 always pass, looser files are refused).
[refs."mpk-keyproxy-self"]
backend = "file"
source = "~/.config/keyproxy/keyproxy.env"
key = "KEYPROXY_TOKEN"
[refs."mpk-example"]
backend = "file"
source = "~/.config/keyproxy/example.env"
key = "EXAMPLE_API_KEY"
# --- env backend: process-environment indirection. source IS the
# variable name (no key field); useful for container-injected values.
[refs."mpk-example-env"]
backend = "env"
source = "EXAMPLE_API_KEY"
# --- bitwarden backend: Bitwarden Secrets Manager REST (machine
# accounts). NOT IMPLEMENTED in v0 — resolves fail loudly with 501
# not_implemented so phase 3 is a drop-in behind the same interface.
[refs."mpk-example-bitwarden"]
backend = "bitwarden"
source = "sm://projects/example"
key = "EXAMPLE_API_KEY"
# --- vault backend: HashiCorp Vault KV v2 + AppRole. NOT IMPLEMENTED in
# v0 — same explicit 501 stub as bitwarden.
[refs."mpk-example-vault"]
backend = "vault"
source = "secret/data/example"
key = "EXAMPLE_API_KEY"
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""keyproxy resolve smoke probe.
Drives `keyproxy serve` on BASE (default http://127.0.0.1:8082) with
plain stdlib urllib: the host has no curl by policy. Checks: healthz,
401 without/with wrong token, 200 file + env resolves, 404 unknown ref,
400 invalid ref format (without echo), 501 bitwarden/vault stubs, 502
empty-value failure. Prints one PASS/FAIL line per check; exits non-zero
on any failure.
"""
import json
import os
import sys
import time
import urllib.error
import urllib.request
BASE = os.environ.get("BASE_URL", "http://127.0.0.1:8082")
TOKEN = os.environ.get("KEYPROXY_TOKEN", "smoke-bearer-token-1111")
failures = 0
def req(method, path, body=None, headers=None):
r = urllib.request.Request(BASE + path, data=body, method=method,
headers=headers or {})
try:
with urllib.request.urlopen(r, timeout=5) as resp:
return resp.status, resp.read().decode()
except urllib.error.HTTPError as e:
return e.code, e.read().decode()
except Exception as e: # transport errors surface as FAIL lines, not tracebacks
return 0, f"transport error: {e}"
def check(name, cond, detail=""):
global failures
print(("PASS" if cond else "FAIL"), name, ("- " + detail) if detail else "")
if not cond:
failures += 1
def wait_healthz(tries=50):
for _ in range(tries):
try:
code, _ = req("GET", "/healthz")
if code == 200:
return True
except Exception:
pass
time.sleep(0.2)
return False
def resolve(token, ref):
body = json.dumps({"ref": ref}).encode()
headers = {"Authorization": "Bearer " + token} if token else {}
return req("POST", "/v1/resolve", body, headers)
def main():
check("healthz answers (unauthenticated)", wait_healthz())
code, text = resolve(None, "mpk-smoke")
check("no bearer token rejected 401", code == 401, text.strip())
code, text = resolve("wrong-token", "mpk-smoke")
check("wrong bearer token rejected 401", code == 401, text.strip())
code, text = resolve(TOKEN, "mpk-smoke")
obj = json.loads(text) if text else {}
check("file ref resolves 200 with value",
code == 200 and obj.get("value") == "smoke-secret-material-2222",
text.strip())
code, text = resolve(TOKEN, "mpk-smoke-env")
obj = json.loads(text) if text else {}
check("env ref resolves 200 with value",
code == 200 and obj.get("value") == "smoke-env-material-4444",
text.strip())
code, text = resolve(TOKEN, "mpk-ghost")
check("unknown ref 404", code == 404 and "mpk-ghost" in text, text.strip())
code, text = resolve(TOKEN, "sk-live-pasted-secret")
check("invalid ref format 400 without echo",
code == 400 and "pasted" not in text, text.strip())
code, text = resolve(TOKEN, "mpk-smoke-bw")
check("bitwarden stub 501 not_implemented",
code == 501 and "not_implemented" in text and "bitwarden" in text,
text.strip())
code, text = resolve(TOKEN, "mpk-smoke-vault")
check("vault stub 501 not_implemented",
code == 501 and "not_implemented" in text and "vault" in text,
text.strip())
code, text = req("GET", "/v1/resolve")
check("GET resolve rejected 405", code == 405, text.strip())
print("probe:", "OK" if failures == 0 else f"{failures} FAILURE(S)")
return 0 if failures == 0 else 1
if __name__ == "__main__":
sys.exit(main())
Executable
+94
View File
@@ -0,0 +1,94 @@
#!/bin/sh
# keyproxy end-to-end smoke: builds in the digest-pinned builder, starts
# `keyproxy serve` in a container on host port 8082 with a throwaway
# config + throwaway 0600 env files, drives it from the host with python
# urllib (curl is banned on the host by policy), checks the 401/200/404/
# 400/501/502 paths AND that the server log is redacted (no material,
# every ref masked as <ref>=***), then tears everything down.
set -e
cd "$(dirname "$0")/.."
IMAGE="golang@sha256:e8c859f5632dcfde7b32d2012b4351728f6437930887c2f6a91ea242459e5514" # = golang:1.26-bookworm
NAME="keyproxy-smoke"
PORT="${PORT:-8082}"
./dev.sh build
rm -rf .smoke
mkdir -p .smoke
chmod 0700 .smoke
# Throwaway material (smoke values only; never real secrets).
cat > .smoke/keyproxy.env <<'EOF'
KEYPROXY_TOKEN=smoke-bearer-token-1111
EOF
cat > .smoke/creds.env <<'EOF'
API_KEY=smoke-secret-material-2222
OTHER=smoke-other-3333
EOF
chmod 0600 .smoke/keyproxy.env .smoke/creds.env
# In-container bind is all-interfaces ONLY because docker -p publishing
# is the boundary here; the host default (and example config) stays
# loopback.
cat > .smoke/keyproxy.toml <<EOF
listen = ":8082"
[auth]
token_ref = "mpk-keyproxy-self"
[refs."mpk-keyproxy-self"]
backend = "file"
source = "/h/.smoke/keyproxy.env"
key = "KEYPROXY_TOKEN"
[refs."mpk-smoke"]
backend = "file"
source = "/h/.smoke/creds.env"
key = "API_KEY"
[refs."mpk-smoke-env"]
backend = "env"
source = "KEYPROXY_SMOKE_ENV"
[refs."mpk-smoke-bw"]
backend = "bitwarden"
source = "sm://p/smoke"
key = "API_KEY"
[refs."mpk-smoke-vault"]
backend = "vault"
source = "secret/data/smoke"
key = "API_KEY"
EOF
docker rm -f "$NAME" >/dev/null 2>&1 || true
docker run -d --name "$NAME" -p "$PORT:8082" \
-v "$PWD:/h" -w /h -u "$(id -u):$(id -g)" -e HOME=/tmp \
-e KEYPROXY_SMOKE_ENV=smoke-env-material-4444 \
"$IMAGE" /h/bin/keyproxy serve -config /h/.smoke/keyproxy.toml
cleanup() {
docker rm -f "$NAME" >/dev/null 2>&1 || true
}
trap cleanup EXIT
echo "--- probing http://127.0.0.1:$PORT (python urllib; curl banned on host) ---"
BASE_URL="http://127.0.0.1:$PORT" KEYPROXY_TOKEN="smoke-bearer-token-1111" \
python3 smoke/probe.py
probe_rc=$?
echo "--- server log (redaction check) ---"
LOG="$(docker logs "$NAME" 2>&1)"
echo "$LOG"
if echo "$LOG" | grep -q 'smoke-secret-material-2222\|smoke-env-material-4444\|smoke-bearer-token-1111'; then
echo "FAIL: server log contains material"
exit 1
fi
if ! echo "$LOG" | grep -q 'ref=mpk-smoke=\*\*\*'; then
echo "FAIL: server log does not mask refs as <ref>=***"
exit 1
fi
exit "$probe_rc"