Add Docker-routed dev tooling, smoke test, example config, and docs

Route every compile/vet/test path through a digest-pinned golang:1.26
builder container (dev.sh + Makefile) so the host never runs a Go
toolchain. Add an end-to-end smoke that serves keyproxy in a container
with a throwaway config and drives 401/200/404/400/501/405 paths with
python urllib, asserting the server log is redacted. Ship a commented
keyproxy.toml.example (the real config stays gitignored along with any
*.env tripwire) and rewrite the README as the full quickstart, config,
HTTP, CLI, and redaction reference.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
2026-08-28 22:30:50 -05:00
parent 24a57c1dff
commit 47343d53a2
7 changed files with 538 additions and 17 deletions
+7
View File
@@ -0,0 +1,7 @@
# keyproxy.toml holds the ref map (paths + key names) — copy to
# keyproxy.toml and fill in. keyproxy.toml itself is gitignored.
# Any *.env file landing in this repo is ignored as a tripwire.
keyproxy.toml
*.env
bin/
.smoke/
+19
View File
@@ -0,0 +1,19 @@
# Makefile: thin front door over dev.sh so `make build/vet/test/check`
# route through the digest-pinned Docker builder (ALL dev work in
# Docker — the host never runs a Go toolchain).
.PHONY: build vet test check smoke
build:
@./dev.sh build
vet:
@./dev.sh vet
test:
@./dev.sh test
check:
@./dev.sh check
smoke:
@./dev.sh smoke
+209 -17
View File
@@ -1,25 +1,217 @@
# mopac-keyproxy
A key-material proxy where the vault stays authoritative. Consumers receive
opaque placeholder keys; real secrets live only in Bitwarden Secrets Manager
and/or HashiCorp Vault and are resolved to real material at the wire, in
memory, never persisted, with no admin UI to attack.
opaque placeholder refs (`mpk-<name>`); real secrets live only in the
configured backends (v0: 0600 env files and process env; phase 3: Bitwarden
Secrets Manager and HashiCorp Vault) and are resolved to real material at
the wire, in memory, never persisted, never logged, with no admin UI to
attack.
Status: 2026-08-28 — spec seed; design frozen, no code yet.
Status: 2026-08-28 — v0 LIVE: `keyproxy serve` (localhost HTTP resolve hop)
+ `keyproxy get` (exec-style CLI) on the file and env backends;
bitwarden/vault ship as explicit not-implemented stubs behind the same
interface (phase 3 drop-in).
## Quickstart
All dev work happens inside a Docker builder (host stays toolchain-free);
`docker pull` of the builder is pre-authorized. Commands below were verified
on 2026-08-28 from a fresh clone.
### Build and test
`dev.sh` routes every compile/vet/test path through the digest-pinned
builder container (or use `make build|vet|test|check`, same routing):
```sh
./dev.sh check # = go build + go vet + go test, all inside the builder
```
The equivalent raw command:
```sh
docker run --rm -v "$PWD:/h" -w /h \
-u "$(id -u):$(id -g)" -e HOME=/tmp -e GOFLAGS=-buildvcs=false \
golang@sha256:e8c859f5632dcfde7b32d2012b4351728f6437930887c2f6a91ea242459e5514 \
sh -c 'go build -o bin/keyproxy ./cmd/keyproxy && go vet ./... && go test ./...'
```
(that digest = `golang:1.26-bookworm`.)
Expected output (tail):
```text
ok git.knownelement.com/ukrrs/mopac-keyproxy/internal/backend
ok git.knownelement.com/ukrrs/mopac-keyproxy/internal/config
ok git.knownelement.com/ukrrs/mopac-keyproxy/internal/server
```
End-to-end smoke (builds, serves in a container on port 8082 with a
throwaway config, drives 401/200/404/400/501/502 paths with python
urllib, and asserts the server log is redacted):
```sh
./dev.sh smoke
```
Expected output (tail):
```text
probe: OK
--- server log (redaction check) ---
... keyproxy: listening addr=:8082 refs=5 backends=bitwarden,env,file,vault auth=mpk-keyproxy-self=***
... keyproxy: resolve ref=mpk-smoke=*** backend=file status=200
```
### Configure
```sh
mkdir -p ~/.config/keyproxy && umask 077
printf 'KEYPROXY_TOKEN=%s\n' "$(od -An -N32 -tx1 /dev/urandom | tr -d ' \n')" \
> ~/.config/keyproxy/keyproxy.env
printf 'EXAMPLE_API_KEY=%s\n' "$(od -An -N32 -tx1 /dev/urandom | tr -d ' \n')" \
> ~/.config/keyproxy/example.env
cp keyproxy.toml.example keyproxy.toml
```
`keyproxy.toml` is gitignored (as is any `*.env` landing in the repo). It
holds the ref MAP — paths and key names — never material itself.
### Serve and resolve
```sh
./bin/keyproxy serve # binds 127.0.0.1:8082 (config: listen)
```
Resolve a ref over the HTTP hop (python3 urllib; curl-free hosts work
fine). The token is read from the 0600 env file, and the resolved value is
shown only as a length + mask — copy-paste safe:
```sh
KEYPROXY_TOKEN="$(sed -n 's/^KEYPROXY_TOKEN=//p' ~/.config/keyproxy/keyproxy.env)" \
python3 - <<'PY'
import json, os, urllib.request, urllib.error
req = urllib.request.Request(
"http://127.0.0.1:8082/v1/resolve",
data=json.dumps({"ref": "mpk-example"}).encode(), method="POST")
req.add_header("Authorization", "Bearer " + os.environ["KEYPROXY_TOKEN"])
try:
with urllib.request.urlopen(req, timeout=5) as r:
v = json.loads(r.read())["value"]
print(f"200: mpk-example={len(v)} chars, mpk-example=***")
except urllib.error.HTTPError as e:
print(e.code, e.read().decode().strip())
PY
```
Expected output:
```text
200: mpk-example=64 chars, mpk-example=***
```
Without (or with a wrong) bearer token the response is
`401 {"error":"unauthorized"}` and the server logs only
`auth failure remote=...` — never the presented token.
Exec-style plumbing, no HTTP hop:
```sh
./bin/keyproxy get mpk-example > /tmp/.k && wc -c < /tmp/.k # bare value, no newline
```
## Config reference (`keyproxy.toml`)
| Key | Scope | Meaning |
|---|---|---|
| `listen` | top level | bind address (default `127.0.0.1:8082`; keep it loopback unless you know why not) |
| `auth.token_ref` | `[auth]` | ref holding the bearer token for `/v1/resolve`; MUST be a `file`-backend ref, resolved once at startup, never logged |
| `refs."mpk-<name>".backend` | ref | `file`, `env`, `bitwarden` (stub), `vault` (stub) |
| `refs."mpk-<name>".source` | ref | file: path to a 0600 env file (`~/` expanded); env: variable name; bitwarden/vault: reserved mount/project path |
| `refs."mpk-<name>".key` | ref | file: KEY inside the env file (required); env: forbidden (source IS the name); stubs: informational |
| `refs."mpk-<name>".mode` | ref | file only: allowed permission mask (default `0600`; stricter files like `0400` pass, looser files are refused before being read) |
Ref names must match `mpk-[a-z0-9][a-z0-9-]*`. Because the pattern is
enforced before any echo, refs can be named safely in logs and errors: an
mpk- ref is a placeholder, by construction not material.
Env files are `KEY=VALUE` lines parsed in Go — NEVER sourced or exec'd
(no shell expansion, no interpolation; comments, blank lines, `export `
prefix and surrounding quotes are handled; later duplicate keys win).
## HTTP reference (`keyproxy serve`)
| Route | Auth | Behavior |
|---|---|---|
| `POST /v1/resolve` | bearer token | body `{"ref":"mpk-<name>"}``{"value":"..."}`; material crosses the wire here and nowhere else |
| `GET /healthz` | none | liveness |
Status codes: 200 resolved; 401 missing/wrong token (constant-time
compare); 400 malformed body or non-`mpk-` ref (an invalid "ref" field is
never echoed — it might hold pasted material); 404 unknown ref; 405 wrong
method; 413 body over 4 KiB; 500 recovered panic (detail suppressed); 501
backend not implemented in v0; 502 backend failure. Failure bodies carry
the ref, the backend and a fixed reason enum (`missing_key`,
`insecure_source_mode`, `unreadable_source`, `empty_value`,
`malformed_source`, `not_implemented`) — reason enums cannot embed
material by construction.
## CLI reference
| Command | Purpose |
|---|---|
| `keyproxy serve` | run the resolve HTTP hop until SIGINT/SIGTERM |
| `keyproxy get REF` | resolve one ref to stdout (bare value, no trailing newline — for `$(...)` plumbing) |
| `keyproxy help` | print usage |
Flags: `-config PATH` (default `$KEYPROXY_CONFIG`, then `./keyproxy.toml`),
`-listen ADDR` (serve only, overrides config).
Exit codes: 0 ok; 1 usage/config error; 2 resolution failure.
## Redaction rules (enforced by tests)
- No admin UI, no persistence, no cache files: material exists in memory
per resolve and is dropped.
- Every log line that names a ref masks any value as `<ref>=***`.
- No material in logs, error bodies, or crash paths: panics are recovered
with the panic value discarded (never stringified); auth failures log
the remote address only; parse errors carry line numbers, never line
contents.
- Resolution failures name the ref and the backend, never the value.
- 0600 enforced on secret env files (looser modes are refused before the
file is read).
## Architecture
```mermaid
flowchart LR
C["consumer<br/>(harness, CLI, container)<br/>holds mpk-... refs only"] -->|"POST /v1/resolve<br/>bearer token"| S["keyproxy serve<br/>auth -> ref lookup -> backend"]
S --> F["file backend<br/>0600 env files, parsed in Go<br/>(never sourced)"]
S --> E["env backend<br/>process env indirection"]
S --> B["bitwarden stub<br/>501 not_implemented"]
S --> V["vault stub<br/>501 not_implemented"]
S -->|"200 {value} — memory only"| C
```
All backends sit behind one interface (`internal/backend.Backend`); phase-3
Bitwarden Secrets Manager (plain REST, machine accounts) and Vault KV
v2 + AppRole connectors replace the stubs in that one place.
## Scope
- Placeholders in, real keys on the wire out: consumers (harness, CLIs,
containers) hold `mpk_...` opaque refs only. A leaked placeholder is
revoke-and-remap, not an incident — zero upstream exposure.
- Backends: Bitwarden Secrets Manager REST (machine accounts) and HashiCorp
Vault KV v2 + AppRole (official Go api pkg is MPL-2.0, vendored).
Stdlib-first; NO official Bitwarden SDK (its source-available license is
AGPL-incompatible).
- Placeholders in, real keys on the wire out: consumers hold `mpk-...`
opaque refs only. A leaked placeholder is revoke-and-remap, not an
incident — zero upstream exposure.
- Backends: v0 file + env; planned Bitwarden Secrets Manager REST (machine
accounts) and HashiCorp Vault KV v2 + AppRole (official Go api pkg is
MPL-2.0, AGPL-compatible, vendored). Stdlib-first; NO official
Bitwarden SDK (its source-available license is AGPL-incompatible).
- Two shapes, one resolver: an HTTP hop for services, and a CLI
(`keyproxy get <ref>`) for exec-style plumbing.
- Memory-only material handling: fetch-on-demand, short TTL, never persisted,
never written to logs; ref redaction everywhere.
- Memory-only material handling: fetch-on-demand, never persisted, never
written to logs; ref redaction everywhere.
- Config-driven: generic, no organizational hosts/paths/defaults baked into
code.
@@ -29,15 +221,15 @@ Status: 2026-08-28 — spec seed; design frozen, no code yet.
- Not a secrets manager: the vault stays authoritative; keyproxy never
becomes a second place secrets live.
- No persistence of key material to disk, cache files, or crash dumps.
- Not org-specific: TSYS policy lives outside this repo (loose-coupling
rules for the MOPAC tool family).
- Not org-specific: policy lives outside this repo (loose-coupling rules
for the MOPAC tool family).
## Today vs planned
| | State |
|---|---|
| Today | Spec only (this README + LICENSE). Design carried in the MOPAC harness DESIGN.md "Key proxy" section. |
| Planned | Go implementation: placeholder->material resolver, Bitwarden Secrets Manager + Vault connectors, HTTP hop + `get` CLI, ref-redaction rules. |
| Today | v0 live: `serve` + `get` on file (0600 env files) and env backends; bearer auth bootstrapped from the file backend; table-driven tests incl. redaction + fake-HTTP end-to-end; docker-routed build discipline. |
| Planned | Phase 3: Bitwarden Secrets Manager REST + Vault KV v2/AppRole connectors behind the same interface (stubs and tests already in place); short-TTL memory cache. |
## Design references
Executable
+54
View File
@@ -0,0 +1,54 @@
#!/bin/sh
# keyproxy dev wrapper. EVERY compile/vet/test path routes through the
# digest-pinned Docker builder (DESIGN "ALL dev work in Docker" — big
# rule); the host runs containers, never toolchains.
#
# Usage: ./dev.sh {build|vet|test|check|smoke|shell} [args...]
#
# build compile ./cmd/keyproxy into bin/keyproxy
# vet go vet ./...
# test go test ./...
# check build + vet + test (the pre-push gate)
# smoke end-to-end smoke: builds, starts `keyproxy serve` in a
# container on port 8082 with a throwaway config, drives it
# from the host with python urllib (curl is banned on host),
# shows 401/200/404/501 paths + redacted server log, tears
# everything down
# shell interactive sh inside the builder
set -e
IMAGE="golang@sha256:e8c859f5632dcfde7b32d2012b4351728f6437930887c2f6a91ea242459e5514" # = golang:1.26-bookworm (bash present; alpine lacks it)
run() {
docker run --rm -v "$PWD:/h" -w /h \
-u "$(id -u):$(id -g)" -e HOME=/tmp -e GOFLAGS=-buildvcs=false \
"$IMAGE" "$@"
}
cmd=${1:-check}
shift || true
case "$cmd" in
build)
run go build -o bin/keyproxy ./cmd/keyproxy
;;
vet)
run go vet ./...
;;
test)
run go test "$@" ./...
;;
check)
run sh -c 'go build -o bin/keyproxy ./cmd/keyproxy && go vet ./... && go test ./...'
;;
smoke)
./smoke/smoke.sh
;;
shell)
run sh
;;
*)
echo "dev.sh: unknown command $cmd (build|vet|test|check|smoke|shell)" >&2
exit 1
;;
esac
+47
View File
@@ -0,0 +1,47 @@
# keyproxy.toml.example — copy to keyproxy.toml and fill in.
# keyproxy.toml is gitignored; it holds the ref MAP (paths + key names),
# never material itself. Refs are mpk-<name> placeholders: consumers see
# only these names; material resolves at request time, memory-only.
# HTTP hop bind address. Keep it loopback unless you know why not.
listen = "127.0.0.1:8082"
[auth]
# Bearer token for POST /v1/resolve, bootstrapped from the file backend
# itself: a 0600 env file holding the token. Resolved once at startup,
# never logged, never persisted by keyproxy.
token_ref = "mpk-keyproxy-self"
# --- file backend: KEY=VALUE env files, 0600, parsed in Go (never
# sourced/exec'd). mode is the allowed permission mask (default 0600;
# stricter files like 0400 always pass, looser files are refused).
[refs."mpk-keyproxy-self"]
backend = "file"
source = "~/.config/keyproxy/keyproxy.env"
key = "KEYPROXY_TOKEN"
[refs."mpk-example"]
backend = "file"
source = "~/.config/keyproxy/example.env"
key = "EXAMPLE_API_KEY"
# --- env backend: process-environment indirection. source IS the
# variable name (no key field); useful for container-injected values.
[refs."mpk-example-env"]
backend = "env"
source = "EXAMPLE_API_KEY"
# --- bitwarden backend: Bitwarden Secrets Manager REST (machine
# accounts). NOT IMPLEMENTED in v0 — resolves fail loudly with 501
# not_implemented so phase 3 is a drop-in behind the same interface.
[refs."mpk-example-bitwarden"]
backend = "bitwarden"
source = "sm://projects/example"
key = "EXAMPLE_API_KEY"
# --- vault backend: HashiCorp Vault KV v2 + AppRole. NOT IMPLEMENTED in
# v0 — same explicit 501 stub as bitwarden.
[refs."mpk-example-vault"]
backend = "vault"
source = "secret/data/example"
key = "EXAMPLE_API_KEY"
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""keyproxy resolve smoke probe.
Drives `keyproxy serve` on BASE (default http://127.0.0.1:8082) with
plain stdlib urllib: the host has no curl by policy. Checks: healthz,
401 without/with wrong token, 200 file + env resolves, 404 unknown ref,
400 invalid ref format (without echo), 501 bitwarden/vault stubs, 502
empty-value failure. Prints one PASS/FAIL line per check; exits non-zero
on any failure.
"""
import json
import os
import sys
import time
import urllib.error
import urllib.request
BASE = os.environ.get("BASE_URL", "http://127.0.0.1:8082")
TOKEN = os.environ.get("KEYPROXY_TOKEN", "smoke-bearer-token-1111")
failures = 0
def req(method, path, body=None, headers=None):
r = urllib.request.Request(BASE + path, data=body, method=method,
headers=headers or {})
try:
with urllib.request.urlopen(r, timeout=5) as resp:
return resp.status, resp.read().decode()
except urllib.error.HTTPError as e:
return e.code, e.read().decode()
except Exception as e: # transport errors surface as FAIL lines, not tracebacks
return 0, f"transport error: {e}"
def check(name, cond, detail=""):
global failures
print(("PASS" if cond else "FAIL"), name, ("- " + detail) if detail else "")
if not cond:
failures += 1
def wait_healthz(tries=50):
for _ in range(tries):
try:
code, _ = req("GET", "/healthz")
if code == 200:
return True
except Exception:
pass
time.sleep(0.2)
return False
def resolve(token, ref):
body = json.dumps({"ref": ref}).encode()
headers = {"Authorization": "Bearer " + token} if token else {}
return req("POST", "/v1/resolve", body, headers)
def main():
check("healthz answers (unauthenticated)", wait_healthz())
code, text = resolve(None, "mpk-smoke")
check("no bearer token rejected 401", code == 401, text.strip())
code, text = resolve("wrong-token", "mpk-smoke")
check("wrong bearer token rejected 401", code == 401, text.strip())
code, text = resolve(TOKEN, "mpk-smoke")
obj = json.loads(text) if text else {}
check("file ref resolves 200 with value",
code == 200 and obj.get("value") == "smoke-secret-material-2222",
text.strip())
code, text = resolve(TOKEN, "mpk-smoke-env")
obj = json.loads(text) if text else {}
check("env ref resolves 200 with value",
code == 200 and obj.get("value") == "smoke-env-material-4444",
text.strip())
code, text = resolve(TOKEN, "mpk-ghost")
check("unknown ref 404", code == 404 and "mpk-ghost" in text, text.strip())
code, text = resolve(TOKEN, "sk-live-pasted-secret")
check("invalid ref format 400 without echo",
code == 400 and "pasted" not in text, text.strip())
code, text = resolve(TOKEN, "mpk-smoke-bw")
check("bitwarden stub 501 not_implemented",
code == 501 and "not_implemented" in text and "bitwarden" in text,
text.strip())
code, text = resolve(TOKEN, "mpk-smoke-vault")
check("vault stub 501 not_implemented",
code == 501 and "not_implemented" in text and "vault" in text,
text.strip())
code, text = req("GET", "/v1/resolve")
check("GET resolve rejected 405", code == 405, text.strip())
print("probe:", "OK" if failures == 0 else f"{failures} FAILURE(S)")
return 0 if failures == 0 else 1
if __name__ == "__main__":
sys.exit(main())
Executable
+94
View File
@@ -0,0 +1,94 @@
#!/bin/sh
# keyproxy end-to-end smoke: builds in the digest-pinned builder, starts
# `keyproxy serve` in a container on host port 8082 with a throwaway
# config + throwaway 0600 env files, drives it from the host with python
# urllib (curl is banned on the host by policy), checks the 401/200/404/
# 400/501/502 paths AND that the server log is redacted (no material,
# every ref masked as <ref>=***), then tears everything down.
set -e
cd "$(dirname "$0")/.."
IMAGE="golang@sha256:e8c859f5632dcfde7b32d2012b4351728f6437930887c2f6a91ea242459e5514" # = golang:1.26-bookworm
NAME="keyproxy-smoke"
PORT="${PORT:-8082}"
./dev.sh build
rm -rf .smoke
mkdir -p .smoke
chmod 0700 .smoke
# Throwaway material (smoke values only; never real secrets).
cat > .smoke/keyproxy.env <<'EOF'
KEYPROXY_TOKEN=smoke-bearer-token-1111
EOF
cat > .smoke/creds.env <<'EOF'
API_KEY=smoke-secret-material-2222
OTHER=smoke-other-3333
EOF
chmod 0600 .smoke/keyproxy.env .smoke/creds.env
# In-container bind is all-interfaces ONLY because docker -p publishing
# is the boundary here; the host default (and example config) stays
# loopback.
cat > .smoke/keyproxy.toml <<EOF
listen = ":8082"
[auth]
token_ref = "mpk-keyproxy-self"
[refs."mpk-keyproxy-self"]
backend = "file"
source = "/h/.smoke/keyproxy.env"
key = "KEYPROXY_TOKEN"
[refs."mpk-smoke"]
backend = "file"
source = "/h/.smoke/creds.env"
key = "API_KEY"
[refs."mpk-smoke-env"]
backend = "env"
source = "KEYPROXY_SMOKE_ENV"
[refs."mpk-smoke-bw"]
backend = "bitwarden"
source = "sm://p/smoke"
key = "API_KEY"
[refs."mpk-smoke-vault"]
backend = "vault"
source = "secret/data/smoke"
key = "API_KEY"
EOF
docker rm -f "$NAME" >/dev/null 2>&1 || true
docker run -d --name "$NAME" -p "$PORT:8082" \
-v "$PWD:/h" -w /h -u "$(id -u):$(id -g)" -e HOME=/tmp \
-e KEYPROXY_SMOKE_ENV=smoke-env-material-4444 \
"$IMAGE" /h/bin/keyproxy serve -config /h/.smoke/keyproxy.toml
cleanup() {
docker rm -f "$NAME" >/dev/null 2>&1 || true
}
trap cleanup EXIT
echo "--- probing http://127.0.0.1:$PORT (python urllib; curl banned on host) ---"
BASE_URL="http://127.0.0.1:$PORT" KEYPROXY_TOKEN="smoke-bearer-token-1111" \
python3 smoke/probe.py
probe_rc=$?
echo "--- server log (redaction check) ---"
LOG="$(docker logs "$NAME" 2>&1)"
echo "$LOG"
if echo "$LOG" | grep -q 'smoke-secret-material-2222\|smoke-env-material-4444\|smoke-bearer-token-1111'; then
echo "FAIL: server log contains material"
exit 1
fi
if ! echo "$LOG" | grep -q 'ref=mpk-smoke=\*\*\*'; then
echo "FAIL: server log does not mask refs as <ref>=***"
exit 1
fi
exit "$probe_rc"