Add end-to-end smoke and rewrite the README for v0

The smoke boots the fake Secrets Manager in a container and drives the real binary from the host through a 0600 env file: login, listings, get by name and uuid, failure paths, and a redaction sweep over every captured output. README documents the implemented wire protocol, the library surface keyproxy will call, the verified quickstart, and the config table.
2026-08-29 00:08:38 -05:00
parent edf056eb9f
commit 9b730afd12
4 changed files with 329 additions and 38 deletions
@@ -1,52 +1,164 @@
 # mopac-bitwarden-go

-A 100% Go CLI for Bitwarden / Bitwarden Secrets Manager, replacing the Node
-`bw` CLI in supply-chain-sensitive (CMMC L3/TS posture) environments. One
-static binary, one auditable vendored module tree, no Node runtime.
+A 100% Go client for the Bitwarden Secrets Manager REST API, replacing the
+Node `bw` CLI in supply-chain-sensitive (CMMC L3/TS posture) environments.
+One static binary, zero third-party modules (stdlib only), no Node
+runtime, no official SDK (source-available, AGPL-incompatible).

-Status: 2026-08-28 — spec seed; reference study complete, open questions
-pending Charles, no code yet.
+Status: 2026-08-28 — v0 complete and green: machine-account auth
+(client_credentials, `encrypted_payload` organization-key unwrap),
+refresh-before-expiry, secret/project reads with full client-side
+decryption (type-2 EncStrings), 0600 env-file config, thin CLI. Built and
+tested entirely against a fake Secrets Manager (no live vault access on
+this account yet); live credentials attach with zero code change.

-## Scope
+## What it implements

- Talks to the Bitwarden Secrets Manager REST API directly with stdlib —
-  NO official SDK (its source-available license is AGPL-incompatible).
- Machine-account auth (access-token flow) for headless/agent use; human
-  auth flows where needed.
- Porcelain/plumbing model: `bwg get|set|list|sync ...` with JSON out for
-  scripting, plain text for humans.
- Memory-only secrets handling: session token held in memory for the
-  process lifetime and zeroed on exit; values never written to logs, disk
-  cache, REPORT files, or crash dumps; refs logged only in redacted form.
- Pairs with [ukrrs/mopac-keyproxy](https://git.knownelement.com/ukrrs/mopac-keyproxy)
-  (placeholder keys to consumers; this CLI is how material gets INTO the
-  vault) but fully standalone.
+The wire protocol, in plain REST with stdlib:
+
+| Step | Wire |
+|---|---|
+| login | `POST /identity/connect/token` — `grant_type=client_credentials`, `scope=api.secrets`, machine client_id/secret. Response carries a JWT access token plus `encrypted_payload` |
+| org key | `encrypted_payload` is a type-2 EncString sealed for the credential's embedded 16-byte key (HKDF-SHA256, salt `bitwarden-accesstoken`, info `sm-access-token`); it unwraps to the organization key |
+| refresh | `POST /identity/connect/token` — `grant_type=refresh_token`, fired automatically 30 s before expiry |
+| secrets | `GET /api/accounts/{id}/secrets`, `GET /api/secrets/{id}`, `GET /api/projects/{id}/secrets` — names/values arrive as EncStrings, decrypted in memory with the org key |
+| projects | `GET /api/accounts/{id}/projects` |
+
+The crypto construction is pinned to the official clients by published
+SDK test vectors (key derivation, EncString AE, credential parsing) — see
+`encstring_test.go`.
+
+## Library surface (what keyproxy calls)
+
+```go
+import bw "git.knownelement.com/ukrrs/mopac-bitwarden-go"
+
+tok, err := bw.Authenticate(ctx, bw.Credentials{BaseURL: url, AccessToken: cred})
+defer tok.Zero()
+value, err := bw.GetSecret(ctx, tok, "redmine-api-key") // name or uuid
+secrets, err := bw.ListSecrets(ctx, tok)                // metadata only
+projects, err := bw.ListProjects(ctx, tok)
+```
+
+Tokens and keys are memory-only; `Token.String()` renders expiry and ids,
+never material; error strings carry fixed reason enums plus uuids/names,
+never values. Errors classify as `ErrAuthFailed`, `ErrTokenExpired`,
+`ErrSecretNotFound`, `ErrMalformedResponse`, `ErrDecrypt`, `ErrServer`,
+`ErrUnreachable`, `ErrInvalidCredentials` (`errors.Is` friendly).
+
+## Quickstart (verified 2026-08-28, all against the fake server)
+
+All dev work happens inside a Docker builder (host stays toolchain-free).
+```sh
+./dev.sh check   # = go build + go vet + go test, inside golang:1.26-bookworm
+```
+Expected output (tail):
+```text
+ok  	git.knownelement.com/ukrrs/mopac-bitwarden-go
+ok  	git.knownelement.com/ukrrs/mopac-bitwarden-go/internal/cli
+ok  	git.knownelement.com/ukrrs/mopac-bitwarden-go/internal/config
+```
+
+End-to-end smoke — builds the CLI, boots the fake Secrets Manager in a
+container on `127.0.0.1:8600`, drives the real binary from the host
+through a 0600 env file (login / projects / secrets list / get by name and
+uuid / failure paths / redaction sweep):
+```sh
+./dev.sh smoke
+```
+Expected output (tail):
+```text
+--- redaction: no material in any captured output
+smoke: OK
+```
+
+### Configure
+
+Credentials NEVER arrive via flags or arguments:
+```sh
+mkdir -p ~/.config/bitwarden-go && umask 077
+cp env.example ~/.config/bitwarden-go/env
+$EDITOR ~/.config/bitwarden-go/env   # fill BW_ACCESS_TOKEN etc.
+```
+
+### Use
+
+```sh
+./bin/bitwarden-go login                    # credential check, prints summary only
+./bin/bitwarden-go projects                 # "<id> <name>" per line
+./bin/bitwarden-go secrets list             # "<id> <name>" per line
+./bin/bitwarden-go get smoke-redmine-key    # bare value, no newline ($(...) plumbing)
+```
+`get` accepts a secret NAME or a secret UUID. Exit codes: 0 ok, 1
+usage/config, 2 auth or resolution failure.
+
+## Config reference
+
+| Key | Where | Meaning |
+|---|---|---|
+| `BW_SERVER_URL` | env or env file | server root (`<url>/identity/connect/token`, `<url>/api/...`); default `https://vault.bitwarden.com`; self-hosted Vaultwarden sets this |
+| `BW_ACCESS_TOKEN` | env or env file | full machine credential `0.<uuid>.<secret>:<key>` as printed by Secrets Manager; unlocks full decryption |
+| `BW_CLIENTID` / `BW_CLIENTSECRET` | env or env file | split form; works only against servers returning unencrypted payloads (test doubles) |
+| `-config PATH` / `$BITWARDENGO_CONFIG` | CLI | env-file location; default `~/.config/bitwarden-go/env` |
+
+Env-file discipline (same as mopac-keyproxy): `KEY=VALUE` parsed in pure
+Go — never sourced or exec'd; mode MUST be 0600 or stricter, checked
+BEFORE the file is read; process env wins over the file; `*.env` files
+are gitignored anywhere in this repo. Any `*.env` landing in the repo is a
+tripwire.
+
+## Security rules (enforced by tests)
+
+- Tokens, keys and values are memory-only: no disk cache, no persistence,
+  no admin UI, nothing written to disk — ever.
+- Errors carry fixed reason enums plus non-secret identifiers (uuids,
+  secret names); response bodies, credentials, tokens and ciphertext are
+  never embedded. Transport errors are scrubbed of URLs and peer text.
+- `get` prints the value to stdout bare — that is the one place material
+  appears; stderr and every other command stay material-free (asserted in
+  tests and smoke).
+- Type-2 EncStrings fail closed: MAC verified before decryption; tampered
+  MAC/ciphertext is a typed `ErrDecrypt`, never passthrough.
+- JWT signatures are not verified locally (same posture as the official
+  clients; the endpoint is reached over TLS).
+
+## Architecture
+
+```mermaid
+flowchart LR
+    C["consumer<br/>(keyproxy bitwarden backend, CLI)"] --> L["library: Authenticate / GetSecret<br/>stdlib-only REST + decrypt"]
+    L -->|"POST /identity/connect/token<br/>client_credentials"| I["Bitwarden / Vaultwarden<br/>/identity"]
+    L -->|"GET /api/accounts/{id}/secrets ...<br/>bearer"| A["Bitwarden / Vaultwarden<br/>/api"]
+    L -.-> F["fakesm (tests + smoke only)<br/>same wire protocol + crypto"]
+```
+
+- `bitwarden.go` — public surface: `Authenticate`, `GetSecret`,
+  `ListSecrets`, `ListProjects`, error sentinels.
+- `token.go`, `api.go` — token lifecycle (refresh-before-expiry) and REST
+  transport.
+- `cred.go`, `encstring.go` — credential parsing/key derivation and
+  EncString crypto (AES-256-CBC + HMAC-SHA256, PKCS#7).
+- `internal/config` — 0600 env-file + env resolution.
+- `internal/cli`, `cmd/bitwarden-go` — the thin CLI.
+- `internal/fakesm`, `smoke/` — the fake Secrets Manager (tests run
+  in-process; smoke boots it in a container).

 ## Non-goals

- No admin UI; a CLI and nothing else.
- Not a vault server — Vaultwarden/Bitwarden stays the store of record.
- No disk cache of secret values; no long-lived persisted sessions by
-  default (open question 4 below may change that, founder's call).
- Not org-specific: hosts/credentials come from config and environment,
-  never baked in.
-
-## Today vs planned
-
-| | State |
-|---|---|
-| Today | Spec only (this README + LICENSE). Interim in production: the KNELSecretsManager containerized `bw` wrapper (ADR-002) — plain `bw` behind docker, plaintext env on disk, full login/unlock/sync per call. |
-| Planned | Go CLI per the reference study: `bw:` key-ref resolution for the MOPAC harness, lookup by item name (password field), per-process unlock, never `bw logout`, typed not-found/unlock errors, fake-bw test stub. |
+- No writes: v0 is the read path keyproxy needs (create/update/delete,
+  human auth flows: later).
+- Not a vault server; Bitwarden/Vaultwarden stays the store of record.
+- No persisted sessions (memory-only by design; see open question on
+  session lifetime in the porting notes).
+- Not org-specific: hosts and credentials come from config, never baked in.

 ## Design references

- [KNELSecretsManager study — current surface, replacement design sketch, open questions](https://git.knownelement.com/ukrrs/MOPAC/src/branch/main/docs/PORTING-NOTES-secrets.md)
-  (section 3 lists seven open questions for Charles: substrate, machine
-  account, ref syntax, session lifetime, secret names, scope, subprocess
-  injection)
- [MOPAC harness DESIGN.md — Toolchain policy, 100% Go HARD RULE](https://git.knownelement.com/ukrrs/MOPAC/src/branch/main/DESIGN.md#user-content-toolchain-policy-charles-2026-08-28-evening--hard-rule)
- [MOPAC harness DESIGN.md — Key proxy](https://git.knownelement.com/ukrrs/MOPAC/src/branch/main/DESIGN.md#user-content-key-proxy-placeholders-only-material-never-leaves-the-vault-charles-2026-08-28-evening)
- Parent: [ukrrs/MOPAC](https://git.knownelement.com/ukrrs/MOPAC) — the harness whose `bw:` key refs this unblocks
+- [Porting notes: KNELSecretsManager](https://git.knownelement.com/ukrrs/MOPAC/src/branch/main/docs/PORTING-NOTES-secrets.md)
+- [ukrrs/mopac-keyproxy](https://git.knownelement.com/ukrrs/mopac-keyproxy) — the `bitwarden:` backend that consumes this library (phase 3)
+- Bitwarden public API behavior (identity connect/token, machine
+  accounts, `/api/accounts/{id}/secrets`) — reimplemented from protocol
+  observation; the official SDK is NOT imported.

 ## License

@@ -0,0 +1,16 @@
+# bitwarden-go machine credentials (copy to a 0600 file, e.g.
+# ~/.config/bitwarden-go/env, and fill in). Values are NEVER passed as
+# flags or arguments. Any *.env file in this repo is gitignored.
+#
+# Server root; identity endpoint is <url>/identity/connect/token and the
+# API is <url>/api/... . Default when unset: https://vault.bitwarden.com
+BW_SERVER_URL=https://vault.example.com
+
+# Machine access token as printed by Bitwarden Secrets Manager
+# ("0.<uuid>.<secret>:<key>"). This single value unlocks full decryption.
+BW_ACCESS_TOKEN=0.00000000-0000-4000-8000-000000000000.replace-me:AAAAAAAAAAAAAAAAAAAAAA==
+
+# OR the split form (no local key: only servers returning unencrypted
+# payloads work with it):
+# BW_CLIENTID=00000000-0000-4000-8000-000000000000
+# BW_CLIENTSECRET=replace-me
@@ -0,0 +1,50 @@
+// Command fakesm boots the fake Bitwarden Secrets Manager on a plain TCP
+// listener for the smoke script (tests use httptest via fakesm.Server
+// directly). The credential identity below is the published SDK sample;
+// the smoke script configures the CLI with the exact same string, and the
+// organization key is random per boot so the full decrypt chain runs.
+package main
+
+import (
+	"crypto/rand"
+	"encoding/base64"
+	"flag"
+	"log"
+	"time"
+
+	"git.knownelement.com/ukrrs/mopac-bitwarden-go/internal/fakesm"
+)
+
+const smokeCredential = "0.ec2c1d46-6a4b-4751-a310-af9601317f2d.C2IgxjjLF7qSshsbwe8JGcbM075YXw:X8vbvA0bduihIDe/qrzIQQ=="
+
+func main() {
+	addr := flag.String("addr", ":8600", "listen address")
+	flag.Parse()
+	tokenKey, err := base64.StdEncoding.DecodeString("X8vbvA0bduihIDe/qrzIQQ==")
+	if err != nil || len(tokenKey) != 16 {
+		log.Fatal("fakesm: bad token key part")
+	}
+	orgKey := make([]byte, 64)
+	if _, err := rand.Read(orgKey); err != nil {
+		log.Fatal("fakesm: entropy")
+	}
+	s := &fakesm.Server{
+		ClientID:     "ec2c1d46-6a4b-4751-a310-af9601317f2d",
+		ClientSecret: "C2IgxjjLF7qSshsbwe8JGcbM075YXw",
+		TokenKey:     tokenKey,
+		OrgKey:       orgKey,
+		OrgID:        "3fb1c0de-0000-4000-8000-000000000000",
+		TokenTTL:     time.Hour,
+		Projects: []fakesm.Project{
+			{ID: "ac1d0000-0000-4000-8000-000000000001", Name: "harness"},
+		},
+		Secrets: []fakesm.Secret{
+			{ID: "5ec1e700-0000-4000-8000-00000000000a", Name: "smoke-redmine-key", Value: "smoke-redmine-value-0123456789abcdef"},
+			{ID: "5ec1e700-0000-4000-8000-00000000000b", Name: "smoke-litellm-key", Value: "smoke-litellm-value-fedcba9876543210"},
+		},
+	}
+	log.Printf("fakesm: listening addr=%s org=%s", *addr, s.OrgID)
+	if err := s.ListenAndServe(*addr); err != nil {
+		log.Fatal(err)
+	}
+}
@@ -0,0 +1,113 @@
+#!/bin/sh
+# End-to-end smoke for bitwarden-go: builds the CLI in the Docker builder,
+# boots the FAKE Secrets Manager in a container on 127.0.0.1:8600, drives
+# the real binary from the host through a 0600 env file, and asserts the
+# happy paths plus redaction. No real vault is ever contacted. Only exact
+# container IDs / PIDs spawned here are killed.
+set -e
+
+cd "$(dirname "$0")/.."
+
+IMAGE="golang@sha256:e8c859f5632dcfde7b32d2012b4351728f6437930887c2f6a91ea242459e5514"
+PORT=8600
+CID=""
+
+cleanup() {
+    if [ -n "$CID" ]; then
+        docker rm -f "$CID" >/dev/null 2>&1 || true
+    fi
+    rm -rf .smoke
+}
+trap cleanup EXIT INT TERM
+
+mkdir -p .smoke
+umask 077
+
+echo "--- build CLI (docker builder)"
+docker run --rm -v "$PWD:/h" -w /h \
+    -u "$(id -u):$(id -g)" -e HOME=/tmp -e GOFLAGS=-buildvcs=false \
+    "$IMAGE" go build -o bin/bitwarden-go ./cmd/bitwarden-go
+
+echo "--- boot fake Secrets Manager (container, port $PORT)"
+CID=$(docker run -d --rm \
+    -v "$PWD:/h" -w /h \
+    -u "$(id -u):$(id -g)" -e HOME=/tmp -e GOFLAGS=-buildvcs=false \
+    -p 127.0.0.1:$PORT:8600 \
+    "$IMAGE" go run ./smoke/fakesm -addr :8600)
+
+# wait for the fake to answer (any HTTP response, even 400, proves it is up)
+i=0
+until [ -n "$CID" ] && [ "$(docker inspect -f '{{.State.Running}}' "$CID" 2>/dev/null)" = "true" ] && \
+      printf 'grant_type=client_credentials&client_id=probe&client_secret=probe&scope=api.secrets' \
+      | python3 -c "
+import sys, urllib.request, urllib.error
+req = urllib.request.Request('http://127.0.0.1:$PORT/identity/connect/token',
+    data=sys.stdin.buffer.read(), headers={'Content-Type':'application/x-www-form-urlencoded'})
+try:
+    urllib.request.urlopen(req, timeout=2)
+except urllib.error.HTTPError:
+    sys.exit(0)   # got an HTTP answer: server is up
+except Exception:
+    sys.exit(1)   # not yet
+sys.exit(0)
+"; do
+    i=$((i+1))
+    if [ "$i" -ge 60 ]; then
+        echo "smoke: fake server did not come up; logs:" >&2
+        docker logs "$CID" >&2 || true
+        exit 1
+    fi
+    sleep 1
+done
+
+CRED='0.ec2c1d46-6a4b-4751-a310-af9601317f2d.C2IgxjjLF7qSshsbwe8JGcbM075YXw:X8vbvA0bduihIDe/qrzIQQ=='
+printf 'BW_SERVER_URL=http://127.0.0.1:%s\nBW_ACCESS_TOKEN=%s\n' "$PORT" "$CRED" > .smoke/env
+chmod 600 .smoke/env
+
+export BITWARDENGO_CONFIG="$PWD/.smoke/env"
+
+echo "--- login (credential check; prints summary only)"
+./bin/bitwarden-go login | tee .smoke/login.out
+grep -q "authenticated: account ec2c1d46-6a4b-4751-a310-af9601317f2d" .smoke/login.out
+grep -q "expires" .smoke/login.out
+
+echo "--- projects"
+./bin/bitwarden-go projects > .smoke/projects.out
+grep -q "ac1d0000-0000-4000-8000-000000000001 harness" .smoke/projects.out
+
+echo "--- secrets list"
+./bin/bitwarden-go secrets list > .smoke/secrets.out
+grep -q "5ec1e700-0000-4000-8000-00000000000a smoke-redmine-key" .smoke/secrets.out
+grep -q "5ec1e700-0000-4000-8000-00000000000b smoke-litellm-key" .smoke/secrets.out
+
+echo "--- get (bare value, no newline)"
+V="$(./bin/bitwarden-go get smoke-redmine-key)"
+[ "$V" = "smoke-redmine-value-0123456789abcdef" ] || { echo "smoke: wrong value: $V" >&2; exit 1; }
+
+echo "--- get by uuid"
+V="$(./bin/bitwarden-go get 5ec1e700-0000-4000-8000-00000000000b)"
+[ "$V" = "smoke-litellm-value-fedcba9876543210" ] || { echo "smoke: wrong value: $V" >&2; exit 1; }
+
+echo "--- failure path: missing secret (exit code + redacted stderr)"
+if ./bin/bitwarden-go get no-such-secret > .smoke/missing.out 2> .smoke/missing.err; then
+    echo "smoke: missing secret should fail" >&2; exit 1
+fi
+grep -q "secret not found" .smoke/missing.err
+[ ! -s .smoke/missing.out ] || { echo "smoke: stdout not empty on failure" >&2; exit 1; }
+
+echo "--- failure path: bad credential (exit code)"
+printf 'BW_SERVER_URL=http://127.0.0.1:%s\nBW_ACCESS_TOKEN=0.ec2c1d46-6a4b-4751-a310-af9601317f2d.wrong-secret:X8vbvA0bduihIDe/qrzIQQ==\n' "$PORT" > .smoke/bad.env
+chmod 600 .smoke/bad.env
+if BITWARDENGO_CONFIG="$PWD/.smoke/bad.env" ./bin/bitwarden-go login 2> .smoke/bad.err; then
+    echo "smoke: bad credential should fail" >&2; exit 1
+fi
+grep -q "auth failed" .smoke/bad.err
+
+echo "--- redaction: no material in any captured output"
+for f in .smoke/login.out .smoke/projects.out .smoke/secrets.out .smoke/missing.err .smoke/bad.err; do
+    grep -qF 'C2IgxjjLF7qSshsbwe8JGcbM075YXw' "$f" && { echo "smoke: client secret leaked into $f" >&2; exit 1; }
+    grep -qF 'X8vbvA0bduihIDe/qrzIQQ==' "$f" && { echo "smoke: credential key leaked into $f" >&2; exit 1; }
+done
+grep -qF 'smoke-litellm-value' .smoke/secrets.out && { echo "smoke: secret value leaked into listing" >&2; exit 1; }
+
+echo "smoke: OK"