Files

T

ci / audit (push) Failing after 54s

Details

[#800 ] ACME e2e proof: IP-identifier issuance validated on sectestbed 5104

http-01 validation, issuance, and chain all verified against the fleet
root via the prod endpoint; 5104 shut back down after the test.
https://projects.knownelement.com/issues/800

2026-09-05 10:16:30 -05:00

2.8 KiB

Raw Blame History

Prod ACME endpoint runbook — step-ca on tsys-ca

Ticket #800 · parent #697 · lane index https://community.turnsys.com/t/297

Live endpoint (2026-09-05)

Directory: https://100.102.96.24:8443/acme/acme/directory (tailnet only)
Chain: ECDSA KNEL Fleet ACME Intermediate ← fleet RSA root (/root/ca-root, stays on the host; key never leaves)
Container: ukrrs-oam-ca-stepca, host networking, listen :8443 (ca.json)
State: /opt/stepca-prod/ (data/ + secrets/password, root-owned)
Repo files: docker/stepca-prod/{compose.yaml,init-stepca.sh}
Re-init: bash init-stepca.sh --reinit (wipes data/, new intermediate)

Client usage (HTTP-01)

lego --server https://100.102.96.24:8443/acme/acme/directory \
     --email agent@knel.net --accept-tos \
     --domains <fqdn> --http run
# trust: install fleet root from http://tsys-ca.knel.net/ca/dist/

DEBUG LOG — http-01 validation (2026-09-05)

Root cause of the "could not connect to validation target" loop on the workstation test target: ultix-streaming (the workstation) is a joined k8s worker node. kube-proxy installs CNI portmap rules that DNAT all tcp/80 traffic to the node's IPs (incl. 127.0.0.1 and the tailnet IP) into a cluster ingress pod — so lego's challenge listener never receives the CA's fetch (it gets the ingress pod's Go-404 instead). strace proved step-ca DID resolve and dial correctly; the fetch landed on the pod.

Consequences / rules for ACME http-01 across the fleet:

http-01 targets must have a FREE, un-intercepted port 80. Any host running a k3s worker + ingress (the workstation, future k8s nodes) CANNOT answer http-01 for its own names.
For such hosts use DNS-01 (pending TSIG approval, #800 Q1) or a k8s-native cert solution (cert-manager) — do NOT fight the port map.
Plain fleet VMs (no k8s) are fine for http-01.

E2E proof plan: boot sectestbed VM (5104), run lego there with an IP identifier (fresh VM, port 80 free), issue via the prod endpoint.

E2E PROOF (2026-09-05) — http-01 issuance VERIFIED

sectestbed VM 5104 (dev PBS, tailnet 100.127.238.29, port 80 free):

lego --server https://100.102.96.24:8443/acme/acme/directory \
     --email agent@knel.net --accept-tos \
     --domains 100.127.238.29 --http run
→ Served key authentication → The server validated our request
→ Server responded with a certificate
Issuer: CN=KNEL Fleet ACME Intermediate (ECDSA, under fleet RSA root)
SAN: IP:100.127.238.29

IP identifiers work. DNS identifiers work when the name resolves from tsys-ca (rooted FQDN — add /etc/hosts entry on tsys-ca for test names). Verify chain: openssl verify -CAfile root.crt -untrusted issuer.crt leaf.crt VM shut back down after the test (lane rule).

2.8 KiB Raw Blame History