SKILL: fleet-proven procedure — per-app SSO scripts, admin key minting, access restrictions, vault fallbacks

Absorbs the 2026-09-09 fleet-wide rollout learnings: per-app SSO entry
points and quirks (discourse /auth/oidc + CSRF + confirm, redmine
oauth_provider=1, gitea username normalization), Cloudron app
accessRestriction management, central admin-lever key minting for
gitea/redmine/discourse (GLPI deferred on #947), and the
setfield-verify-recreate seed-storage pattern.
https://projects.knownelement.com/issues/942

skills/agent-provisioning/SKILL.md

@@ -7,7 +7,7 @@ description: >
   schema, and verification. Use whenever asked to "stand up" an identity or
   agent, create/fix Cloudron users, wire SSO, debug identity logins or 2FA,
   or store identity credentials — even if the word "provisioning" is not used.
-  Policy-critical: encodes reachableceo rulings #942 (unique passwords, Cloudron-only
+  Policy-critical: encodes founder rulings #942 (unique passwords, Cloudron-only
   accounts, no reachableceo credentials).
 ---
 
@@ -19,17 +19,17 @@ Bitwarden vault. The Cloudron is the ONLY identity source: never create local
 accounts in Redmine, Gitea, Discourse, or GLPI — the system account comes into
 existence when the identity completes its first SSO login there.
 
-## Non-negotiable policy (reachableceo rulings — violating these invalidates the work)
+## Non-negotiable policy (founder rulings — violating these invalidates the work)
 
 1. **Unique password per identity.** One identity, one password, used only for
    its Cloudron account. Shared passwords are banned (Redmine #942).
 2. **Cloudron-only accounts.** System access = SSO first-login. No local
    accounts, no admin-created users in the downstream systems.
 3. **Never use reachableceo credentials.** Items like `creds/gitea-admin`,
-   `creds/gitea`, `creds/redmine` are reachableceo's personal accounts and
+   `creds/gitea`, `creds/redmine` are the founder's personal accounts and
    carry RESTRICTED notes. If an auth test resolves to user `reachableceo`,
-   stop — you are holding reachableceo's identity.
+   stop — you are holding the founder's identity.
-4. **TOTP seeds MUST be stored in Bitwarden** (reachableceo ruling 2026-09-08).
+4. **TOTP seeds MUST be stored in Bitwarden** (founder ruling 2026-09-08).
    Enroll TOTP and write the base32 seed to the vault item in the same
    breath — a seed that is not in the vault is treated as lost, because
    there is no admin 2FA reset on this Cloudron and no other record exists.
@@ -42,9 +42,9 @@ existence when the identity completes its first SSO login there.
 
 ## Scope check
 
-Identity stand-up is the COO C2 lane's assigned program (reachableceo directive,
+Identity stand-up is the COO C2 lane's assigned program (founder directive,
 2026-09-08). If your chat runs in another lane (SRE, security, OAM), confirm
-reachableceo assigned the stand-up to you before executing, or route a ticket
+the founder assigned the stand-up to you before executing, or route a ticket
 to the COO C2 lane instead.
 
 ## Admin lever: Cloudron API token
@@ -70,8 +70,8 @@ seed was lost, then re-enroll with capture. Admin password set:
 ### 0. Prerequisites
 
 - Vault reachable: `sm get creds/cloudron` works (see references/vault.md).
-- For a NEW Cloudron user: reachableceo generates the invite, or you create the
+- For a NEW Cloudron user: the founder generates the invite, or you create the
-  user via the admin token if reachableceo has authorized self-service.
+  user via the admin token if the founder has authorized self-service.
 - A Redmine ticket to record the work (create one if none exists).
 
 ### 1. Cloudron account (unique password)
@@ -114,7 +114,7 @@ by a real TOTP login (a totpToken in a login is silently ignored for users
 without 2FA — "success" there proves nothing).
 
 If the identity already has 2FA enforced and no seed exists anywhere: STOP and
-ask reachableceo. There is no programmatic recovery (verified 2026-09-08).
+ask the founder. There is no programmatic recovery (verified 2026-09-08).
 Compute codes offline when needed: `scripts/totp.sh <seed>`.
 
 ### 3. TOTP enrollment (in the identity's dashboard session)
@@ -130,36 +130,72 @@ and is the ONLY trustworthy success signal.
 
 ### 4. SSO first-login into each system
 
-The interactive login is scriptable with a cookie jar — no browser needed.
+The interactive login is fully scriptable per app — use the per-app scripts;
-`scripts/oidc-login.sh <system> <login>` runs the whole flow and leaves a
+each runs the whole flow (entry → Cloudron interaction → TOTP from the vault
-session cookie jar at `/tmp/oidc-<login>-<system>.txt`. Entry points:
+seed → resume → consent → callback) with a cookie jar and verifies a live
+session:
 
-| System | SSO start point | After login |
+```bash
+scripts/gitea-sso-login.sh     <username> [vault-item]   # /user/oauth2/cloudron
+scripts/redmine-sso-login.sh   <username> [vault-item]   # /oauth?oauth_provider=1
+scripts/discourse-sso-login.sh <username> [vault-item]   # /auth/oidc (+ CSRF + confirm form)
+```
+
+| System | SSO start point | Gotchas |
 |---|---|---|
-| GLPI | `https://cmdb.knownelement.com/plugins/singlesignon/front/callback.php/provider/1?remember=1` | account auto-provisioned |
+| GLPI | `https://cmdb.knownelement.com/plugins/singlesignon/front/callback.php/provider/1?remember=1` | account auto-provisioned; no scripted flow yet |
-| Gitea | `https://git.knownelement.com/user/oauth2/cloudron` | account auto-provisioned |
+| Gitea | `https://git.knownelement.com/user/oauth2/cloudron` | redirects to `client_id=<app-uuid>-oidc`; gitea may NORMALIZE the username (e.g. `tsgstaff-coo-vpperf` → `vpperf`) — verify which login got created |
-| Discourse | `https://community.turnsys.com/auth/openid_connect` | account auto-provisioned |
+| Discourse | `https://community.turnsys.com/auth/oidc` | hostname is community.**turnsys.com**; provider name is `oidc`, NOT openid_connect; needs `/session/csrf` token then a confirm-page authenticity_token POST |
-| Redmine | `https://projects.knownelement.com/oauth` | account auto-provisioned (if the bounce in references/systems.md is hit, see that file) |
+| Redmine | `https://projects.knownelement.com/oauth?oauth_provider=1` | the "bounce to /login" mystery = the plugin's GET form needs `oauth_provider=1` |
+
+A failed session-verify can still mean the ACCOUNT was created (common) —
+check via the admin API before retrying. App-level access control can deny
+the whole flow with `access_denied: User has no access to this app`: the
+Cloudron app's `accessRestriction` (per-user/group allow-list) must include
+the identity. Set it with
+`POST /api/v1/apps/<app-id>/configure/access_restriction` body
+`{"accessRestriction":{"users":[uids…],"groups":[gids…]}}` — the `groups`
+array is REQUIRED even when empty. Gitea (1c91ff45-…) and Discourse
+(b506a0c1-…) both carry allow-lists.
 
 Key gotcha (cost a debugging session — do not rediscover): the OIDC
 interaction login is `POST /openid/interaction/<uid>/login` with JSON body
 `{"username": "...", "password": "...", "totpToken": "..."}` — the field is
 **`username`**, not `login`, and the response is `{"redirectTo": ...}` on
-success.
+success (absolute URL — never concatenate a base onto it).
 
-### 5. Mint per-system API keys (as the identity, via its session)
+### 5. Mint per-system API keys
 
-- **Redmine**: My account → API key (visible to self); also returned by
+vptechops holds ADMIN in all four systems (2026-09-09), so keys are minted
-  `GET /users/current.json` with the session.
+centrally — no per-identity session needed:
-- **Gitea**: Settings → Applications → new token with write scopes; or
-  `POST /api/v1/users/<login>/tokens` using the session (requires basic auth
-  of the same user for token endpoints).
-- **Discourse**: user API key flow; or admin-generated key if an admin lever
-  is ever provided (none currently is — do not use reachableceo keys).
-- **GLPI**: My account → API token (personal token); pair with the app token
-  from `creds/glpi` (password field) for REST calls.
 
-Store each key in the vault item immediately (schema below).
+- **Gitea** (admin basic auth; the token route REQUIRES basic auth — a token
+  works as the basic password):
+  `POST /api/v1/users/<login>/tokens -u "vptechops:$admin_token"` with
+  `{"name":"agent-<date>","scopes":["all"]}` → `.sha1`. Minting is scoped to
+  the authenticating token's scopes, so use the full-scope admin token
+  (`creds/vptechops-gitea` `admin_token`, generated via
+  `gitea admin user generate-access-token` in the container with
+  `--config /run/gitea/app.ini`).
+- **Redmine**: api_key is not in `GET /users/:id.json` for others — dump via
+  rails runner in the app container (`User.where(status:1).each { |u| puts
+  "#{u.login}|#{u.api_key}" }`; needs `HOME=/tmp`, `SECRET_KEY_BASE` from
+  `/app/data/secrets.yml`, rbenv shims in PATH, `bundle exec rails runner`).
+- **Discourse**: rails runner `ApiKey.create!(user_id:, description:,
+  created_by_id:)` per user — `.key` readable only at creation. Note:
+  Discourse usernames can differ from Cloudron usernames (legacy
+  `coo1coo`, prefix-stripped `vpengops`…) — resolve via
+  `/admin/users/list/active.json` first. Admin Api-Key minting uses the same
+  model; admin key for vptechops is in `creds/vptechops-discourse`
+  (`admin_api_key`).
+- **GLPI**: DEFERRED on #947 — REST `user_token` auth 401s even for a
+  super-admin with api_token/personal_token set; do not burn time re-deriving
+  that failure.
+
+Store each key in the vault item immediately (schema below), and always
+verify the write (`sm get … --field …` readback) — `sm setfield` 400s
+("Data missing") on some ciphers; fall back to a full item recreate that
+PRESERVES every existing field (username, password, uri, totp_seed, tokens).
 
 ### 6. Vault item schema — `creds/<login>`
 
@@ -191,7 +227,7 @@ After provisioning changes the identity estate:
 
 Details: `references/org-chart-update.md`.
 
-### 7. Verify the trail
+### 8. Verify the trail
 
 Prove each system knows the identity (expected: 200 + the identity's own
 name; a 200 from the wrong identity is a FAIL):
@@ -219,9 +255,15 @@ the workspace STATUS/memory.
 - `scripts/cloudron-oidc-login.sh <username> [vault-item]` — full OIDC
   dashboard login (PKCE + interaction + consent); Bearer at
   `/tmp/.oidc-bearer-<username>`.
-- `scripts/cloudron-totp-enroll.sh <username> [vault-item]` — enroll TOTP,
+- `scripts/cloudron-totp-enroll.sh <username> [vault-item]` — enroll TOTP
-  capture the seed to the vault, verify end-to-end.
+  (Cloudron 10 profile routes: POST `/api/v1/profile/totp_secret` +
-- `scripts/oidc-login.sh <glpi|gitea|discourse|redmine> <login>` — system SSO
+  `/api/v1/profile/totp_enable`), capture the seed to the vault
-  login via cookie jar.
+  (setfield → verify → full-recreate fallback), verify end-to-end.
+- `scripts/gitea-sso-login.sh <username> [vault-item]` — Gitea SSO first
+  login.
+- `scripts/redmine-sso-login.sh <username> [vault-item]` — Redmine SSO first
+  login.
+- `scripts/discourse-sso-login.sh <username> [vault-item]` — Discourse SSO
+  first login.
 - `scripts/totp.sh <base32-seed>` — current 6-digit TOTP code (pure bash,
   RFC-7636-vector tested).