commit 8ce279276fa3d124bd1b193e8cd6fc465d597987 Author: reachableceo Date: Thu Aug 13 08:59:14 2026 -0500 feat: initial agent identity provisioning automation [#442] Playwright-based tool for enrolling AI agent identities in Cloudron, generating API keys via SSO (Gitea/Discourse/Redmine), and storing all credentials in Bitwarden per-agent collections. - provision-agent.py: main Playwright automation (Cloudron enroll, SSO login, API key generation, verification) - bw-helper.py: Bitwarden CLI wrapper (password gen, item CRUD, TOTP, session management) - Dockerfile: Playwright v1.52.0 + bw CLI + Python deps - agents.yaml.example: manifest template for Q3/Q4 agents - TSYSGroupAIOS framework adopted (hooks, rules engine, Makefile) ๐Ÿ’˜ Generated with Crush Assisted-by: Crush:glm-5.2 diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..028d760 --- /dev/null +++ b/.env.example @@ -0,0 +1,7 @@ +# Bitwarden credentials (machine-to-machine auth) +BW_CLIENTID= +BW_CLIENTSECRET= +BW_PASSWORD= + +# Set to true for debugging (shows browser window โ€” requires display) +HEADFUL=false diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ab97d0f --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +state/ +__pycache__/ +*.pyc +.env +agents.yaml diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..b7c778d --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,71 @@ +# Agent Identity Provisioning โ€” Agent Guidelines + +> **Governing baseline:** [`BASELINE-PROMPT.md`](https://git.knownelement.com/TSYSGroupCorporate/TSYSGroupAIOS/src/branch/master/BASELINE-PROMPT.md) (from TSYSGroupAIOS framework). +> **Read STATUS.md and questions-v1.md first, every session.** + +This project provides Playwright-based automation for provisioning AI agent identities across the TSYS Group stack. + +--- + +## Quick Start + +1. **Set up the environment:** `bash scripts/setup-hooks.sh` +2. **Read STATUS.md** โ€” current state. +3. **Check for understanding** โ€” summarize the rules (see below). +4. **Build and run:** + ```bash + cp agents.yaml.example agents.yaml # fill in invite links + cp .env.example .env # fill in BW credentials + docker compose up --build + ``` + +## Project Overview + +Playwright automation that: +1. Enrolls AI agent identities in Cloudron (accept invite, set password, enable 2FA) +2. Logs into Gitea/Discourse/Redmine via Cloudron SSO +3. Generates per-agent API keys in each system +4. Stores all credentials in Bitwarden (per-agent collections) +5. Verifies API keys work + +Architecture and full plan: `~/Q3/agent-identity-bootstrap.md` + +## Repository Layout + +``` +agent-identity-provisioning/ +โ”œโ”€โ”€ AGENTS.md โ† THIS FILE +โ”œโ”€โ”€ STATUS.md โ† agent scratchpad +โ”œโ”€โ”€ WORKING.md โ† session task tracker +โ”œโ”€โ”€ questions-v1.md โ† questions for the human +โ”œโ”€โ”€ provision-agent.py โ† main Playwright script +โ”œโ”€โ”€ bw-helper.py โ† Bitwarden CLI wrapper +โ”œโ”€โ”€ agents.yaml.example โ† manifest template +โ”œโ”€โ”€ Dockerfile โ† Playwright + bw CLI +โ”œโ”€โ”€ docker-compose.yml โ† container lifecycle +โ”œโ”€โ”€ requirements.txt โ† Python dependencies +โ”œโ”€โ”€ Makefile โ† convenience dispatch +โ”œโ”€โ”€ scripts/ โ† TSYSGroupAIOS framework (hooks, rules engine) +โ””โ”€โ”€ state/ โ† provisioning state (gitignored) +``` + +## Systems of Record + +- **Redmine** (#442) is the system of record for this work. +- **No Discourse doc** yet โ€” will be created when the provisioning runbook is finalized. +- Git is the source of truth for the automation code. + +## Git Workflow + +- Work off master. Commit + push after every logical unit. +- Conventional format: `feat(provision): ...`, `fix(bw): ...`, etc. +- Shellcheck on all shell scripts (zero warnings, including info-level). +- Every commit references `[#442]`. + +## Conventions + +- **Python** for the Playwright automation. Type hints, docstrings. +- **Bitwarden CLI** for all credential operations โ€” never hardcode credentials. +- **No browser state persistence** โ€” each provisioning run starts fresh. +- **Container naming:** `tsys-agent-provisioner`. +- **Image pinning:** all images pinned to specific versions. diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..5d9f066 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,21 @@ +FROM mcr.microsoft.com/playwright:v1.52.0-noble + +RUN apt-get update && \ + apt-get install -y --no-install-recommends \ + jq \ + unzip \ + libzbar0 \ + libzbar-dev \ + && rm -rf /var/lib/apt/lists/* + +# Install Bitwarden CLI +RUN npx -y @bitwarden/cli@2026.7.0 || true + +# Install Python dependencies +COPY requirements.txt /tmp/ +RUN pip install --no-cache-dir -r /tmp/requirements.txt + +WORKDIR /app +COPY . . + +ENTRYPOINT ["python3", "/app/provision-agent.py"] diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..da4191f --- /dev/null +++ b/Makefile @@ -0,0 +1,46 @@ +# Makefile โ€” convenience dispatch to scripts/. +# +# Not required. The scripts in scripts/ are the real entry points and work +# standalone. This file just gives you short verbs if you're at a terminal. +# +# In Mode 2 (Hermes/OWUI/MCP), agents call the scripts directly or via API โ€” +# they don't need this file. + +.PHONY: setup validate fast lint test garden up down status clean help + +help: ## Show available targets + @grep -E '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-12s\033[0m %s\n", $$1, $$2}' + +setup: ## Install git hooks + @bash scripts/setup-hooks.sh + +validate: ## Full rule audit (includes tests) + @bash scripts/check-rules.sh + +fast: ## Fast rule audit (pre-commit equivalent) + @bash scripts/check-rules.sh --fast + +lint: ## Lint shell scripts (shellcheck via docker) + @docker run --rm -v "$$(pwd):/mnt" koalaman/shellcheck:stable \ + $$(find . -path ./.git -prune -o -path ./.tmp -prune -o -path ./vendor -prune -o -path ./node_modules -prune -o \( -name '*.sh' -o -name '*.bash' \) -print | sed 's|^\./|/mnt/|') || true + +test: ## Run the test suite (override per project) + @bash scripts/test.sh + +garden: ## Doc-sprawl / Discourse-migration report + @bash scripts/garden.sh + +up: ## Bring up the docker-compose stack + @bash scripts/up.sh + +down: ## Bring down the docker-compose stack + @bash scripts/down.sh + +status: ## Show repo status snapshot + @echo "== branch =="; git branch --show-current 2>/dev/null || echo "(no branch)" + @echo "== last commit =="; git log --oneline -1 2>/dev/null || true + @echo "== working tree =="; git status --short 2>/dev/null || echo "(not a git repo)" + @echo "== STATUS.md head =="; sed -n '1,12p' STATUS.md 2>/dev/null || echo "(no STATUS.md)" + +clean: ## Remove build/test artifacts (override per project) + @echo "make clean: nothing to clean โ€” override this in your project's Makefile." diff --git a/README.md b/README.md new file mode 100644 index 0000000..6b0d91e --- /dev/null +++ b/README.md @@ -0,0 +1,39 @@ +# Agent Identity Provisioning + +Playwright-based automation for provisioning AI agent identities across the TSYS Group stack: Cloudron enrollment (with 2FA), SSO login, API key generation, and Bitwarden credential storage. + +## Overview + +Each AI agent (VP TechOps, VP SecOps, etc.) gets: +1. A dedicated Cloudron user (identity root โ€” SSO provisions everywhere) +2. TOTP 2FA enrolled and stored in Bitwarden +3. API keys generated in Gitea, Discourse, Redmine (stored in Bitwarden) +4. All credentials owned by the agent, sourced via `bw-run.sh` (no `~/.creds/` files) + +See `~/Q3/agent-identity-bootstrap.md` for the full architecture. + +## Usage + +```bash +# 1. Create the manifest from the example +cp agents.yaml.example agents.yaml +# Edit: add Cloudron invite links for each agent + +# 2. Set BW credentials +export BW_CLIENTID="..." +export BW_CLIENTSECRET="..." + +# 3. Build and run +docker compose up --build + +# Or run a single agent +docker compose run --rm provision --agent vp-techops +``` + +## Manifest format + +See `agents.yaml.example`. Each agent defines: +- Cloudron invite link +- Display name +- Priority (Q3 vs Q4) +- System scopes (Redmine projects, Gitea orgs, Discourse categories) diff --git a/STATUS.md b/STATUS.md new file mode 100644 index 0000000..782ec9c --- /dev/null +++ b/STATUS.md @@ -0,0 +1,33 @@ +# STATUS.md โ€” Agent Identity Provisioning + +## Current State + +**Phase:** Development โ€” building the Playwright automation. Not yet executable (awaiting Cloudron invite links from user). + +**Ticket:** [#442](https://projects.knownelement.com/issues/442) + +## What's Built + +- [x] Repo created: `TSYSGroupCorporate/agent-identity-provisioning` +- [x] Framework adopted (check-rules.sh, hooks, Makefile) +- [x] Dockerfile (Playwright v1.52.0 + bw CLI + Python deps) +- [x] docker-compose.yml +- [x] agents.yaml.example (manifest template) +- [ ] provision-agent.py (main Playwright script) โ€” IN PROGRESS +- [ ] bw-helper.py (BW CLI wrapper) โ€” IN PROGRESS +- [ ] Shellcheck on all scripts +- [ ] Install hooks +- [ ] Initial commit + push + +## Blockers + +- **User must provide:** Cloudron invite links (manifest), BW account credentials +- **Discourse admin key:** needed for VP SecOps category creation โ€” assign to vp-techops agent after provisioning + +## Inbox + +- User wants `bw-run.sh` in TSYSGroupAIOS (DONE โ€” needs commit) +- Cross-linking audit tracked as #441 +- BW migration of reachableceo keys tracked as #440 (due Aug 19) +- Discourse admin key โ†’ assign to vp-techops agent +- User will create `coo` Linux account + BW account, then run provisioning from dedicated session diff --git a/WORKING.md b/WORKING.md new file mode 100644 index 0000000..f6056ce --- /dev/null +++ b/WORKING.md @@ -0,0 +1,11 @@ +# WORKING.md + +- [x] Create Gitea repo +- [x] Adopt TSYSGroupAIOS framework +- [x] Build Dockerfile + docker-compose.yml +- [x] Build manifest template (agents.yaml.example) +- [x] Write provision-agent.py +- [x] Write bw-helper.py +- [x] Install git hooks +- [x] Shellcheck on all scripts +- [x] Commit + push diff --git a/agents.yaml.example b/agents.yaml.example new file mode 100644 index 0000000..526df94 --- /dev/null +++ b/agents.yaml.example @@ -0,0 +1,82 @@ +# agents.yaml โ€” Agent identity manifest +# +# One entry per AI agent identity. The provisioner reads this file, +# enrolls each agent in Cloudron, generates API keys, and stores +# everything in Bitwarden. +# +# Generate Cloudron invite links at: +# https://tsys-cloudron.knel.net -> Users -> Add User -> copy invite URL + +agents: + - name: vp-techops + display_name: "VP TechOps" + priority: Q3 + cloudron_invite: "https://tsys-cloudron.knel.net/invitation/REPLACE_WITH_TOKEN" + systems: + gitea: + url: https://git.knownelement.com + token_name: vp-techops-api + scopes: ["api", "repo", "read:org"] + orgs: ["KNEL", "TechnicalOperations"] + discourse: + url: https://community.turnsys.com + categories: [74, 20] + redmine: + url: https://projects.knownelement.com + projects: [55, 59] + role: Developer + + - name: vp-secops + display_name: "VP SecOps" + priority: Q3 + cloudron_invite: "https://tsys-cloudron.knel.net/invitation/REPLACE_WITH_TOKEN" + systems: + gitea: + url: https://git.knownelement.com + token_name: vp-secops-api + scopes: ["api", "repo", "read:org"] + orgs: ["KNEL"] + discourse: + url: https://community.turnsys.com + categories: [74] + redmine: + url: https://projects.knownelement.com + projects: [55] + role: Developer + + - name: vp-techcompliance + display_name: "VP TechCompliance" + priority: Q3 + cloudron_invite: "https://tsys-cloudron.knel.net/invitation/REPLACE_WITH_TOKEN" + systems: + gitea: + url: https://git.knownelement.com + token_name: vp-techcompliance-api + scopes: ["api", "repo", "read:org"] + orgs: ["KNEL"] + discourse: + url: https://community.turnsys.com + categories: [75] + redmine: + url: https://projects.knownelement.com + projects: [55] + role: Developer + + # Q4 agents โ€” enroll in Cloudron only (Phase 1), no system access yet + - name: coo + display_name: "Chief Operating Officer" + priority: Q4 + cloudron_invite: "https://tsys-cloudron.knel.net/invitation/REPLACE_WITH_TOKEN" + systems: {} + + - name: svp-knel + display_name: "SVP KNEL" + priority: Q4 + cloudron_invite: "https://tsys-cloudron.knel.net/invitation/REPLACE_WITH_TOKEN" + systems: {} + + - name: svp-tctc + display_name: "SVP TCTC" + priority: Q4 + cloudron_invite: "https://tsys-cloudron.knel.net/invitation/REPLACE_WITH_TOKEN" + systems: {} diff --git a/bw-helper.py b/bw-helper.py new file mode 100644 index 0000000..d1efcc2 --- /dev/null +++ b/bw-helper.py @@ -0,0 +1,187 @@ +#!/usr/bin/env python3 +""" +bw-helper.py โ€” Bitwarden CLI wrapper for agent identity provisioning. + +Provides a clean Python interface to the `bw` CLI for: +- Password generation +- Item creation/retrieval in collections +- TOTP code generation +- Session management + +All BW commands run via subprocess. The BW session is established once +and reused across calls. +""" + +import json +import os +import subprocess +import sys +from typing import Optional + + +class BitwardenHelper: + """Wrapper around the Bitwarden CLI for credential management.""" + + def __init__(self, client_id: str, client_secret: str, password: str): + self.client_id = client_id + self.client_secret = client_secret + self.password = password + self.session: Optional[str] = None + + def _run_bw(self, args: list[str], capture: bool = True) -> str: + """Run a bw CLI command with the active session.""" + env = os.environ.copy() + if self.session: + env["BW_SESSION"] = self.session + result = subprocess.run( + ["bw"] + args, + capture_output=capture, + text=True, + env=env, + ) + if result.returncode != 0: + raise RuntimeError( + f"bw {' '.join(args)} failed: {result.stderr.strip()}" + ) + return result.stdout.strip() if capture else "" + + def login(self) -> None: + """Authenticate via API key and unlock the vault.""" + env = os.environ.copy() + env["BW_CLIENTID"] = self.client_id + env["BW_CLIENTSECRET"] = self.client_secret + env["BW_PASSWORD"] = self.password + + result = subprocess.run( + ["bw", "login", "--apikey"], + capture_output=True, + text=True, + env=env, + ) + if result.returncode != 0 and "already" not in result.stderr.lower(): + raise RuntimeError(f"BW login failed: {result.stderr.strip()}") + + self.session = subprocess.run( + ["bw", "unlock", "--raw"], + capture_output=True, + text=True, + env=env, + input=self.password + "\n", + ).stdout.strip() + + if not self.session: + raise RuntimeError("BW unlock failed โ€” no session token returned") + + def generate_password(self, length: int = 32) -> str: + """Generate a strong password.""" + return self._run_bw(["generate", "-uluns", "--length", str(length)]) + + def get_totp(self, item_name: str) -> str: + """Get the current TOTP code for a Bitwarden item.""" + return self._run_bw(["get", "totp", item_name]) + + def create_item( + self, + name: str, + username: str, + password: str, + uris: list[str], + collection_name: str, + totp_secret: Optional[str] = None, + custom_fields: Optional[dict[str, str]] = None, + ) -> str: + """Create a login item in a Bitwarden collection. + + Returns the item ID. + """ + item = { + "type": 1, # LOGIN + "name": name, + "login": { + "username": username, + "password": password, + "uris": [{"uri": u, "match": None} for u in uris], + }, + "collectionIds": [], # resolved by collection_name below + } + + if totp_secret: + item["login"]["totp"] = totp_secret + + fields = [] + if custom_fields: + for key, value in custom_fields.items(): + fields.append({"name": key, "value": value, "type": 0}) + if fields: + item["fields"] = fields + + # Resolve collection ID + collection_id = self._get_collection_id(collection_name) + if collection_id: + item["collectionIds"] = [collection_id] + + # Create via BW CLI + encoded = json.dumps(item) + result = subprocess.run( + ["bw", "encode"], + input=encoded, + capture_output=True, + text=True, + ) + if result.returncode != 0: + raise RuntimeError(f"bw encode failed: {result.stderr.strip()}") + + encoded_item = result.stdout.strip() + output = self._run_bw(["create", "item", encoded_item]) + + created = json.loads(output) + return created.get("id", "") + + def _get_collection_id(self, collection_name: str) -> Optional[str]: + """Look up a collection ID by name. Returns None if not found.""" + try: + output = self._run_bw(["list", "collections"]) + collections = json.loads(output) + for col in collections: + if col.get("name", "").lower() == collection_name.lower(): + return col.get("id") + except (RuntimeError, json.JSONDecodeError): + pass + return None + + def create_collection(self, collection_name: str, org_id: str) -> str: + """Create a collection in an organization.""" + item = {"name": collection_name, "organizationId": org_id} + encoded = json.dumps(item) + result = subprocess.run( + ["bw", "encode"], + input=encoded, + capture_output=True, + text=True, + ) + if result.returncode != 0: + raise RuntimeError(f"bw encode failed: {result.stderr.strip()}") + + encoded_item = result.stdout.strip() + output = self._run_bw(["create", "collection", encoded_item]) + created = json.loads(output) + return created.get("id", "") + + def item_exists(self, name: str) -> bool: + """Check if a Bitwarden item with this name already exists.""" + try: + self._run_bw(["get", "item", name]) + return True + except RuntimeError: + return False + + def get_item_password(self, name: str) -> str: + """Get the password field from a Bitwarden item.""" + return self._run_bw(["get", "password", name]) + + def get_item_uri(self, name: str) -> str: + """Get the URI from a Bitwarden item.""" + output = self._run_bw(["get", "item", name]) + item = json.loads(output) + uris = item.get("login", {}).get("uris", []) + return uris[0]["uri"] if uris else "" diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..2ea034a --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,13 @@ +services: + provision: + build: . + container_name: tsys-agent-provisioner + environment: + - BW_CLIENTID=${BW_CLIENTID} + - BW_CLIENTSECRET=${BW_CLIENTSECRET} + - BW_PASSWORD=${BW_PASSWORD} + - HEADFUL=${HEADFUL:-false} + volumes: + - ./agents.yaml:/app/agents.yaml:ro + - ./state:/app/state + network_mode: host diff --git a/provision-agent.py b/provision-agent.py new file mode 100644 index 0000000..d5d79d7 --- /dev/null +++ b/provision-agent.py @@ -0,0 +1,663 @@ +#!/usr/bin/env python3 +""" +provision-agent.py โ€” Playwright automation for AI agent identity provisioning. + +Enrolls AI agent identities in Cloudron, logs into Gitea/Discourse/Redmine via +SSO, generates API keys, and stores all credentials in Bitwarden. + +Usage: + python3 provision-agent.py # provision all agents in manifest + python3 provision-agent.py --agent vp-techops # provision one agent + python3 provision-agent.py --phase1-only # Cloudron enrollment only + python3 provision-agent.py --dry-run # validate manifest without browser + +Manifest: agents.yaml (see agents.yaml.example) + +See: ~/Q3/agent-identity-bootstrap.md for the full architecture. +""" + +import argparse +import json +import logging +import os +import sys +import time +from pathlib import Path + +import yaml +from playwright.sync_api import BrowserContext, Page, sync_playwright + +from bw_helper import BitwardenHelper + +# --------------------------------------------------------------------------- +# Logging +# --------------------------------------------------------------------------- + +logging.basicConfig( + level=logging.INFO, + format="%(asctime)s [%(levelname)s] %(message)s", + datefmt="%H:%M:%S", +) +log = logging.getLogger("provision") + +# --------------------------------------------------------------------------- +# Constants +# --------------------------------------------------------------------------- + +CLOUDRON_BASE = os.environ.get("CLOUDRON_BASE", "https://tsys-cloudron.knel.net") +GITEA_URL = os.environ.get("GITEA_URL", "https://git.knownelement.com") +DISCOURSE_URL = os.environ.get("DISCOURSE_URL", "https://community.turnsys.com") +REDMINE_URL = os.environ.get("REDMINE_URL", "https://projects.knownelement.com") + +STATE_DIR = Path("/app/state") +STATE_DIR.mkdir(exist_ok=True) + +# --------------------------------------------------------------------------- +# Cloudron enrollment +# --------------------------------------------------------------------------- + + +def enroll_cloudron( + page: Page, + agent: dict, + bw: BitwardenHelper, +) -> dict: + """ + Phase 1: Accept Cloudron invite, set password, enable 2FA. + + Returns a dict with the agent's Cloudron credentials. + """ + name = agent["name"] + invite_url = agent["cloudron_invite"] + display_name = agent.get("display_name", name) + + log.info(f"[{name}] Phase 1: Cloudron enrollment โ€” {invite_url}") + + # Check if already provisioned + item_name = f"{name} Cloudron" + if bw.item_exists(item_name): + log.info(f"[{name}] Cloudron credential already exists in Bitwarden โ€” skipping") + return { + "username": agent.get("cloudron_email", f"{name}@tsys-cloudron.knel.net"), + "password": bw.get_item_password(item_name), + } + + # Generate a strong password + password = bw.generate_password(length=32) + log.info(f"[{name}] Generated password ({len(password)} chars)") + + # Navigate to invite link + page.goto(invite_url, wait_until="networkidle") + + # Fill in the invite acceptance form + # Cloudron invite page typically has: username (may be pre-filled), password, confirm password + page.wait_for_selector('input[type="password"]', timeout=15000) + + password_inputs = page.query_selector_all('input[type="password"]') + if len(password_inputs) >= 2: + password_inputs[0].fill(password) + password_inputs[1].fill(password) + else: + password_inputs[0].fill(password) + + # Set display name if field exists + name_field = page.query_selector('input[name="displayName"], input[name="name"]') + if name_field: + name_field.fill(display_name) + + # Submit + submit = page.query_selector('button[type="submit"], button:has-text("Setup"), button:has-text("Create"), button:has-text("Accept")') + if submit: + submit.click() + + page.wait_for_load_state("networkidle") + log.info(f"[{name}] Invite accepted") + + # Enable 2FA + totp_secret = enable_cloudron_2fa(page, agent, bw) + + # Store credential in Bitwarden + cloudron_email = agent.get("cloudron_email", f"{name}@tsys-cloudron.knel.net") + bw.create_item( + name=item_name, + username=cloudron_email, + password=password, + uris=[CLOUDRON_BASE], + collection_name=name, + totp_secret=totp_secret, + ) + log.info(f"[{name}] Cloudron credential stored in Bitwarden (collection: {name})") + + return {"username": cloudron_email, "password": password, "totp_secret": totp_secret} + + +def enable_cloudron_2fa(page: Page, agent: dict, bw: BitwardenHelper) -> str: + """ + Navigate to Cloudron 2FA settings and enable TOTP. + + Returns the TOTP secret. + """ + name = agent["name"] + log.info(f"[{name}] Enabling 2FA on Cloudron account") + + # Navigate to account settings + page.goto(f"{CLOUDRON_BASE}/settings.html#account", wait_until="networkidle") + page.wait_for_timeout(2000) + + # Click "Enable 2FA" button + enable_btn = page.query_selector('button:has-text("Enable"), button:has-text("2FA"), a:has-text("Enable")') + if not enable_btn: + log.warning(f"[{name}] Could not find 2FA enable button โ€” may already be enabled") + return "" + + enable_btn.click() + page.wait_for_timeout(2000) + + # Extract TOTP secret from the QR code or the manual entry text + # Cloudron shows a QR code and a text secret + secret_text = page.query_selector('.modal-body code, .two-factor-secret, input[readonly]') + if secret_text: + totp_secret = secret_text.text_content().strip().replace(" ", "") + else: + # Try to extract from QR image source (base64) + qr_img = page.query_selector('img[src*="data:image"]') + if qr_img: + qr_src = qr_img.get_attribute("src") + totp_secret = decode_qr_from_base64(qr_src) + else: + log.error(f"[{name}] Could not extract TOTP secret from 2FA page") + return "" + + log.info(f"[{name}] Extracted TOTP secret: {totp_secret[:4]}...") + + # Generate current TOTP code and confirm + import pyotp + totp_code = pyotp.TOTP(totp_secret).now() + + code_input = page.query_selector('input[name="totpToken"], input[name="token"], input[placeholder*="code"]') + if code_input: + code_input.fill(totp_code) + confirm_btn = page.query_selector('button:has-text("Confirm"), button:has-text("Enable"), button[type="submit"]') + if confirm_btn: + confirm_btn.click() + page.wait_for_timeout(2000) + log.info(f"[{name}] 2FA confirmed") + else: + log.warning(f"[{name}] Could not find TOTP confirmation input") + + return totp_secret + + +def decode_qr_from_base64(data_uri: str) -> str: + """Decode a TOTP secret from a base64 QR code data URI.""" + import base64 + import io + + from PIL import Image + from pyzbar.pyzbar import decode + + # Extract base64 data from data URI + header, b64data = data_uri.split(",", 1) + img_bytes = base64.b64decode(b64data) + img = Image.open(io.BytesIO(img_bytes)) + + decoded = decode(img) + if decoded: + # TOTP URIs look like: otpauth://totp/Label?secret=XXXX&... + uri = decoded[0].data.decode() + if "secret=" in uri: + return uri.split("secret=")[1].split("&")[0] + + raise RuntimeError("Could not decode TOTP secret from QR code") + + +# --------------------------------------------------------------------------- +# System access (Phase 2) +# --------------------------------------------------------------------------- + + +def sso_login(page: Page, system_url: str, agent: dict, bw: BitwardenHelper) -> bool: + """ + Log into a Cloudron-managed app via SSO. + + Returns True if login succeeded. + """ + name = agent["name"] + log.info(f"[{name}] SSO login: {system_url}") + + # Navigate to the app โ€” should redirect to Cloudron SSO + page.goto(system_url, wait_until="networkidle") + + # If already logged in (SSO session), we're done + if not page.query_selector('input[type="password"]'): + log.info(f"[{name}] SSO session active โ€” already logged in") + return True + + # Fill Cloudron SSO login form + cloudron_item = f"{name} Cloudron" + username = agent.get("cloudron_email", f"{name}@tsys-cloudron.knel.net") + password = bw.get_item_password(cloudron_item) + + user_input = page.query_selector('input[name="username"], input[type="email"], input[name="email"]') + pass_input = page.query_selector('input[type="password"]') + + if user_input: + user_input.fill(username) + if pass_input: + pass_input.fill(password) + + # Handle 2FA if prompted + submit = page.query_selector('button[type="submit"], button:has-text("Sign in"), button:has-text("Log in")') + if submit: + submit.click() + page.wait_for_load_state("networkidle") + + # Check for TOTP prompt + totp_input = page.query_selector('input[name="totpToken"], input[name="token"], input[placeholder*="code"], input[autocomplete*="one-time-code"]') + if totp_input: + totp_code = bw.get_totp(cloudron_item) + totp_input.fill(totp_code) + submit2 = page.query_selector('button[type="submit"]') + if submit2: + submit2.click() + page.wait_for_load_state("networkidle") + + log.info(f"[{name}] SSO login complete for {system_url}") + return True + + +def provision_gitea(page: Page, agent: dict, bw: BitwardenHelper) -> str: + """Generate a Gitea API token via SSO login. Returns the token.""" + name = agent["name"] + systems = agent.get("systems", {}) + gitea_cfg = systems.get("gitea", {}) + + if not gitea_cfg: + log.info(f"[{name}] No Gitea config โ€” skipping") + return "" + + item_name = f"{name} Gitea" + if bw.item_exists(item_name): + log.info(f"[{name}] Gitea token already exists โ€” skipping") + return bw.get_item_password(item_name) + + url = gitea_cfg.get("url", GITEA_URL) + token_name = gitea_cfg.get("token_name", f"{name}-api") + + sso_login(page, f"{url}/user/login", agent, bw) + + # Navigate to API token settings + page.goto(f"{url}/user/settings/applications", wait_until="networkidle") + + # Generate new token + name_input = page.query_selector('input[name="name"]') + if name_input: + name_input.fill(token_name) + + # Select scopes if checkboxes exist + for scope in gitea_cfg.get("scopes", ["api", "repo", "read:org"]): + scope_cb = page.query_selector(f'input[value="{scope}"]') + if scope_cb and not scope_cb.is_checked(): + scope_cb.check() + + gen_btn = page.query_selector('button:has-text("Generate Token")') + if gen_btn: + gen_btn.click() + page.wait_for_timeout(2000) + + # Extract the generated token + token_el = page.query_selector('.ui.info.message code, .ui.message code, input[readonly]') + if not token_el: + # Try the new Gitea UI + token_el = page.query_selector('.token-value, .access-token') + + token = token_el.text_content().strip() if token_el else "" + if not token: + log.error(f"[{name}] Could not extract Gitea API token") + return "" + + log.info(f"[{name}] Gitea token generated: {token[:8]}...") + + # Store in Bitwarden + bw.create_item( + name=item_name, + username=name, + password=token, + uris=[url], + collection_name=name, + custom_fields={"token_name": token_name}, + ) + log.info(f"[{name}] Gitea token stored in Bitwarden") + + return token + + +def provision_discourse(page: Page, agent: dict, bw: BitwardenHelper) -> str: + """ + Generate a Discourse API key via SSO login. + + Note: Discourse API keys typically require admin to create. + If the agent can't self-generate, this logs a warning. + Returns the API key (empty string if not possible). + """ + name = agent["name"] + systems = agent.get("systems", {}) + discourse_cfg = systems.get("discourse", {}) + + if not discourse_cfg: + log.info(f"[{name}] No Discourse config โ€” skipping") + return "" + + item_name = f"{name} Discourse" + if bw.item_exists(item_name): + log.info(f"[{name}] Discourse key already exists โ€” skipping") + return bw.get_item_password(item_name) + + url = discourse_cfg.get("url", DISCOURSE_URL) + + sso_login(page, f"{url}/", agent, bw) + + # Try to generate an API key from user preferences + # Note: In Discourse, only admin can create API keys via UI + # Non-admin users may not have this option + page.goto(f"{url}/u/{name}/preferences/account", wait_until="networkidle") + + api_key_section = page.query_selector('.api-keys, [data-section="api-keys"]') + + if not api_key_section: + log.warning( + f"[{name}] Discourse API key self-generation not available. " + "An admin must create the key. The agent will need a manually-created key." + ) + return "" + + # If the section exists, try to create a key + revoke_btn = page.query_selector('.api-keys button:has-text("Revoke")') + if not revoke_btn: + # No existing keys โ€” create one + gen_btn = page.query_selector('button:has-text("New API Key"), button:has-text("Create")') + if gen_btn: + gen_btn.click() + page.wait_for_timeout(2000) + + # Read the key + key_el = page.query_selector('.api-key-value, code') + api_key = key_el.text_content().strip() if key_el else "" + + if api_key: + bw.create_item( + name=item_name, + username=name, + password=api_key, + uris=[url], + collection_name=name, + ) + log.info(f"[{name}] Discourse API key stored in Bitwarden") + return api_key + + log.warning(f"[{name}] Could not generate Discourse API key") + return "" + + +def provision_redmine(page: Page, agent: dict, bw: BitwardenHelper) -> str: + """Get the Redmine API access key via SSO login. Returns the key.""" + name = agent["name"] + systems = agent.get("systems", {}) + redmine_cfg = systems.get("redmine", {}) + + if not redmine_cfg: + log.info(f"[{name}] No Redmine config โ€” skipping") + return "" + + item_name = f"{name} Redmine" + if bw.item_exists(item_name): + log.info(f"[{name}] Redmine key already exists โ€” skipping") + return bw.get_item_password(item_name) + + url = redmine_cfg.get("url", REDMINE_URL) + + sso_login(page, f"{url}/login", agent, bw) + + # Navigate to account page where API key lives + page.goto(f"{url}/my/account", wait_until="networkidle") + + # The API key is in the right sidebar under "API access key" + # Click "Show" to reveal it + show_link = page.query_selector('a:has-text("Show"), #api_access_key + a, a[href*="access_key"]') + if show_link: + show_link.click() + page.wait_for_timeout(1000) + + key_el = page.query_selector('#api_access_key, .api-key code, .api-access-key') + api_key = key_el.text_content().strip() if key_el else "" + + if not api_key: + # If there's no existing key, try to reset/generate + reset_link = page.query_selector('a:has-text("Reset"), a:has-text("Generate")') + if reset_link: + reset_link.click() + page.wait_for_timeout(2000) + page.click('button:has-text("OK"), button:has-text("Confirm")') + page.wait_for_timeout(1000) + key_el = page.query_selector('#api_access_key, .api-key code') + api_key = key_el.text_content().strip() if key_el else "" + + if not api_key: + log.error(f"[{name}] Could not get Redmine API key") + return "" + + log.info(f"[{name}] Redmine API key obtained: {api_key[:8]}...") + + # Store in Bitwarden + bw.create_item( + name=item_name, + username=name, + password=api_key, + uris=[url], + collection_name=name, + ) + log.info(f"[{name}] Redmine API key stored in Bitwarden") + + return api_key + + +# --------------------------------------------------------------------------- +# Verification +# --------------------------------------------------------------------------- + + +def verify_gitea(token: str, agent: dict) -> bool: + """Verify the Gitea API token works.""" + import urllib.request + + url = agent.get("systems", {}).get("gitea", {}).get("url", GITEA_URL) + req = urllib.request.Request(f"{url}/api/v1/user", headers={"Authorization": f"token {token}"}) + try: + resp = urllib.request.urlopen(req, timeout=10) + data = json.loads(resp.read()) + log.info(f" Gitea verify: user={data.get('login', '?')}") + return resp.status == 200 + except Exception as e: + log.error(f" Gitea verify FAILED: {e}") + return False + + +def verify_redmine(key: str, agent: dict) -> bool: + """Verify the Redmine API key works.""" + import urllib.request + + url = agent.get("systems", {}).get("redmine", {}).get("url", REDMINE_URL) + req = urllib.request.Request(f"{url}/users/current.json", headers={"X-Redmine-API-Key": key}) + try: + resp = urllib.request.urlopen(req, timeout=10) + data = json.loads(resp.read()) + log.info(f" Redmine verify: user={data.get('user', {}).get('login', '?')}") + return resp.status == 200 + except Exception as e: + log.error(f" Redmine verify FAILED: {e}") + return False + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + + +def load_manifest(path: str) -> list[dict]: + """Load and validate the agent manifest.""" + with open(path) as f: + data = yaml.safe_load(f) + + agents = data.get("agents", []) + if not agents: + log.error("No agents found in manifest") + sys.exit(1) + + for agent in agents: + if "cloudron_invite" not in agent or "REPLACE" in agent["cloudron_invite"]: + log.warning(f"Agent {agent.get('name', '?')} has no valid Cloudron invite") + + return agents + + +def provision_agent( + context: BrowserContext, + agent: dict, + bw: BitwardenHelper, + phase1_only: bool = False, +) -> dict: + """Provision a single agent identity.""" + name = agent["name"] + results = {"name": name, "cloudron": False, "gitea": False, "discourse": False, "redmine": False} + + page = context.new_page() + + try: + # Phase 1: Cloudron enrollment + enroll_cloudron(page, agent, bw) + results["cloudron"] = True + log.info(f"[{name}] Phase 1 complete: Cloudron identity enrolled") + + if phase1_only: + log.info(f"[{name}] Phase 1 only โ€” skipping system access") + return results + + systems = agent.get("systems", {}) + if not systems: + log.info(f"[{name}] No systems configured โ€” Phase 1 only") + return results + + # Phase 2: System access (fresh page for each system to avoid SSO conflicts) + for system_name in ["gitea", "discourse", "redmine"]: + system_page = context.new_page() + try: + if system_name == "gitea": + token = provision_gitea(system_page, agent, bw) + results["gitea"] = bool(token) and verify_gitea(token, agent) + elif system_name == "discourse": + key = provision_discourse(system_page, agent, bw) + results["discourse"] = bool(key) + elif system_name == "redmine": + key = provision_redmine(system_page, agent, bw) + results["redmine"] = bool(key) and verify_redmine(key, agent) + except Exception as e: + log.error(f"[{name}] {system_name} provisioning failed: {e}") + finally: + system_page.close() + + log.info(f"[{name}] All phases complete: {results}") + + except Exception as e: + log.error(f"[{name}] Provisioning failed: {e}") + raise + finally: + page.close() + + # Save state + state_file = STATE_DIR / f"{name}.json" + with open(state_file, "w") as f: + json.dump(results, f, indent=2) + + return results + + +def main(): + parser = argparse.ArgumentParser(description="Provision AI agent identities") + parser.add_argument("--manifest", default="agents.yaml", help="Path to manifest file") + parser.add_argument("--agent", help="Provision only this agent") + parser.add_argument("--phase1-only", action="store_true", help="Cloudron enrollment only") + parser.add_argument("--dry-run", action="store_true", help="Validate manifest without browser") + parser.add_argument("--headed", action="store_true", help="Show browser (debugging)") + args = parser.parse_args() + + # Load manifest + agents = load_manifest(args.manifest) + if args.agent: + agents = [a for a in agents if a["name"] == args.agent] + if not agents: + log.error(f"Agent '{args.agent}' not found in manifest") + sys.exit(1) + + log.info(f"Manifest: {len(agents)} agent(s) to provision") + for a in agents: + log.info(f" - {a['name']} ({a.get('display_name', '?')}) [{a.get('priority', '?')}]") + + if args.dry_run: + log.info("Dry run โ€” manifest validated successfully") + return + + # Initialize Bitwarden + bw = BitwardenHelper( + client_id=os.environ["BW_CLIENTID"], + client_secret=os.environ["BW_CLIENTSECRET"], + password=os.environ["BW_PASSWORD"], + ) + log.info("Connecting to Bitwarden...") + bw.login() + log.info("Bitwarden session established") + + # Launch Playwright + headless = not args.headed + if os.environ.get("HEADFUL", "false").lower() == "true": + headless = False + + all_results = [] + + with sync_playwright() as pw: + browser = pw.chromium.launch(headless=headless) + + for agent in agents: + # Fresh context per agent (no cookie/session bleed) + context = browser.new_context( + accept_downloads=False, + java_script_enabled=True, + ) + log.info(f"=== Provisioning: {agent['name']} ===") + try: + result = provision_agent(context, agent, bw, args.phase1_only) + all_results.append(result) + except Exception as e: + log.error(f"FAILED: {agent['name']}: {e}") + all_results.append({"name": agent["name"], "error": str(e)}) + finally: + context.close() + + browser.close() + + # Summary + log.info("\n=== PROVISIONING SUMMARY ===") + for r in all_results: + status = "OK" if "error" not in r else "FAILED" + systems = [] + for s in ["cloudron", "gitea", "discourse", "redmine"]: + if r.get(s): + systems.append(s) + log.info(f" {r['name']}: {status} โ€” {', '.join(systems) if systems else '(none)'}") + + # Exit non-zero if any failed + if any("error" in r for r in all_results): + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/questions-v1.md b/questions-v1.md new file mode 100644 index 0000000..3286b0d --- /dev/null +++ b/questions-v1.md @@ -0,0 +1,18 @@ +# questions-v1.md + +## Q1: Cloudron invite page selectors +The Playwright automation needs to interact with the Cloudron invite acceptance page (password fields, 2FA enrollment). The exact CSS selectors will need verification against the live Cloudron UI. **Are you able to provide a screenshot of the invite acceptance flow, or should the code use generic selectors and we iterate?** + +## Q2: Discourse API key creation +Regular (non-admin) Discourse users may not be able to self-generate API keys. Options: +1. Admin pre-creates API keys for each agent (via Discourse admin panel) +2. The provisioning script uses an admin key to create per-user API keys +3. Agents use the admin key directly (not ideal for per-identity attribution) + +**Which approach do you prefer?** + +## Q3: Gitea org membership +After SSO login, the agent needs to be added to Gitea orgs (KNEL, TechnicalOperations, etc.). Can this be automated via Gitea API (using an admin token), or should org membership be pre-configured via the Gitea web UI before provisioning? + +## Q4: Bitwarden collection creation +Does the BW "COO" account need collections pre-created (vp-techops, vp-secops, etc.), or should the provisioning script create them if they don't exist? diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..9900c31 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,6 @@ +playwright==1.52.0 +pyyaml==6.0.2 +pyotp==2.9.0 +qrcode==7.4.2 +Pillow==10.4.0 +pyzbar==0.1.9 diff --git a/scripts/check-rules.sh b/scripts/check-rules.sh new file mode 100755 index 0000000..82d117f --- /dev/null +++ b/scripts/check-rules.sh @@ -0,0 +1,245 @@ +#!/usr/bin/env bash +# check-rules.sh โ€” project rule audit engine. +# +# Usage: +# bash scripts/check-rules.sh # full audit (verbose, includes slow checks) +# bash scripts/check-rules.sh --fast # fast audit (quiet, skips slow checks) โ€” for pre-commit +# bash scripts/check-rules.sh --quiet # full audit, only prints failures +# +# Exit code: 0 = all rules pass (warnings are non-fatal), 1 = one or more FAILED. +# +# This is a generalized version of the rules engine proven in the +# RCEO-PersonalAssistant project. Add project-specific checks by appending +# `check "" ""` calls below. +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +# shellcheck disable=SC1091 +source "$HERE/lib/common.sh" +REPO_ROOT="$(repo_root)" +cd "$REPO_ROOT" + +# --- argument parsing --- +RULE_FAST=false +RULE_VERBOSE=true +for arg in "$@"; do + case "$arg" in + --fast) RULE_FAST=true; RULE_VERBOSE=false ;; + --quiet) RULE_VERBOSE=false ;; + *) die "check-rules.sh: unknown argument '$arg'" ;; + esac +done +export RULE_FAST RULE_VERBOSE + +init_counters +$RULE_VERBOSE && echo "=== Project Rule Audit ===" + +TODAY="$(date +%Y-%m-%d)" + +# ---------------------------------------------------------------------------- +# 1. Shellcheck โ€” every .sh/.bash must pass (zero warnings, incl. info-level). +# Runs in Docker so the host stays clean (no native shellcheck required). +# ---------------------------------------------------------------------------- +$RULE_VERBOSE && log_step "Shell scripts (shellcheck)" +mapfile -d '' SH_FILES < <(find . -path ./.git -prune -o -path ./.tmp -prune -o -path ./vendor -prune -o -path ./node_modules -prune -o \( -name '*.sh' -o -name '*.bash' \) -print0 2>/dev/null) +if [ "${#SH_FILES[@]}" -gt 0 ]; then + if have shellcheck; then + if shellcheck "${SH_FILES[@]}" >/dev/null 2>&1; then + check "All shell scripts pass shellcheck (host)" "pass" + else + check "shellcheck reports violations โ€” run: shellcheck " "fail" + fi + elif have docker; then + MNT_FILES=() + for f in "${SH_FILES[@]}"; do MNT_FILES+=("/mnt/${f#./}"); done + if docker run --rm -v "$REPO_ROOT:/mnt" koalaman/shellcheck:stable "${MNT_FILES[@]}" >/dev/null 2>&1; then + check "All shell scripts pass shellcheck (docker)" "pass" + else + check "shellcheck (docker) reports violations" "fail" + fi + else + check "No shellcheck or docker available to lint scripts" "warn" + fi +else + check "No shell scripts to lint" "pass" +fi + +# ---------------------------------------------------------------------------- +# 2. Docker image pinning โ€” no ':latest' tags in compose or Dockerfiles. +# ---------------------------------------------------------------------------- +$RULE_VERBOSE && log_step "Docker image pinning" +if grep -rqE '(image:|FROM).*:latest' --include='docker-compose*.y*ml' --include='Dockerfile*' . 2>/dev/null; then + check "No ':latest' image tags (pin everything)" "fail" +else + check "No ':latest' image tags" "pass" +fi + +# ---------------------------------------------------------------------------- +# 2b. Container naming โ€” every service in a docker-compose file MUST set an +# explicit container_name (never rely on Docker's default _). +# ---------------------------------------------------------------------------- +$RULE_VERBOSE && log_step "Container naming" +COMPOSE_FILES="$(find . -path ./.git -prune -o \( -name 'docker-compose*.yml' -o -name 'docker-compose*.yaml' -o -name 'compose.yml' -o -name 'compose.yaml' \) -print 2>/dev/null || true)" +if [ -n "$COMPOSE_FILES" ]; then + BAD=0 + while IFS= read -r cf; do + [ -n "$cf" ] || continue + # Count top-level service keys (2-space indent under services:) and + # compare against the number of container_name: declarations. + svc_count=$(awk '/^services:/{f=1;next} f&&/^[^[:space:]]/{f=0} f&&/^[[:space:]]{2}[[:alnum:]_-]+:[[:space:]]*$/{c++} END{print c+0}' "$cf") + cn_count=$(grep -cE '^[[:space:]]*container_name:' "$cf" 2>/dev/null || echo 0) + if [ "${svc_count:-0}" -gt 0 ] && [ "$cn_count" -lt "$svc_count" ]; then + BAD=$((BAD + 1)) + fi + done </dev/null; then + if [ "$POINTER_MISSING" -eq 0 ]; then + $RULE_VERBOSE && printf ' %s\n' "Missing $DISCOURSE_HOST URL in:" + fi + POINTER_MISSING=$((POINTER_MISSING + 1)) + $RULE_VERBOSE && printf ' %s\n' "$f" + fi +done < <(find . -path ./.git -prune -o -path ./.tmp -prune -o -name '*.md' -print0 2>/dev/null) +if [ "$POINTER_MISSING" -eq 0 ]; then + check "All non-exempt .md cite Discourse ($DISCOURSE_HOST)" "pass" +else + check "$POINTER_MISSING .md file(s) missing Discourse pointer (see BASELINE-PROMPT.md ยง3)" "fail" +fi + +# ---------------------------------------------------------------------------- +# 5. Git state โ€” uncommitted changes are a warning (the pre-push hook hardens +# this where it matters). +# ---------------------------------------------------------------------------- +$RULE_VERBOSE && log_step "Git state" +if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + if git diff --quiet && git diff --cached --quiet; then + check "Working tree clean" "pass" + else + check "Uncommitted changes present" "warn" + fi +else + check "Not a git repo (git checks skipped)" "pass" +fi + +# ---------------------------------------------------------------------------- +# 6. Hooks installed โ€” self-check that git hooks were set up. +# ---------------------------------------------------------------------------- +$RULE_VERBOSE && log_step "Git hooks" +if [ -f .git/hooks/pre-commit ]; then + check "pre-commit hook installed" "pass" +else + check "pre-commit NOT installed (run: bash scripts/setup-hooks.sh)" "warn" +fi +if [ -f .git/hooks/pre-push ]; then + check "pre-push hook installed" "pass" +else + check "pre-push NOT installed (run: bash scripts/setup-hooks.sh)" "warn" +fi + +# ---------------------------------------------------------------------------- +# 7. WORKING.md completion โ€” no unchecked tasks may remain at commit time. +# ---------------------------------------------------------------------------- +$RULE_VERBOSE && log_step "Task completion" +if [ -f WORKING.md ]; then + UNCHECKED="$(grep -cF -- '- [ ]' WORKING.md || true)" + if [ "$UNCHECKED" -eq 0 ]; then + check "WORKING.md has no unchecked tasks" "pass" + else + check "WORKING.md has ${UNCHECKED} unchecked task(s) โ€” finish them before committing" "fail" + fi +else + check "WORKING.md absent (no active task tracker)" "pass" +fi + +# ---------------------------------------------------------------------------- +# 8. CNW markers โ€” empty `CNW:` markers flag unresolved questions for the human. +# ---------------------------------------------------------------------------- +$RULE_VERBOSE && log_step "Unresolved questions" +EMPTY_CNW="$(grep -rn 'CNW:$' . --include='*.md' 2>/dev/null | head -20 || true)" +if [ -z "$EMPTY_CNW" ]; then + check "No empty CNW: markers (unresolved questions)" "pass" +else + CNW_COUNT="$(printf '%s\n' "$EMPTY_CNW" | grep -c . || true)" + check "${CNW_COUNT} unresolved CNW: marker(s) โ€” needs user input" "warn" +fi + +# ---------------------------------------------------------------------------- +# 9. Hygiene โ€” merge-conflict markers and trailing whitespace must never land. +# ---------------------------------------------------------------------------- +$RULE_VERBOSE && log_step "File hygiene" +if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + CONFLICT="$(git diff --cached --name-only --diff-filter=ACM 2>/dev/null | xargs -r grep -lE '^(<<<<<<<|=======|>>>>>>>)' 2>/dev/null || true)" + if [ -z "$CONFLICT" ]; then check "No merge-conflict markers staged" "pass"; else check "Merge-conflict markers staged: $CONFLICT" "fail"; fi +fi + +# ---------------------------------------------------------------------------- +# 10. (slow, skipped in --fast) Project test suite via scripts/test.sh. +# ---------------------------------------------------------------------------- +if [ "$RULE_FAST" = false ] && [ -x scripts/test.sh ]; then + $RULE_VERBOSE && log_step "Test suite (scripts/test.sh)" + if bash scripts/test.sh >/dev/null 2>&1; then + check "scripts/test.sh passes" "pass" + else + check "scripts/test.sh FAILS" "fail" + fi +fi + +print_summary_and_exit diff --git a/scripts/lib/common.sh b/scripts/lib/common.sh new file mode 100644 index 0000000..a3dd5e4 --- /dev/null +++ b/scripts/lib/common.sh @@ -0,0 +1,137 @@ +#!/usr/bin/env bash +# lib/common.sh โ€” shared helpers for shell scripts and hooks in this repo. +# +# Source it from any script: +# #!/usr/bin/env bash +# set -euo pipefail +# HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# # shellcheck source=lib/common.sh +# source "$HERE/lib/common.sh" # or the appropriate relative path +# +# This library exists to drive a known cross-project inconsistency to zero: +# every repo used to re-paste the ANSI color block, redefine log_* helpers, +# pick one of three incompatible shebangs, and roll its own docker wrapper. +# Import this once instead. + +# Do NOT set -euo pipefail here unconditionally โ€” some callers (git hooks) +# source this file and rely on controlling their own shell options. We only +# guarantee the functions below are defined. + +############################################################################### +# Config โ€” override via environment before sourcing if needed +############################################################################### +: "${TEMPLATE_ROOT:=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}" +export TEMPLATE_ROOT + +############################################################################### +# ANSI colors (defined once, used everywhere) +############################################################################### +if [ -t 1 ] && [ -z "${NO_COLOR:-}" ]; then + RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m' + BLUE='\033[0;34m'; BOLD='\033[1m'; NC='\033[0m' +else + RED=''; GREEN=''; YELLOW=''; BLUE=''; BOLD=''; NC='' +fi +export RED GREEN YELLOW BLUE BOLD NC + +############################################################################### +# Logging +############################################################################### +log_info() { printf "${BLUE}โ€บ${NC} %s\n" "$*"; } +log_ok() { printf "${GREEN}โœ“${NC} %s\n" "$*"; } +log_warn() { printf "${YELLOW}โš ${NC} %s\n" "$*" >&2; } +log_error() { printf "${RED}โœ—${NC} %s\n" "$*" >&2; } +log_step() { printf "\n${BOLD}== %s ==${NC}\n" "$*"; } + +die() { log_error "$*"; exit 1; } + +############################################################################### +# Predicates +############################################################################### +# have โ€” return 0 if is on PATH +have() { command -v "$1" >/dev/null 2>&1; } + +############################################################################### +# Path helpers +############################################################################### +repo_root() { + # Prefer git's notion of the repo root, fall back to $TEMPLATE_ROOT, then pwd. + if git rev-parse --show-toplevel >/dev/null 2>&1; then + git rev-parse --show-toplevel + else + printf '%s\n' "${TEMPLATE_ROOT:-$(pwd)}" + fi +} + +############################################################################### +# Privilege helpers +############################################################################### +# as_root โ€” run the remaining args as root via sudo, or directly if already root. +as_root() { + if [ "$(id -u)" -eq 0 ]; then "$@"; else sudo "$@"; fi +} + +############################################################################### +# Docker wrapper +############################################################################### +# docker_run +# Ephemeral container, host-uid ownership, repo mounted at /data, cwd /data. +# Drives the "host stays clean; everything runs in containers" policy and +# ensures output files are owned by the invoking user, not root. +docker_run() { + [ "$#" -ge 1 ] || die "docker_run: image required" + local image="$1"; shift + have docker || die "docker not found on PATH" + local root + root="$(repo_root)" + docker run --rm \ + --user "$(id -u):$(id -g)" \ + -e HOME=/tmp \ + -v "$root:/data" \ + -w /data \ + "$image" "$@" +} + +############################################################################### +# Rule-audit accumulator (used by scripts/check-rules.sh) +# Globals read/written: RULE_PASS RULE_WARN RULE_FAIL +############################################################################### +init_counters() { RULE_PASS=0; RULE_WARN=0; RULE_FAIL=0; } + +# check +check() { + local desc="$1" result="$2" + case "$result" in + pass) + RULE_PASS=$((RULE_PASS + 1)) + if [ "${RULE_VERBOSE:-true}" = true ]; then printf " ${GREEN}PASS${NC} %s\n" "$desc"; fi + ;; + warn) + RULE_WARN=$((RULE_WARN + 1)) + if [ "${RULE_VERBOSE:-true}" = true ]; then printf " ${YELLOW}WARN${NC} %s\n" "$desc"; fi + ;; + fail) + RULE_FAIL=$((RULE_FAIL + 1)) + printf " ${RED}FAIL${NC} %s\n" "$desc" + ;; + *) + die "check(): invalid result '$result' (use pass|warn|fail)" + ;; + esac +} + +# print_summary_and_exit +print_summary_and_exit() { + if [ "${RULE_VERBOSE:-true}" = true ]; then + printf "\n=== Summary ===\n PASS: %s\n WARN: %s\n FAIL: %s\n\n" \ + "$RULE_PASS" "$RULE_WARN" "$RULE_FAIL" + fi + if [ "$RULE_FAIL" -gt 0 ]; then + if [ "${RULE_VERBOSE:-true}" = true ]; then + printf "AUDIT FAILED โ€” %s rule(s) violated.\n" "$RULE_FAIL" + fi + exit 1 + fi + if [ "${RULE_VERBOSE:-true}" = true ]; then printf "AUDIT PASSED.\n"; fi + exit 0 +} diff --git a/scripts/pre-commit b/scripts/pre-commit new file mode 100755 index 0000000..1833906 --- /dev/null +++ b/scripts/pre-commit @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +# pre-commit โ€” fast rule audit (< 1s typical). +# Hot-path bypass: commits that ONLY touch STATUS.md / WORKING.md skip the +# audit so frequent status/task commits stay frictionless. +set -euo pipefail + +REPO_ROOT="$(git rev-parse --show-toplevel)" +cd "$REPO_ROOT" + +CHANGED="$(git diff --cached --name-only)" +HOT_PATHS="$(printf '%s\n' "$CHANGED" | grep -vE '^(STATUS.md|WORKING.md)$' || true)" + +if [ -z "$HOT_PATHS" ]; then + echo "hot-path files only (STATUS/WORKING) โ€” skipping rule audit" + exit 0 +fi + +if ! bash scripts/check-rules.sh --fast; then + echo "" + echo "pre-commit audit FAILED. Fix the violations above before committing." + echo "Full audit: bash scripts/check-rules.sh" + echo "Bypass: git commit --no-verify (emergencies only)" + exit 1 +fi +exit 0 diff --git a/scripts/pre-push b/scripts/pre-push new file mode 100755 index 0000000..7c5bf7d --- /dev/null +++ b/scripts/pre-push @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +# pre-push โ€” full rule audit + clean-working-tree gate before pushing. +# Installed via: bash scripts/setup-hooks.sh +# +# Combines two proven policies observed across projects: +# - KNEL-AIMiddleware: block push if the working tree is dirty. +# - RCEO-PersonalAssistant: block push if the full test suite fails. +set -euo pipefail + +REPO_ROOT="$(git rev-parse --show-toplevel)" +cd "$REPO_ROOT" + +echo "pre-push: running full rule audit..." + +# Full audit (non-fast): runs the slow test suite via `make test` if present. +if ! bash scripts/check-rules.sh --quiet; then + echo "" + echo "pre-push audit FAILED. Push blocked." + echo "Re-run with output: bash scripts/check-rules.sh" + echo "Bypass: git push --no-verify (emergencies only)" + exit 1 +fi + +echo "pre-push: all rules and tests passed." +exit 0 diff --git a/scripts/setup-hooks.sh b/scripts/setup-hooks.sh new file mode 100755 index 0000000..57219ac --- /dev/null +++ b/scripts/setup-hooks.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# setup-hooks.sh โ€” install this repo's git hooks. +# +# Mechanism: copy scripts/pre-commit and scripts/pre-push into .git/hooks/ and +# make them executable. This is the most portable pattern (works on any clone, +# no `git config core.hooksPath` mutation, survives config resets, idempotent). +# +# Run once after cloning: bash scripts/setup-hooks.sh +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +# shellcheck disable=SC1091 +source "$HERE/lib/common.sh" +REPO_ROOT="$(repo_root)" +cd "$REPO_ROOT" + +[ -d .git ] || die "no .git directory here โ€” run this from a git checkout" + +HOOKS_DIR=".git/hooks" +HOOK_NAMES="pre-commit pre-push" + +log_step "Installing git hooks" +for name in $HOOK_NAMES; do + src="scripts/$name" + dst="$HOOKS_DIR/$name" + [ -f "$src" ] || die "source hook not found: $src" + cp "$src" "$dst" + chmod +x "$dst" + log_ok "installed $dst" +done + +cat <