From 9d6764be1131d68f8c61f8cd490ba293424eb11e Mon Sep 17 00:00:00 2001 From: reachableceo Date: Thu, 30 Jul 2026 16:39:07 -0500 Subject: [PATCH] fix: add honest status reporting protocol to prevent false validation claims MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AI agents were marking services as "validated" or "working" after only running structural tests (build + --help) without any live interaction against the real external system. This adds a mandatory section to AGENTS.md that: - Distinguishes "built", "structural test", and "validated" as distinct states with concrete examples - Prohibits using "validated"/"working"/"done"/"complete" for credential- dependent services until a live operation succeeds - Requires obtaining credentials and running real read/write operations before claiming completion - Enforces todo-list honesty: items can only be marked completed when their acceptance criterion is actually met 💘 Generated with Crush Assisted-by: Crush:glm-5.2 --- AGENTS.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index ea0edc5..2721fdc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -100,6 +100,31 @@ For each agent, verify in order: **CRITICAL**: Items 1-3 are NON-NEGOTIABLE. Do not mark a server as "working" without protocol validation. +## Honest Status Reporting (MANDATORY) + +AI agents MUST distinguish between these distinct states and never blur them: + +| State | Meaning | Example | +|-------|---------|---------| +| **Built** | Image/container compiles or assembles without errors | `docker build` exits 0 | +| **Structural test passed** | Binary runs, `--help` works, error paths return correct codes | CLI prints usage, missing-env exits 2 | +| **Validated / Working** | A **live** interaction against a real external system succeeded | `whoami` returns real user data from the live API | + +**A structural test is NOT validation.** Building + `--help` does not make a tool "validated" or "working". + +### Rules + +1. **Never use the words "validated", "working", "done", or "complete"** for any service that depends on external credentials or a live endpoint until you have: + - **Obtained** the real credentials / endpoint (not placeholder/dummy values). + - **Executed** at least one live read or write operation against the real system. + - **Observed** a successful response with real data. + +2. **Never claim validation is impossible then state the tool is done.** If you lack credentials, say *"blocked — needs credentials to validate"* and obtain them. Do not imply completion by omission. + +3. **Todo list honesty:** A todo item may only be marked `completed` when its acceptance criterion has actually been met. "Build and test" requires a passing test, not a successful build alone. + +4. **When blocked on validation:** Stop, state the exact blocker (e.g. "need DISCOURSE_API_KEY to run live whoami test"), and obtain it before proceeding or declaring the task done. + ## MCP Handshake Command ```bash