docs: add TODO.md with automated E2E testing roadmap

Capture plan for fully automated VM testing using test-specific ISO variant with hardcoded credentials and automated test runner. Includes safety rails to prevent test artifacts from leaking to production builds. 💵 Generated with Crush Assisted-by: GLM-4 via Crush <crush@charm.land>
2026-02-19 09:43:33 -05:00
parent 747d58e6ec
commit 5306942e45
1 changed files with 106 additions and 0 deletions
--- a/TODO.md
+++ b/TODO.md
@@ -0,0 +1,106 @@
+# KNEL-Football Secure OS - Backlog
+
+Items here are not actively being worked. Once work begins, move to STATUS.md Active Tasks.
+
+---
+
+## Automated End-to-End VM Testing
+
+### Goal
+Fully automated testing of the complete boot-to-desktop flow, including FDE passphrase entry, using a test-specific ISO variant.
+
+### Components
+
+#### 1. Test ISO Build Pipeline
+- [ ] Add `build_test` function to `run.sh`
+- [ ] Output file: `knel-football-secure-test.iso` (distinct name prevents confusion)
+- [ ] Build flag toggles test-only artifacts
+
+#### 2. Test Preseed Configuration
+- [ ] Create `config/preseed-test.cfg`
+- [ ] Hardcode encryption passphrase: `TestPassphrase2026!Secure`
+- [ ] Hardcode test user credentials
+- [ ] Add file checksum validation (detect if test preseed leaks to production build)
+
+#### 3. Automated Test Runner
+- [ ] Create `config/hooks/live/automated-test-runner.sh`
+- [ ] Runs on first boot (systemd service or rc.local)
+- [ ] Tests to include:
+  - [ ] FDE: Verify LUKS container is active (`cryptsetup status`)
+  - [ ] Services: auditd, aide, nftables running
+  - [ ] Configs: password policy, SSH client-only, WiFi/Bluetooth blocked
+  - [ ] Write results to `/var/log/knel-test-results.log`
+  - [ ] Optional: shutdown after tests complete
+
+#### 4. Safety Rails
+- [ ] Production build (`./run.sh iso`) explicitly excludes:
+  - `config/preseed-test.cfg`
+  - `config/hooks/live/automated-test-runner.sh`
+- [ ] Add build-time assertion: fail if test artifacts detected in production build
+- [ ] Add checksum comparison in build script
+
+#### 5. CI Integration
+- [ ] `./run.sh test:iso automated` - boots test ISO, waits, extracts results
+- [ ] Parse `/var/log/knel-test-results.log` via serial console or shared folder
+- [ ] Exit with appropriate code (0=pass, non-zero=fail)
+
+### Architecture
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    BUILD PIPELINE                           │
+├─────────────────────────────────────────────────────────────┤
+│                                                             │
+│  ./run.sh iso          ./run.sh build-test                  │
+│       │                      │                              │
+│       ▼                      ▼                              │
+│  Production ISO         Test ISO                            │
+│  (clean)                (includes test artifacts)           │
+│       │                      │                              │
+│       │                      ▼                              │
+│       │               Boot in VM                            │
+│       │                      │                              │
+│       │                      ▼                              │
+│       │               Preseed auto-answers                  │
+│       │               FDE passphrase                        │
+│       │                      │                              │
+│       │                      ▼                              │
+│       │               Test Suite Runs                       │
+│       │               (automated-test-runner.sh)            │
+│       │                      │                              │
+│       │                      ▼                              │
+│       │               Results logged to                     │
+│       │               /var/log/knel-test-results.log        │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Commands
+
+```bash
+# Build test ISO (includes test artifacts, hardcoded passphrase)
+./run.sh build-test
+
+# Run automated test (boot, validate, report)
+./run.sh test:iso automated
+
+# Build production ISO (safety-checked, no test artifacts)
+./run.sh iso
+```
+
+### Security Considerations
+
+- Test passphrase is publicly documented in this file (acceptable for test ISO)
+- Test ISO must NEVER be used in production
+- Consider adding MOTD warning on test ISO: "THIS IS A TEST BUILD - NOT FOR PRODUCTION"
+- Build script should grep for test artifacts and fail production build if found
+
+---
+
+## Other Backlog Items
+
+*(Add future items here)*
+
+---
+
+*Last updated: 2026-02-19*