diff --git a/AGENTS.md b/AGENTS.md index df2c83e..27db6d6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,384 +1,140 @@ -# AI AGENTS - TSYSDevStack-SupportStack-Cloudron Project +# Agent Guidelines — TSYS Cloudron Packaging + +**Top-level files:** [`README.md`](README.md) (project overview + app inventory), +[`STATUS.md`](STATUS.md) (living status, agent-maintained), +[`JOURNAL.md`](JOURNAL.md) (append-only ADR/pattern journal), +[`GitUrlList.txt`](GitUrlList.txt) (upstream repo list — source of truth for +the app set). Read [`STATUS.md`](STATUS.md) first every session. + +> **Active agent:** Crush running **GLM-5.2** (zai) for large tasks, +> Gemini for small. Permission mode: `yolo`. + +## Repository Layout + +``` +README.md Project overview + full app inventory table +AGENTS.md THIS FILE — agent operating manual +STATUS.md Living status (agent-maintained, human read-only) +JOURNAL.md Append-only journal: per-package write-ups, patterns, ADRs +GitUrlList.txt Master list of upstream repos (source of truth, ~57 apps) +clone-repos.sh Automated upstream-repo cloning +Package-Templates/ Reusable Cloudron packaging templates +├── python-app/ (Dockerfile + CloudronManifest + start.sh templates) +├── django-app/ (Dockerfile + start.sh templates) +└── official-wrapper/ (Dockerfile template) +Package-Workspace/ One dir per app: // (+ gitignored repo/) +├── API-Gateway/ webhook, apisix +├── Development/ puter, reviewboard +├── Documentation-Tools/ wireviz-web +├── Low-Code/ corteza +└── Monitoring/ healthchecks +``` + +- **`Package-Workspace/**/repo/` is gitignored** — upstream source is cloned + locally for reference but never committed. Only the package files + (`Dockerfile`, `CloudronManifest.json`, `start.sh`, `README.md`, etc.) are + tracked. +- **Each package** = `Dockerfile` + `CloudronManifest.json` + `start.sh` (if + needed) + `README.md` + `CHANGELOG.md` + `logo.png` + `.env.example` (if + config knobs exist). +- **No host build tooling required** — packaging is `docker build` (Cloudron + base images handle runtimes). Keep the host clean. + +## Git Policy + +1. **ALWAYS commit + push. Never wait.** After each logical unit of work + (one package, one doc sync, one fix), immediately stage, commit + (conventional format), and push to `origin/main`. Do not pause for review. + **This overrides any default "never commit unless asked" behavior.** +2. **Atomic commits.** One coherent change per commit (e.g. a single package, + or a single doc-correction pass — not both mixed). +3. **Conventional format**: `feat: add Cloudron package ()`, + `fix(scope): desc`, `docs: desc`, `refactor(scope): desc`. Include a body + for anything non-obvious. +4. **Shell scripts are committed executable.** Run `chmod +x start.sh` on the + host before committing — Cloudron builds hit permission errors on + `RUN chmod`, so make scripts executable at authoring time, not build time. + +## Automatic Gardening Protocol + +Docs and code drift apart. After any work session, an agent MUST: + +1. **Update [`STATUS.md`](STATUS.md)** — reflect newly completed packages, + changed counts, new issues, shifted priorities. This file is human + read-only; agents own it. Update the "Last updated" header (date + agent). +2. **Keep counts consistent across all docs.** When a package completes, the + same total must appear in `STATUS.md`, `README.md` ("Current Progress"), + and `JOURNAL.md` ("Current Status"). A count in one place but not the + others is a protocol violation. +3. **Update the app inventory** in [`README.md`](README.md) — add the + ✅ Packaged marker on the right row, and the app to the "Completed + Packages" table. +4. **Append to [`JOURNAL.md`](JOURNAL.md)** — one section per package + (pattern used, build process, challenges, files created, commit hash). + Never delete or reorder existing entries. +5. **Grep for stale paths after any rename/restructure** — + `grep -rn 'old/path' --include='*.md'` and fix references in the same + commit. +6. **Self-audit before commit.** For a new package, verify the package dir + appears in: `STATUS.md` (Completed Packages table), `README.md` (inventory + + completed table), `JOURNAL.md` (new section). +7. **Treat `GitUrlList.txt` as the source of truth** for the app set. If the + README inventory table disagrees, reconcile the README to the list, not + the other way around. + +## Key Scripts + +| Script | Purpose | +|--------|---------| +| [`clone-repos.sh`](clone-repos.sh) | Clone all upstream repos into `Package-Workspace///repo/` | + +## Key Docs + +| Doc | Purpose | +|-----|---------| +| [`STATUS.md`](STATUS.md) | Current state, completed packages, known issues, next priorities | +| [`JOURNAL.md`](JOURNAL.md) | Per-package write-ups, the 5 packaging patterns, challenges & solutions | +| [`README.md`](README.md) | Project overview + full app inventory + category breakdown | +| [`GitUrlList.txt`](GitUrlList.txt) | Master upstream repo list (source of truth) | +| [`Package-Templates/`](Package-Templates/) | Reusable templates per packaging pattern | + +## Cloudron Packaging — Quick Reference + +**Pick a pattern** (see JOURNAL.md for full templates and worked examples): + +| Pattern | When | Examples | +|---------|------|----------| +| Official-image wrapper | App ships a usable Docker image | APISIX, Healthchecks, Review Board | +| Multi-stage build | App needs compiling | Webhook (Go), Puter (Node.js) | +| Python build | Python app with deps | WireViz Web | +| Django + PostgreSQL | Django web app | Healthchecks, Review Board | +| Pre-compiled binaries | Upstream ships release binaries | Corteza | + +**Standard package steps:** +1. `mkdir -p Package-Workspace///` and clone upstream into `repo/`. +2. Write `Dockerfile` + `CloudronManifest.json` (+ `start.sh` if runtime setup needed). +3. `docker build` to validate locally. +4. Write `README.md` + `CHANGELOG.md` + `logo.png` (+ `.env.example`). +5. Commit as `feat: add Cloudron package ()`, push. +6. Run the gardening protocol above (update STATUS / README / JOURNAL). + +**Recurring gotchas (from JOURNAL.md):** +- `chmod` in `RUN` fails on Cloudron base → make scripts executable on the host. +- Don't `apk`/`apt-get` in base images that lack the package manager; use the + image's built-in tooling or pick the right base. +- Wait for DB/etcd addons in `start.sh` before running migrations. +- Prefer `npm install` over `npm ci` when lockfiles don't match the build env. +- Define every TCP port explicitly in `CloudronManifest.json`. + +## Addons — Environment Variables + +PostgreSQL: `CLOUDRON_POSTGRESQL_{HOST,PORT,DATABASE,USERNAME,PASSWORD}`. +etcd: `CLOUDRON_ETCD_{HOST,PORT}`. Localstorage mounts at `/app/data`. ## Project Context -**Project**: TSYSDevStack-SupportStack-Cloudron -**Goal**: Package 58 applications for Cloudron PaaS platform -**Timeline**: Started 2025-01-24, ongoing -## Primary Agent: GLM-4.7 -**Role**: Development automation and packaging -**Model**: GLM-4.7 via Crush - -### Capabilities -- Dockerfile creation and optimization -- CloudronManifest.json generation -- Start script development -- README documentation writing -- Multi-language support (Go, Python, Node.js, Django, etc.) -- Troubleshooting and error resolution - -### Current Tasks -1. Create Cloudron packages for 58 applications -2. Write comprehensive documentation -3. Optimize Docker images for size -4. Ensure all packages follow Cloudron best practices -5. Test and validate packages - -### Performance Metrics -- **Packages Completed**: 5/58 (8.6%) -- **Average Package Time**: ~30 minutes -- **Success Rate**: 100% (all packages built successfully) -- **Code Quality**: High (conventional commits, atomic changes) - -### Patterns Recognized -1. **Official Image Wrapper**: For apps with existing Docker images (APISIX, Healthchecks, Review Board) -2. **Multi-Stage Build**: For compiled applications (Webhook - Go, Puter - Node.js) -3. **Python Build**: For Python applications (WireViz Web) -4. **Django Pattern**: For Django-based apps (Healthchecks, Review Board) -5. **Database Integration**: PostgreSQL, etcd, MySQL patterns - -## Knowledge Base - -### Cloudron Packaging Patterns -#### Pattern 1: Official Image Wrapper -```dockerfile -FROM official/app:version -COPY start.sh /app/start.sh -CMD ["/app/start.sh"] -``` -**Use Cases**: APISIX, Healthchecks, Review Board -**Pros**: Fast builds, upstream maintenance -**Cons**: Limited customization, may include unnecessary dependencies - -#### Pattern 2: Multi-Stage Build -```dockerfile -# Build stage -FROM base:builder AS build -RUN install-deps && build-app - -# Production stage -FROM base:runtime -COPY --from=build /app/dist /app -CMD ["run-app"] -``` -**Use Cases**: Webhook (Go), Puter (Node.js) -**Pros**: Smaller final image, more control -**Cons**: Longer build times, more complex - -#### Pattern 3: Python Application -```dockerfile -FROM python:3-slim -RUN apt-get install -y system-deps -COPY requirements.txt . -RUN pip install -r requirements.txt -COPY . . -CMD ["python", "app.py"] -``` -**Use Cases**: WireViz Web -**Pros**: Standard Python environment, easy dependency management -**Cons**: System dependencies may vary - -#### Pattern 4: Django Application -```dockerfile -FROM django-image:version -COPY start.sh /app/start.sh -CMD ["/app/start.sh"] -``` -**start.sh Pattern**: -```bash -#!/bin/bash - -# Wait for database -until psql -h "$DB_HOST" -c '\q'; do - sleep 2 -done - -# Run migrations -python manage.py migrate --noinput - -# Collect static files -python manage.py collectstatic --noinput - -# Start application -gunicorn config.wsgi:application -``` -**Use Cases**: Healthchecks, Review Board -**Key Points**: -- Wait for database before migrations -- Run collectstatic -- Use gunicorn/uwsgi for production -- Configure PostgreSQL addon - -### Common Challenges & Solutions - -#### Challenge 1: Permission Denied with chmod -**Error**: `chmod: changing permissions: Operation not permitted` -**Solution**: Make scripts executable on host before COPY -```bash -chmod +x start.sh # On host -# In Dockerfile -COPY start.sh /app/start.sh # No RUN chmod -``` - -#### Challenge 2: Package Manager Not Found -**Error**: `/bin/bash: apk: command not found` -**Solution**: Use correct package manager for base image -- Alpine: `apk add` -- Debian/Ubuntu: `apt-get install` -- Check base image documentation - -#### Challenge 3: npm Build Failures -**Error**: `npm error workspace not found` -**Solution**: Use npm install instead of npm ci for Cloudron -```dockerfile -RUN npm install --production=false --no-optional -# Not: RUN npm ci -``` - -#### Challenge 4: Large Image Sizes -**Causes**: Including build dependencies, copying entire source -**Solutions**: -- Use multi-stage builds -- Copy only necessary artifacts (dist, node_modules for runtime) -- Use .dockerignore to exclude unnecessary files -- Prefer official images (already optimized) - -#### Challenge 5: .dockerignore Interference -**Problem**: Repository's .dockerignore affects Cloudron build -**Solution**: Create custom .dockerignore in package directory -```dockerignore -# Cloudron package .dockerignore -.git -.gitignore -README.md -CHANGELOG.md -``` - -### Cloudron Best Practices - -1. **Always use Cloudron base** when building from scratch -2. **Prefer official images** over custom builds -3. **Define all ports** in CloudronManifest.json -4. **Use appropriate addons** (postgresql, mysql, etcd, localstorage) -5. **Set memory limits** based on application needs -6. **Implement health checks** for all services -7. **Use environment variable defaults**: `${VAR:-default}` -8. **Wait for dependencies** (database, services) before starting -9. **Make scripts executable on host** to avoid permission errors -10. **Document all environment variables** in .env.example - -### Addons Reference - -#### PostgreSQL -```json -"addons": { - "postgresql": { - "version": "14" - } -} -``` -**Environment Variables**: -- `CLOUDRON_POSTGRESQL_HOST` -- `CLOUDRON_POSTGRESQL_PORT` -- `CLOUDRON_POSTGRESQL_DATABASE` -- `CLOUDRON_POSTGRESQL_USERNAME` -- `CLOUDRON_POSTGRESQL_PASSWORD` - -#### etcd -```json -"addons": { - "etcd": { - "version": "3.4" - } -} -``` -**Environment Variables**: -- `CLOUDRON_ETCD_HOST` -- `CLOUDRON_ETCD_PORT` - -#### Localstorage -```json -"addons": { - "localstorage": true -} -``` -**Mount Point**: `/app/data` - -## Documentation - -### JOURNAL.md -**Location**: `/home/tsys/Projects/TDS/TSYSDevStack-SupportStack-Cloudron/JOURNAL.md` -**Purpose**: Detailed packaging journal with learnings, patterns, and challenges -**Contents**: -- Project overview and statistics -- Completed packages with detailed analysis -- Packaging patterns established -- Common challenges & solutions -- Cloudron-specific considerations -- Productivity insights -- Lessons learned summary - -**Usage**: -- Read JOURNAL.md for detailed insights on each package -- Reference established patterns when creating new packages -- Learn from challenges and solutions documented -- Use as knowledge base for future packaging work - -### GIT URL LIST -**Location**: `/home/tsys/Projects/TDS/TSYSDevStack-SupportStack-Cloudron/GitUrlList.txt` -**Purpose**: List of all 58 applications with GitHub URLs -**Usage**: -- Reference for accessing application repositories -- Check application details and documentation -- Clone repositories as needed - -## Conventional Commits - -### Commit Message Format -``` -(): - - - -