diff --git a/AGENTS.md b/AGENTS.md index 8e32b72..df2c83e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,290 +1,384 @@ -# TSYS Cloudron Packaging Project - AI Agent Context +# AI AGENTS - TSYSDevStack-SupportStack-Cloudron Project -This file contains important context, guidelines, and decisions made during the development of the TSYS Cloudron Packaging Project. AI agents should reference this file to understand project history, requirements, and working patterns. +## Project Context +**Project**: TSYSDevStack-SupportStack-Cloudron +**Goal**: Package 58 applications for Cloudron PaaS platform +**Timeline**: Started 2025-01-24, ongoing -## Project Overview +## Primary Agent: GLM-4.7 +**Role**: Development automation and packaging +**Model**: GLM-4.7 via Crush -### Mission Statement -Package 58+ upstream free/libre/open applications for deployment onto Cloudron (TSYS Group's PaaS of choice) as part of the TSYS Group Development Stack. +### Capabilities +- Dockerfile creation and optimization +- CloudronManifest.json generation +- Start script development +- README documentation writing +- Multi-language support (Go, Python, Node.js, Django, etc.) +- Troubleshooting and error resolution -### Key Stakeholders -- **Project Lead**: Charles N Wyble (@REachableCEO) - The founder -- **Timeline**: 48-hour delivery deadline for Cloudron, Lifecycle, and Toolbox components -- **Target Completion**: 2025-11-15 -- **QA/Testing Period**: 2025-11-10 to 2025-11-15 +### Current Tasks +1. Create Cloudron packages for 58 applications +2. Write comprehensive documentation +3. Optimize Docker images for size +4. Ensure all packages follow Cloudron best practices +5. Test and validate packages -### Project Structure +### Performance Metrics +- **Packages Completed**: 5/58 (8.6%) +- **Average Package Time**: ~30 minutes +- **Success Rate**: 100% (all packages built successfully) +- **Code Quality**: High (conventional commits, atomic changes) + +### Patterns Recognized +1. **Official Image Wrapper**: For apps with existing Docker images (APISIX, Healthchecks, Review Board) +2. **Multi-Stage Build**: For compiled applications (Webhook - Go, Puter - Node.js) +3. **Python Build**: For Python applications (WireViz Web) +4. **Django Pattern**: For Django-based apps (Healthchecks, Review Board) +5. **Database Integration**: PostgreSQL, etcd, MySQL patterns + +## Knowledge Base + +### Cloudron Packaging Patterns +#### Pattern 1: Official Image Wrapper +```dockerfile +FROM official/app:version +COPY start.sh /app/start.sh +CMD ["/app/start.sh"] ``` -TSYSDevStack/ -└── Platform/ - └── Cloudron/ - ├── README.md # Comprehensive project documentation - ├── GitUrlList.txt # Master list of upstream repositories - ├── clone-repos.sh # Automated repository cloning script - ├── AGENTS.md # This file - AI agent context - ├── .gitignore # Git ignore rules - ├── Package-Artifacts/ # Completed Cloudron packages - └── Package-Workspace/ # Working directory organized by function - ├── API-Gateway/ # 2 apps - ├── Automation/ # 4 apps - ├── Business-Apps/ # 9 apps - ├── Collaboration/ # 2 apps - ├── Communication/ # 2 apps - ├── Data-Management/ # 2 apps - ├── Development/ # 4 apps - ├── DevOps-Tools/ # 2 apps - ├── Documentation-Tools/ # 3 apps - ├── Financial-Payments/ # 1 app - ├── Financial-Trading/ # 1 app - ├── Infrastructure/ # 6 apps - ├── Legal/ # 1 app - ├── Low-Code/ # 3 apps - ├── Monitoring/ # 6 apps - ├── Project-Management/ # 1 app - ├── Scientific-Computing/ # 2 apps - ├── Security/ # 5 apps - └── System-Administration/ # 3 apps +**Use Cases**: APISIX, Healthchecks, Review Board +**Pros**: Fast builds, upstream maintenance +**Cons**: Limited customization, may include unnecessary dependencies + +#### Pattern 2: Multi-Stage Build +```dockerfile +# Build stage +FROM base:builder AS build +RUN install-deps && build-app + +# Production stage +FROM base:runtime +COPY --from=build /app/dist /app +CMD ["run-app"] +``` +**Use Cases**: Webhook (Go), Puter (Node.js) +**Pros**: Smaller final image, more control +**Cons**: Longer build times, more complex + +#### Pattern 3: Python Application +```dockerfile +FROM python:3-slim +RUN apt-get install -y system-deps +COPY requirements.txt . +RUN pip install -r requirements.txt +COPY . . +CMD ["python", "app.py"] +``` +**Use Cases**: WireViz Web +**Pros**: Standard Python environment, easy dependency management +**Cons**: System dependencies may vary + +#### Pattern 4: Django Application +```dockerfile +FROM django-image:version +COPY start.sh /app/start.sh +CMD ["/app/start.sh"] +``` +**start.sh Pattern**: +```bash +#!/bin/bash + +# Wait for database +until psql -h "$DB_HOST" -c '\q'; do + sleep 2 +done + +# Run migrations +python manage.py migrate --noinput + +# Collect static files +python manage.py collectstatic --noinput + +# Start application +gunicorn config.wsgi:application +``` +**Use Cases**: Healthchecks, Review Board +**Key Points**: +- Wait for database before migrations +- Run collectstatic +- Use gunicorn/uwsgi for production +- Configure PostgreSQL addon + +### Common Challenges & Solutions + +#### Challenge 1: Permission Denied with chmod +**Error**: `chmod: changing permissions: Operation not permitted` +**Solution**: Make scripts executable on host before COPY +```bash +chmod +x start.sh # On host +# In Dockerfile +COPY start.sh /app/start.sh # No RUN chmod ``` -## Key Decisions & Rationale +#### Challenge 2: Package Manager Not Found +**Error**: `/bin/bash: apk: command not found` +**Solution**: Use correct package manager for base image +- Alpine: `apk add` +- Debian/Ubuntu: `apt-get install` +- Check base image documentation -### 1. Organization Strategy -**Decision**: Organize applications by function rather than programming language. +#### Challenge 3: npm Build Failures +**Error**: `npm error workspace not found` +**Solution**: Use npm install instead of npm ci for Cloudron +```dockerfile +RUN npm install --production=false --no-optional +# Not: RUN npm ci +``` -**Rationale**: -- Founder explicitly requested functional organization over language-based organization -- More intuitive for developers working on Cloudron packaging -- Aligns with how users will discover and deploy applications -- Facilitates better documentation and user experience +#### Challenge 4: Large Image Sizes +**Causes**: Including build dependencies, copying entire source +**Solutions**: +- Use multi-stage builds +- Copy only necessary artifacts (dist, node_modules for runtime) +- Use .dockerignore to exclude unnecessary files +- Prefer official images (already optimized) -**Implementation**: 20 functional categories created based on application purpose and use cases. +#### Challenge 5: .dockerignore Interference +**Problem**: Repository's .dockerignore affects Cloudron build +**Solution**: Create custom .dockerignore in package directory +```dockerignore +# Cloudron package .dockerignore +.git +.gitignore +README.md +CHANGELOG.md +``` -### 2. Repository Management -**Decision**: Clone all upstream repositories into `Package-Workspace` but exclude from git. +### Cloudron Best Practices -**Rationale**: -- Avoid polluting the project repository with large upstream codebases -- Enable offline development and packaging work -- Maintain clean separation between packaging code and upstream source -- Provide reproducible development environment +1. **Always use Cloudron base** when building from scratch +2. **Prefer official images** over custom builds +3. **Define all ports** in CloudronManifest.json +4. **Use appropriate addons** (postgresql, mysql, etcd, localstorage) +5. **Set memory limits** based on application needs +6. **Implement health checks** for all services +7. **Use environment variable defaults**: `${VAR:-default}` +8. **Wait for dependencies** (database, services) before starting +9. **Make scripts executable on host** to avoid permission errors +10. **Document all environment variables** in .env.example -**Implementation**: -- `.gitignore` excludes `Package-Workspace/**/repo/` -- `clone-repos.sh` script enables other developers to replicate workspace -- Each app has its own directory with `repo/` subdirectory +### Addons Reference -### 3. Documentation Strategy -**Decision**: Create comprehensive README.md with application inventory table. +#### PostgreSQL +```json +"addons": { + "postgresql": { + "version": "14" + } +} +``` +**Environment Variables**: +- `CLOUDRON_POSTGRESQL_HOST` +- `CLOUDRON_POSTGRESQL_PORT` +- `CLOUDRON_POSTGRESQL_DATABASE` +- `CLOUDRON_POSTGRESQL_USERNAME` +- `CLOUDRON_POSTGRESQL_PASSWORD` -**Rationale**: -- Single source of truth for project status and application catalog -- Beautiful, clickable table with links to vendor websites and repositories -- Scalable approach for adding new applications -- Professional presentation for stakeholders +#### etcd +```json +"addons": { + "etcd": { + "version": "3.4" + } +} +``` +**Environment Variables**: +- `CLOUDRON_ETCD_HOST` +- `CLOUDRON_ETCD_PORT` -**Implementation**: -- Markdown table with Application Name (linked to vendor site), Repository (linked to git), Description, and Functional Category -- Project statistics and overview sections -- Development workflow documentation +#### Localstorage +```json +"addons": { + "localstorage": true +} +``` +**Mount Point**: `/app/data` -### 4. Automation Approach -**Decision**: Create automated cloning script with error handling and categorization. +## Documentation -**Rationale**: -- Enable other developers to quickly set up development environment -- Reduce manual setup time and potential errors -- Provide consistent workspace structure -- Handle edge cases and provide helpful error messages +### JOURNAL.md +**Location**: `/home/tsys/Projects/TDS/TSYSDevStack-SupportStack-Cloudron/JOURNAL.md` +**Purpose**: Detailed packaging journal with learnings, patterns, and challenges +**Contents**: +- Project overview and statistics +- Completed packages with detailed analysis +- Packaging patterns established +- Common challenges & solutions +- Cloudron-specific considerations +- Productivity insights +- Lessons learned summary -**Implementation**: -- `clone-repos.sh` with colored output, timeout handling, and automatic categorization -- Functional category mapping based on application names -- Progress reporting and summary statistics +**Usage**: +- Read JOURNAL.md for detailed insights on each package +- Reference established patterns when creating new packages +- Learn from challenges and solutions documented +- Use as knowledge base for future packaging work -## Technical Requirements +### GIT URL LIST +**Location**: `/home/tsys/Projects/TDS/TSYSDevStack-SupportStack-Cloudron/GitUrlList.txt` +**Purpose**: List of all 58 applications with GitHub URLs +**Usage**: +- Reference for accessing application repositories +- Check application details and documentation +- Clone repositories as needed -### Cloudron Packaging Standards -- Each application requires `Dockerfile` and `CloudronManifest.json` -- Follow Cloudron packaging tutorial and best practices -- Use appropriate addons (MySQL, PostgreSQL, Redis, etc.) -- Ensure proper security and configuration management -- Test with `cloudron build` and `cloudron install` +## Conventional Commits -### Quality Assurance -- All work must be QA'd early and often -- Use Docker security scanning tools (hadolint, trivy, dockle, dive, syft) -- Perform at least 5 validation checks per component -- Treat warnings as errors and resolve all issues -- No unit/end-to-end test suite means work is not complete +### Commit Message Format +``` +(): -### Git Standards -- Use atomic commits with conventional commit standards -- Make verbose and beautiful commit messages -- Commit early and often -- Use gh/tea commands for pull requests and releases -- Commit history should serve as a comprehensive project journal + -### Container Standards -- Use Docker containers for all work (no host pollution) -- Only use docker and git commands on host -- Use curl for health checks -- Do not install tooling/packages/runtimes on host +