From ac7e644ba3a2edee87adb16411ed54c9d6910126 Mon Sep 17 00:00:00 2001 From: ReachableCEO Date: Sat, 24 Jan 2026 10:47:15 -0500 Subject: [PATCH] docs: add root-level AGENTS.md for agent guidance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Create comprehensive agent guide at repository root documenting: - Project overview and architecture - Essential commands (deployment, testing, validation) - Code organization and service categories - Naming conventions and style patterns - Testing approach and quality standards - Critical gotchas (process management, demo security) - Development workflow and best practices This provides quick reference for AI agents working in this repo, complementing the detailed development guidelines in demo/AGENTS.md. 💘 Generated with Crush Assisted-by: GLM-4.7 via Crush --- AGENTS.md | 389 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 389 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..4aa180b --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,389 @@ +# TSYS Developer Support Stack - Agent Guide + +## Project Overview + +This repository contains a Docker Compose-based multi-service stack that provides off-the-shelf applications running locally on developer workstations. It's designed to enhance developer productivity and quality of life for the TSYS engineering team. + +### Project Type +- **Infrastructure as Code**: Docker Compose with shell orchestration +- **Multi-Service Stack**: 13 services across 4 categories +- **Demo-First Architecture**: All configurations for demonstration purposes only + +### Directory Structure +``` +TSYSDevStack-SupportStack-LocalWorkstation/ +├── LICENSE +├── README.md # Root-level documentation +├── AGENTS.md # This file - agent guide +├── demo/ # Demo environment (primary focus) +│ ├── AGENTS.md # Detailed development guidelines +│ ├── PRD.md # Product Requirements Document +│ ├── README.md # Demo-specific documentation +│ ├── demo.env # Environment variables (dynamic) +│ ├── docker-compose.yml # Generated compose file +│ ├── docker-compose.yml.template # Template for compose file +│ ├── scripts/ # Orchestration scripts +│ │ ├── demo-stack.sh # Deployment/management +│ │ ├── demo-test.sh # QA/testing suite +│ │ └── validate-all.sh # Pre-deployment validation +│ ├── tests/ # Test suite +│ │ ├── unit/ # Unit tests +│ │ ├── integration/ # Integration tests +│ │ └── e2e/ # End-to-end tests +│ ├── config/ # Service configurations +│ │ ├── homepage/ # Homepage dashboard config +│ │ ├── grafana/ # Grafana dashboards/datasources +│ │ ├── pihole/ # Pi-hole configuration +│ │ ├── portainer/ # Portainer configuration +│ │ ├── influxdb/ # InfluxDB configuration +│ │ ├── drawio/ # Draw.io configuration +│ │ ├── kroki/ # Kroki configuration +│ │ ├── atomictracker/ # Atomic Tracker configuration +│ │ ├── archivebox/ # ArchiveBox configuration +│ │ ├── tubearchivist/ # Tube Archivist configuration +│ │ ├── wakapi/ # Wakapi configuration +│ │ ├── mailhog/ # MailHog configuration +│ │ └── atuin/ # Atuin configuration +│ └── docs/ # Additional documentation +│ ├── service-guides/ # Service-specific guides +│ ├── troubleshooting/ # Troubleshooting documentation +│ └── api-docs/ # API documentation +└── prod/ # Production environment (placeholder) +``` + +## Essential Commands + +### Deployment & Management +```bash +# Deploy the demo stack +./demo/scripts/demo-stack.sh deploy + +# Stop all services +./demo/scripts/demo-stack.sh stop + +# Restart all services +./demo/scripts/demo-stack.sh restart + +# Show service status +./demo/scripts/demo-stack.sh status +``` + +### Testing & Validation +```bash +# Run comprehensive validation (before deployment) +./demo/scripts/validate-all.sh + +# Run full test suite +./demo/scripts/demo-test.sh full + +# Run security compliance tests only +./demo/scripts/demo-test.sh security + +# Run permission validation tests only +./demo/scripts/demo-test.sh permissions + +# Run network isolation tests only +./demo/scripts/demo-test.sh network +``` + +### Docker Operations +```bash +# View logs for all services +docker compose -f demo/docker-compose.yml logs -f + +# View logs for specific service +docker compose -f demo/docker-compose.yml logs -f + +# Execute command in container +docker exec -it + +# Check service health +docker compose -f demo/docker-compose.yml ps +``` + +### Code Quality Validation +```bash +# Validate YAML files (yamllint) +docker run --rm -v "$(pwd):/data" cytopia/yamllint demo/docker-compose.yml.template +docker run --rm -v "$(pwd):/data" cytopia/yamllint demo/config/homepage/docker.yaml +docker run --rm -v "$(pwd):/data" cytopia/yamllint demo/config/grafana/datasources.yml +docker run --rm -v "$(pwd):/data" cytopia/yamllint demo/config/grafana/dashboards.yml + +# Validate shell scripts (shellcheck) +docker run --rm -v "$(pwd):/data" koalaman/shellcheck demo/scripts/*.sh +docker run --rm -v "$(pwd):/data" koalaman/shellcheck demo/tests/**/*.sh + +# Validate Dockerfiles (hadolint) +docker run --rm -v "$(pwd):/workdir" hadolint/hadolint +``` + +## Code Organization & Structure + +### Service Categories +1. **Infrastructure Services** (ports 4000-4007) + - Homepage (4000) - Central dashboard for service discovery + - Docker Socket Proxy (4005) - Security layer for Docker API access + - Pi-hole (4006) - DNS management with ad blocking + - Portainer (4007) - Web-based container management + +2. **Monitoring & Observability** (ports 4008-4009) + - InfluxDB (4008) - Time series database for metrics + - Grafana (4009) - Visualization platform + +3. **Documentation & Diagramming** (ports 4010-4011) + - Draw.io (4010) - Web-based diagramming application + - Kroki (4011) - Diagrams as a service + +4. **Developer Tools** (ports 4012, 4013, 4014, 4015, 4017, 4018) + - Atomic Tracker (4012) - Habit tracking and personal dashboard + - ArchiveBox (4013) - Web archiving solution + - Tube Archivist (4014) - YouTube video archiving + - Wakapi (4015) - Open-source WakaTime alternative (time tracking) + - MailHog (4017) - Web and API based SMTP testing + - Atuin (4018) - Magical shell history synchronization + +### Configuration Management +- **Environment Variables**: All configuration via `demo/demo.env` +- **Template-Based**: `docker-compose.yml` generated from `docker-compose.yml.template` using `envsubst` +- **Dynamic User Detection**: UID/GID automatically detected and applied +- **Service Discovery**: Automatic via Homepage labels in docker-compose.yml + +## Naming Conventions & Style Patterns + +### Service Naming +- **Container Names**: `tsysdevstack-supportstack-demo-` +- **Volume Names**: `tsysdevstack-supportstack-demo-_data` +- **Network Name**: `tsysdevstack-supportstack-demo-network` +- **Project Name**: `tsysdevstack-supportstack-demo` + +### Port Assignment +- **Range**: 4000-4099 +- **Sequential Allocation**: Groups assigned consecutive ports +- **Documented**: All ports documented in README.md port table + +### Docker Labels (Service Discovery) +```yaml +labels: + homepage.group: "Infrastructure" # Category + homepage.name: "Service Display Name" # Human-readable name + homepage.icon: "icon-name" # Icon identifier + homepage.href: "http://localhost:PORT" # Access URL + homepage.description: "Brief description" # Tool tip +``` + +### Shell Script Patterns +- **Strict Mode**: `set -euo pipefail` at top of all scripts +- **Color Output**: Consistent color codes for logging (RED, GREEN, YELLOW, BLUE, NC) +- **Log Functions**: `log_info()`, `log_success()`, `log_warning()`, `log_error()` +- **Function Organization**: Clear function headers with purpose description +- **Error Handling**: Check prerequisites before execution + +### YAML Patterns +- **Header Comments**: Include purpose and version +- **Structured Sections**: Clear separation of networks, volumes, services +- **Consistent Indentation**: 2-space indentation +- **Environment Variables**: All via `${VARIABLE_NAME}` format +- **Health Checks**: All services include healthcheck section + +## Testing Approach & Patterns + +### Test Organization +- **Unit Tests**: `/demo/tests/unit/` - Test individual components +- **Integration Tests**: `/demo/tests/integration/` - Test service interactions +- **E2E Tests**: `/demo/tests/e2e/` - Test complete workflows + +### Test Categories (demo-test.sh) +1. **File Ownership**: Verify no root-owned files on host +2. **User Mapping**: Validate UID/GID detection and application +3. **Docker Group**: Confirm docker group access +4. **Service Health**: All services passing health checks +5. **Port Accessibility**: Verify all ports accessible from host +6. **Network Isolation**: Confirm services isolated in demo network +7. **Volume Permissions**: Validate Docker volume permissions +8. **Security Compliance**: Docker socket proxy restrictions enforced + +### Validation Before Deployment +Run `./demo/scripts/validate-all.sh` to check: +- YAML syntax (yamllint) +- Shell script syntax (shellcheck) +- Docker image availability +- Port availability +- Environment variables +- Health endpoints +- Service dependencies +- Resource requirements + +## Important Gotchas & Non-Obvious Patterns + +### ⚠️ CRITICAL: Process Management +**NEVER interfere with existing processes**: +1. **Always check first**: Run `screen -ls` and `ps aux | grep demo-stack` before starting new work +2. **Unique identifiers**: Use unique session names with timestamps: `demo-deploy-$(date +%Y%m%d-%H%M%S)` +3. **Ask before acting**: Never kill or modify existing processes without explicit permission +4. **Respect concurrent work**: Other users/processes may be running - do not assume exclusive access + +### Demo-Only Security Model +- **Hardcoded Credentials**: All passwords set to `demo_password` or similar +- **No Encryption**: No TLS, no security hardening +- **Isolated Network**: Services contained in Docker network only +- **Not for Production**: Clear separation required for production deployments +- **Documentation Required**: All security assumptions must be documented + +### Zero-Tolerance Quality Standards +Before ANY file is created or modified: +1. ✅ YAML syntax validated with yamllint +2. ✅ Shell script validated with shellcheck +3. ✅ Environment variables verified +4. ✅ Port availability confirmed +5. ✅ Docker image existence verified +6. ✅ Service dependencies validated +7. ✅ Health check endpoints confirmed +8. ✅ Resource requirements assessed + +**Failure on any validation must block deployment.** + +### Dynamic Environment Strategy +- **User Detection**: UID/GID automatically detected and written to `demo.env` +- **Template Generation**: `docker-compose.yml` generated from template using `envsubst` +- **Variable Expansion**: All configuration via environment variables +- **Source demo.env**: Always source `demo.env` before accessing variables + +### Docker Socket Proxy Requirements +- **Mandatory**: All container operations must go through `docker-socket-proxy` +- **No Direct Access**: Never mount Docker socket directly in containers (except proxy) +- **Restricted API**: Minimal permissions per service requirements +- **Group-Based**: Dynamic docker group ID assignment + +### Volume vs Bind Mount Strategy +- **Prefer Volumes**: Use Docker volumes for data storage +- **Minimal Bind Mounts**: Use host bind mounts only for configuration that needs persistence +- **Dynamic Naming**: Volume names follow pattern: `tsysdevstack-supportstack-demo-_data` +- **Permission Mapping**: UID/GID mapped via environment variables + +### Service Discovery Mechanism +- **Homepage Labels**: Services automatically discovered via Docker labels +- **No Manual Config**: Don't manually add services to Homepage configuration +- **Group-Based**: Services organized by group (Infrastructure, Monitoring, Documentation, Developer Tools) +- **Real-Time**: Homepage updates automatically as services start/stop + +### FOSS Only Policy +- **No Proprietary**: Exclusively use free/libre/open source software +- **Official Images**: Prefer official Docker images +- **Verify Licenses**: Check license compatibility before adding services +- **Document Exceptions**: Any proprietary dependencies must be documented + +## Project-Specific Context + +### Current State +- **Demo Environment**: Fully configured with 13 services +- **Production Environment**: Placeholder only, not yet implemented +- **Documentation**: Comprehensive (AGENTS.md, PRD.md, README.md) +- **Scripts**: Complete orchestration and testing scripts available + +### Architecture Principles +1. **Demo-First**: All configurations for demonstration purposes +2. **No Persistence**: Zero data persistence between demo sessions +3. **Dynamic User Handling**: Automatic UID/GID detection +4. **Security-First**: Docker socket proxy for all container operations +5. **FOSS Only**: Exclusively open source software +6. **Inner Loop Focus**: Support daily development workflows +7. **Workstation Local**: Run locally on developer machines + +### Development Workflow +1. **Research**: Verify FOSS status and official Docker image availability +2. **Plan**: Determine port assignment and service group +3. **Template Configuration**: Add to docker-compose.yml.template with variables +4. **Environment Setup**: Add service variables to demo.env +5. **Security Integration**: Configure docker-socket-proxy permissions +6. **Dynamic Testing**: Validate with demo-stack.sh and demo-test.sh +7. **Documentation Update**: Update README.md, PRD.md, and AGENTS.md +8. **Atomic Commit**: Conventional commit with detailed description + +### Service Health Standards +- **All Services**: Must include healthcheck section +- **Completion**: Health checks must complete within 10 seconds +- **HTTP Endpoints**: Preferred health check method +- **Fallback**: Container status checks when HTTP not available +- **Monitoring**: All health checks visible via `docker compose ps` + +### Resource Limits +- **Memory**: < 512MB per service (where applicable) +- **CPU**: < 25% per service (idle) +- **Startup Time**: < 60 seconds for full stack +- **Disk Usage**: Temporary volumes only (demo mode) + +## Environment Variables (demo.env) + +### Required Variables +```bash +COMPOSE_PROJECT_NAME=tsysdevstack-supportstack-demo +COMPOSE_NETWORK_NAME=tsysdevstack-supportstack-demo-network + +# User Detection (Auto-populated by demo-stack.sh) +DEMO_UID= +DEMO_GID= +DEMO_DOCKER_GID= + +# Service Ports (4000-4099 range) +HOMEPAGE_PORT=4000 +DOCKER_SOCKET_PROXY_PORT=4005 +PIHOLE_PORT=4006 +PORTAINER_PORT=4007 +INFLUXDB_PORT=4008 +GRAFANA_PORT=4009 +DRAWIO_PORT=4010 +KROKI_PORT=4011 +ATOMIC_TRACKER_PORT=4012 +ARCHIVEBOX_PORT=4013 +TUBE_ARCHIVIST_PORT=4014 +WAKAPI_PORT=4015 +MAILHOG_PORT=4017 +ATUIN_PORT=4018 + +# Demo Credentials (NOT FOR PRODUCTION) +DEMO_ADMIN_USER=admin +DEMO_ADMIN_PASSWORD=demo_password +``` + +## Key Files Reference + +| File | Purpose | Read Before... | +|------|---------|----------------| +| `demo/AGENTS.md` | Detailed development guidelines | Making any changes | +| `demo/PRD.md` | Product Requirements Document | Adding/removing services | +| `demo/README.md` | Demo-specific documentation | Deploying or troubleshooting | +| `demo/demo.env` | Environment configuration | Modifying service settings | +| `demo/docker-compose.yml.template` | Service definitions | Adding/modifying services | +| `demo/scripts/demo-stack.sh` | Deployment orchestration | Understanding deployment flow | +| `demo/scripts/demo-test.sh` | Test suite | Verifying changes | +| `demo/scripts/validate-all.sh` | Pre-deployment validation | Making any commits | + +## Troubleshooting Quick Reference + +### Common Issues +1. **Port Conflicts**: Check with `netstat -tulpn | grep :` +2. **Permission Issues**: Verify UID/GID in demo.env match current user +3. **Image Pull Failures**: Run `docker pull ` manually +4. **Health Check Failures**: Check service logs with `docker compose logs ` +5. **Network Issues**: Verify network exists: `docker network ls | grep tsysdevstack` + +### Getting Help +1. Check troubleshooting section in demo/README.md +2. Review service logs: `docker compose -f demo/docker-compose.yml logs {service}` +3. Consult individual service documentation +4. Check health status: `docker compose -f demo/docker-compose.yml ps` +5. **CRITICAL**: Always check for existing processes first + +## Related Documentation + +- **demo/AGENTS.md**: Detailed development guidelines and standards +- **demo/PRD.md**: Product Requirements Document +- **demo/README.md**: Demo-specific documentation and quick start +- **docs/service-guides/**: Service-specific guides +- **docs/troubleshooting/**: Detailed troubleshooting procedures +- **docs/api-docs/**: API documentation + +--- + +**Last Updated**: 2025-01-24 +**Version**: 1.0