TSYSDevStack-SupportStack-LocalWorkstation/AGENTS.md

# TSYS Developer Support Stack - Agent Guide

## Project Overview

This repository contains a Docker Compose-based multi-service stack that provides off-the-shelf applications running locally on developer workstations. It's designed to enhance developer productivity and quality of life for the TSYS engineering team.

### Project Type
- **Infrastructure as Code**: Docker Compose with shell orchestration
- **Multi-Service Stack**: 16 services across 4 categories
- **Demo-First Architecture**: All configurations for demonstration purposes only

### Directory Structure
```
TSYSDevStack-SupportStack-LocalWorkstation/
├── LICENSE
├── README.md                      # Root-level documentation
├── AGENTS.md                      # This file - agent guide
├── demo/                          # Demo environment (primary focus)
│   ├── AGENTS.md                  # Detailed development guidelines
│   ├── PRD.md                     # Product Requirements Document
│   ├── README.md                  # Demo-specific documentation
│   ├── demo.env                   # Environment variables (dynamic)
│   ├── docker-compose.yml         # Generated compose file
│   ├── docker-compose.yml.template # Template for compose file
│   ├── scripts/                   # Orchestration scripts
│   │   ├── demo-stack.sh         # Deployment/management
│   │   ├── demo-test.sh          # QA/testing suite
│   │   └── validate-all.sh       # Pre-deployment validation
│   ├── tests/                     # Test suite
│   │   ├── unit/                 # Unit tests
│   │   ├── integration/          # Integration tests
│   │   └── e2e/                  # End-to-end tests
│   ├── config/                    # Service configurations
│   │   ├── homepage/            # Homepage dashboard config
│   │   ├── grafana/             # Grafana dashboards/datasources
│   │   ├── pihole/              # Pi-hole configuration
│   │   ├── dockhand/            # Dockhand configuration
│   │   ├── influxdb/            # InfluxDB configuration
│   │   ├── drawio/              # Draw.io configuration
│   │   ├── kroki/               # Kroki configuration
│   │   ├── atomictracker/       # Atomic Tracker configuration
│   │   ├── archivebox/          # ArchiveBox configuration
│   │   ├── tubearchivist/       # Tube Archivist configuration
│   │   ├── wakapi/              # Wakapi configuration
│   │   ├── mailhog/             # MailHog configuration
│   │   └── atuin/               # Atuin configuration
│   └── docs/                     # Additional documentation
│       ├── service-guides/      # Service-specific guides
│       ├── troubleshooting/     # Troubleshooting documentation
│       └── api-docs/            # API documentation
└── prod/                         # Production environment (placeholder)
```

## Essential Commands

### Deployment & Management
```bash
# Deploy the demo stack
./demo/scripts/demo-stack.sh deploy

# Stop all services
./demo/scripts/demo-stack.sh stop

# Restart all services
./demo/scripts/demo-stack.sh restart

# Show service status
./demo/scripts/demo-stack.sh status
```

### Testing & Validation
```bash
# Run comprehensive validation (before deployment)
./demo/scripts/validate-all.sh

# Run full test suite
./demo/scripts/demo-test.sh full

# Run security compliance tests only
./demo/scripts/demo-test.sh security

# Run permission validation tests only
./demo/scripts/demo-test.sh permissions

# Run network isolation tests only
./demo/scripts/demo-test.sh network
```

### Docker Operations
```bash
# View logs for all services
docker compose -f demo/docker-compose.yml logs -f

# View logs for specific service
docker compose -f demo/docker-compose.yml logs -f <service-name>

# Execute command in container
docker exec -it <container-name> <command>

# Check service health
docker compose -f demo/docker-compose.yml ps
```

### Code Quality Validation
```bash
# Validate YAML files (yamllint)
docker run --rm -v "$(pwd):/data" cytopia/yamllint demo/docker-compose.yml.template
docker run --rm -v "$(pwd):/data" cytopia/yamllint demo/config/homepage/docker.yaml
docker run --rm -v "$(pwd):/data" cytopia/yamllint demo/config/grafana/datasources.yml
docker run --rm -v "$(pwd):/data" cytopia/yamllint demo/config/grafana/dashboards.yml

# Validate shell scripts (shellcheck)
docker run --rm -v "$(pwd):/data" koalaman/shellcheck demo/scripts/*.sh
docker run --rm -v "$(pwd):/data" koalaman/shellcheck demo/tests/**/*.sh

# Validate Dockerfiles (hadolint)
docker run --rm -v "$(pwd):/workdir" hadolint/hadolint <path-to-dockerfile>
```

## Code Organization & Structure

### Service Categories
1. **Infrastructure Services** (ports 4005-4007)
   - Docker Socket Proxy (4005) - Security layer for Docker API access (internal only)
   - Pi-hole (4006) - DNS management with ad blocking
   - Dockhand (4007) - Web-based container management

2. **Monitoring & Observability** (ports 4008-4009)
   - InfluxDB (4008) - Time series database for metrics
   - Grafana (4009) - Visualization platform

3. **Documentation & Diagramming** (ports 4010-4011)
   - Draw.io (4010) - Web-based diagramming application
   - Kroki (4011) - Diagrams as a service

4. **Developer Tools** (ports 4000, 4012-4018)
   - Homepage (4000) - Central dashboard for service discovery
   - Atomic Tracker (4012) - Habit tracking and personal dashboard
   - ArchiveBox (4013) - Web archiving solution
   - Tube Archivist (4014) - YouTube video archiving (requires ta-redis + ta-elasticsearch)
   - Wakapi (4015) - Open-source WakaTime alternative (time tracking)
   - MailHog (4017) - Web and API based SMTP testing
   - Atuin (4018) - Magical shell history synchronization

5. **Companion Services** (internal only, no host ports)
   - ta-redis - Redis cache for Tube Archivist
   - ta-elasticsearch - Elasticsearch index for Tube Archivist

### Configuration Management
- **Environment Variables**: All configuration via `demo/demo.env`
- **Template-Based**: `docker-compose.yml` generated from `docker-compose.yml.template` using `envsubst`
- **Dynamic User Detection**: UID/GID automatically detected and applied
- **Service Discovery**: Automatic via Homepage labels in docker-compose.yml

## Naming Conventions & Style Patterns

### Service Naming
- **Container Names**: `kneldevstack-supportstack-demo-<service-name>`
- **Volume Names**: `kneldevstack-supportstack-demo_<service>_data`
- **Network Name**: `kneldevstack-supportstack-demo-network`
- **Project Name**: `kneldevstack-supportstack-demo`

### Port Assignment
- **Range**: 4000-4099
- **Sequential Allocation**: Groups assigned consecutive ports
- **Documented**: All ports documented in README.md port table

### Docker Labels (Service Discovery)
```yaml
labels:
  homepage.group: "Infrastructure"      # Category
  homepage.name: "Service Display Name" # Human-readable name
  homepage.icon: "icon-name"           # Icon identifier
  homepage.href: "http://localhost:PORT" # Access URL
  homepage.description: "Brief description" # Tool tip
```

### Shell Script Patterns
- **Strict Mode**: `set -euo pipefail` at top of all scripts
- **Color Output**: Consistent color codes for logging (RED, GREEN, YELLOW, BLUE, NC)
- **Log Functions**: `log_info()`, `log_success()`, `log_warning()`, `log_error()`
- **Function Organization**: Clear function headers with purpose description
- **Error Handling**: Check prerequisites before execution

### YAML Patterns
- **Header Comments**: Include purpose and version
- **Structured Sections**: Clear separation of networks, volumes, services
- **Consistent Indentation**: 2-space indentation
- **Environment Variables**: All via `${VARIABLE_NAME}` format
- **Health Checks**: All services include healthcheck section

## Testing Approach & Patterns

### Test Organization
- **Unit Tests**: `/demo/tests/unit/` - Test individual components
- **Integration Tests**: `/demo/tests/integration/` - Test service interactions
- **E2E Tests**: `/demo/tests/e2e/` - Test complete workflows

### Test Categories (demo-test.sh)
1. **File Ownership**: Verify no root-owned files on host
2. **User Mapping**: Validate UID/GID detection and application
3. **Docker Group**: Confirm docker group access
4. **Service Health**: All services passing health checks
5. **Port Accessibility**: Verify all ports accessible from host
6. **Network Isolation**: Confirm services isolated in demo network
7. **Volume Permissions**: Validate Docker volume permissions
8. **Security Compliance**: Docker socket proxy restrictions enforced

### Validation Before Deployment
Run `./demo/scripts/validate-all.sh` to check:
- YAML syntax (yamllint)
- Shell script syntax (shellcheck)
- Docker image availability
- Port availability
- Environment variables
- Health endpoints
- Service dependencies
- Resource requirements

## Important Gotchas & Non-Obvious Patterns

### ⚠️ CRITICAL: Process Management
**NEVER interfere with existing processes**:
1. **Always check first**: Run `screen -ls` and `ps aux | grep demo-stack` before starting new work
2. **Unique identifiers**: Use unique session names with timestamps: `demo-deploy-$(date +%Y%m%d-%H%M%S)`
3. **Ask before acting**: Never kill or modify existing processes without explicit permission
4. **Respect concurrent work**: Other users/processes may be running - do not assume exclusive access

### Demo-Only Security Model
- **Hardcoded Credentials**: All passwords set to `demo_password` or similar
- **No Encryption**: No TLS, no security hardening
- **Isolated Network**: Services contained in Docker network only
- **Not for Production**: Clear separation required for production deployments
- **Documentation Required**: All security assumptions must be documented

### Zero-Tolerance Quality Standards
Before ANY file is created or modified:
1. ✅ YAML syntax validated with yamllint
2. ✅ Shell script validated with shellcheck
3. ✅ Environment variables verified
4. ✅ Port availability confirmed
5. ✅ Docker image existence verified
6. ✅ Service dependencies validated
7. ✅ Health check endpoints confirmed
8. ✅ Resource requirements assessed

**Failure on any validation must block deployment.**

### Dynamic Environment Strategy
- **User Detection**: UID/GID automatically detected and written to `demo.env`
- **Template Generation**: `docker-compose.yml` generated from template using `envsubst`
- **Variable Expansion**: All configuration via environment variables
- **Source demo.env**: Always source `demo.env` before accessing variables

### Docker Socket Proxy Requirements
- **Mandatory**: All container operations must go through `docker-socket-proxy`
- **No Direct Access**: Never mount Docker socket directly in containers (except proxy)
- **Restricted API**: Minimal permissions per service requirements
- **Group-Based**: Dynamic docker group ID assignment

### Volume vs Bind Mount Strategy
- **Prefer Volumes**: Use Docker volumes for data storage
- **Minimal Bind Mounts**: Use host bind mounts only for configuration that needs persistence
- **Dynamic Naming**: Volume names follow pattern: `kneldevstack-supportstack-demo_<service>_data`
- **Permission Mapping**: UID/GID mapped via environment variables

### Service Discovery Mechanism
- **Homepage Labels**: Services automatically discovered via Docker labels
- **No Manual Config**: Don't manually add services to Homepage configuration
- **Group-Based**: Services organized by group (Infrastructure, Monitoring, Documentation, Developer Tools)
- **Real-Time**: Homepage updates automatically as services start/stop

### FOSS Only Policy
- **No Proprietary**: Exclusively use free/libre/open source software
- **Official Images**: Prefer official Docker images
- **Verify Licenses**: Check license compatibility before adding services
- **Document Exceptions**: Any proprietary dependencies must be documented

## Project-Specific Context

### Current State
- **Demo Environment**: Fully configured with 16 services
- **Production Environment**: Placeholder only, not yet implemented
- **Documentation**: Comprehensive (AGENTS.md, PRD.md, README.md)
- **Scripts**: Complete orchestration and testing scripts available

### Architecture Principles
1. **Demo-First**: All configurations for demonstration purposes
2. **No Persistence**: Zero data persistence between demo sessions
3. **Dynamic User Handling**: Automatic UID/GID detection
4. **Security-First**: Docker socket proxy for all container operations
5. **FOSS Only**: Exclusively open source software
6. **Inner Loop Focus**: Support daily development workflows
7. **Workstation Local**: Run locally on developer machines

### Development Workflow
1. **Research**: Verify FOSS status and official Docker image availability
2. **Plan**: Determine port assignment and service group
3. **Template Configuration**: Add to docker-compose.yml.template with variables
4. **Environment Setup**: Add service variables to demo.env
5. **Security Integration**: Configure docker-socket-proxy permissions
6. **Dynamic Testing**: Validate with demo-stack.sh and demo-test.sh
7. **Documentation Update**: Update README.md, PRD.md, and AGENTS.md
8. **Atomic Commit**: Conventional commit with detailed description

### Service Health Standards
- **All Services**: Must include healthcheck section
- **Completion**: Health checks must complete within 10 seconds
- **HTTP Endpoints**: Preferred health check method
- **Fallback**: Container status checks when HTTP not available
- **Monitoring**: All health checks visible via `docker compose ps`

### Resource Limits
- **Memory**: < 512MB per service (where applicable)
- **CPU**: < 25% per service (idle)
- **Startup Time**: < 60 seconds for full stack
- **Disk Usage**: Temporary volumes only (demo mode)

## Environment Variables (demo.env)

### Required Variables
```bash
COMPOSE_PROJECT_NAME=kneldevstack-supportstack-demo
COMPOSE_NETWORK_NAME=kneldevstack-supportstack-demo-network

# User Detection (Auto-populated by demo-stack.sh)
DEMO_UID=
DEMO_GID=
DEMO_DOCKER_GID=

# Service Ports (4000-4099 range)
HOMEPAGE_PORT=4000
DOCKER_SOCKET_PROXY_PORT=4005
PIHOLE_PORT=4006
DOCKHAND_PORT=4007
INFLUXDB_PORT=4008
GRAFANA_PORT=4009
DRAWIO_PORT=4010
KROKI_PORT=4011
ATOMIC_TRACKER_PORT=4012
ARCHIVEBOX_PORT=4013
TUBE_ARCHIVIST_PORT=4014
WAKAPI_PORT=4015
MAILHOG_PORT=4017
ATUIN_PORT=4018

# Demo Credentials (NOT FOR PRODUCTION)
DEMO_ADMIN_USER=admin
DEMO_ADMIN_PASSWORD=demo_password
```

## Key Files Reference

| File | Purpose | Read Before... |
|------|---------|----------------|
| `demo/AGENTS.md` | Detailed development guidelines | Making any changes |
| `demo/PRD.md` | Product Requirements Document | Adding/removing services |
| `demo/README.md` | Demo-specific documentation | Deploying or troubleshooting |
| `demo/demo.env` | Environment configuration | Modifying service settings |
| `demo/docker-compose.yml.template` | Service definitions | Adding/modifying services |
| `demo/scripts/demo-stack.sh` | Deployment orchestration | Understanding deployment flow |
| `demo/scripts/demo-test.sh` | Test suite | Verifying changes |
| `demo/scripts/validate-all.sh` | Pre-deployment validation | Making any commits |

## Troubleshooting Quick Reference

### Common Issues
1. **Port Conflicts**: Check with `netstat -tulpn | grep :<port>`
2. **Permission Issues**: Verify UID/GID in demo.env match current user
3. **Image Pull Failures**: Run `docker pull <image>` manually
4. **Health Check Failures**: Check service logs with `docker compose logs <service>`
5. **Network Issues**: Verify network exists: `docker network ls | grep kneldevstack`

### Getting Help
1. Check troubleshooting section in demo/README.md
2. Review service logs: `docker compose -f demo/docker-compose.yml logs {service}`
3. Consult individual service documentation
4. Check health status: `docker compose -f demo/docker-compose.yml ps`
5. **CRITICAL**: Always check for existing processes first

## Related Documentation

- **demo/AGENTS.md**: Detailed development guidelines and standards
- **demo/PRD.md**: Product Requirements Document
- **demo/README.md**: Demo-specific documentation and quick start
- **docs/service-guides/**: Service-specific guides
- **docs/troubleshooting/**: Detailed troubleshooting procedures
- **docs/api-docs/**: API documentation

---

**Last Updated**: 2025-01-24
**Version**: 1.0