Add Reactive Resume, Metrics, Kiwix, Resume Matcher, and Apple Health from the earlier SelfStack project. Rewrite Apple Health collector to use InfluxDB v2 with proper error handling. Update all tests, scripts, Homepage config, env template, and documentation for the expanded stack. New services: - Reactive Resume (4016) + Postgres/Minio/Chrome companions - Metrics (4021) - GitHub metrics visualization - Kiwix (4022) - offline wiki reader - Resume Matcher (4023) - AI resume screening - Apple Health (4024) - health data collector → InfluxDB v2 Also adds git policy to AGENTS.md: always commit and push automatically. 💘 Generated with Crush Assisted-by: GLM-5.1 via Crush <crush@charm.land>
13 KiB
13 KiB
TSYS Developer Support Stack - Development Guidelines
🎯 Development Principles
Demo-First Architecture
- Demo-Only Configuration: All services configured for demonstration purposes only
- No Persistent Data: Zero data persistence between demo sessions
- Dynamic User Handling: Automatic UID/GID detection and application
- Security-First: Docker socket proxy for all container operations
- Minimal Bind Mounts: Prefer Docker volumes over host bind mounts. Use host bind mounts only for minimal bootstrap purposes of configuration data that needs to be persistent.
- Consistent Naming:
kneldevstack-supportstack-demo-prefix everywhere including in the docker-compose file for the service names. - One-Command Deployment: Single script deployment with full validation
Dynamic Environment Strategy
- User Detection: Automatic current user and group ID detection
- Docker Group Handling: Dynamic docker group ID resolution
- Variable-Driven Configuration: All settings via environment variables
- Template-Based Compose: Generate docker-compose.yml from templates
- Environment Isolation: Separate demo.env for all configuration
FOSS Only Policy
- Exclusively use free/libre/open source software
- Verify license compatibility
- Prefer official Docker images
- Document any proprietary dependencies
Inner Loop Focus
- Support daily development workflows
- Avoid project-specific dependencies
- Prioritize developer productivity
- Maintain workstation-local deployment
Code Organization Policy
- Mandatory Code Subdirectory: ALL created files, configurations, scripts, and code MUST be placed in the
code/subdirectory - No Root-Level Code: Absolutely NO code files shall be created at the project root level
- Structured Organization: All implementation artifacts belong under
code/with proper subdirectory organization
System Interference Policy
- NEVER interfere with existing processes: Do not kill, stop, or modify any running processes without explicit permission
- Check before acting: Always verify what processes/screen sessions are running before taking any action
- Use unique identifiers: Create uniquely named sessions/processes to avoid conflicts
- Ask first: Always request permission before touching any existing work on the system
- Respect concurrent work: Other users/processes may be running - do not assume exclusive access
🛡️ Quality Assurance Standards
Mandatory Tool Validation
- yamllint: ALL YAML files MUST pass yamllint validation before commit
- shellcheck: ALL shell scripts MUST pass shellcheck validation before commit
- hadolint: ALL Dockerfiles MUST pass hadolint validation before commit
- Pre-commit Hooks: Automated validation on every commit attempt
Zero-Tolerance Policy
- No YAML syntax errors: Prevents Docker Compose failures
- No shell script errors: Prevents deployment script failures
- No Docker image issues: Prevents container startup failures
- No port conflicts: Prevents service accessibility issues
- No permission problems: Prevents file ownership issues
Proactive Validation Checklist
Before ANY file is created or modified:
- ✅ YAML syntax validated with yamllint
- ✅ Shell script validated with shellcheck
- ✅ Environment variables verified
- ✅ Port availability confirmed
- ✅ Docker image existence verified
- ✅ Service dependencies validated
- ✅ Health check endpoints confirmed
- ✅ Resource requirements assessed
🏗️ Architecture Guidelines
Service Categories
- Infrastructure Services: Core platform services
- Monitoring & Observability: Metrics, visualization, and health data
- Documentation & Diagramming: Knowledge management
- Developer Tools: Productivity enhancers
- Productivity: Resume building and screening tools
Design Patterns
- Service Discovery: Automatic via Homepage dashboard
- Health Checks: Comprehensive for all services
- Network Isolation: Docker network per stack
- Resource Limits: Memory and CPU constraints
🔧 Technical Standards
Docker Configuration Standards
Demo Service Template
# Standard service template (docker-compose.yml.template)
services:
service-name:
image: official/image:tag
user: "${UID}:${GID}"
container_name: "${COMPOSE_PROJECT_NAME}-service-name"
restart: unless-stopped
networks:
- ${COMPOSE_NETWORK_NAME}
volumes:
- "${COMPOSE_PROJECT_NAME}_service_data:/path"
environment:
- PUID=${UID}
- PGID=${GID}
labels:
homepage.group: "Group Name"
homepage.name: "Display Name"
homepage.icon: "icon-name"
homepage.href: "http://localhost:${SERVICE_PORT}"
homepage.description: "Brief description"
Dynamic Variable Requirements
- UID/GID: Current user and group detection
- DOCKER_GID: Docker group ID for socket access
- COMPOSE_PROJECT_NAME:
kneldevstack-supportstack-demo - COMPOSE_NETWORK_NAME:
kneldevstack-supportstack-demo-network - Service Ports: All configurable via environment variables
Port Assignment Strategy
- Range: 4000-4099
- Groups: Sequential allocation
- Document in README.md port table
- Avoid conflicts with host services
Network Configuration
- Network name:
kneldevstack-supportstack-demo - IP binding:
192.168.3.6:{port}where applicable - Inter-service communication via container names
- Only necessary ports exposed to host
📋 Quality Assurance
Testing Requirements
- Automated health check validation
- Port accessibility verification
- Service discovery functionality
- Resource usage monitoring
- User workflow validation
Code Quality Standards
- Clear, commented configurations
- Consistent naming conventions
- Comprehensive documentation
- Atomic commits with conventional messages
Security Guidelines
Demo Security Model
- Demo-Hardened Configurations: All settings optimized for demonstration
- No External Network Access: Isolated except for image pulls
- Production Separation: Clear distinction from production deployments
- Security Documentation: All assumptions clearly documented
Docker Socket Security
- Mandatory Proxy: All container operations through docker-socket-proxy
- Restricted API Access: Minimal permissions per service requirements
- No Direct Socket Access: Prevent direct Docker socket mounting
- Group-Based Access: Dynamic docker group ID assignment
File System Security
- Dynamic User Mapping: Automatic UID/GID detection prevents ownership issues
- Volume-First Storage: Prefer Docker volumes over bind mounts
- Read-Only Bind Mounts: Minimal host filesystem access
- Permission Validation: Automated file ownership verification
🔄 Development Workflow
Demo-First Service Addition
- Research: Verify FOSS status and official Docker image availability
- Plan: Determine port assignment and service group
- Template Configuration: Add to docker-compose.yml.template with variables
- Environment Setup: Add service variables to demo.env
- Security Integration: Configure docker-socket-proxy permissions
- Dynamic Testing: Validate with demo-stack.sh and demo-test.sh
- Documentation Update: Update README.md, PRD.md, and AGENTS.md
- Atomic Commit: Conventional commit with detailed description
Process Management Guidelines
- Screen Sessions: Use descriptive, unique names (e.g.,
demo-deploy-YYYYMMDD-HHMMSS) - Background Processes: Always use logging to track progress
- Process Discovery: Use
ps aux | grepandscreen -lsto check existing work - Safe Termination: Only terminate processes you explicitly started
- Permission First: Always ask before modifying/killing any existing process
Template-Driven Development
- Variable Configuration: All settings via environment variables
- Naming Convention: Consistent
kneldevstack-supportstack-demo-prefix - User Handling: Dynamic UID/GID detection in all services
- Security Integration: Docker socket proxy for container operations
- Volume Strategy: Docker volumes with dynamic naming
Service Removal Process
- Deprecate: Mark service for removal in documentation
- Test: Verify stack functionality without service
- Remove: Delete from docker-compose.yml
- Update: Clean up documentation and port assignments
- Commit: Document removal in commit message
Configuration Changes
- Plan: Document change rationale and impact
- Test: Validate in development environment
- Update: Apply changes to configuration files
- Verify: Run full test suite
- Document: Update relevant documentation
- Commit: Atomic commit with detailed description
📊 Monitoring & Observability
Health Check Standards
- All services must include health checks
- Health checks complete within 10 seconds
- HTTP endpoints preferred
- Fallback to container status checks
Resource Limits
- Memory: < 512MB per service (where applicable)
- CPU: < 25% per service (idle)
- Startup time: < 60 seconds for full stack
- Disk usage: Temporary volumes only
Logging Standards
- Structured logging where possible
- Log levels: INFO, WARN, ERROR
- Container logs accessible via
docker compose logs - No persistent log storage in demo mode
🧪 Testing Guidelines
Demo Testing Framework
# ALWAYS check for existing work first
screen -ls
ps aux | grep demo-stack
# Dynamic deployment and testing (use unique session names)
screen -S demo-deploy-$(date +%Y%m%d-%H%M%S) -dm -L -Logfile deploy-$(date +%Y%m%d-%H%M%S).log ./scripts/demo-stack.sh deploy
./scripts/demo-test.sh full # Comprehensive QA/validation
./scripts/demo-test.sh security # Security compliance validation
./scripts/demo-test.sh permissions # File ownership validation
./scripts/demo-test.sh network # Network isolation validation
Automated Validation Suite
- File Ownership: Verify no root-owned files on host
- User Mapping: Validate UID/GID detection and application
- Docker Group: Confirm docker group access for socket proxy
- Service Health: All services passing health checks
- Port Accessibility: Verify all ports accessible from host
- Network Isolation: Confirm services isolated in demo network
- Volume Permissions: Validate Docker volume permissions
- Security Compliance: Docker socket proxy restrictions enforced
Manual Testing Checklist
- All web interfaces accessible via browser
- Demo credentials work correctly
- Service discovery functional in Homepage
- Inter-service communication working through proxy
- Resource usage within defined limits
- No port conflicts on host system
- All health checks passing
- No root-owned files created on host
- Docker socket proxy functioning correctly
- Dynamic user detection working properly
Performance Testing
- Startup time measurement
- Memory usage monitoring
- CPU usage validation
- Network connectivity testing
- Resource leak detection
📚 Documentation Standards
README.md Requirements
- Quick start instructions
- Service overview table
- Technical configuration details
- Troubleshooting guide
- Security notes and warnings
PRD.md Requirements
- Product vision and goals
- Functional requirements
- User experience requirements
- Acceptance criteria
- Success metrics
AGENTS.md Requirements
- Development principles
- Technical standards
- Quality assurance guidelines
- Development workflow
- Testing procedures
🔒 Security Considerations
Demo Security Model
- Hardcoded credentials clearly marked
- No encryption or security hardening
- Network isolation within Docker
- No external access except image pulls
Security Checklist
- All services use demo credentials
- No persistent sensitive data
- Network properly isolated
- Only necessary ports exposed
- Security warnings documented
- Production deployment guidance included
🚀 Deployment Guidelines
Local Development
# Check for existing work BEFORE starting
screen -ls
ps aux | grep demo-stack
# Start development stack with unique session name
screen -S demo-deploy-$(date +%Y%m%d-%H%M%S) -dm -L -Logfile deploy-$(date +%Y%m%d-%H%M%S).log ./scripts/demo-stack.sh deploy
# Monitor startup
docker compose logs -f
# Validate deployment
./scripts/demo-test.sh full
Demo Preparation
- Clean all containers and volumes
- Pull latest images
- Verify all health checks
- Test complete user workflows
- Document any known issues
Production Migration
- Replace demo credentials with secure ones
- Implement persistent data storage
- Add encryption and security hardening
- Configure backup and recovery
- Set up monitoring and alerting
📞 Development Support
Getting Help
- Check troubleshooting section in README.md
- Review service logs:
docker compose logs {service} - Consult individual service documentation
- Check health status:
docker compose ps - CRITICAL: Always check for existing processes before starting new ones:
screen -lsandps aux | grep demo-stack
Issue Reporting
- Include full error messages
- Provide system information
- Document reproduction steps
- Include relevant configuration snippets
- Specify demo vs production context
Last updated: 2025-11-13