KNEL/KNEL-AIMiddleware
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2cb5f29683092bb52e3df461a77a4ab7f4cabe39
KNEL-AIMiddleware/JOURNAL.md

22 KiB
Raw Blame History

Project Journal - Append Only

This is an append-only journal documenting architectural decisions, insights, patterns, and work done on this project. Never delete entries from this journal.

2026-01-23

[ADR-001] Establish Project Journal

Date: 2026-01-23 Type: Architecture Decision Record Status: Accepted

Context: Working with AI automation in the KNEL-AIMiddleware project requires tracking decisions, insights, and patterns that emerge during development. Without proper documentation, valuable knowledge and reasoning may be lost.

Decision: Created this append-only journal (JOURNAL.md) to maintain a permanent record of:

  • Architecture Decision Records (ADRs)
  • Insights and patterns discovered during development
  • Work performed and reasoning behind it
  • Lessons learned and best practices

Rationale:

  • Append-only ensures historical context is never lost
  • Date/time stamped entries provide chronological progression
  • AGENTS.md updated to require journal maintenance as a critical project practice
  • Centralizes knowledge that would otherwise be scattered

Consequences:

  • All future work must append to this journal with timestamp and reasoning
  • AGENTS.md must reference this journal as required reading
  • Journal entries must never be deleted (append-only policy)
  • Provides traceability for decisions made during AI-assisted development

Housekeeping: Created JOURNAL.md and Updated AGENTS.md

Date: 2026-01-23 Work: Project housekeeping

What was done:

  1. Created JOURNAL.md - an append-only journal for tracking work, decisions, and patterns
  2. Added ADR-001 documenting the creation of the journal itself
  3. Updated AGENTS.md with instruction requiring journal maintenance
  4. Added prominent reference to JOURNAL.md at the top of AGENTS.md

Why: To ensure all work performed on this project is documented with reasoning and context, providing traceability and knowledge retention for AI-assisted development.

Pattern established: Housekeeping tasks like this should also be logged to the journal with clear "what" and "why" sections.

[Project Overview] KNEL-AIMiddleware Architecture and Evolution

Date: 2026-01-23 Type: Project Orientation Status: Documented

Context: Deep review of project structure, git history, and current state to understand the complete evolution and architectural patterns of KNEL-AIMiddleware.

Project Purpose: KNEL-AIMiddleware is a comprehensive Docker-based infrastructure that provides Model Context Protocol (MCP) servers and Language Server Protocol (LSP) providers for AI assistants (OpenWebUI, Crush). The project enables seamless integration with 27+ external tools and services.

Core Architecture Patterns:

  1. Container Naming Convention:

    • All containers use kneldevstack-aimiddleware- prefix (lowercase)
    • Standardized across all services for easy identification
    • Evolved from initial mixed case through commits ab21749 and 338094f

  2. Dockerfile Management Pattern:

    • Custom Dockerfiles stored in dockerfiles/ directory (tracked in git)
    • Vendor repositories cloned into vendor/ (gitignored)
    • Build context points to vendor directory with custom Dockerfile path
    • Example: dockerfile: ../../dockerfiles/bash-language-server/Dockerfile

  3. Service Communication Modes:

    • stdio-based: Run on-demand by Crush via docker run -i --rm (LSP + some MCP)
    • Long-running: Services with exposed ports (HTTP-based MCP, some stdio with ports)
    • Restart policy differs: "no" for stdio-based, "unless-stopped" for long-running

  4. Package Management Ecosystem:

    • Node.js/TypeScript (13 servers): npx for runtime installation
    • Python (10 servers): uvx for runtime, multi-stage with uv sync for builds
    • Go (2 servers): Binary compilation (kubernetes-mcp, terraform-mcp)
    • Rust (1 server): cargo build with multi-stage (elasticsearch-mcp)
    • Prebuilt: bash-language-server (npm), docker-language-server (binary), marksman (binary)

  5. Profile Organization:

    • dev profile: Development tools, design tools (LSP + certain MCPs)
    • ops profile: Operations, infrastructure, production services

  6. Wrapper Script Pattern:

    • LSP servers use wrapper scripts in project root
    • Pattern: Force remove existing container, then run with explicit name
    • Prevents container name conflicts during repeated Crush invocations
    • Scripts: lsp-bash-wrapper.sh, lsp-docker-wrapper.sh, lsp-marksman-wrapper.sh

  7. Environment Variable Management:

    • .env file (gitignored) for secrets
    • .env.example template with dummy values
    • Services have defaults where appropriate
    • Crush integration can inject env vars dynamically

Project Evolution Timeline:

Phase 1: Initial Setup (commits 226a37a to e03266d)

  • Project initialization with .gitignore
  • Initial docker-compose.yml with 27 MCP servers
  • Added STATUS.md for validation tracking
  • Added AGENTS.md with comprehensive documentation

Phase 2: Expansion and Refinement (commits 2792fdf to bc96cb4)

  • Added KiCAD host-only configuration
  • Added dockerfiles directory structure
  • Added LSP services (bash, docker, marksman)
  • Added LSP Dockerfiles for each service

Phase 3: Crush Integration (commits 23bc39f to 576a582)

  • Created crush.json with LSP configurations
  • Documented Crush integration in AGENTS.md
  • Standardized container naming to lowercase

Phase 4: Service Enhancements (commits cd5754e to 9e88f14)

  • Added custom Dockerfiles for multiple services
  • Fixed Python MCP startup issues
  • Updated STATUS.md with progress
  • Added imap-mcp Dockerfile

Phase 5: Documentation and Tooling (commits 30cbbeb to 2c0e19a)

  • Added setup scripts and Makefile
  • Created comprehensive README.md
  • Organized scripts into scripts/ directory
  • Replaced Makefile with maintenance.sh (commit ea1c90d)
  • Added .env.example template
  • Added .env to gitignore

Phase 6: LSP Configuration Refinement (commits 3484451 to 2c0e19a)

  • Updated LSP configs to use wrapper scripts
  • Added wrapper scripts for clean container management
  • Added multiple MCP servers to crush.json
  • Reset STATUS.md and started validation

Phase 7: Systematic Validation (commits 1638fff to c5393f9 - current)

  • Validated bash-language-server
  • Validated docker-language-server
  • Validated marksman
  • Systematically validating MCP servers:
    • audiobook-mcp (with Dockerfile fix: npm install instead of npm ci)
    • bitwarden-mcp (multi-stage distroless build)
    • blender-mcp (Python with uv package manager)
    • cloudron-mcp (fixed CMD to use dist/server.js)
    • docker-mcp (Python with uv, requires Docker socket)
    • drawio-mcp (TypeScript with pnpm)
    • elasticsearch-mcp (Rust, fixed to pass "stdio" subcommand)

Key Architectural Decisions:

ADR-002: stdio vs Long-running Services

  • Decision: Split services into stdio-based (on-demand) and long-running (with ports)
  • Rationale: stdio is more resource-efficient for LSP and occasional MCP use; long-running needed for HTTP endpoints and services that maintain state
  • Status: Implemented via restart policy ("no" vs "unless-stopped") and port configuration

ADR-003: Wrapper Scripts for LSP

  • Decision: Use wrapper scripts instead of direct docker run commands in crush.json
  • Rationale: Prevents container name conflicts, ensures clean state for each LSP session, handles edge cases (exited containers)
  • Status: Implemented for all three LSP servers (bash, docker, marksman)

ADR-004: Dockerfile Location Strategy

  • Decision: Store custom Dockerfiles in tracked dockerfiles/, use vendor/ for source code
  • Rationale: Keeps customizations version controlled, allows updates to vendor repos, clean separation of concerns
  • Status: Implemented for 20+ services

ADR-005: Container Naming Convention

  • Decision: Standardize to lowercase kneldevstack-aimiddleware- prefix
  • Rationale: Docker best practices, case-insensitivity issues, consistency with Docker Compose defaults
  • Status: Fully implemented across all services

ADR-006: Validation Script Approach

  • Decision: Create dedicated validate-mcp.sh script with proper JSON-RPC protocol testing
  • Rationale: Manual testing insufficient, need automated validation of MCP protocol handshake, catch configuration issues early
  • Status: Implemented and used for systematic validation

Current State:

Working Servers (validated):

  • LSP: bash-language-server, docker-language-server, marksman
  • MCP: audiobook, bitwarden, blender, cloudron, docker, drawio, elasticsearch

Recently Built (validation in progress):

  • All 7 MCP servers listed above successfully built and validated
  • Fixed Dockerfiles for cloudron-mcp (CMD), audiobook-mcp (npm install), elasticsearch-mcp (stdio subcommand)

Pending Build:

  • Remaining MCP servers from initial 27 (nextcloud, ghost, discourse, etc.)

Key Insights and Patterns:

  1. Multi-stage builds essential for Python and Rust to minimize image size
  2. uv package manager provides faster Python builds than pip
  3. npx -y enables consistent runtime installation without version conflicts
  4. Container name cleanup is critical for stdio-based services (wrapper scripts solve this)
  5. Environment variable injection must be handled carefully in Crush integration
  6. Validation early and often prevents accumulation of misconfigured services
  7. STATUS.md as single source of truth for operational status works well
  8. Sequential validation approach prevents getting overwhelmed by configuration issues

Lessons Learned:

  1. Dockerfile fixes are common: Most services required custom Dockerfiles (CMD, entrypoint, build process)
  2. Environment variable naming varies: Check upstream docs carefully (e.g., ES_URL vs ELASTICSEARCH_URL)
  3. Port mapping not always needed: stdio-based services don't expose ports
  4. Restart policy matters: "no" for on-demand, "unless-stopped" for long-running
  5. Documentation drift: STATUS.md must be updated immediately after each milestone
  6. Wrapper scripts edge cases: Container removal timing matters (sleep needed)
  7. Validation protocol: Must send proper JSON-RPC initialize message, not just check if container runs

Technical Debt / Future Work:

  1. Build and validate remaining MCP servers (20+ pending)
  2. Standardize environment variable naming across services
  3. Add health check configurations to docker-compose.yml
  4. Consider container resource limits
  5. Add automated testing beyond validation script
  6. Document Crush integration patterns more thoroughly

Why This Matters: This systematic approach to building and validating MCP/LSP infrastructure creates a reliable foundation for AI assistant integrations. The combination of custom Dockerfiles, wrapper scripts, validation tools, and status tracking ensures that services work correctly before deployment.

[ADR-007] Sequential Server Validation Pattern

Date: 2026-01-23 Type: Architecture Decision Record Status: Adopted

Context: With 27 MCP servers to build and validate, attempting to do all at once creates an overwhelming amount of work and makes it impossible to track issues systematically.

Decision: Adopt sequential validation approach:

  1. Build one MCP server at a time
  2. Fix Dockerfile issues as they arise
  3. Validate with proper MCP protocol handshake
  4. Update STATUS.md immediately after validation
  5. Commit changes for each server before moving to next
  6. Document findings in commit messages

Rationale:

  • Isolation: Issues in one server don't block others
  • Traceability: Each server's configuration history is clear
  • Documentation: STATUS.md stays accurate
  • Manageable: One server at a time is cognitively manageable
  • Early feedback: Patterns emerge that apply to future servers

Implementation:

  • Commit sequence: c5393f9 → 5398e4b → fa0b2b9 → 6017d99 → 8c67bbc → 15c5cd8 → 2c0e19a
  • Each commit includes: docker-compose.yml changes, Dockerfile (if needed), STATUS.md update
  • Validation performed using scripts/validate-mcp.sh with proper JSON-RPC initialize request

Results:

  • Successfully validated 7 MCP servers sequentially
  • Fixed Dockerfile issues: npm install vs npm ci, CMD corrections, stdio subcommand passing
  • Identified patterns: Python + uv, TypeScript + pnpm, Rust multi-stage
  • STATUS.md remains accurate and up-to-date

Consequences:

  • Progress is slower initially but more reliable
  • Each server gets full attention and proper validation
  • Pattern recognition helps accelerate future builds
  • STATUS.md becomes trusted source of truth
  • Git history provides clear audit trail

Future: Apply same pattern to remaining 20+ MCP servers and to LSP servers if needed

[Refactoring] Makefile to maintenance.sh Migration

Date: 2026-01-23 Type: Refactoring Status: Completed

Context: Project initially used Makefile for build automation (commits 3c2ee58, 49256e2). While functional, Makefiles have limitations for complex Docker Compose orchestration and can be less accessible to team members unfamiliar with Make syntax.

What Was Done:

  1. Replaced Makefile with maintenance.sh bash script (commit ea1c90d)
  2. Organized helper scripts into scripts/ directory (commit 7c583e2)
  3. Moved scripts: CloneVendorRepos.sh, BuildAll.sh, CleanVendor.sh, StatusCheck.sh, validate-mcp.sh
  4. Added comprehensive usage help and colored output
  5. Implemented all Makefile targets with bash function equivalents

Why This Change:

  • Accessibility: Bash is more universally understood than Make
  • Flexibility: Easier to add complex logic and error handling
  • Integration: Better with Docker Compose workflows
  • Maintainability: Single file with clear function structure
  • Portability: Works consistently across Linux/macOS without make dependency

Command Mapping:

Makefile Target maintenance.sh Command
make clone-vendors ./maintenance.sh clone-vendors
make build-all ./maintenance.sh build-all
make build SERVICE=xxx ./maintenance.sh build xxx
make clean-vendor ./maintenance.sh clean-vendor
make status ./maintenance.sh status
make up ./maintenance.sh up [profile]
make down ./maintenance.sh down
make rebuild SERVICE=xxx ./maintenance.sh rebuild xxx
New functionality ./maintenance.sh validate
New functionality ./maintenance.sh logs
New functionality ./maintenance.sh ps

Features Added:

  • Interactive confirmation for destructive operations (clean-vendor)
  • Profile support for selective service startup (up dev, up ops)
  • Validation command for MCP server testing
  • Logs and process status commands for monitoring
  • Comprehensive help system

Pattern: The refactoring maintained all existing functionality while adding new capabilities, proving the value of a migration rather than a rewrite. The bash script uses colored output and clear messaging for better user experience.

[Pattern] Documentation Philosophy and STATUS.md Maintenance

Date: 2026-01-23 Type: Pattern Documentation Status: Established

Context: Complex multi-service infrastructure requires accurate documentation for both operational status and architecture decisions. Without rigorous maintenance, documentation drifts and becomes unreliable.

Established Pattern: STATUS.md MUST always be kept fully up to date as it is single source of truth for operational status of all MCP servers.

Documentation Hierarchy:

  1. README.md: Entry point for new users

    • Project overview and purpose
    • Quick start guide
    • Basic usage examples
    • Links to detailed docs

  2. AGENTS.md: Development workflow and conventions

    • Agent/server catalog with capabilities
    • Environment variables reference
    • Development notes and critical instructions
    • Crush integration patterns
    • Reference to STATUS.md and JOURNAL.md (newly added)

  3. STATUS.md: Operational status (single source of truth)

    • Current validation results
    • Working vs non-working servers
    • Build status and version info
    • Known issues
    • MUST be updated immediately after each milestone

  4. JOURNAL.md: Historical record and decisions (newly created)

    • Append-only ADRs and insights
    • Project evolution and patterns
    • Reasoning behind work performed
    • NEVER delete entries

  5. .env.example: Configuration template

    • All environment variables with dummy values
    • Comments explaining purpose
    • Service-specific configuration

Maintenance Workflow:

When working on any MCP server:

  1. Before starting work: Review current status in STATUS.md
  2. During work: Update STATUS.md immediately after each milestone (build, start, validation, issues)
  3. After completing work: Ensure STATUS.md accurately reflects final status with relevant notes
  4. Commit STATUS.md changes: Always commit STATUS.md updates in separate atomic commits with clear commit messages

Rationale for Separate Commits:

  • Clear audit trail of when each server became operational
  • Easier to bisect issues if they arise later
  • Commit history matches project timeline
  • STATUS.md changes are independent of code changes

Examples from Git History:

  • c5393f9: "feat: add elasticsearch-mcp validation" - STATUS.md + docker-compose.yml + Dockerfile
  • 5398e4b: "feat: add drawio-mcp validation" - STATUS.md only (validation completed)
  • 2c0e19a: "feat: add audiobook-mcp validation and fix Dockerfile" - STATUS.md + docker-compose.yml + Dockerfile

Validation Script Integration: The scripts/validate-mcp.sh script provides automated validation that should be used before updating STATUS.md. This ensures that "Working" status is based on actual protocol handshake verification, not just container startup.

Why This Matters:

  • Prevents documentation drift from reality
  • Enables quick assessment of project readiness
  • Provides clear communication about what's working
  • Creates trust in documentation as a reliable resource
  • Historical record in JOURNAL.md preserves institutional knowledge

Anti-Patterns to Avoid:

  • Updating STATUS.md in bulk after multiple servers
  • Marking servers as "Working" without validation
  • Committing STATUS.md changes mixed with unrelated code changes
  • Skipping STATUS.md updates when "just making a quick fix"

[Project State] Deep Orientation Complete

Date: 2026-01-23 Type: Project Orientation Status: Complete

What Was Done: Conducted comprehensive review of KNEL-AIMiddleware project including:

  • Full git log analysis (50+ commits)
  • Review of all key files (crush.json, STATUS.md, README.md, AGENTS.md, docker-compose.yml)
  • Examination of dockerfiles/ directory structure (23 custom Dockerfiles)
  • Analysis of wrapper scripts (lsp-*-wrapper.sh)
  • Review of maintenance scripts (maintenance.sh, scripts/*.sh)
  • Understanding of build patterns across different languages

Current Project Understanding:

Purpose: KNEL-AIMiddleware is a Docker-based infrastructure providing 27 MCP servers and 3 LSP servers for AI assistant integration with OpenWebUI and Crush.

Key Statistics:

  • 27 MCP servers across 7 categories (Design, Hosting, Development, Content, Communication, Analytics, Productivity)
  • 3 LSP servers (bash, docker, markdown)
  • 7 MCP servers fully validated and working
  • 20+ MCP servers pending build/validation
  • All LSP servers validated and working
  • 23 custom Dockerfiles in dockerfiles/ directory

Build Progress:

  • Phase 1: Initial setup (27 MCP servers defined in docker-compose.yml)
  • Phase 2: LSP integration (3 LSP servers + Crush configuration)
  • Phase 3: Documentation (README.md, AGENTS.md, STATUS.md)
  • Phase 4: Tooling (scripts, maintenance.sh)
  • Phase 5: Systematic validation (7 MCP servers completed, 20+ remaining)

Technical Patterns Identified:

  1. Multi-language ecosystem: Node.js (npx), Python (uvx), Go (binary), Rust (cargo), Prebuilt binaries
  2. Two communication modes: stdio (on-demand) vs long-running (with ports)
  3. Container naming: lowercase kneldevstack-aimiddleware-* prefix
  4. Dockerfile strategy: Custom in tracked dockerfiles/, vendor source in gitignored vendor/
  5. Environment management: .env file with .env.example template
  6. Service organization: dev and ops profiles for selective startup
  7. Wrapper pattern: LSP servers use wrapper scripts for clean container management
  8. Validation approach: Sequential build/validate/commit cycle

Architecture Decisions Documented:

  • ADR-002: stdio vs long-running services
  • ADR-003: Wrapper scripts for LSP
  • ADR-004: Dockerfile location strategy
  • ADR-005: Container naming convention
  • ADR-006: Validation script approach
  • ADR-007: Sequential server validation pattern

Documentation Structure Established:

  1. README.md - User-facing overview and quick start
  2. AGENTS.md - Development workflow with journal instruction
  3. STATUS.md - Single source of truth for operational status (must stay current)
  4. JOURNAL.md - Append-only historical record and ADRs (newly created)

Recent Work Pattern (last 7 commits): Sequential validation of MCP servers with pattern:

  1. Build container (fix Dockerfile if needed)
  2. Validate MCP protocol handshake using validate-mcp.sh
  3. Update STATUS.md with version and working status
  4. Commit with descriptive message

Pending Work:

  1. Build and validate remaining 20+ MCP servers
  2. Consider adding health checks to docker-compose.yml
  3. Add resource limits to service definitions
  4. Explore automation opportunities for validation
  5. Document Crush integration patterns more thoroughly

Why This Orientation Matters:

  • Deep understanding prevents unintended breaks during future work
  • Pattern recognition accelerates future builds
  • Documentation ensures knowledge preservation
  • Historical context informs decision-making
  • Clear understanding of what works and what doesn't

Key Takeaways:

  1. Project is well-organized with clear conventions
  2. Sequential validation approach is working well
  3. Documentation is comprehensive and maintained
  4. STATUS.md is accurate and trusted
  5. Patterns are consistent across services
  6. Tooling supports efficient workflows

Ready For:

  • Building and validating remaining MCP servers
  • Making architectural improvements
  • Onboarding additional team members
  • Integrating additional services
  • Scaling the infrastructure
Reference in New Issue View Git Blame Copy Permalink