KNEL-AIMiddleware/README.md

# KNEL-AIMiddleware

[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![MCP](https://img.shields.io/badge/MCP-Compatible-orange.svg)](https://modelcontextprotocol.io/)
[![LSP](https://img.shields.io/badge/LSP-Compatible-purple.svg)](https://microsoft.github.io/language-server-protocol/)

> MCP and LSP infrastructure for OpenWebUI and Crush to utilize

## 📖 Overview

KNEL-AIMiddleware is a comprehensive Docker-based infrastructure for running **Model Context Protocol (MCP)** servers and **Language Server Protocol (LSP)** providers. It enables AI assistants like OpenWebUI and Crush to seamlessly integrate with external tools, services, and code intelligence.

### 🎯 Use Cases

- **OpenWebUI Integration**: Provide MCP servers for enhanced AI context and tooling
- **Crush Integration**: LSP servers for intelligent code completion and analysis
- **Development**: Individual server isolation for easy debugging and updates
- **Production**: Scalable, containerized service deployment

## 🚀 Features

- ✅ **Modular Architecture**: Each service runs in its own container
- ✅ **Docker Compose**: Easy orchestration and management
- ✅ **Prebuilt Images**: Optimized container images with minimal dependencies
- ✅ **Crush Ready**: Pre-configured LSP servers for `crush.json`
- ✅ **Environment Variables**: Secure configuration via `.env` file
- ✅ **Health Monitoring**: Built-in logging and container health checks

## 📊 Server Status

For detailed build and configuration status of all MCP/LSP servers, see [STATUS.md](STATUS.md).

## 📦 Installation

### Prerequisites

- [Docker](https://docs.docker.com/get-docker/) 20.10+
- [Docker Compose](https://docs.docker.com/compose/install/) 2.0+
- (Optional) [Crush](https://github.com/charmbracelet/crush) for LSP integration

### Quick Start

1. **Clone the repository**
   ```bash
   git clone https://github.com/KNEL/KNEL-AIMiddleware.git
   cd KNEL-AIMiddleware
   ```

2. **Configure environment variables**
   ```bash
   cp .env.example .env
   # Edit .env with your API keys and configurations
   ```

3. **Build and start services**
   ```bash
   # Start all built services
   docker compose up -d

   # Or start specific service
   docker compose up -d <service-name>
   ```

### Setup Scripts

The repository includes helpful scripts to simplify setup and management:

#### CloneVendorRepos.sh
Clone all upstream vendor MCP/LSP repositories:

```bash
./CloneVendorRepos.sh
```

This script clones all 27 vendor repositories into `vendor/` directory.

#### BuildAll.sh
Build all MCP/LSP services:

```bash
./BuildAll.sh
```

This script iterates through all services in `docker-compose.yml` and builds each one.

#### CleanVendor.sh
Remove all cloned vendor repositories:

```bash
./CleanVendor.sh
```

⚠️ **Warning**: This permanently deletes all cloned vendor repositories. You'll need to re-clone them.

#### StatusCheck.sh
Check build status of all services:

```bash
./StatusCheck.sh
```

This script checks which Docker images exist and compares with `STATUS.md`.

### Makefile Targets

For a more command-line oriented approach, use the included Makefile:

```bash
# Show all available targets
make help

# Clone all vendor repositories
make clone-vendors

# Build all services
make build-all

# Clean vendor directory
make clean-vendor

# Check service status
make status

# Start all services
make up

# Stop all services
make down

# Rebuild specific service
make rebuild SERVICE=context7-mcp
```

## 💻 Usage

### Docker Compose Commands

#### Build and Start
```bash
# Build specific service
docker compose build <service-name>

# Build without cache
docker compose build --no-cache <service-name>

# Start service (detached mode)
docker compose up -d <service-name>

# Start all services
docker compose up -d
```

#### View Logs
```bash
# Follow logs for service
docker compose logs -f <service-name>

# View last 100 lines
docker compose logs --tail 100 <service-name>
```

#### Stop and Clean
```bash
# Stop service
docker compose stop <service-name>

# Remove service (also deletes container)
docker compose rm -f <service-name>

# Stop all services
docker compose down

# Remove all volumes
docker compose down -v
```

### Service Profiles

Services are organized into profiles for selective startup:

```bash
# Development services (LSP + MCP)
docker compose --profile dev up

# Production services
docker compose --profile prod up

# Design tools (KiCAD, Blender, etc.)
docker compose --profile design up
```

## 🔧 Crush Integration

All LSP and MCP instances must be configured in `crush.json` for Crush to use them.

### crush.json Format

```json
{
  "$schema": "https://charm.land/crush.json",
  "lsp": {
    "bash": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "KNELDevStack-AIMiddleware-bash-language-server", "start"]
    },
    "docker": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "KNELDevStack-AIMiddleware-docker-language-server", "start", "--stdio"]
    },
    "markdown": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "KNELDevStack-AIMiddleware-marksman", "server"]
    }
  }
}
```

### Configuration Priority

Crush looks for configuration files in this order:
1. `.crush.json` (project-local)
2. `crush.json` (project-local)
3. `$HOME/.config/crush/crush.json` (global)

### Docker-based LSP Configuration

For Docker-based LSP instances, use this pattern:
```json
{
  "command": "docker",
  "args": [
    "run",
    "-i",           // Interactive mode (required for stdio)
    "--rm",         // Auto-cleanup after use
    "KNELDevStack-AIMiddleware-<service-name>",
    "<server-args>" // e.g., "start", "--stdio"
  ]
}
```

## 🏗️ Project Structure

```
KNEL-AIMiddleware/
├── docker-compose.yml          # Service orchestration
├── .env                      # Environment variables (not committed)
├── .env.example              # Environment template
├── dockerfiles/              # Custom Dockerfiles (tracked in git)
│   ├── bash-language-server/
│   ├── docker-language-server/
│   └── marksman/
├── vendor/                   # Cloned repositories (gitignored)
│   ├── bash-language-server/
│   ├── docker-language-server/
│   └── marksman/
├── AGENTS.md                 # Development workflow and conventions
├── STATUS.md                 # Server status and progress tracking
└── README.md                 # This file
```

### Dockerfile Management

Custom Dockerfiles are created in `dockerfiles/` and referenced in `docker-compose.yml`:

```yaml
service-name:
  build:
    context: ./vendor/service-name
    dockerfile: ../../dockerfiles/service-name/Dockerfile
  container_name: KNELDevStack-AIMiddleware-service-name
  restart: unless-stopped
```

This approach:
- Keeps custom Dockerfiles under version control
- Uses vendor repository as build context
- Maintains clean separation of concerns

## 🌍 Environment Variables

Required variables (see `.env.example`):

| Variable | Description | Required |
|----------|-------------|-----------|
| `OPENAI_API_KEY` | OpenAI API key for LLM services | Yes (for OpenAI) |
| `KICAD_HOST` | KiCAD server host | Yes (for KiCAD MCP) |
| `KICAD_PORT` | KiCAD server port | Yes (for KiCAD MCP) |
| `BITWARDEN_*` | Bitwarden credentials | Yes (for Bitwarden MCP) |
| `MATOMO_*` | Matomo API credentials | Yes (for Matomo MCP) |
| `*_API_KEY` | Service-specific API keys | Varies |

**⚠️ Security Note**: Never commit `.env` file to version control.

## 🔍 Troubleshooting

### Container Not Starting

```bash
# Check container logs
docker compose logs <service-name>

# Inspect container state
docker inspect <container-name>
```

### MCP Server Not Connecting

1. Verify service is running: `docker ps`
2. Check logs for errors: `docker compose logs <service>`
3. Verify environment variables in `.env`
4. Test connectivity: `docker compose exec <service> curl localhost:8080`

### LSP Server Not Responding (Crush)

1. Verify Docker image exists: `docker images | grep knel`
2. Test container manually:
   ```bash
   echo '{"jsonrpc":"2.0","method":"initialize","params":{}}' | \
   docker run -i --rm KNELDevStack-AIMiddleware-<lsp-name>
   ```
3. Check `crush.json` configuration
4. Verify container runs with `-i` flag for stdio

### Port Conflicts

If ports are already in use, modify `docker-compose.yml`:

```yaml
ports:
  - "8081:8080"  # Change host port
```

## 🤝 Contributing

Contributions are welcome! Please see [AGENTS.md](AGENTS.md) for development guidelines.

### Development Workflow

1. Review [AGENTS.md](AGENTS.md) for conventions
2. Check [STATUS.md](STATUS.md) for current project status
3. Create feature branch
4. Add service following existing patterns
5. Update `docker-compose.yml`, `STATUS.md`, and `crush.json`
6. Test thoroughly
7. Submit pull request

## 📄 License

This project is licensed under the MIT License - see [LICENSE](LICENSE) file for details.

## 🙏 Acknowledgments

- [Model Context Protocol](https://modelcontextprotocol.io/)
- [Language Server Protocol](https://microsoft.github.io/language-server-protocol/)
- [Docker](https://www.docker.com/)
- [Charmbracelet Crush](https://github.com/charmbracelet/crush)

---

**Maintained by**: KNEL Development Team
**Last Updated**: 2024-01-21