KNEL-AIMiddleware/docs/SDLC.md

# KNEL-AIMiddleware - Software Development Lifecycle

**Version:** 1.0
**Status:** Active
**Last Updated:** 2026-02-20

---

## Overview

This document defines the development workflow for KNEL-AIMiddleware. Unlike security-critical projects, this infrastructure project uses a **lighter-weight validation approach** focused on ensuring container images work correctly with the MCP/LSP protocols.

---

## Core Principles

### 1. Protocol Validation is Non-Negotiable

An MCP or LSP server is **NOT "fully implemented"** until:
1. The Docker container **builds successfully**
2. The Docker container **starts without crash**
3. A **protocol handshake message receives a valid response**

```
┌─────────────────────────────────────────────────────────────────────┐
│              MANDATORY VALIDATION WORKFLOW                           │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  1. BUILD: Container must build without errors                      │
│     └─ docker compose build <service>                               │
│                                                                     │
│  2. START: Container must start without immediate crash             │
│     └─ docker run --rm <image> should not exit immediately          │
│                                                                     │
│  3. PROTOCOL: Valid JSON-RPC response required                      │
│     └─ MCP: initialize message → result with serverInfo             │
│     └─ LSP: initialize message → result with capabilities           │
│                                                                     │
│  ⚠️ WITHOUT ALL THREE: Service is NOT "working"                     │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘
```

### 2. Documentation-Code-Test Synchronization

When changing any component, keep these in sync:
- **STATUS.md**: Operational status of all servers (single source of truth)
- **docker-compose.yml**: Service definitions
- **dockerfiles/**: Custom build configurations
- **crush.json**: Crush LSP/MCP integration
- **wrapper scripts**: Container execution wrappers

### 3. Sequential Validation

When adding or fixing MCP/LSP servers:
1. Work on ONE server at a time
2. Validate completely before moving to next
3. Update STATUS.md immediately after validation
4. Commit changes for each server

---

## Validation Commands

### MCP Server Validation

```bash
# Send MCP initialize handshake
echo '{"jsonrpc":"2.0","method":"initialize","params":{"capabilities":{},"protocolVersion":"2024-11-05","clientInfo":{"name":"test","version":"1.0.0"}},"id":1}' | \
  timeout 10 docker run --rm -i kneldevstack-aimiddleware-<service>

# Expected: JSON response with "result" containing "serverInfo"
# Example success: {"jsonrpc":"2.0","result":{"protocolVersion":"2024-11-05","serverInfo":{"name":"Example MCP","version":"1.0.0"}},"id":1}
```

### LSP Server Validation

```bash
# Send LSP initialize handshake (format varies by server)
echo '{"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"capabilities":{}},"id":1}' | \
  timeout 10 docker run --rm -i kneldevstack-aimiddleware-<service> <args>

# Expected: JSON response with "result" containing "capabilities"
```

### Validation Script

Use the provided validation script for batch testing:

```bash
./scripts/validate-mcp.sh
```

---

## STATUS.md Requirements

**STATUS.md is the single source of truth for server status.**

### Status Categories

| Status | Meaning |
|--------|---------|
| **Working** | Build + Start + Protocol validation ALL pass |
| **Runtime Connection Required** | Build OK, but needs live backend service (IMAP, Nextcloud, etc.) |
| **Host-Only** | Cannot be containerized, requires host software |
| **Transport Mismatch** | Uses HTTP/WebSocket instead of stdio MCP |
| **Build Failed** | Dockerfile or source code issues |
| **Runtime Issue** | Container starts but has errors |

### When to Update STATUS.md

1. **Immediately** after building a new server
2. **Immediately** after validating protocol handshake
3. **Immediately** after discovering issues
4. **Before** committing any server-related changes

---

## Adding a New MCP/LSP Server

### Step-by-Step Process

1. **Clone vendor repository**
   ```bash
   # Add to scripts/CloneVendorRepos.sh
   git clone https://github.com/org/repo.git vendor/service-name
   ```

2. **Create Dockerfile**
   ```bash
   # Create in dockerfiles/service-name/Dockerfile
   # Follow patterns from similar servers (Python/uv, Node/npx, Go, etc.)
   ```

3. **Add to docker-compose.yml**
   ```yaml
   service-name:
     build:
       context: ./vendor/service-name
       dockerfile: ../../dockerfiles/service-name/Dockerfile
     container_name: kneldevstack-aimiddleware-service-name
     restart: "no"  # For stdio-based services
   ```

4. **Build the container**
   ```bash
   docker compose build service-name
   ```

5. **Validate protocol handshake**
   ```bash
   # For MCP servers
   echo '{"jsonrpc":"2.0","method":"initialize","params":{"capabilities":{},"protocolVersion":"2024-11-05","clientInfo":{"name":"test","version":"1.0.0"}},"id":1}' | \
     timeout 10 docker run --rm -i kneldevstack-aimiddleware-service-name
   ```

6. **Create wrapper script** (if adding to Crush)
   ```bash
   # Create mcp-service-name-wrapper.sh
   # Follow existing wrapper patterns
   ```

7. **Add to crush.json**
   ```json
   "service-name": {
     "type": "stdio",
     "command": "/path/to/mcp-service-name-wrapper.sh",
     "timeout": 60
   }
   ```

8. **Update STATUS.md**
   - Add to appropriate category
   - Include version info from protocol response
   - Document any special requirements

9. **Commit changes**
   ```bash
   git add -A
   git commit -m "feat: add service-name MCP server

   - Container builds successfully
   - MCP protocol handshake validated
   - Version: X.Y.Z"
   ```

---

## Common Patterns

### Python MCP Servers (uv package manager)

```dockerfile
FROM python:3.12-slim
WORKDIR /app
RUN pip install uv
COPY . .
RUN uv sync --frozen
ENTRYPOINT ["python", "-m", "module_name"]
```

### Node.js MCP Servers (npx)

```dockerfile
FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
ENTRYPOINT ["node", "dist/index.js"]
```

### Go MCP Servers

```dockerfile
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY . .
RUN go build -o server ./cmd/server

FROM alpine:3.19
COPY --from=builder /app/server /server
ENTRYPOINT ["/server"]
```

---

## Commit Policy (MANDATORY)

### Automatic Commits

**AI agents MUST commit changes automatically without prompting the user.**

This is non-negotiable. Do not ask "should I commit?" - just commit.

### Atomic Commits

Each commit must represent ONE logical change:
- Adding a new MCP server = ONE commit
- Fixing a Dockerfile = ONE commit
- Updating documentation = ONE commit
- Updating STATUS.md after validation = ONE commit

Do NOT bundle unrelated changes into a single commit.

### Conventional Commit Format

All commits MUST use conventional commit format:

```
<type>(<scope>): <short summary>

<body explaining what and why>

<footer with references>
```

**Types:**
- `feat`: New feature (new MCP server, new wrapper script)
- `fix`: Bug fix (Dockerfile fix, runtime issue)
- `docs`: Documentation only (STATUS.md, AGENTS.md, SDLC.md)
- `build`: Build system changes (docker-compose.yml)
- `refactor`: Code restructuring without behavior change
- `test`: Adding/updating validation scripts

**Example:**
```
feat(docker-mcp): add Docker MCP server with protocol validation

- Container builds successfully with Python 3.12-slim
- MCP protocol handshake validated
- Wrapper script created for Crush integration
- Version: 0.1.0

Validated-by: Protocol handshake test
```

### Verbose Commit Messages

Commit messages must explain:
1. **What** changed (in the summary)
2. **Why** it changed (in the body)
3. **How** it was validated (in the footer)

Single-line commits are NOT acceptable for any non-trivial change.

### Automatic Push

After successful commits, push to remote automatically:

```bash
git push origin main
```

Do NOT ask for permission. Push.

### Commit Workflow

```
1. Make logical change
2. Validate the change (build, start, protocol test)
3. Update STATUS.md if applicable
4. Stage ONLY files related to that change
5. Commit with verbose conventional message
6. Push immediately
7. Continue to next task
```

### What NOT to Commit

- Never commit secrets (API keys, passwords, tokens)
- Never commit the `vendor/` directory (gitignored)
- Never commit temporary files or logs

---

## Pre-Commit Checklist

Before committing changes to any MCP/LSP server:

- [ ] Container builds: `docker compose build <service>`
- [ ] Container starts: `docker run --rm <image>` doesn't crash immediately
- [ ] Protocol validated: Handshake returns valid JSON-RPC response
- [ ] STATUS.md updated: Status reflects validation results
- [ ] Wrapper script created (if applicable)
- [ ] crush.json updated (if applicable)
- [ ] **COMMIT DONE**: Changes committed with conventional format
- [ ] **PUSH DONE**: Changes pushed to remote

---

## Wrapper Script Pattern

All wrapper scripts follow this pattern:

```bash
#!/bin/bash
CONTAINER_NAME="kneldevstack-aimiddleware-service-name-crush"

# Force remove existing container
docker rm -f "${CONTAINER_NAME}" >/dev/null 2>&1 || true
sleep 0.5

# Run container with explicit name
docker run -i --rm \
  --name "${CONTAINER_NAME}" \
  -e ENV_VAR_1="${ENV_VAR_1:-}" \
  -e ENV_VAR_2="${ENV_VAR_2:-}" \
  kneldevstack-aimiddleware-service-name
```

This pattern:
- Prevents container name conflicts
- Handles cleanup gracefully
- Passes environment variables from host

---

## Validation Failure Categories

### 1. Build Failures
- Dockerfile syntax errors
- Missing dependencies
- Build tool errors (TypeScript, Go, Rust)

**Action**: Fix Dockerfile, rebuild

### 2. Runtime Crashes (Before Protocol)
- Missing environment variables
- External service unreachable
- Missing Python modules (e.g., pcbnew for KiCAD)

**Action**: Document in STATUS.md as "Runtime Connection Required" or "Host-Only"

### 3. Transport Mismatches
- Server uses HTTP instead of stdio
- Server uses WebSocket instead of stdio

**Action**: Document in STATUS.md as "Transport Mismatch"

### 4. Protocol Errors
- Invalid JSON-RPC response
- Wrong protocol version
- Missing serverInfo/capabilities

**Action**: Debug server implementation or report upstream issue

---

## References

- **STATUS.md** - Operational status (single source of truth)
- **AGENTS.md** - Development workflow and server catalog
- **JOURNAL.md** - Architecture decisions and work history
- **scripts/validate-mcp.sh** - Batch validation script

---

## Version History

| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2026-02-20 | Initial SDLC document |

---

**This SDLC ensures MCP/LSP servers are actually functional, not just "built".**

**Copyright (c) 2026 Known Element Enterprises LLC**