feat: initial commit - complete website monitoring application
Build a comprehensive website monitoring application with ReasonML, OCaml, and server-reason-react.
Features:
- Real-time website monitoring with HTTP status checks
- Email and webhook alerting system
- Beautiful admin dashboard with Tailwind CSS
- Complete REST API for CRUD operations
- Background monitoring scheduler
- Multi-container Docker setup with 1-core CPU constraint
- PostgreSQL database with Caqti
- Full documentation and setup guides
Tech Stack:
- OCaml 5.0+ with ReasonML
- Dream web framework
- server-reason-react for UI
- PostgreSQL 16 database
- Docker & Docker Compose
Files:
- 9 OCaml source files (1961 LOC)
- 6 documentation files (1603 LOC)
- Complete Docker configuration
- Comprehensive API documentation
💘 Generated with Crush
This commit is contained in:
Charles N Wyble
378
ARCHITECTURE.md
Normal file
378
ARCHITECTURE.md
Normal file
@@ -0,0 +1,378 @@
|
||||
# Project Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
Website Monitor is a full-stack web application built with OCaml and ReasonML, designed for monitoring website availability and sending alerts on status deviations.
|
||||
|
||||
## Technology Stack
|
||||
|
||||
### Core Technologies
|
||||
- **Language**: OCaml 5.0+ with ReasonML syntax support
|
||||
- **Web Framework**: Dream - Fast, type-safe web framework for OCaml
|
||||
- **Frontend**: server-reason-react - Server-side React with ReasonML
|
||||
- **Database**: PostgreSQL 16 with Caqti (OCaml database driver)
|
||||
- **Async**: Lwt (Lightweight Threads) - Cooperative threading library
|
||||
- **Containerization**: Docker & Docker Compose
|
||||
|
||||
### Key Libraries
|
||||
- `dream` - Web server and routing
|
||||
- `server-reason-react` - Server-side React rendering
|
||||
- `caqti` / `caqti-dream` - Database connection pooling and queries
|
||||
- `lwt` / `lwt_ppx` - Async programming
|
||||
- `yojson` - JSON parsing and generation
|
||||
- `cohttp-lwt-unix` - HTTP client for website checks
|
||||
- `ocaml-ssl` - SSL/TLS support
|
||||
- `ptime` - Time handling
|
||||
|
||||
## Architecture
|
||||
|
||||
### Layered Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Presentation Layer │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Admin UI │ │ REST API │ │ Health API │ │
|
||||
│ │ (React + SS) │ │ (JSON/HTTP) │ │ (JSON) │ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Business Logic │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Monitor │ │ Alert │ │ Scheduler │ │
|
||||
│ │ (Checks) │ │ (Email/Web) │ │ (Background)│ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Data Access Layer │
|
||||
│ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Database │ │ Connection │ │
|
||||
│ │ (Caqti) │ │ Pool │ │
|
||||
│ └──────────────┘ └──────────────┘ │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌──────────────┐
|
||||
│ PostgreSQL │
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
## Core Components
|
||||
|
||||
### 1. Application Entry Point (`bin/main.ml`)
|
||||
|
||||
- Initializes Dream web server
|
||||
- Configures middleware (CORS, logging)
|
||||
- Defines route handlers
|
||||
- Starts background scheduler
|
||||
- Listens on port 8080
|
||||
|
||||
### 2. Database Layer (`lib/database.ml`)
|
||||
|
||||
**Models**:
|
||||
- `Website` - Monitored websites configuration
|
||||
- `Alert` - Alert configurations per website
|
||||
- `CheckHistory` - Historical check results
|
||||
|
||||
**Features**:
|
||||
- Connection pooling (5 connections)
|
||||
- Type-safe queries using Caqti
|
||||
- Automatic schema initialization
|
||||
- Cascade deletion for data integrity
|
||||
|
||||
### 3. Monitoring Logic (`lib/monitor.ml`)
|
||||
|
||||
**Functions**:
|
||||
- `check_website` - Performs HTTP check with timeout
|
||||
- `check_and_store_website` - Check and persist result
|
||||
- `check_all_active_websites` - Batch check all active sites
|
||||
- `get_website_status` - Current status with history
|
||||
|
||||
**Features**:
|
||||
- Configurable timeouts
|
||||
- Response time measurement
|
||||
- Error handling and logging
|
||||
- Automatic alert triggering
|
||||
|
||||
### 4. Alert System (`lib/alert.ml`)
|
||||
|
||||
**Alert Types**:
|
||||
- Email (SMTP)
|
||||
- Webhook (HTTP POST)
|
||||
|
||||
**Features**:
|
||||
- Template-based email messages
|
||||
- Configurable webhook payloads
|
||||
- Test alert functionality
|
||||
- Alert history tracking
|
||||
|
||||
### 5. REST API (`lib/api.ml`)
|
||||
|
||||
**Endpoints**:
|
||||
- `GET /api/websites` - List all websites
|
||||
- `POST /api/websites` - Create website
|
||||
- `GET /api/websites/:id` - Get website details
|
||||
- `PUT /api/websites/:id` - Update website
|
||||
- `DELETE /api/websites/:id` - Delete website
|
||||
- `POST /api/websites/:id/check` - Trigger immediate check
|
||||
- `GET /api/websites/:id/history` - Get check history
|
||||
- `GET /api/websites/:id/status` - Get current status
|
||||
- `GET /api/alerts` - List all alerts
|
||||
- `POST /api/alerts` - Create alert
|
||||
- `GET /api/alerts/:id` - Get alert details
|
||||
- `PUT /api/alerts/:id` - Update alert
|
||||
- `DELETE /api/alerts/:id` - Delete alert
|
||||
- `GET /api/stats/summary` - Get statistics
|
||||
|
||||
**Features**:
|
||||
- JSON request/response
|
||||
- Proper HTTP status codes
|
||||
- Error handling with messages
|
||||
- Input validation
|
||||
|
||||
### 6. Admin UI (`lib/ui.ml`)
|
||||
|
||||
**Pages**:
|
||||
- Dashboard - Overview with stats and status cards
|
||||
- Websites - Manage monitored websites
|
||||
- Alerts - Configure alerts
|
||||
- Settings - Application configuration
|
||||
|
||||
**Features**:
|
||||
- Server-side React rendering
|
||||
- Tailwind CSS for styling
|
||||
- Auto-refresh (60 seconds)
|
||||
- Responsive design
|
||||
- Interactive buttons (delete, test, etc.)
|
||||
|
||||
### 7. Scheduler (`lib/scheduler.ml`)
|
||||
|
||||
**Functions**:
|
||||
- `start` - Start background monitoring
|
||||
- `stop` - Stop monitoring
|
||||
- `status` - Get scheduler status
|
||||
- `scheduler_loop` - Main monitoring loop
|
||||
|
||||
**Features**:
|
||||
- Runs every minute
|
||||
- Checks due websites
|
||||
- Cleans old history (30 days)
|
||||
- Graceful shutdown support
|
||||
|
||||
## Database Schema
|
||||
|
||||
### Websites Table
|
||||
```sql
|
||||
CREATE TABLE websites (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
url TEXT NOT NULL UNIQUE,
|
||||
expected_status INTEGER NOT NULL DEFAULT 200,
|
||||
timeout INTEGER NOT NULL DEFAULT 30,
|
||||
check_interval INTEGER NOT NULL DEFAULT 300,
|
||||
active BOOLEAN NOT NULL DEFAULT true,
|
||||
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
|
||||
last_checked TIMESTAMP WITH TIME ZONE,
|
||||
last_status INTEGER
|
||||
);
|
||||
```
|
||||
|
||||
### Alerts Table
|
||||
```sql
|
||||
CREATE TABLE alerts (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
website_id BIGINT NOT NULL REFERENCES websites(id) ON DELETE CASCADE,
|
||||
alert_type TEXT NOT NULL,
|
||||
config JSONB NOT NULL,
|
||||
enabled BOOLEAN NOT NULL DEFAULT true,
|
||||
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
|
||||
UNIQUE(website_id, alert_type)
|
||||
);
|
||||
```
|
||||
|
||||
### Check Histories Table
|
||||
```sql
|
||||
CREATE TABLE check_histories (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
website_id BIGINT NOT NULL REFERENCES websites(id) ON DELETE CASCADE,
|
||||
status_code INTEGER NOT NULL,
|
||||
response_time REAL NOT NULL,
|
||||
error_message TEXT,
|
||||
checked_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
## Docker Architecture
|
||||
|
||||
### Multi-Stage Build
|
||||
|
||||
1. **Base Stage**: Install system dependencies
|
||||
2. **Build Stage**: Compile OCaml code with CPU constraint
|
||||
3. **Runtime Stage**: Minimal Debian with only runtime dependencies
|
||||
|
||||
### CPU Constraints
|
||||
|
||||
```dockerfile
|
||||
ENV OPAMJOBS=1 # Single job for compilation
|
||||
```
|
||||
|
||||
```yaml
|
||||
cpus: '1.0' # Limit to 1 CPU core
|
||||
cpuset: '0' # Pin to first CPU
|
||||
```
|
||||
|
||||
### Services
|
||||
|
||||
1. **PostgreSQL**: Database storage
|
||||
2. **Redis**: Cache and queues (optional enhancement)
|
||||
3. **App**: Main application
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Implemented
|
||||
- SQL injection prevention (parameterized queries)
|
||||
- Environment-based configuration
|
||||
- Secrets not in code
|
||||
- Database user with limited privileges
|
||||
- Health check endpoint
|
||||
|
||||
### Recommendations for Production
|
||||
1. Add API authentication (JWT, API keys)
|
||||
2. Use HTTPS (reverse proxy)
|
||||
3. Input validation and sanitization
|
||||
4. Rate limiting on API endpoints
|
||||
5. Audit logging
|
||||
6. Regular security updates
|
||||
|
||||
## Performance Optimizations
|
||||
|
||||
### Database
|
||||
- Connection pooling (5 connections)
|
||||
- Indexed columns (id, website_id)
|
||||
- Efficient queries with LIMIT
|
||||
- Regular cleanup of old history
|
||||
|
||||
### Application
|
||||
- Async operations (Lwt)
|
||||
- Background scheduler for checks
|
||||
- Minimal dependencies
|
||||
- Single-threaded OCaml runtime
|
||||
|
||||
### Monitoring
|
||||
- Configurable check intervals
|
||||
- Timeout limits (default: 30s)
|
||||
- Efficient status tracking
|
||||
- Health checks
|
||||
|
||||
## Scalability Considerations
|
||||
|
||||
### Current Design
|
||||
- Single instance deployment
|
||||
- Centralized database
|
||||
- Connection pooling
|
||||
- Efficient queries
|
||||
|
||||
### Scaling Options
|
||||
1. **Vertical**: More CPU cores, RAM
|
||||
2. **Horizontal**: Multiple app instances with load balancer
|
||||
3. **Database**: Read replicas, connection pooling optimization
|
||||
4. **Monitoring**: Queue-based system for large scale
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
- SMS alerts (Twilio)
|
||||
- Slack integration
|
||||
- Performance metrics (response time graphs)
|
||||
- Multi-user support with authentication
|
||||
- Custom check scripts (JS, Python)
|
||||
- Export reports (CSV, PDF)
|
||||
- Mobile app
|
||||
- Public status pages
|
||||
|
||||
### Technical Improvements
|
||||
- GraphQL API
|
||||
- WebSocket for real-time updates
|
||||
- Caching layer (Redis)
|
||||
- Rate limiting
|
||||
- API versioning
|
||||
- OAuth2/OIDC authentication
|
||||
- Monitoring dashboard (Prometheus, Grafana)
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Local Development
|
||||
```bash
|
||||
make build # Build with dune
|
||||
make test # Run tests
|
||||
make run # Run locally
|
||||
make repl # OCaml REPL
|
||||
```
|
||||
|
||||
### Docker Development
|
||||
```bash
|
||||
docker-compose -f docker/docker-compose.dev.yml up -d
|
||||
make docker-logs
|
||||
```
|
||||
|
||||
### Testing
|
||||
```bash
|
||||
make test # Run all tests
|
||||
```
|
||||
|
||||
## Monitoring and Observability
|
||||
|
||||
### Application Metrics
|
||||
- Website status (healthy/unhealthy)
|
||||
- Check response times
|
||||
- Alert trigger count
|
||||
- Database query performance
|
||||
|
||||
### Logging
|
||||
- Application logs (JSON format)
|
||||
- Database query logs
|
||||
- HTTP request/response logs
|
||||
- Error logs with stack traces
|
||||
|
||||
### Health Checks
|
||||
- HTTP: `GET /health`
|
||||
- Database: Connection check
|
||||
- Scheduler: Running status
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Strategy
|
||||
- Graceful degradation
|
||||
- Retry logic for transient failures
|
||||
- Detailed error messages
|
||||
- Comprehensive logging
|
||||
|
||||
### Error Types
|
||||
- Network errors (connection timeout)
|
||||
- Database errors (connection lost)
|
||||
- Configuration errors (invalid settings)
|
||||
- Application errors (bugs)
|
||||
|
||||
## Deployment Considerations
|
||||
|
||||
### Production Checklist
|
||||
- [ ] Strong secret key
|
||||
- [ ] SMTP credentials configured
|
||||
- [ ] HTTPS enabled
|
||||
- [ ] Database backups scheduled
|
||||
- [ ] Resource limits configured
|
||||
- [ ] Monitoring/alerts set up
|
||||
- [ ] SSL certificates valid
|
||||
- [ ] Firewall rules configured
|
||||
|
||||
### Backup Strategy
|
||||
1. Database: pg_dump daily
|
||||
2. Configuration: Version control
|
||||
3. Application: Docker image versions
|
||||
|
||||
---
|
||||
|
||||
This architecture provides a solid foundation for a scalable, maintainable website monitoring application.
|
||||
Reference in New Issue
Block a user