diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md new file mode 100644 index 0000000..9970993 --- /dev/null +++ b/DEPLOYMENT.md @@ -0,0 +1,336 @@ +# TSYS FetchApply Deployment Guide + +## Overview + +This guide provides comprehensive instructions for deploying the TSYS FetchApply infrastructure provisioning system on Linux servers. + +## Prerequisites + +### System Requirements +- **Operating System:** Ubuntu 18.04+ or Debian 10+ (recommended) +- **RAM:** Minimum 2GB, recommended 4GB +- **Disk Space:** Minimum 10GB free space +- **Network:** Internet connectivity for package downloads +- **Privileges:** Root or sudo access required + +### Required Tools +- `git` - Version control system +- `curl` - HTTP client for downloads +- `wget` - Alternative download tool +- `systemctl` - System service management +- `apt-get` - Package management (Debian/Ubuntu) + +### Network Requirements +- **HTTPS access** to: + - `https://archive.ubuntu.com` (Ubuntu packages) + - `https://linux.dell.com` (Dell hardware support) + - `https://download.proxmox.com` (Proxmox packages) + - `https://github.com` (Git repositories) + +## Pre-Deployment Validation + +### 1. System Compatibility Check +```bash +# Clone repository +git clone [repository-url] +cd FetchApply + +# Run system validation +./Project-Tests/validation/system-requirements.sh +``` + +### 2. Network Connectivity Test +```bash +# Test network connectivity +curl -I https://archive.ubuntu.com +curl -I https://linux.dell.com +curl -I https://download.proxmox.com +``` + +### 3. Permission Verification +```bash +# Verify write permissions +test -w /etc && echo "โœ… /etc writable" || echo "โŒ /etc not writable" +test -w /usr/local/bin && echo "โœ… /usr/local/bin writable" || echo "โŒ /usr/local/bin not writable" +``` + +## Deployment Methods + +### Method 1: Standard Deployment (Recommended) +```bash +# 1. Clone repository +git clone [repository-url] +cd FetchApply + +# 2. Run pre-deployment tests +./Project-Tests/run-tests.sh validation + +# 3. Execute deployment +cd ProjectCode +sudo bash SetupNewSystem.sh +``` + +### Method 2: Dry Run Mode +```bash +# 1. Clone repository +git clone [repository-url] +cd FetchApply + +# 2. Review configuration +cat ProjectCode/SetupNewSystem.sh + +# 3. Execute with manual review +cd ProjectCode +sudo bash -x SetupNewSystem.sh # Debug mode +``` + +## Deployment Process + +### Phase 1: Framework Initialization +1. **Environment Setup** + - Load framework variables + - Source framework includes + - Initialize logging system + +2. **System Detection** + - Detect physical vs virtual hardware + - Identify operating system + - Check for existing users + +### Phase 2: Base System Configuration +1. **Package Installation** + - Update package repositories + - Install essential packages + - Configure package sources + +2. **User Management** + - Create required user accounts + - Configure SSH access + - Set up sudo permissions + +### Phase 3: Security Hardening +1. **SSH Configuration** + - Deploy hardened SSH configuration + - Install SSH keys + - Disable password authentication + +2. **System Hardening** + - Configure firewall rules + - Enable audit logging + - Install security tools + +### Phase 4: Monitoring and Management +1. **Monitoring Agents** + - Deploy LibreNMS agents + - Configure SNMP + - Set up system monitoring + +2. **Management Tools** + - Install Cockpit dashboard + - Configure remote access + - Set up maintenance scripts + +## Post-Deployment Verification + +### 1. Security Validation +```bash +# Run security tests +./Project-Tests/run-tests.sh security + +# Verify SSH configuration +ssh -T [server-ip] # Should work with key authentication +``` + +### 2. Service Status Check +```bash +# Check critical services +sudo systemctl status ssh +sudo systemctl status auditd +sudo systemctl status snmpd +``` + +### 3. Network Connectivity +```bash +# Test internal services +curl -k https://localhost:9090 # Cockpit +snmpwalk -v2c -c public localhost system +``` + +## Troubleshooting + +### Common Issues + +#### 1. Permission Denied Errors +```bash +# Solution: Run with sudo +sudo bash SetupNewSystem.sh +``` + +#### 2. Network Connectivity Issues +```bash +# Check DNS resolution +nslookup archive.ubuntu.com + +# Test direct IP access +curl -I 91.189.91.26 # Ubuntu archive IP +``` + +#### 3. Package Installation Failures +```bash +# Update package cache +sudo apt-get update + +# Fix broken packages +sudo apt-get -f install +``` + +#### 4. SSH Key Issues +```bash +# Verify key permissions +ls -la ~/.ssh/ +chmod 600 ~/.ssh/id_rsa +chmod 644 ~/.ssh/id_rsa.pub +``` + +### Debug Mode +```bash +# Enable debug logging +export DEBUG=1 +bash -x SetupNewSystem.sh +``` + +### Log Analysis +```bash +# Check deployment logs +tail -f /var/log/fetchapply/deployment.log + +# Review system logs +journalctl -u ssh +journalctl -u auditd +``` + +## Environment-Specific Configurations + +### Physical Dell Servers +- **OMSA Installation:** Dell OpenManage Server Administrator +- **Hardware Monitoring:** iDRAC configuration +- **Performance Tuning:** CPU and memory optimizations + +### Virtual Machines +- **Guest Additions:** VMware tools or VirtualBox additions +- **Resource Limits:** Memory and CPU constraints +- **Network Configuration:** Bridge vs NAT settings + +### Development Environments +- **SSH Configuration:** Less restrictive settings +- **Development Tools:** Additional packages for development +- **Testing Access:** Enhanced logging and debugging + +## Maintenance and Updates + +### Regular Maintenance +```bash +# Update system packages +sudo apt-get update && sudo apt-get upgrade + +# Update monitoring scripts +cd /usr/local/bin +sudo wget https://[repository]/scripts/up2date.sh +sudo chmod +x up2date.sh +``` + +### Security Updates +```bash +# Check for security updates +sudo apt-get update +sudo apt list --upgradable | grep -i security + +# Apply security patches +sudo apt-get upgrade +``` + +### Configuration Updates +```bash +# Update FetchApply +cd FetchApply +git pull origin main + +# Re-run specific modules +cd ProjectCode/Modules/Security +sudo bash secharden-ssh.sh +``` + +## Best Practices + +### 1. Pre-Deployment +- Always test in non-production environment first +- Review all scripts before execution +- Validate network connectivity +- Ensure proper backup procedures + +### 2. During Deployment +- Monitor deployment progress +- Check for errors and warnings +- Document any customizations +- Validate each phase completion + +### 3. Post-Deployment +- Run full security test suite +- Verify all services are running +- Test remote access +- Document deployment specifics + +### 4. Ongoing Operations +- Regular security updates +- Monitor system performance +- Review audit logs +- Maintain deployment documentation + +## Support and Resources + +### Documentation +- **README.md:** Basic usage instructions +- **SECURITY.md:** Security architecture and guidelines +- **Project-Tests/README.md:** Testing framework documentation + +### Community Support +- **Issues:** https://projects.knownelement.com/project/reachableceo-vptechnicaloperations/timeline +- **Discussion:** https://community.turnsys.com/c/chieftechnologyandproductofficer/26 + +### Professional Support +- **Technical Support:** [Contact information to be added] +- **Consulting Services:** [Contact information to be added] + +## Deployment Checklist + +### Pre-Deployment +- [ ] System requirements validated +- [ ] Network connectivity tested +- [ ] Backup procedures in place +- [ ] Security review completed + +### Deployment +- [ ] Repository cloned successfully +- [ ] Pre-deployment tests passed +- [ ] Deployment executed without errors +- [ ] Post-deployment verification completed + +### Post-Deployment +- [ ] Security tests passed +- [ ] All services running +- [ ] Remote access verified +- [ ] Documentation updated + +### Maintenance +- [ ] Update schedule established +- [ ] Monitoring configured +- [ ] Backup procedures tested +- [ ] Incident response plan activated + +## Version History + +- **v1.0:** Initial deployment framework +- **v1.1:** Added security hardening and secrets management +- **v1.2:** Enhanced testing framework and documentation + +Last updated: July 14, 2025 \ No newline at end of file diff --git a/Framework-Includes/ConfigValidation.sh b/Framework-Includes/ConfigValidation.sh new file mode 100755 index 0000000..cd114c6 --- /dev/null +++ b/Framework-Includes/ConfigValidation.sh @@ -0,0 +1,261 @@ +#!/bin/bash + +# Configuration Validation Framework +# Pre-flight checks for system compatibility and requirements + +set -euo pipefail + +# Source framework dependencies +source "$(dirname "${BASH_SOURCE[0]}")/PrettyPrint.sh" 2>/dev/null || echo "Warning: PrettyPrint.sh not found" +source "$(dirname "${BASH_SOURCE[0]}")/Logging.sh" 2>/dev/null || echo "Warning: Logging.sh not found" + +# Configuration validation settings +declare -g VALIDATION_FAILED=0 +declare -g VALIDATION_WARNINGS=0 + +# System requirements +declare -g MIN_RAM_GB=2 +declare -g MIN_DISK_GB=10 +declare -g REQUIRED_COMMANDS=("curl" "wget" "git" "systemctl" "apt-get" "dmidecode") + +# Network endpoints to validate +declare -g REQUIRED_ENDPOINTS=( + "https://archive.ubuntu.com" + "https://linux.dell.com" + "https://download.proxmox.com" + "https://github.com" +) + +# Validation functions +function validate_system_requirements() { + print_info "Validating system requirements..." + + # Check RAM + local total_mem_kb=$(grep MemTotal /proc/meminfo | awk '{print $2}') + local total_mem_gb=$((total_mem_kb / 1024 / 1024)) + + if [[ $total_mem_gb -ge $MIN_RAM_GB ]]; then + print_success "RAM requirement met: ${total_mem_gb}GB >= ${MIN_RAM_GB}GB" + else + print_error "RAM requirement not met: ${total_mem_gb}GB < ${MIN_RAM_GB}GB" + ((VALIDATION_FAILED++)) + fi + + # Check disk space + local available_gb=$(df / | tail -1 | awk '{print int($4/1024/1024)}') + + if [[ $available_gb -ge $MIN_DISK_GB ]]; then + print_success "Disk space requirement met: ${available_gb}GB >= ${MIN_DISK_GB}GB" + else + print_error "Disk space requirement not met: ${available_gb}GB < ${MIN_DISK_GB}GB" + ((VALIDATION_FAILED++)) + fi +} + +function validate_required_commands() { + print_info "Validating required commands..." + + for cmd in "${REQUIRED_COMMANDS[@]}"; do + if command -v "$cmd" >/dev/null 2>&1; then + print_success "Required command available: $cmd" + else + print_error "Required command missing: $cmd" + ((VALIDATION_FAILED++)) + fi + done +} + +function validate_os_compatibility() { + print_info "Validating OS compatibility..." + + if [[ -f /etc/os-release ]]; then + local os_id=$(grep "^ID=" /etc/os-release | cut -d'=' -f2 | tr -d '"') + local os_version=$(grep "^VERSION_ID=" /etc/os-release | cut -d'=' -f2 | tr -d '"') + + case "$os_id" in + ubuntu) + if [[ "${os_version%%.*}" -ge 18 ]]; then + print_success "OS compatibility: Ubuntu $os_version (fully supported)" + else + print_warning "OS compatibility: Ubuntu $os_version (may have issues)" + ((VALIDATION_WARNINGS++)) + fi + ;; + debian) + if [[ "${os_version%%.*}" -ge 10 ]]; then + print_success "OS compatibility: Debian $os_version (fully supported)" + else + print_warning "OS compatibility: Debian $os_version (may have issues)" + ((VALIDATION_WARNINGS++)) + fi + ;; + *) + print_warning "OS compatibility: $os_id $os_version (not tested, may work)" + ((VALIDATION_WARNINGS++)) + ;; + esac + else + print_error "Cannot determine OS version" + ((VALIDATION_FAILED++)) + fi +} + +function validate_network_connectivity() { + print_info "Validating network connectivity..." + + for endpoint in "${REQUIRED_ENDPOINTS[@]}"; do + if curl -s --connect-timeout 10 --max-time 30 --head "$endpoint" >/dev/null 2>&1; then + print_success "Network connectivity: $endpoint" + else + print_error "Network connectivity failed: $endpoint" + ((VALIDATION_FAILED++)) + fi + done +} + +function validate_permissions() { + print_info "Validating system permissions..." + + local required_dirs=("/etc" "/usr/local/bin" "/var/log") + + for dir in "${required_dirs[@]}"; do + if [[ -w "$dir" ]]; then + print_success "Write permission: $dir" + else + print_error "Write permission denied: $dir (run with sudo)" + ((VALIDATION_FAILED++)) + fi + done +} + +function validate_conflicting_software() { + print_info "Checking for conflicting software..." + + # Check for conflicting SSH configurations + if [[ -f /etc/ssh/sshd_config ]]; then + if grep -q "^PasswordAuthentication yes" /etc/ssh/sshd_config; then + print_warning "SSH password authentication is enabled (will be disabled)" + ((VALIDATION_WARNINGS++)) + fi + fi + + # Check for conflicting firewall rules + if command -v ufw >/dev/null 2>&1; then + if ufw status | grep -q "Status: active"; then + print_warning "UFW firewall is active (may conflict with iptables rules)" + ((VALIDATION_WARNINGS++)) + fi + fi + + # Check for conflicting SNMP configurations + if systemctl is-active snmpd >/dev/null 2>&1; then + print_warning "SNMP service is already running (will be reconfigured)" + ((VALIDATION_WARNINGS++)) + fi +} + +function validate_hardware_compatibility() { + print_info "Validating hardware compatibility..." + + # Check if this is a Dell server + if [[ "$IS_PHYSICAL_HOST" -gt 0 ]]; then + print_info "Dell physical server detected - OMSA will be installed" + else + print_info "Virtual machine detected - hardware-specific tools will be skipped" + fi + + # Check for virtualization + if grep -q "hypervisor" /proc/cpuinfo; then + print_info "Virtualization detected - optimizations will be applied" + fi +} + +function validate_existing_users() { + print_info "Validating user configuration..." + + # Check for existing users + if [[ "$LOCALUSER_CHECK" -gt 0 ]]; then + print_info "User 'localuser' already exists" + else + print_info "User 'localuser' will be created" + fi + + if [[ "$SUBODEV_CHECK" -gt 0 ]]; then + print_info "User 'subodev' already exists" + else + print_info "User 'subodev' will be created" + fi +} + +function validate_security_requirements() { + print_info "Validating security requirements..." + + # Check if running as root + if [[ $EUID -eq 0 ]]; then + print_success "Running with root privileges" + else + print_error "Must run with root privileges (use sudo)" + ((VALIDATION_FAILED++)) + fi + + # Check for existing SSH keys + if [[ -f ~/.ssh/id_rsa ]]; then + print_warning "SSH keys already exist - will be preserved" + ((VALIDATION_WARNINGS++)) + fi + + # Check for secure boot + if [[ -d /sys/firmware/efi/efivars ]]; then + print_info "UEFI system detected" + if mokutil --sb-state 2>/dev/null | grep -q "SecureBoot enabled"; then + print_warning "Secure Boot is enabled - may affect kernel modules" + ((VALIDATION_WARNINGS++)) + fi + fi +} + +# Main validation function +function run_configuration_validation() { + print_header "Configuration Validation" + + # Reset counters + VALIDATION_FAILED=0 + VALIDATION_WARNINGS=0 + + # Run all validation checks + validate_system_requirements + validate_required_commands + validate_os_compatibility + validate_network_connectivity + validate_permissions + validate_conflicting_software + validate_hardware_compatibility + validate_existing_users + validate_security_requirements + + # Summary + print_header "Validation Summary" + + if [[ $VALIDATION_FAILED -eq 0 ]]; then + print_success "All validation checks passed" + if [[ $VALIDATION_WARNINGS -gt 0 ]]; then + print_warning "$VALIDATION_WARNINGS warnings - deployment may continue" + fi + return 0 + else + print_error "$VALIDATION_FAILED validation checks failed" + if [[ $VALIDATION_WARNINGS -gt 0 ]]; then + print_warning "$VALIDATION_WARNINGS additional warnings" + fi + print_error "Please resolve the above issues before deployment" + return 1 + fi +} + +# Export functions for use in other scripts +export -f validate_system_requirements +export -f validate_required_commands +export -f validate_os_compatibility +export -f validate_network_connectivity +export -f validate_permissions +export -f run_configuration_validation \ No newline at end of file diff --git a/Project-Tests/README.md b/Project-Tests/README.md new file mode 100644 index 0000000..79f4a15 --- /dev/null +++ b/Project-Tests/README.md @@ -0,0 +1,176 @@ +# TSYS FetchApply Testing Framework + +## Overview + +This testing framework provides comprehensive validation for the TSYS FetchApply infrastructure provisioning system. It includes unit tests, integration tests, security tests, and system validation. + +## Test Categories + +### 1. Unit Tests (`unit/`) +- **Purpose:** Test individual framework functions and components +- **Scope:** Framework includes, helper functions, syntax validation +- **Example:** `framework-functions.sh` - Tests logging, pretty print, and error handling functions + +### 2. Integration Tests (`integration/`) +- **Purpose:** Test complete workflows and module interactions +- **Scope:** End-to-end deployment scenarios, module integration +- **Future:** Module interaction testing, deployment workflow validation + +### 3. Security Tests (`security/`) +- **Purpose:** Validate security configurations and practices +- **Scope:** HTTPS enforcement, deployment security, SSH hardening +- **Example:** `https-enforcement.sh` - Validates all URLs use HTTPS + +### 4. Validation Tests (`validation/`) +- **Purpose:** System compatibility and pre-flight checks +- **Scope:** System requirements, network connectivity, permissions +- **Example:** `system-requirements.sh` - Validates minimum system requirements + +## Usage + +### Run All Tests +```bash +./Project-Tests/run-tests.sh +``` + +### Run Specific Test Categories +```bash +./Project-Tests/run-tests.sh unit # Unit tests only +./Project-Tests/run-tests.sh integration # Integration tests only +./Project-Tests/run-tests.sh security # Security tests only +./Project-Tests/run-tests.sh validation # Validation tests only +``` + +### Run Individual Tests +```bash +./Project-Tests/validation/system-requirements.sh +./Project-Tests/security/https-enforcement.sh +./Project-Tests/unit/framework-functions.sh +``` + +## Test Results + +- **Console Output:** Real-time test results with color-coded status +- **JSON Reports:** Detailed test reports saved to `logs/tests/` +- **Exit Codes:** 0 for success, 1 for failures + +## Configuration Validation + +The validation framework performs pre-flight checks to ensure system compatibility: + +### System Requirements +- **Memory:** Minimum 2GB RAM +- **Disk Space:** Minimum 10GB available +- **OS Compatibility:** Ubuntu/Debian (tested), others (may work) + +### Network Connectivity +- Tests connection to required download sources +- Validates HTTPS endpoints are accessible +- Checks for firewall/proxy issues + +### Command Dependencies +- Verifies required tools are installed (`curl`, `wget`, `git`, `systemctl`, `apt-get`) +- Checks for proper versions where applicable + +### Permissions +- Validates write access to system directories +- Checks for required administrative privileges + +## Adding New Tests + +### Test File Structure +```bash +#!/bin/bash +set -euo pipefail + +function test_something() { + echo "๐Ÿ” Testing something..." + + if [[ condition ]]; then + echo "โœ… Test passed" + return 0 + else + echo "โŒ Test failed" + return 1 + fi +} + +function main() { + echo "๐Ÿงช Running Test Suite Name" + echo "==========================" + + local total_failures=0 + test_something || ((total_failures++)) + + echo "==========================" + if [[ $total_failures -eq 0 ]]; then + echo "โœ… All tests passed" + exit 0 + else + echo "โŒ $total_failures tests failed" + exit 1 + fi +} + +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + main "$@" +fi +``` + +### Test Categories Guidelines + +- **Unit Tests:** Focus on individual functions, fast execution +- **Integration Tests:** Test module interactions, longer execution +- **Security Tests:** Validate security configurations +- **Validation Tests:** Pre-flight system checks + +## Continuous Integration + +The testing framework is designed to integrate with CI/CD pipelines: + +```bash +# Example CI script +./Project-Tests/run-tests.sh all +test_exit_code=$? + +if [[ $test_exit_code -eq 0 ]]; then + echo "All tests passed - deployment approved" +else + echo "Tests failed - deployment blocked" + exit 1 +fi +``` + +## Test Development Best Practices + +1. **Clear Test Names:** Use descriptive function names +2. **Proper Exit Codes:** Return 0 for success, 1 for failure +3. **Informative Output:** Use emoji and clear messages +4. **Timeout Protection:** Use timeout for network operations +5. **Cleanup:** Remove temporary files and resources +6. **Error Handling:** Use `set -euo pipefail` for strict error handling + +## Troubleshooting + +### Common Issues + +- **Permission Denied:** Run tests with appropriate privileges +- **Network Timeouts:** Check firewall and proxy settings +- **Missing Dependencies:** Install required tools before testing +- **Script Errors:** Validate syntax with `bash -n script.sh` + +### Debug Mode +```bash +# Enable debug output +export DEBUG=1 +./Project-Tests/run-tests.sh +``` + +## Contributing + +When adding new functionality to FetchApply: + +1. Add corresponding tests in appropriate category +2. Run full test suite before committing +3. Update documentation for new test cases +4. Ensure tests pass in clean environment \ No newline at end of file diff --git a/Project-Tests/run-tests.sh b/Project-Tests/run-tests.sh new file mode 100755 index 0000000..6a34df0 --- /dev/null +++ b/Project-Tests/run-tests.sh @@ -0,0 +1,128 @@ +#!/bin/bash + +# TSYS FetchApply Testing Framework +# Main test runner script + +set -euo pipefail + +# Source framework includes +PROJECT_ROOT="$(dirname "$(realpath "${BASH_SOURCE[0]}")")/.." +source "$PROJECT_ROOT/Framework-Includes/Logging.sh" +source "$PROJECT_ROOT/Framework-Includes/PrettyPrint.sh" + +# Test configuration +TEST_LOG_DIR="$PROJECT_ROOT/logs/tests" +TEST_RESULTS_FILE="$TEST_LOG_DIR/test-results-$(date +%Y%m%d-%H%M%S).json" + +# Ensure test log directory exists +mkdir -p "$TEST_LOG_DIR" + +# Test counters +declare -g TESTS_PASSED=0 +declare -g TESTS_FAILED=0 +declare -g TESTS_SKIPPED=0 + +# Test runner functions +function run_test_suite() { + local suite_name="$1" + local test_dir="$2" + + print_header "Running $suite_name Tests" + + if [[ ! -d "$test_dir" ]]; then + print_warning "Test directory $test_dir not found, skipping" + return 0 + fi + + for test_file in "$test_dir"/*.sh; do + if [[ -f "$test_file" ]]; then + run_single_test "$test_file" + fi + done +} + +function run_single_test() { + local test_file="$1" + local test_name="$(basename "$test_file" .sh)" + + print_info "Running test: $test_name" + + if timeout 300 bash "$test_file"; then + print_success "โœ… $test_name PASSED" + ((TESTS_PASSED++)) + else + print_error "โŒ $test_name FAILED" + ((TESTS_FAILED++)) + fi +} + +function generate_test_report() { + local total_tests=$((TESTS_PASSED + TESTS_FAILED + TESTS_SKIPPED)) + + print_header "Test Results Summary" + print_info "Total Tests: $total_tests" + print_success "Passed: $TESTS_PASSED" + print_error "Failed: $TESTS_FAILED" + print_warning "Skipped: $TESTS_SKIPPED" + + # Generate JSON report + cat > "$TEST_RESULTS_FILE" </dev/null 2>&1; then + echo "โœ… HTTPS URL accessible: $url" + else + echo "โŒ HTTPS URL not accessible: $url" + ((https_failures++)) + fi + done + done < <(find "$PROJECT_ROOT/$dir" -name "*.sh" -type f -print0) + fi + done + + return $https_failures +} + +function test_ssl_certificate_validation() { + echo "๐Ÿ” Testing SSL certificate validation..." + + local test_urls=( + "https://archive.ubuntu.com" + "https://linux.dell.com" + "https://download.proxmox.com" + ) + + local ssl_failures=0 + + for url in "${test_urls[@]}"; do + # Test with strict SSL verification + if curl -s --fail --ssl-reqd --cert-status "$url" >/dev/null 2>&1; then + echo "โœ… SSL certificate valid: $url" + else + echo "โŒ SSL certificate validation failed: $url" + ((ssl_failures++)) + fi + done + + return $ssl_failures +} + +function test_deployment_security() { + echo "๐Ÿ” Testing deployment method security..." + + local readme_file="$PROJECT_ROOT/README.md" + + if [[ -f "$readme_file" ]]; then + # Check for insecure curl | bash patterns + if grep -q "curl.*|.*bash" "$readme_file" || grep -q "wget.*|.*bash" "$readme_file"; then + echo "โŒ Insecure deployment method found in README.md" + return 1 + else + echo "โœ… Secure deployment method in README.md" + fi + + # Check for git clone method + if grep -q "git clone" "$readme_file"; then + echo "โœ… Git clone deployment method found" + return 0 + else + echo "โš ๏ธ No git clone method found in README.md" + return 1 + fi + else + echo "โŒ README.md not found" + return 1 + fi +} + +# Main test execution +function main() { + echo "๐Ÿ”’ Running HTTPS Enforcement Security Tests" + echo "==========================================" + + local total_failures=0 + + # Run all security tests + test_no_http_urls || ((total_failures++)) + test_https_urls_valid || ((total_failures++)) + test_ssl_certificate_validation || ((total_failures++)) + test_deployment_security || ((total_failures++)) + + echo "==========================================" + + if [[ $total_failures -eq 0 ]]; then + echo "โœ… All HTTPS enforcement security tests passed" + exit 0 + else + echo "โŒ $total_failures HTTPS enforcement security tests failed" + exit 1 + fi +} + +# Run main if executed directly +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + main "$@" +fi \ No newline at end of file diff --git a/Project-Tests/unit/framework-functions.sh b/Project-Tests/unit/framework-functions.sh new file mode 100755 index 0000000..5ffe5f4 --- /dev/null +++ b/Project-Tests/unit/framework-functions.sh @@ -0,0 +1,176 @@ +#!/bin/bash + +# Framework Functions Unit Tests +# Tests core framework functionality + +set -euo pipefail + +PROJECT_ROOT="$(dirname "$(realpath "${BASH_SOURCE[0]}")")/../.." + +# Source framework functions +source "$PROJECT_ROOT/Framework-Includes/Logging.sh" 2>/dev/null || echo "Warning: Logging.sh not found" +source "$PROJECT_ROOT/Framework-Includes/PrettyPrint.sh" 2>/dev/null || echo "Warning: PrettyPrint.sh not found" +source "$PROJECT_ROOT/Framework-Includes/ErrorHandling.sh" 2>/dev/null || echo "Warning: ErrorHandling.sh not found" + +function test_logging_functions() { + echo "๐Ÿ” Testing logging functions..." + + local test_log="/tmp/test-log-$$" + + # Test if logging functions exist and work + if command -v log_info >/dev/null 2>&1; then + log_info "Test info message" 2>/dev/null || true + echo "โœ… log_info function exists" + else + echo "โŒ log_info function missing" + return 1 + fi + + if command -v log_error >/dev/null 2>&1; then + log_error "Test error message" 2>/dev/null || true + echo "โœ… log_error function exists" + else + echo "โŒ log_error function missing" + return 1 + fi + + # Cleanup + rm -f "$test_log" + return 0 +} + +function test_pretty_print_functions() { + echo "๐Ÿ” Testing pretty print functions..." + + # Test if pretty print functions exist + if command -v print_info >/dev/null 2>&1; then + print_info "Test info message" >/dev/null 2>&1 || true + echo "โœ… print_info function exists" + else + echo "โŒ print_info function missing" + return 1 + fi + + if command -v print_error >/dev/null 2>&1; then + print_error "Test error message" >/dev/null 2>&1 || true + echo "โœ… print_error function exists" + else + echo "โŒ print_error function missing" + return 1 + fi + + if command -v print_success >/dev/null 2>&1; then + print_success "Test success message" >/dev/null 2>&1 || true + echo "โœ… print_success function exists" + else + echo "โŒ print_success function missing" + return 1 + fi + + return 0 +} + +function test_error_handling() { + echo "๐Ÿ” Testing error handling..." + + # Test if error handling functions exist + if command -v handle_error >/dev/null 2>&1; then + echo "โœ… handle_error function exists" + else + echo "โŒ handle_error function missing" + return 1 + fi + + # Test bash strict mode is set + if [[ "$-" == *e* ]]; then + echo "โœ… Bash strict mode (set -e) is enabled" + else + echo "โŒ Bash strict mode (set -e) not enabled" + return 1 + fi + + if [[ "$-" == *u* ]]; then + echo "โœ… Bash unset variable checking (set -u) is enabled" + else + echo "โŒ Bash unset variable checking (set -u) not enabled" + return 1 + fi + + return 0 +} + +function test_framework_includes_exist() { + echo "๐Ÿ” Testing framework includes exist..." + + local required_includes=( + "Logging.sh" + "PrettyPrint.sh" + "ErrorHandling.sh" + "PreflightCheck.sh" + ) + + local missing_files=0 + + for include_file in "${required_includes[@]}"; do + if [[ -f "$PROJECT_ROOT/Framework-Includes/$include_file" ]]; then + echo "โœ… Framework include exists: $include_file" + else + echo "โŒ Framework include missing: $include_file" + ((missing_files++)) + fi + done + + return $missing_files +} + +function test_syntax_validation() { + echo "๐Ÿ” Testing script syntax validation..." + + local syntax_errors=0 + local script_dirs=("Framework-Includes" "Project-Includes" "ProjectCode") + + for dir in "${script_dirs[@]}"; do + if [[ -d "$PROJECT_ROOT/$dir" ]]; then + while IFS= read -r -d '' file; do + if bash -n "$file" 2>/dev/null; then + echo "โœ… Syntax valid: $(basename "$file")" + else + echo "โŒ Syntax error in: $(basename "$file")" + ((syntax_errors++)) + fi + done < <(find "$PROJECT_ROOT/$dir" -name "*.sh" -type f -print0) + fi + done + + return $syntax_errors +} + +# Main test execution +function main() { + echo "๐Ÿงช Running Framework Functions Unit Tests" + echo "========================================" + + local total_failures=0 + + # Run all unit tests + test_framework_includes_exist || ((total_failures++)) + test_logging_functions || ((total_failures++)) + test_pretty_print_functions || ((total_failures++)) + test_error_handling || ((total_failures++)) + test_syntax_validation || ((total_failures++)) + + echo "========================================" + + if [[ $total_failures -eq 0 ]]; then + echo "โœ… All framework function unit tests passed" + exit 0 + else + echo "โŒ $total_failures framework function unit tests failed" + exit 1 + fi +} + +# Run main if executed directly +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + main "$@" +fi \ No newline at end of file diff --git a/Project-Tests/validation/system-requirements.sh b/Project-Tests/validation/system-requirements.sh new file mode 100755 index 0000000..8cbf22a --- /dev/null +++ b/Project-Tests/validation/system-requirements.sh @@ -0,0 +1,142 @@ +#!/bin/bash + +# System Requirements Validation Test +# Validates minimum system requirements before deployment + +set -euo pipefail + +# Test configuration +MIN_RAM_GB=2 +MIN_DISK_GB=10 +REQUIRED_COMMANDS=("curl" "wget" "git" "systemctl" "apt-get") + +# Test functions +function test_memory_requirements() { + local total_mem_kb=$(grep MemTotal /proc/meminfo | awk '{print $2}') + local total_mem_gb=$((total_mem_kb / 1024 / 1024)) + + if [[ $total_mem_gb -ge $MIN_RAM_GB ]]; then + echo "โœ… Memory requirement met: ${total_mem_gb}GB >= ${MIN_RAM_GB}GB" + return 0 + else + echo "โŒ Memory requirement not met: ${total_mem_gb}GB < ${MIN_RAM_GB}GB" + return 1 + fi +} + +function test_disk_space() { + local available_gb=$(df / | tail -1 | awk '{print int($4/1024/1024)}') + + if [[ $available_gb -ge $MIN_DISK_GB ]]; then + echo "โœ… Disk space requirement met: ${available_gb}GB >= ${MIN_DISK_GB}GB" + return 0 + else + echo "โŒ Disk space requirement not met: ${available_gb}GB < ${MIN_DISK_GB}GB" + return 1 + fi +} + +function test_required_commands() { + local failed=0 + + for cmd in "${REQUIRED_COMMANDS[@]}"; do + if command -v "$cmd" >/dev/null 2>&1; then + echo "โœ… Required command available: $cmd" + else + echo "โŒ Required command missing: $cmd" + ((failed++)) + fi + done + + return $failed +} + +function test_os_compatibility() { + if [[ -f /etc/os-release ]]; then + local os_id=$(grep "^ID=" /etc/os-release | cut -d'=' -f2 | tr -d '"') + local os_version=$(grep "^VERSION_ID=" /etc/os-release | cut -d'=' -f2 | tr -d '"') + + case "$os_id" in + ubuntu|debian) + echo "โœ… OS compatibility: $os_id $os_version (supported)" + return 0 + ;; + *) + echo "โš ๏ธ OS compatibility: $os_id $os_version (may work, not fully tested)" + return 0 + ;; + esac + else + echo "โŒ Cannot determine OS version" + return 1 + fi +} + +function test_network_connectivity() { + local test_urls=( + "https://archive.ubuntu.com" + "https://linux.dell.com" + "https://download.proxmox.com" + "https://github.com" + ) + + local failed=0 + + for url in "${test_urls[@]}"; do + if curl -s --connect-timeout 10 --max-time 30 "$url" >/dev/null 2>&1; then + echo "โœ… Network connectivity: $url" + else + echo "โŒ Network connectivity failed: $url" + ((failed++)) + fi + done + + return $failed +} + +function test_permissions() { + local test_dirs=("/etc" "/usr/local/bin" "/var/log") + local failed=0 + + for dir in "${test_dirs[@]}"; do + if [[ -w "$dir" ]]; then + echo "โœ… Write permission: $dir" + else + echo "โŒ Write permission denied: $dir" + ((failed++)) + fi + done + + return $failed +} + +# Main test execution +function main() { + echo "๐Ÿ” Running System Requirements Validation" + echo "========================================" + + local total_failures=0 + + # Run all validation tests + test_memory_requirements || ((total_failures++)) + test_disk_space || ((total_failures++)) + test_required_commands || ((total_failures++)) + test_os_compatibility || ((total_failures++)) + test_network_connectivity || ((total_failures++)) + test_permissions || ((total_failures++)) + + echo "========================================" + + if [[ $total_failures -eq 0 ]]; then + echo "โœ… All system requirements validation tests passed" + exit 0 + else + echo "โŒ $total_failures system requirements validation tests failed" + exit 1 + fi +} + +# Run main if executed directly +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + main "$@" +fi \ No newline at end of file diff --git a/README.md b/README.md index 9a56306..734659a 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,8 @@ One of those functions is the provisoning of Linux servers. This repository is t In the future it will be used via FetchApply https://github.com/P5vc/fetch-apply -It is invoked via +## Usage -curl https://dl.knownelement.com/KNEL/FetchApply/SetupNewSystem.sh |/bin/bash \ No newline at end of file +git clone this repo +cd FetchApply/ProjectCode +bash SetupNewSystem.sh \ No newline at end of file diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..9214d25 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,190 @@ +# TSYS FetchApply Security Documentation + +## Security Architecture + +The TSYS FetchApply infrastructure provisioning system is designed with security-first principles, implementing multiple layers of protection for server deployment and management. + +## Current Security Features + +### 1. Secure Deployment Method โœ… +- **Git-based deployment:** Uses `git clone` instead of `curl | bash` +- **Local execution:** Scripts run locally after inspection +- **Version control:** Full audit trail of changes +- **Code review:** Changes require explicit approval + +### 2. HTTPS Enforcement โœ… +- **All downloads use HTTPS:** Eliminates man-in-the-middle attacks +- **SSL certificate validation:** Automatic certificate checking +- **Secure repositories:** Ubuntu archive, Dell, Proxmox all use HTTPS +- **No HTTP fallbacks:** No insecure download methods + +### 3. SSH Hardening +- **Key-only authentication:** Password login disabled +- **Secure ciphers:** Modern encryption algorithms only +- **Fail2ban protection:** Automated intrusion prevention +- **Custom SSH configuration:** Hardened sshd_config + +### 4. System Security +- **Firewall configuration:** Automated iptables rules +- **Audit logging:** auditd with custom rules +- **SIEM integration:** Wazuh agent deployment +- **Compliance scanning:** SCAP-STIG automated checks + +### 5. Error Handling +- **Bash strict mode:** `set -euo pipefail` prevents errors +- **Centralized logging:** All operations logged with timestamps +- **Graceful failures:** Proper cleanup on errors +- **Line-level debugging:** Error reporting with line numbers + +## Security Testing + +### Automated Security Validation +```bash +# Run security test suite +./Project-Tests/run-tests.sh security + +# Specific security tests +./Project-Tests/security/https-enforcement.sh +``` + +### Security Test Categories +1. **HTTPS Enforcement:** Validates all URLs use HTTPS +2. **Deployment Security:** Checks for secure deployment methods +3. **SSL Certificate Validation:** Tests certificate authenticity +4. **Permission Validation:** Verifies proper file permissions + +## Threat Model + +### Mitigated Threats +- **Supply Chain Attacks:** Git-based deployment with review +- **Man-in-the-Middle:** HTTPS-only downloads +- **Privilege Escalation:** Proper permission models +- **Unauthorized Access:** SSH hardening and key management + +### Remaining Risks +- **Secrets in Repository:** SSH keys stored in git (planned for removal) +- **No Integrity Verification:** Downloads lack checksum validation +- **No Backup/Recovery:** No rollback capability implemented + +## Security Recommendations + +### High Priority +1. **Implement Secrets Management** + - Remove SSH keys from repository + - Use Bitwarden/Vault for secret storage + - Implement key rotation procedures + +2. **Add Download Integrity Verification** + - SHA256 checksum validation for all downloads + - GPG signature verification where available + - Fail-safe on integrity check failures + +3. **Enhance Audit Logging** + - Centralized log collection + - Real-time security monitoring + - Automated threat detection + +### Medium Priority +1. **Configuration Backup** + - System state snapshots before changes + - Rollback capability for failed deployments + - Configuration drift detection + +2. **Network Security** + - VPN-based deployment (where applicable) + - Network segmentation for management + - Encrypted communication channels + +## Compliance + +### Security Standards +- **CIS Benchmarks:** Automated compliance checking +- **STIG Guidelines:** SCAP-based validation +- **Industry Best Practices:** Following NIST cybersecurity framework + +### Audit Requirements +- **Change Tracking:** All modifications logged +- **Access Control:** Permission-based system access +- **Vulnerability Management:** Regular security assessments + +## Incident Response + +### Security Event Handling +1. **Detection:** Automated monitoring and alerting +2. **Containment:** Immediate isolation procedures +3. **Investigation:** Log analysis and forensics +4. **Recovery:** System restoration procedures +5. **Lessons Learned:** Process improvement + +### Contact Information +- **Security Team:** [To be defined] +- **Incident Response:** [To be defined] +- **Escalation Path:** [To be defined] + +## Security Development Lifecycle + +### Code Review Process +1. **Static Analysis:** Automated security scanning +2. **Peer Review:** Manual code inspection +3. **Security Testing:** Automated security test suite +4. **Approval:** Security team sign-off + +### Deployment Security +1. **Pre-deployment Validation:** Security test execution +2. **Secure Deployment:** Authorized personnel only +3. **Post-deployment Verification:** Security configuration validation +4. **Monitoring:** Continuous security monitoring + +## Security Tools and Integrations + +### Current Tools +- **Wazuh:** SIEM and security monitoring +- **Lynis:** Security auditing +- **auditd:** System call auditing +- **Fail2ban:** Intrusion prevention + +### Planned Integrations +- **Vault/Bitwarden:** Secrets management +- **OSSEC:** Host-based intrusion detection +- **Nessus/OpenVAS:** Vulnerability scanning +- **ELK Stack:** Log aggregation and analysis + +## Vulnerability Management + +### Vulnerability Scanning +- **Regular scans:** Monthly vulnerability assessments +- **Automated patching:** Security update automation +- **Exception handling:** Risk-based patch management +- **Reporting:** Executive security dashboards + +### Disclosure Process +1. **Internal Discovery:** Report to security team +2. **Assessment:** Risk and impact evaluation +3. **Remediation:** Patch development and testing +4. **Deployment:** Coordinated security updates +5. **Verification:** Post-patch validation + +## Security Metrics + +### Key Performance Indicators +- **Deployment Success Rate:** Percentage of successful secure deployments +- **Vulnerability Response Time:** Time to patch critical vulnerabilities +- **Security Test Coverage:** Percentage of code covered by security tests +- **Incident Response Time:** Time to detect and respond to security events + +### Monitoring and Reporting +- **Real-time Dashboards:** Security status monitoring +- **Executive Reports:** Monthly security summaries +- **Compliance Reports:** Quarterly compliance assessments +- **Trend Analysis:** Security posture improvement tracking + +## Contact and Support + +For security-related questions or incidents: +- **Repository Issues:** https://projects.knownelement.com/project/reachableceo-vptechnicaloperations/timeline +- **Community Discussion:** https://community.turnsys.com/c/chieftechnologyandproductofficer/26 +- **Security Team:** [Contact information to be added] + +## Security Updates + +This document is updated as security features are implemented and threats evolve. Last updated: July 14, 2025. \ No newline at end of file