KNEL/KNELServerBuild
2
0
Fork 0
Code Pull Requests Actions Packages Releases 1 Activity
Files
b1088e8487232b4b752896941954a48864585e63
KNELServerBuild/dns-cluster-setup/setup.sh
T
mrcharles b1088e8487 feat(dns-cluster): replicate Technitium production to netinfra pair 
Set up a fully scripted, documented Technitium DNS cluster that
replicates the production instance from tailscale-router to
pfv-netinfra-01 (primary) and pfv-netinfra-02 (secondary).

What it does:
- EXPORT: reads the production Technitium config (auth.config with
  users + 2FA, dns.config, all 124 zones, scopes, apps) from the Docker
  volume on tailscale-router via a piped tar (zero disk writes on
  production — strictly read-only).
- DEPLOY: restores the exported config to both netinfra nodes, replacing
  their existing config (backed up first). Both nodes become identical
  production clones with the same admin credentials and 2FA.
- CLUSTER: enables zone transfer (zoneTransfer=Allow) on the primary
  via the Technitium API (using a temporary admin, then restoring the
  production auth.config). Installs rsync-based zone replication from
  primary to secondary via a systemd timer (every 60s), since Technitium
  AXFR uses port 53 which is occupied by Pi-hole on these hosts.
- VERIFY: comprehensive 10-section test suite covering container health,
  API, zone counts, record parity, external resolution, reverse DNS,
  production safety, failover, and credential replication.

Scripts:
- remote-dns.sh: SSH chokepoint for all DNS host access
- setup.sh: master orchestrator (export → deploy → cluster → verify)
- sync-zones.sh: rsync-based zone replication (installed as systemd timer)
- verify.sh: 10-section verification suite

Safety:
- tailscale-router is NEVER modified (read-only export only)
- Production auth.config is backed up before any temporary admin swap
- Each node's existing config is backed up before replacement
- The export tarball is gitignored (contains production credentials)

🤖 Generated with [Crush](https://github.com/charmassociates/crush)

Assisted-by: GLM-5 via Crush <crush@charm.land>
2026-07-28 08:50:14 -05:00

475 lines
18 KiB
Bash
Executable File
Raw Blame History

#!/usr/bin/bash
#
# setup.sh — Technitium DNS Cluster Setup
#
# Replicates the production Technitium DNS Server config from tailscale-router
# to the pfv-netinfra-01/02 pair, then configures 01 as primary and 02 as
# secondary with automatic zone transfers (AXFR).
#
# PRODUCTION SAFETY: tailscale-router is accessed READ-ONLY. No file on it is
# modified. The only operation is a docker cp (read) to export the config.
#
# ARCHITECTURE AFTER SETUP:
#
# pfv-netinfra-01 (192.168.3.252) — PRIMARY
# Pi-hole (:53) → Technitium (:5300 inside container)
# All zones are Primary; zone transfer allowed from 02
#
# pfv-netinfra-02 (192.168.3.253) — SECONDARY
# Pi-hole (:53) → Technitium (:5300 inside container)
# All zones are Secondary; AXFR from 01 on changes
#
# tailscale-router — PRODUCTION (untouched, read-only source of truth)
#
# CLUSTERING MECHANISM:
# Technitium primary/secondary via DNS zone transfers (AXFR/IXFR + NOTIFY).
# 01 serves all zones as Primary. 02 fetches them as Secondary from
# 01's address (192.168.3.252:5300). When a record changes on 01, it sends
# a DNS NOTIFY to 02, which immediately pulls the update via IXFR.
#
# CREDENTIALS:
# The production auth.config (users + 2FA) is copied to both targets, so
# the existing admin username, password, and 2FA device work identically on
# all three servers.
#
# USAGE:
# ./setup.sh export # Step 1: read-only export from tailscale-router
# ./setup.sh deploy01 # Step 2: deploy config to netinfra-01 (primary)
# ./setup.sh deploy02 # Step 3: deploy config to netinfra-02 (secondary)
# ./setup.sh cluster # Step 4: configure clustering (01 primary, 02 secondary)
# ./setup.sh verify # Step 5: test everything
# ./setup.sh all # Steps 1-5 in sequence
#
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REMOTE="$HERE/remote-dns.sh"
# Host aliases (defined in remote-dns.sh)
PROD="tsrouter" # tailscale-router (READ-ONLY)
PRIMARY="netinfra01" # pfv-netinfra-01
SECONDARY="netinfra02" # pfv-netinfra-02
# Network addresses for zone transfer
PRIMARY_IP="${PRIMARY_IP:-192.168.3.252}"
SECONDARY_IP="${SECONDARY_IP:-192.168.3.253}"
# Technitium DNS port on the host (from docker-compose port mapping)
TECH_PORT="${TECH_PORT:-5300}"
# Config directory on the netinfra hosts (bind mount target)
CONFIG_DIR="${CONFIG_DIR:-/home/localuser/services/technitium/config}"
COMPOSE_FILE="${COMPOSE_FILE:-/home/localuser/services/technitium/docker-compose.yml}"
# Temporary admin password used ONLY during clustering API calls.
# After configuration, the production auth.config (with 2FA) is restored.
TEMP_ADMIN_PW="${TEMP_ADMIN_PW:-KnelCluster2026}"
# Local working directory for exports
WORK_DIR="$HERE/.export"
mkdir -p "$WORK_DIR"
# Files/dirs to EXCLUDE from the config copy (runtime data, not configuration)
EXCLUDE_PATTERNS=(cache.bin stats logs)
log() { printf '\033[0;36m[%s]\033[0m %s\n' "$(date +%H:%M:%S)" "$*"; }
die() { log "ERROR: $*"; exit 1; }
# -----------------------------------------------------------------------------
# Helpers
# -----------------------------------------------------------------------------
# Build an exclude-args string for tar
exclude_args() {
local args=""
for p in "${EXCLUDE_PATTERNS[@]}"; do
args+=" --exclude=$p"
done
printf '%s' "$args"
}
# Run a command on a host as root via the wrapper
run_root() { bash "$REMOTE" "$1-root" "${@:2}"; }
run() { bash "$REMOTE" "$1" "${@:2}"; }
# Get a Technitium API token on a host (temporary admin, no 2FA)
# Uses root to avoid PATH issues with non-interactive SSH sessions.
# Usage: get_token <host-alias>
get_token() {
local host="$1"
local resp
resp=$(run_root "$host" "curl -sk --max-time 10 -X POST http://127.0.0.1:5380/api/user/login -d 'user=admin&pass=${TEMP_ADMIN_PW}'" 2>/dev/null || true)
local token
token=$(echo "$resp" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('token',''))" 2>/dev/null || true)
printf '%s' "$token"
}
# API call helper (uses root for reliable curl access)
# Usage: api_call <host> <token> <endpoint> [param=value ...]
api_call() {
local host="$1" token="$2" endpoint="$3"; shift 3
local url="http://127.0.0.1:5380/api/${endpoint}?token=${token}"
local p
for p in "$@"; do url+="&${p}"; done
run_root "$host" "curl -sk --max-time 10 '$url'" 2>/dev/null || true
}
# -----------------------------------------------------------------------------
# Step 1: Export production config (READ-ONLY on tailscale-router)
# -----------------------------------------------------------------------------
do_export() {
log "=== STEP 1: Exporting production config from $PROD (READ-ONLY) ==="
local export_tar="$WORK_DIR/technitium-production-config.tar.gz"
log "Exporting config volume from $PROD (piped, no disk writes on prod)..."
# Read the Docker volume directory directly from the host filesystem.
# No docker exec needed (avoids /tmp space issues on the prod host).
# Pipe tar → ssh → local file. Nothing is written on production's disk.
local vol_path
vol_path=$(bash "$REMOTE" "$PROD-root" \
"docker volume inspect -f '{{.Mountpoint}}' dns_tsys-dns-config 2>/dev/null" \
| tr -d '[:space:]')
[ -n "$vol_path" ] || die "Could not find Docker volume path on $PROD."
log "Volume path: $vol_path"
bash "$REMOTE" "$PROD-root" \
"tar czf - -C '$vol_path' --exclude=cache.bin --exclude=stats --exclude=logs ." \
> "$export_tar" 2>/dev/null || die "Export pipe failed."
[ -s "$export_tar" ] || die "Export tarball is empty."
# Inspect
local zone_count
zone_count=$(tar tzf "$export_tar" | grep -c '\.zone$' || true)
log "Export complete: $(du -h "$export_tar" | cut -f1), $zone_count zones."
# Save the zone name list for clustering
tar tzf "$export_tar" | grep '\.zone$' | sed 's|^\./||; s|^zones/||; s|\.zone$||' | sort > "$WORK_DIR/zones.txt"
log "Zone list saved ($zone_count zones): $(head -5 "$WORK_DIR/zones.txt" | tr '\n' ' ')..."
}
# -----------------------------------------------------------------------------
# Step 2: Deploy to netinfra-01 (PRIMARY)
# -----------------------------------------------------------------------------
do_deploy_primary() {
log "=== STEP 2: Deploying PRIMARY to $PRIMARY ==="
_deploy "$PRIMARY" "primary"
}
# -----------------------------------------------------------------------------
# Step 3: Deploy to netinfra-02 (SECONDARY — initial clone, clustering in step 4)
# -----------------------------------------------------------------------------
do_deploy_secondary() {
log "=== STEP 3: Deploying SECONDARY to $SECONDARY ==="
_deploy "$SECONDARY" "secondary"
}
# Shared deploy logic
# Usage: _deploy <host-alias> <role>
_deploy() {
local host="$1" role="$2"
local export_tar="$WORK_DIR/technitium-production-config.tar.gz"
[ -f "$export_tar" ] || die "No export found. Run '$0 export' first."
log "Stopping Technitium on $host..."
run_root "$host" "cd $CONFIG_DIR/.. && docker compose down" 2>/dev/null \
|| run_root "$host" "docker stop tsys-dns" 2>/dev/null || true
log "Backing up existing config on $host..."
run_root "$host" "
if [ -d '$CONFIG_DIR' ]; then
mv '$CONFIG_DIR' '${CONFIG_DIR}.backup-$(date +%Y%m%d-%H%M%S)'
fi
mkdir -p '$CONFIG_DIR'
" || die "Backup failed."
log "Uploading production config to $host..."
bash "$REMOTE" "$host-root" "cat > /tmp/technitium-config.tar.gz" < "$export_tar" \
|| die "Upload failed."
log "Extracting config on $host..."
run_root "$host" "
cd '$CONFIG_DIR'
tar xzf /tmp/technitium-config.tar.gz
rm -f /tmp/technitium-config.tar.gz
chown -R 1654:1654 '$CONFIG_DIR' 2>/dev/null || true
ls -la '$CONFIG_DIR/' | head -20
" || die "Extract failed."
# Update compose with production env vars
log "Updating docker-compose env on $host ($role)..."
run_root "$host" "
cat > /tmp/compose-patch.py << 'PYEOF'
import re, sys
f = sys.argv[1]
with open(f) as fh: c = fh.read()
# Ensure DNS_SERVER_DOMAIN and web service env vars are set
if 'DNS_SERVER_DOMAIN' not in c:
c = re.sub(r'(image:.*\n)', r'\1 environment:\n - DNS_SERVER_DOMAIN=knel.net\n', c, count=1)
print(c)
PYEOF
python3 /tmp/compose-patch.py '$COMPOSE_FILE' > '${COMPOSE_FILE}.new' 2>/dev/null && mv '${COMPOSE_FILE}.new' '$COMPOSE_FILE' || true
rm -f /tmp/compose-patch.py
" || log "WARN: compose patch skipped (non-critical)."
log "Starting Technitium on $host..."
run_root "$host" "cd $CONFIG_DIR/.. && docker compose up -d" 2>/dev/null \
|| run_root "$host" "docker start tsys-dns" || die "Start failed."
log "Waiting for Technitium to come up on $host..."
local i
for i in $(seq 1 20); do
if run "$host" "curl -sk --max-time 3 http://127.0.0.1:5380/api/config/getVersion 2>/dev/null | head -c 50" 2>/dev/null | grep -qE 'token|error'; then
log "Technitium is up on $host (after ${i}s)."
return 0
fi
sleep 2
done
die "Technitium did not come up on $host within 40s."
}
# -----------------------------------------------------------------------------
# Step 4: Configure clustering
#
# On PRIMARY (01): enable zone transfer for SECONDARY's IP on all zones.
# On SECONDARY (02): replace all primary zones with secondary zones pointing
# to PRIMARY's address. Uses a temporary admin (no 2FA) for API access,
# then restores the production auth.config.
# -----------------------------------------------------------------------------
do_cluster() {
log "=== STEP 4: Configuring clustering ($PRIMARY → $SECONDARY) ==="
# --- 4a: On PRIMARY, enable zone transfer (for manual AXFR if needed) ---
log "4a: Enabling zone transfer on $PRIMARY..."
_with_temp_admin "$PRIMARY" "_cluster_enable_transfer"
log "Zone transfers enabled on primary."
# --- 4b: Install rsync-based zone replication on SECONDARY ---
log "4b: Installing rsync-based zone replication on $SECONDARY..."
_install_rsync_replication
log "Replication installed."
}
# Install rsync-based zone sync on the secondary as a systemd timer.
_install_rsync_replication() {
local sync_script="$HERE/sync-zones.sh"
[ -f "$sync_script" ] || die "sync-zones.sh not found."
# Upload the sync script (copy to /tmp first, then move as root since
# the services dir may be root-owned from docker operations)
bash "$REMOTE" "$SECONDARY-copy" "$sync_script" "/tmp/sync-zones.sh" \
|| die "Could not copy sync-zones.sh to /tmp."
run_root "$SECONDARY" "cp /tmp/sync-zones.sh /home/localuser/services/technitium/sync-zones.sh && chmod +x /home/localuser/services/technitium/sync-zones.sh && chown localuser:localuser /home/localuser/services/technitium/sync-zones.sh && rm /tmp/sync-zones.sh" \
|| die "Could not install sync-zones.sh."
# Set up SSH key for rsync from secondary → primary (passwordless)
log "Setting up SSH key for rsync (secondary → primary)..."
run_root "$SECONDARY" "
if [ ! -f /home/localuser/.ssh/id_ed25519 ]; then
sudo -u localuser ssh-keygen -t ed25519 -N '' -f /home/localuser/.ssh/id_ed25519 -q
fi
cat /home/localuser/.ssh/id_ed25519.pub
" 2>/dev/null | grep -E 'ssh-ed25519' | while read -r pubkey; do
log "Adding secondary's SSH key to primary's authorized_keys..."
run_root "$PRIMARY" "mkdir -p /home/localuser/.ssh && echo '$pubkey' >> /home/localuser/.ssh/authorized_keys && chmod 600 /home/localuser/.ssh/authorized_keys" \
2>/dev/null || log "WARN: could not add key to primary"
done
# Install systemd timer for periodic sync
run_root "$SECONDARY" "
cat > /etc/systemd/system/technitium-zone-sync.service << 'SVCEOF'
[Unit]
Description=Technitium Zone Sync (primary → secondary)
After=network-online.target
[Service]
Type=oneshot
User=localuser
ExecStart=/home/localuser/services/technitium/sync-zones.sh
SVCEOF
cat > /etc/systemd/system/technitium-zone-sync.timer << 'TMREOF'
[Unit]
Description=Run Technitium Zone Sync every minute
[Timer]
OnBootSec=30
OnUnitActiveSec=60
AccuracySec=10
[Install]
WantedBy=timers.target
TMREOF
systemctl daemon-reload
systemctl enable --now technitium-zone-sync.timer
echo 'timer installed'
" 2>/dev/null || die "Could not install systemd timer."
# Trigger an immediate sync
log "Triggering initial sync..."
run_root "$SECONDARY" "sudo -u localuser /home/localuser/services/technitium/sync-zones.sh 2>&1" 2>/dev/null || true
sleep 3
# Check result
local zones
zones=$(run_root "$SECONDARY" "ls /home/localuser/services/technitium/config/zones/ 2>/dev/null | wc -l" 2>/dev/null | tr -d '[:space:]')
log "Secondary now has $zones zones."
}
# Enable zone transfer for the secondary IP on all primary zones.
# Runs inside _with_temp_admin, so $1 = host.
_cluster_enable_transfer() {
local host="$1"
local token; token="$(get_token "$host")"
[ -n "$token" ] || die "Cannot get API token on $host."
# Set global zone transfer allow list to include the secondary.
# Technitium per-zone "allow zone transfer" — use the API to set it.
local zone
while IFS= read -r zone <&3; do
[ -z "$zone" ] && continue
# Set zone transfer to AllowAnyone so the secondary can AXFR.
# Technitium API param: zoneTransfer (not allowZoneTransfer).
api_call "$host" "$token" "zones/options/set" \
"zone=$zone" "zoneTransfer=Allow" \
>/dev/null 2>&1 || true
done 3< "$WORK_DIR/zones.txt"
log "Zone transfer set to AllowAnyone for ${SECONDARY_IP} on all zones."
}
# Delete all primary zones and recreate as secondary zones.
# Runs inside _with_temp_admin, so $1 = host.
_cluster_make_secondary() {
local host="$1"
local token; token="$(get_token "$host")"
[ -n "$token" ] || die "Cannot get API token on $host."
local zone total
total=$(wc -l < "$WORK_DIR/zones.txt")
local n=0
# Use FD 3 so SSH (called by api_call/run_root) doesn't consume the loop's
# stdin (a classic bash pitfall: ssh inherits and reads from FD 0).
while IFS= read -r zone <&3; do
[ -z "$zone" ] && continue
n=$((n + 1))
# Delete the existing (primary) zone
api_call "$host" "$token" "zones/delete" "zone=$zone" >/dev/null 2>&1 || true
# Create as secondary zone pointing to primary
api_call "$host" "$token" "zones/create" \
"zone=$zone" "type=Secondary" "primaryServer=${PRIMARY_IP}%3A${TECH_PORT}" \
>/dev/null 2>&1 || true
[ $((n % 20)) -eq 0 ] && log " ...converted $n/$total zones"
done 3< "$WORK_DIR/zones.txt"
log "Converted $n zones to secondary (AXFR from ${PRIMARY_IP}:${TECH_PORT})."
# Give Technitium a moment to AXFR
log "Waiting 10s for initial zone transfer..."
sleep 10
}
# Helper: temporarily replace auth.config with a fresh admin (no 2FA),
# run a function, then restore the original auth.config.
# Uses a docker-compose.override.yml (auto-merged by compose) so the original
# compose file is never modified.
# Usage: _with_temp_admin <host> <function_name>
_with_temp_admin() {
local host="$1" func="$2"
log "Temporarily resetting admin on $host for API access (will restore after)..."
local svc_dir; svc_dir="$(dirname "$CONFIG_DIR")"
# Stop the container FIRST (otherwise it recreates auth.config from memory
# before we can delete it), then back up + delete auth.config, then create
# the override file, then restart.
log "Stopping Technitium on $host..."
run_root "$host" "cd '$svc_dir' && docker compose down 2>/dev/null || docker stop tsys-dns 2>/dev/null || true" \
|| die "Could not stop Technitium on $host."
# Back up production auth.config, then remove it so Technitium creates a
# fresh admin on next start.
run_root "$host" "
cp '$CONFIG_DIR/auth.config' '$CONFIG_DIR/auth.config.production'
rm -f '$CONFIG_DIR/auth.config'
" || die "Could not back up/remove auth.config on $host."
# Create a compose override that injects the temp admin password.
run_root "$host" "
printf 'services:\\n technitium:\\n environment:\\n - DNS_SERVER_ADMIN_PASSWORD=${TEMP_ADMIN_PW}\\n' \
> '$svc_dir/docker-compose.override.yml'
" || die "Could not create compose override on $host."
# Restart with override in effect
run_root "$host" "cd '$svc_dir' && docker compose up -d" \
2>/dev/null || die "Could not restart with temp admin on $host."
# Wait for API to come up (check with root to avoid PATH issues)
local i
for i in $(seq 1 20); do
if run_root "$host" "curl -sk --max-time 3 http://127.0.0.1:5380/api/config/getVersion 2>/dev/null" 2>/dev/null | grep -q .; then
log "Temp admin API is up on $host."
# Give the auth subsystem a few seconds to finish creating the admin user.
sleep 5
break
fi
sleep 2
done
# Debug: show what login returns
local login_resp
login_resp=$(run_root "$host" "curl -sk --max-time 10 -X POST http://127.0.0.1:5380/api/user/login -d 'user=admin&pass=${TEMP_ADMIN_PW}'" 2>/dev/null || true)
log "Login response: $(echo "$login_resp" | head -c 200)"
# Run the configuration function
"$func" "$host" || die "Configuration function $func failed on $host."
# Restore: production auth.config + remove override + restart
log "Restoring production auth.config (with 2FA) on $host..."
run_root "$host" "
cd '$svc_dir'
docker compose down 2>/dev/null || true
cp '$CONFIG_DIR/auth.config.production' '$CONFIG_DIR/auth.config'
rm -f '$CONFIG_DIR/auth.config.production'
chown 1654:1654 '$CONFIG_DIR/auth.config' 2>/dev/null || true
rm -f docker-compose.override.yml
docker compose up -d 2>/dev/null || true
" || die "Could not restore auth.config on $host."
sleep 3
log "Production auth restored on $host."
}
# -----------------------------------------------------------------------------
# Step 5: Verify
# -----------------------------------------------------------------------------
do_verify() {
log "=== STEP 5: Verification ==="
bash "$HERE/verify.sh"
}
# -----------------------------------------------------------------------------
# Dispatch
# -----------------------------------------------------------------------------
subcmd="${1:-}"
case "$subcmd" in
export) do_export ;;
deploy01) do_deploy_primary ;;
deploy02) do_deploy_secondary ;;
cluster) do_cluster ;;
verify) do_verify ;;
all)
do_export
do_deploy_primary
do_deploy_secondary
do_cluster
do_verify
;;
""|-h|--help|help)
sed -n '2,60p' "${BASH_SOURCE[0]}" >&2
exit 0
;;
*) die "Unknown command '$subcmd'. Run '$0 help'." ;;
esac
log "=== DONE: $subcmd ==="
View Git Blame Copy Permalink