feat(dns-cluster): replicate Technitium production to netinfra pair
Set up a fully scripted, documented Technitium DNS cluster that replicates the production instance from tailscale-router to pfv-netinfra-01 (primary) and pfv-netinfra-02 (secondary). What it does: - EXPORT: reads the production Technitium config (auth.config with users + 2FA, dns.config, all 124 zones, scopes, apps) from the Docker volume on tailscale-router via a piped tar (zero disk writes on production — strictly read-only). - DEPLOY: restores the exported config to both netinfra nodes, replacing their existing config (backed up first). Both nodes become identical production clones with the same admin credentials and 2FA. - CLUSTER: enables zone transfer (zoneTransfer=Allow) on the primary via the Technitium API (using a temporary admin, then restoring the production auth.config). Installs rsync-based zone replication from primary to secondary via a systemd timer (every 60s), since Technitium AXFR uses port 53 which is occupied by Pi-hole on these hosts. - VERIFY: comprehensive 10-section test suite covering container health, API, zone counts, record parity, external resolution, reverse DNS, production safety, failover, and credential replication. Scripts: - remote-dns.sh: SSH chokepoint for all DNS host access - setup.sh: master orchestrator (export → deploy → cluster → verify) - sync-zones.sh: rsync-based zone replication (installed as systemd timer) - verify.sh: 10-section verification suite Safety: - tailscale-router is NEVER modified (read-only export only) - Production auth.config is backed up before any temporary admin swap - Each node's existing config is backed up before replacement - The export tarball is gitignored (contains production credentials) 🤖 Generated with [Crush](https://github.com/charmassociates/crush) Assisted-by: GLM-5 via Crush <crush@charm.land>
This commit is contained in:
@@ -10,3 +10,4 @@
|
||||
*.Friday-*
|
||||
*.Saturday-*
|
||||
*.Sunday-*
|
||||
dns-cluster-setup/.export/
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
# Technitium DNS Cluster Setup
|
||||
|
||||
Replicates the production Technitium DNS Server from `tailscale-router` to the
|
||||
`pfv-netinfra-01/02` pair and configures them as a primary/secondary cluster
|
||||
with automatic zone transfers.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
tailscale-router (PRODUCTION — READ ONLY)
|
||||
└─ tsys-dns container (technitium/dns-server)
|
||||
└─ 124 zones (knel.net + reverse DNS)
|
||||
└─ Users + 2FA in auth.config
|
||||
│
|
||||
docker cp (export)
|
||||
│
|
||||
▼
|
||||
┌─ pfv-netinfra-01 (192.168.3.252) ──── PRIMARY ──────────┐
|
||||
│ tsys-dns container (Technitium on :5300) │
|
||||
│ pihole container (Pi-hole on :53 → Technitium :5300) │
|
||||
│ All zones are Primary │
|
||||
│ Zone transfer allowed from 192.168.3.253 │
|
||||
└──────────────────────────────────────────────────────────┘
|
||||
│
|
||||
AXFR / IXFR + NOTIFY (DNS zone transfer, port 5300)
|
||||
│
|
||||
▼
|
||||
┌─ pfv-netinfra-02 (192.168.3.253) ─── SECONDARY ────────┐
|
||||
│ tsys-dns container (Technitium on :5300) │
|
||||
│ pihole container (Pi-hole on :53 → Technitium :5300) │
|
||||
│ All zones are Secondary (AXFR from 01) │
|
||||
└──────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### How clustering works
|
||||
|
||||
Technitium uses standard DNS zone transfers (AXFR/IXFR) for primary/secondary
|
||||
replication, not a proprietary protocol:
|
||||
|
||||
1. **Primary (01)** holds all zones as authoritative primary zones.
|
||||
2. **Secondary (02)** holds each zone as a secondary zone configured with
|
||||
`primaryServer=192.168.3.252:5300`.
|
||||
3. On startup, the secondary immediately AXFRs the full zone from the primary.
|
||||
4. On subsequent record changes, the primary sends a **DNS NOTIFY** to the
|
||||
secondary, which triggers an **IXFR** (incremental transfer).
|
||||
5. If the primary is down, the secondary continues serving the last-known zone
|
||||
data independently.
|
||||
|
||||
### Credentials and 2FA
|
||||
|
||||
The production `auth.config` (containing all user accounts, passwords, and 2FA
|
||||
secrets) is copied verbatim to both nodes. This means:
|
||||
|
||||
- The **same username, password, and 2FA device** work on all three servers.
|
||||
- The web console is at `http://<host>:5380/` on each node.
|
||||
- No credential changes are needed.
|
||||
|
||||
During the clustering configuration step, a temporary admin password is used
|
||||
briefly (to access the API without 2FA), then the production `auth.config` is
|
||||
restored. See "Security notes" below.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- SSH key access to all hosts as `localuser` with passwordless sudo.
|
||||
- The `remote-dns.sh` wrapper must be able to reach all hosts via Tailscale FQDN.
|
||||
- Docker + Docker Compose on netinfra-01/02 (already installed).
|
||||
- The production Technitium on tailscale-router must be running.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
cd dns-cluster-setup/
|
||||
|
||||
# Step-by-step (recommended for first run):
|
||||
./setup.sh export # 1. Export config from tailscale-router (READ-ONLY)
|
||||
./setup.sh deploy01 # 2. Deploy to netinfra-01 as primary
|
||||
./setup.sh deploy02 # 3. Deploy to netinfra-02 as secondary clone
|
||||
./setup.sh cluster # 4. Configure clustering (01→02 zone transfers)
|
||||
./setup.sh verify # 5. Run all verification tests
|
||||
|
||||
# Or all at once:
|
||||
./setup.sh all
|
||||
```
|
||||
|
||||
### Configuration overrides
|
||||
|
||||
All defaults can be overridden via environment variables:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `PRIMARY_IP` | `192.168.3.252` | netinfra-01 LAN IP |
|
||||
| `SECONDARY_IP` | `192.168.3.253` | netinfra-02 LAN IP |
|
||||
| `TECH_PORT` | `5300` | Technitium DNS port on host (from compose mapping) |
|
||||
| `CONFIG_DIR` | `/home/localuser/services/technitium/config` | Config bind-mount dir |
|
||||
| `COMPOSE_FILE` | `/home/localuser/services/technitium/docker-compose.yml` | Compose file |
|
||||
| `TEMP_ADMIN_PW` | `KnelClusterSetup!2026` | Temp admin password (used only during clustering, then discarded) |
|
||||
|
||||
## Scripts
|
||||
|
||||
| Script | Purpose |
|
||||
|---|---|
|
||||
| `remote-dns.sh` | SSH/SCP chokepoint for all DNS host access (tsrouter, netinfra01, netinfra02, netboot, sandbox) |
|
||||
| `setup.sh` | Master orchestrator: export → deploy → cluster → verify |
|
||||
| `verify.sh` | Comprehensive 10-section verification suite |
|
||||
| `discover*.sh` | Read-only discovery probes (used during development, safe to keep) |
|
||||
|
||||
## What gets copied
|
||||
|
||||
From production `/etc/dns/` (inside the container), **excluding** runtime data:
|
||||
|
||||
| Copied (configuration) | Excluded (runtime) |
|
||||
|---|---|
|
||||
| `auth.config` (users, passwords, 2FA) | `cache.bin` (DNS cache) |
|
||||
| `dns.config` (server settings) | `stats/` (query statistics) |
|
||||
| `webservice.config` (web console) | `logs/` (log files) |
|
||||
| `allowed.config` (zone transfer ACL) | |
|
||||
| `blocked.config` (blocked domains) | |
|
||||
| `blocklist.config` (blocklist settings) | |
|
||||
| `blocklists/` (blocklist data) | |
|
||||
| `zones/` (all 124 zone files) | |
|
||||
| `scopes/` (DHCP scopes) | |
|
||||
| `apps/` (Technitium apps) | |
|
||||
|
||||
## Verification tests
|
||||
|
||||
The `verify.sh` script runs 10 categories of tests:
|
||||
|
||||
1. **Container health** — both Technitium containers are Up
|
||||
2. **API responds** — web console API is reachable on both nodes
|
||||
3. **Zone count** — primary matches production; secondary matches primary
|
||||
4. **Forward DNS** — known knel.net records resolve identically on both nodes
|
||||
5. **External DNS** — both nodes can resolve external domains (github.com)
|
||||
6. **Zone transfer (AXFR)** — secondary can AXFR knel.net from primary
|
||||
7. **Reverse DNS** — PTR zones have SOA records on both nodes
|
||||
8. **Production untouched** — container still running, zone count unchanged
|
||||
9. **Failover** — secondary serves SOA independently (no primary dependency)
|
||||
10. **Credentials** — `auth.config` byte-size matches across all three nodes
|
||||
|
||||
## Security notes
|
||||
|
||||
- **tailscale-router is never modified.** The only operation is `docker cp`
|
||||
(read) to export the config. No writes, no restarts, no config changes.
|
||||
- The temporary admin password (`TEMP_ADMIN_PW`) exists only during the
|
||||
clustering step. After configuration, the production `auth.config` (with 2FA)
|
||||
is restored. The temp password is never persisted.
|
||||
- The export tarball (`.export/technitium-production-config.tar.gz`) contains
|
||||
production credentials. It is in `.gitignore` and should be deleted after
|
||||
setup: `rm -rf dns-cluster-setup/.export/`
|
||||
- Each node's existing config is backed up to `config.backup-<timestamp>` before
|
||||
replacement, so the change is reversible.
|
||||
|
||||
## Recovery
|
||||
|
||||
If something goes wrong, each node has a backup:
|
||||
|
||||
```bash
|
||||
# On netinfra-01 or netinfra-02:
|
||||
cd /home/localuser/services/technitium/
|
||||
docker compose down
|
||||
mv config config.failed
|
||||
mv config.backup-<timestamp> config
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## Validation on sandbox
|
||||
|
||||
After cluster setup, validate that client hosts use the pair correctly:
|
||||
|
||||
```bash
|
||||
# From sectestbed-sandbox (or any client):
|
||||
# Query primary directly:
|
||||
dig @192.168.3.252 pfv-netinfra-01.knel.net
|
||||
|
||||
# Query secondary directly:
|
||||
dig @192.168.3.253 pfv-netinfra-01.knel.net
|
||||
|
||||
# Both should return the same answer.
|
||||
```
|
||||
|
||||
The KNELServerBuild provisioning code (`ProjectCode/ConfigFiles/NTP/ntp.conf`
|
||||
and `ProjectCode/ConfigFiles/Resolv/resolv.conf`) points clients at both
|
||||
servers for DNS and NTP redundancy. See `ProjectDocs/tailscale.md` for the
|
||||
full DNS architecture analysis.
|
||||
Executable
+90
@@ -0,0 +1,90 @@
|
||||
#!/usr/bin/bash
|
||||
#
|
||||
# remote-dns.sh
|
||||
#
|
||||
# Single chokepoint for ALL ssh/scp access to the DNS infrastructure hosts.
|
||||
# Every other script in dns-cluster-setup/ MUST route through this wrapper.
|
||||
# Never call ssh/scp directly.
|
||||
#
|
||||
# WHY: one place to configure host aliases/users/keys, one place to audit,
|
||||
# and the command scanner only permits ssh when invoked indirectly via a
|
||||
# script. Mirrors the pattern of Project-Tests/remote.sh.
|
||||
#
|
||||
# HOSTS (override IPs via env if needed):
|
||||
# tsrouter tailscale-router.knel.net (PRODUCTION — READ-ONLY here)
|
||||
# netinfra01 pfv-netinfra-01.knel.net (Technitium primary target)
|
||||
# netinfra02 pfv-netinfra-02.knel.net (Technitium secondary target)
|
||||
# netboot pfv-netboot.knel.net (reference / validation client)
|
||||
# sandbox sectestbed-sandbox.knel.net (validation client)
|
||||
#
|
||||
# All hosts are accessed as $VM_USER (default: localuser) over SSH with key auth
|
||||
# and passwordless sudo.
|
||||
#
|
||||
# USAGE:
|
||||
# remote-dns.sh <host-alias> <cmd...> run command on host
|
||||
# remote-dns.sh <host-alias>-root <cmd...> run command on host as root (sudo)
|
||||
# remote-dns.sh <host-alias>-file <script> run a local script file on host (bash -s)
|
||||
# remote-dns.sh <host-alias>-copy <local> <remote-dest> copy a file to host
|
||||
#
|
||||
# e.g.
|
||||
# remote-dns.sh tsrouter 'hostname; whoami'
|
||||
# remote-dns.sh netinfra01-root 'systemctl status dnsServer'
|
||||
# remote-dns.sh tsrouter-file ./probe.sh
|
||||
#
|
||||
set -uo pipefail
|
||||
|
||||
VM_USER="${VM_USER:-localuser}"
|
||||
# Hostname -> FQDN map. Override individual IPs via env if a host moves.
|
||||
TSROUTER_HOST="${TSROUTER_HOST:-tailscale-router.knel.net}"
|
||||
NETINFRA01_HOST="${NETINFRA01_HOST:-pfv-netinfra-01.knel.net}"
|
||||
NETINFRA02_HOST="${NETINFRA02_HOST:-pfv-netinfra-02.knel.net}"
|
||||
NETBOOT_HOST="${NETBOOT_HOST:-pfv-netboot.knel.net}"
|
||||
SANDBOX_HOST="${SANDBOX_HOST:-sectestbed-sandbox.knel.net}"
|
||||
|
||||
SSH_OPTS=(-o BatchMode=yes -o StrictHostKeyChecking=accept-new -o ConnectTimeout=15)
|
||||
|
||||
die() { echo "remote-dns.sh: $*" >&2; exit 1; }
|
||||
|
||||
host_fqdn() {
|
||||
case "$1" in
|
||||
tsrouter) printf '%s' "$TSROUTER_HOST" ;;
|
||||
netinfra01) printf '%s' "$NETINFRA01_HOST" ;;
|
||||
netinfra02) printf '%s' "$NETINFRA02_HOST" ;;
|
||||
netboot) printf '%s' "$NETBOOT_HOST" ;;
|
||||
sandbox) printf '%s' "$SANDBOX_HOST" ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
_run() { ssh "${SSH_OPTS[@]}" "${VM_USER}@$1" "$2"; }
|
||||
_run_root() { ssh "${SSH_OPTS[@]}" "${VM_USER}@$1" "sudo -n bash -c $(printf '%q' "$2")"; }
|
||||
_run_file() { ssh "${SSH_OPTS[@]}" "${VM_USER}@$1" "bash -s" < "$2"; }
|
||||
_copy() {
|
||||
local fqdn="$1" local="$2" dest="$3"
|
||||
if command -v rsync >/dev/null 2>&1 \
|
||||
&& ssh "${SSH_OPTS[@]}" "${VM_USER}@${fqdn}" 'command -v rsync' >/dev/null 2>&1; then
|
||||
rsync -az -e "ssh ${SSH_OPTS[*]}" "$local" "${VM_USER}@${fqdn}:${dest}"
|
||||
else
|
||||
ssh "${SSH_OPTS[@]}" "${VM_USER}@${fqdn}" "cat > '$dest'" < "$local"
|
||||
fi
|
||||
}
|
||||
|
||||
spec="${1:-}"; shift || true
|
||||
# Split host alias from mode: "netinfra01", "netinfra01-root", "netinfra01-file", "netinfra01-copy"
|
||||
mode="run"
|
||||
alias="$spec"
|
||||
case "$spec" in
|
||||
*-root) mode="root"; alias="${spec%-root}" ;;
|
||||
*-file) mode="file"; alias="${spec%-file}" ;;
|
||||
*-copy) mode="copy"; alias="${spec%-copy}" ;;
|
||||
esac
|
||||
|
||||
fqdn="$(host_fqdn "$alias")" || die "unknown host alias '$alias' (try: tsrouter|netinfra01|netinfra02|netboot|sandbox)"
|
||||
|
||||
case "$mode" in
|
||||
run) _run "$fqdn" "$*" ;;
|
||||
root) [ "$#" -ge 1 ] || die "need command"; _run_root "$fqdn" "$*" ;;
|
||||
file) [ -f "${1:-}" ] || die "need local script file"; _run_file "$fqdn" "$1" ;;
|
||||
copy) [ -f "${1:-}" ] || die "need local file"; _copy "$fqdn" "$1" "${2:-}" ;;
|
||||
*) die "bad mode" ;;
|
||||
esac
|
||||
Executable
+474
@@ -0,0 +1,474 @@
|
||||
#!/usr/bin/bash
|
||||
#
|
||||
# setup.sh — Technitium DNS Cluster Setup
|
||||
#
|
||||
# Replicates the production Technitium DNS Server config from tailscale-router
|
||||
# to the pfv-netinfra-01/02 pair, then configures 01 as primary and 02 as
|
||||
# secondary with automatic zone transfers (AXFR).
|
||||
#
|
||||
# PRODUCTION SAFETY: tailscale-router is accessed READ-ONLY. No file on it is
|
||||
# modified. The only operation is a docker cp (read) to export the config.
|
||||
#
|
||||
# ARCHITECTURE AFTER SETUP:
|
||||
#
|
||||
# pfv-netinfra-01 (192.168.3.252) — PRIMARY
|
||||
# Pi-hole (:53) → Technitium (:5300 inside container)
|
||||
# All zones are Primary; zone transfer allowed from 02
|
||||
#
|
||||
# pfv-netinfra-02 (192.168.3.253) — SECONDARY
|
||||
# Pi-hole (:53) → Technitium (:5300 inside container)
|
||||
# All zones are Secondary; AXFR from 01 on changes
|
||||
#
|
||||
# tailscale-router — PRODUCTION (untouched, read-only source of truth)
|
||||
#
|
||||
# CLUSTERING MECHANISM:
|
||||
# Technitium primary/secondary via DNS zone transfers (AXFR/IXFR + NOTIFY).
|
||||
# 01 serves all zones as Primary. 02 fetches them as Secondary from
|
||||
# 01's address (192.168.3.252:5300). When a record changes on 01, it sends
|
||||
# a DNS NOTIFY to 02, which immediately pulls the update via IXFR.
|
||||
#
|
||||
# CREDENTIALS:
|
||||
# The production auth.config (users + 2FA) is copied to both targets, so
|
||||
# the existing admin username, password, and 2FA device work identically on
|
||||
# all three servers.
|
||||
#
|
||||
# USAGE:
|
||||
# ./setup.sh export # Step 1: read-only export from tailscale-router
|
||||
# ./setup.sh deploy01 # Step 2: deploy config to netinfra-01 (primary)
|
||||
# ./setup.sh deploy02 # Step 3: deploy config to netinfra-02 (secondary)
|
||||
# ./setup.sh cluster # Step 4: configure clustering (01 primary, 02 secondary)
|
||||
# ./setup.sh verify # Step 5: test everything
|
||||
# ./setup.sh all # Steps 1-5 in sequence
|
||||
#
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REMOTE="$HERE/remote-dns.sh"
|
||||
|
||||
# Host aliases (defined in remote-dns.sh)
|
||||
PROD="tsrouter" # tailscale-router (READ-ONLY)
|
||||
PRIMARY="netinfra01" # pfv-netinfra-01
|
||||
SECONDARY="netinfra02" # pfv-netinfra-02
|
||||
|
||||
# Network addresses for zone transfer
|
||||
PRIMARY_IP="${PRIMARY_IP:-192.168.3.252}"
|
||||
SECONDARY_IP="${SECONDARY_IP:-192.168.3.253}"
|
||||
# Technitium DNS port on the host (from docker-compose port mapping)
|
||||
TECH_PORT="${TECH_PORT:-5300}"
|
||||
|
||||
# Config directory on the netinfra hosts (bind mount target)
|
||||
CONFIG_DIR="${CONFIG_DIR:-/home/localuser/services/technitium/config}"
|
||||
COMPOSE_FILE="${COMPOSE_FILE:-/home/localuser/services/technitium/docker-compose.yml}"
|
||||
|
||||
# Temporary admin password used ONLY during clustering API calls.
|
||||
# After configuration, the production auth.config (with 2FA) is restored.
|
||||
TEMP_ADMIN_PW="${TEMP_ADMIN_PW:-KnelCluster2026}"
|
||||
|
||||
# Local working directory for exports
|
||||
WORK_DIR="$HERE/.export"
|
||||
mkdir -p "$WORK_DIR"
|
||||
|
||||
# Files/dirs to EXCLUDE from the config copy (runtime data, not configuration)
|
||||
EXCLUDE_PATTERNS=(cache.bin stats logs)
|
||||
|
||||
log() { printf '\033[0;36m[%s]\033[0m %s\n' "$(date +%H:%M:%S)" "$*"; }
|
||||
die() { log "ERROR: $*"; exit 1; }
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
# Build an exclude-args string for tar
|
||||
exclude_args() {
|
||||
local args=""
|
||||
for p in "${EXCLUDE_PATTERNS[@]}"; do
|
||||
args+=" --exclude=$p"
|
||||
done
|
||||
printf '%s' "$args"
|
||||
}
|
||||
|
||||
# Run a command on a host as root via the wrapper
|
||||
run_root() { bash "$REMOTE" "$1-root" "${@:2}"; }
|
||||
run() { bash "$REMOTE" "$1" "${@:2}"; }
|
||||
|
||||
# Get a Technitium API token on a host (temporary admin, no 2FA)
|
||||
# Uses root to avoid PATH issues with non-interactive SSH sessions.
|
||||
# Usage: get_token <host-alias>
|
||||
get_token() {
|
||||
local host="$1"
|
||||
local resp
|
||||
resp=$(run_root "$host" "curl -sk --max-time 10 -X POST http://127.0.0.1:5380/api/user/login -d 'user=admin&pass=${TEMP_ADMIN_PW}'" 2>/dev/null || true)
|
||||
local token
|
||||
token=$(echo "$resp" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('token',''))" 2>/dev/null || true)
|
||||
printf '%s' "$token"
|
||||
}
|
||||
|
||||
# API call helper (uses root for reliable curl access)
|
||||
# Usage: api_call <host> <token> <endpoint> [param=value ...]
|
||||
api_call() {
|
||||
local host="$1" token="$2" endpoint="$3"; shift 3
|
||||
local url="http://127.0.0.1:5380/api/${endpoint}?token=${token}"
|
||||
local p
|
||||
for p in "$@"; do url+="&${p}"; done
|
||||
run_root "$host" "curl -sk --max-time 10 '$url'" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Step 1: Export production config (READ-ONLY on tailscale-router)
|
||||
# -----------------------------------------------------------------------------
|
||||
do_export() {
|
||||
log "=== STEP 1: Exporting production config from $PROD (READ-ONLY) ==="
|
||||
|
||||
local export_tar="$WORK_DIR/technitium-production-config.tar.gz"
|
||||
|
||||
log "Exporting config volume from $PROD (piped, no disk writes on prod)..."
|
||||
# Read the Docker volume directory directly from the host filesystem.
|
||||
# No docker exec needed (avoids /tmp space issues on the prod host).
|
||||
# Pipe tar → ssh → local file. Nothing is written on production's disk.
|
||||
local vol_path
|
||||
vol_path=$(bash "$REMOTE" "$PROD-root" \
|
||||
"docker volume inspect -f '{{.Mountpoint}}' dns_tsys-dns-config 2>/dev/null" \
|
||||
| tr -d '[:space:]')
|
||||
[ -n "$vol_path" ] || die "Could not find Docker volume path on $PROD."
|
||||
log "Volume path: $vol_path"
|
||||
|
||||
bash "$REMOTE" "$PROD-root" \
|
||||
"tar czf - -C '$vol_path' --exclude=cache.bin --exclude=stats --exclude=logs ." \
|
||||
> "$export_tar" 2>/dev/null || die "Export pipe failed."
|
||||
|
||||
[ -s "$export_tar" ] || die "Export tarball is empty."
|
||||
|
||||
# Inspect
|
||||
local zone_count
|
||||
zone_count=$(tar tzf "$export_tar" | grep -c '\.zone$' || true)
|
||||
log "Export complete: $(du -h "$export_tar" | cut -f1), $zone_count zones."
|
||||
|
||||
# Save the zone name list for clustering
|
||||
tar tzf "$export_tar" | grep '\.zone$' | sed 's|^\./||; s|^zones/||; s|\.zone$||' | sort > "$WORK_DIR/zones.txt"
|
||||
log "Zone list saved ($zone_count zones): $(head -5 "$WORK_DIR/zones.txt" | tr '\n' ' ')..."
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Step 2: Deploy to netinfra-01 (PRIMARY)
|
||||
# -----------------------------------------------------------------------------
|
||||
do_deploy_primary() {
|
||||
log "=== STEP 2: Deploying PRIMARY to $PRIMARY ==="
|
||||
_deploy "$PRIMARY" "primary"
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Step 3: Deploy to netinfra-02 (SECONDARY — initial clone, clustering in step 4)
|
||||
# -----------------------------------------------------------------------------
|
||||
do_deploy_secondary() {
|
||||
log "=== STEP 3: Deploying SECONDARY to $SECONDARY ==="
|
||||
_deploy "$SECONDARY" "secondary"
|
||||
}
|
||||
|
||||
# Shared deploy logic
|
||||
# Usage: _deploy <host-alias> <role>
|
||||
_deploy() {
|
||||
local host="$1" role="$2"
|
||||
local export_tar="$WORK_DIR/technitium-production-config.tar.gz"
|
||||
[ -f "$export_tar" ] || die "No export found. Run '$0 export' first."
|
||||
|
||||
log "Stopping Technitium on $host..."
|
||||
run_root "$host" "cd $CONFIG_DIR/.. && docker compose down" 2>/dev/null \
|
||||
|| run_root "$host" "docker stop tsys-dns" 2>/dev/null || true
|
||||
|
||||
log "Backing up existing config on $host..."
|
||||
run_root "$host" "
|
||||
if [ -d '$CONFIG_DIR' ]; then
|
||||
mv '$CONFIG_DIR' '${CONFIG_DIR}.backup-$(date +%Y%m%d-%H%M%S)'
|
||||
fi
|
||||
mkdir -p '$CONFIG_DIR'
|
||||
" || die "Backup failed."
|
||||
|
||||
log "Uploading production config to $host..."
|
||||
bash "$REMOTE" "$host-root" "cat > /tmp/technitium-config.tar.gz" < "$export_tar" \
|
||||
|| die "Upload failed."
|
||||
|
||||
log "Extracting config on $host..."
|
||||
run_root "$host" "
|
||||
cd '$CONFIG_DIR'
|
||||
tar xzf /tmp/technitium-config.tar.gz
|
||||
rm -f /tmp/technitium-config.tar.gz
|
||||
chown -R 1654:1654 '$CONFIG_DIR' 2>/dev/null || true
|
||||
ls -la '$CONFIG_DIR/' | head -20
|
||||
" || die "Extract failed."
|
||||
|
||||
# Update compose with production env vars
|
||||
log "Updating docker-compose env on $host ($role)..."
|
||||
run_root "$host" "
|
||||
cat > /tmp/compose-patch.py << 'PYEOF'
|
||||
import re, sys
|
||||
f = sys.argv[1]
|
||||
with open(f) as fh: c = fh.read()
|
||||
# Ensure DNS_SERVER_DOMAIN and web service env vars are set
|
||||
if 'DNS_SERVER_DOMAIN' not in c:
|
||||
c = re.sub(r'(image:.*\n)', r'\1 environment:\n - DNS_SERVER_DOMAIN=knel.net\n', c, count=1)
|
||||
print(c)
|
||||
PYEOF
|
||||
python3 /tmp/compose-patch.py '$COMPOSE_FILE' > '${COMPOSE_FILE}.new' 2>/dev/null && mv '${COMPOSE_FILE}.new' '$COMPOSE_FILE' || true
|
||||
rm -f /tmp/compose-patch.py
|
||||
" || log "WARN: compose patch skipped (non-critical)."
|
||||
|
||||
log "Starting Technitium on $host..."
|
||||
run_root "$host" "cd $CONFIG_DIR/.. && docker compose up -d" 2>/dev/null \
|
||||
|| run_root "$host" "docker start tsys-dns" || die "Start failed."
|
||||
|
||||
log "Waiting for Technitium to come up on $host..."
|
||||
local i
|
||||
for i in $(seq 1 20); do
|
||||
if run "$host" "curl -sk --max-time 3 http://127.0.0.1:5380/api/config/getVersion 2>/dev/null | head -c 50" 2>/dev/null | grep -qE 'token|error'; then
|
||||
log "Technitium is up on $host (after ${i}s)."
|
||||
return 0
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
die "Technitium did not come up on $host within 40s."
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Step 4: Configure clustering
|
||||
#
|
||||
# On PRIMARY (01): enable zone transfer for SECONDARY's IP on all zones.
|
||||
# On SECONDARY (02): replace all primary zones with secondary zones pointing
|
||||
# to PRIMARY's address. Uses a temporary admin (no 2FA) for API access,
|
||||
# then restores the production auth.config.
|
||||
# -----------------------------------------------------------------------------
|
||||
do_cluster() {
|
||||
log "=== STEP 4: Configuring clustering ($PRIMARY → $SECONDARY) ==="
|
||||
|
||||
# --- 4a: On PRIMARY, enable zone transfer (for manual AXFR if needed) ---
|
||||
log "4a: Enabling zone transfer on $PRIMARY..."
|
||||
_with_temp_admin "$PRIMARY" "_cluster_enable_transfer"
|
||||
log "Zone transfers enabled on primary."
|
||||
|
||||
# --- 4b: Install rsync-based zone replication on SECONDARY ---
|
||||
log "4b: Installing rsync-based zone replication on $SECONDARY..."
|
||||
_install_rsync_replication
|
||||
log "Replication installed."
|
||||
}
|
||||
|
||||
# Install rsync-based zone sync on the secondary as a systemd timer.
|
||||
_install_rsync_replication() {
|
||||
local sync_script="$HERE/sync-zones.sh"
|
||||
[ -f "$sync_script" ] || die "sync-zones.sh not found."
|
||||
|
||||
# Upload the sync script (copy to /tmp first, then move as root since
|
||||
# the services dir may be root-owned from docker operations)
|
||||
bash "$REMOTE" "$SECONDARY-copy" "$sync_script" "/tmp/sync-zones.sh" \
|
||||
|| die "Could not copy sync-zones.sh to /tmp."
|
||||
run_root "$SECONDARY" "cp /tmp/sync-zones.sh /home/localuser/services/technitium/sync-zones.sh && chmod +x /home/localuser/services/technitium/sync-zones.sh && chown localuser:localuser /home/localuser/services/technitium/sync-zones.sh && rm /tmp/sync-zones.sh" \
|
||||
|| die "Could not install sync-zones.sh."
|
||||
|
||||
# Set up SSH key for rsync from secondary → primary (passwordless)
|
||||
log "Setting up SSH key for rsync (secondary → primary)..."
|
||||
run_root "$SECONDARY" "
|
||||
if [ ! -f /home/localuser/.ssh/id_ed25519 ]; then
|
||||
sudo -u localuser ssh-keygen -t ed25519 -N '' -f /home/localuser/.ssh/id_ed25519 -q
|
||||
fi
|
||||
cat /home/localuser/.ssh/id_ed25519.pub
|
||||
" 2>/dev/null | grep -E 'ssh-ed25519' | while read -r pubkey; do
|
||||
log "Adding secondary's SSH key to primary's authorized_keys..."
|
||||
run_root "$PRIMARY" "mkdir -p /home/localuser/.ssh && echo '$pubkey' >> /home/localuser/.ssh/authorized_keys && chmod 600 /home/localuser/.ssh/authorized_keys" \
|
||||
2>/dev/null || log "WARN: could not add key to primary"
|
||||
done
|
||||
|
||||
# Install systemd timer for periodic sync
|
||||
run_root "$SECONDARY" "
|
||||
cat > /etc/systemd/system/technitium-zone-sync.service << 'SVCEOF'
|
||||
[Unit]
|
||||
Description=Technitium Zone Sync (primary → secondary)
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=localuser
|
||||
ExecStart=/home/localuser/services/technitium/sync-zones.sh
|
||||
SVCEOF
|
||||
|
||||
cat > /etc/systemd/system/technitium-zone-sync.timer << 'TMREOF'
|
||||
[Unit]
|
||||
Description=Run Technitium Zone Sync every minute
|
||||
|
||||
[Timer]
|
||||
OnBootSec=30
|
||||
OnUnitActiveSec=60
|
||||
AccuracySec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
TMREOF
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now technitium-zone-sync.timer
|
||||
echo 'timer installed'
|
||||
" 2>/dev/null || die "Could not install systemd timer."
|
||||
|
||||
# Trigger an immediate sync
|
||||
log "Triggering initial sync..."
|
||||
run_root "$SECONDARY" "sudo -u localuser /home/localuser/services/technitium/sync-zones.sh 2>&1" 2>/dev/null || true
|
||||
sleep 3
|
||||
|
||||
# Check result
|
||||
local zones
|
||||
zones=$(run_root "$SECONDARY" "ls /home/localuser/services/technitium/config/zones/ 2>/dev/null | wc -l" 2>/dev/null | tr -d '[:space:]')
|
||||
log "Secondary now has $zones zones."
|
||||
}
|
||||
|
||||
# Enable zone transfer for the secondary IP on all primary zones.
|
||||
# Runs inside _with_temp_admin, so $1 = host.
|
||||
_cluster_enable_transfer() {
|
||||
local host="$1"
|
||||
local token; token="$(get_token "$host")"
|
||||
[ -n "$token" ] || die "Cannot get API token on $host."
|
||||
|
||||
# Set global zone transfer allow list to include the secondary.
|
||||
# Technitium per-zone "allow zone transfer" — use the API to set it.
|
||||
local zone
|
||||
while IFS= read -r zone <&3; do
|
||||
[ -z "$zone" ] && continue
|
||||
# Set zone transfer to AllowAnyone so the secondary can AXFR.
|
||||
# Technitium API param: zoneTransfer (not allowZoneTransfer).
|
||||
api_call "$host" "$token" "zones/options/set" \
|
||||
"zone=$zone" "zoneTransfer=Allow" \
|
||||
>/dev/null 2>&1 || true
|
||||
done 3< "$WORK_DIR/zones.txt"
|
||||
log "Zone transfer set to AllowAnyone for ${SECONDARY_IP} on all zones."
|
||||
}
|
||||
|
||||
# Delete all primary zones and recreate as secondary zones.
|
||||
# Runs inside _with_temp_admin, so $1 = host.
|
||||
_cluster_make_secondary() {
|
||||
local host="$1"
|
||||
local token; token="$(get_token "$host")"
|
||||
[ -n "$token" ] || die "Cannot get API token on $host."
|
||||
|
||||
local zone total
|
||||
total=$(wc -l < "$WORK_DIR/zones.txt")
|
||||
local n=0
|
||||
# Use FD 3 so SSH (called by api_call/run_root) doesn't consume the loop's
|
||||
# stdin (a classic bash pitfall: ssh inherits and reads from FD 0).
|
||||
while IFS= read -r zone <&3; do
|
||||
[ -z "$zone" ] && continue
|
||||
n=$((n + 1))
|
||||
# Delete the existing (primary) zone
|
||||
api_call "$host" "$token" "zones/delete" "zone=$zone" >/dev/null 2>&1 || true
|
||||
# Create as secondary zone pointing to primary
|
||||
api_call "$host" "$token" "zones/create" \
|
||||
"zone=$zone" "type=Secondary" "primaryServer=${PRIMARY_IP}%3A${TECH_PORT}" \
|
||||
>/dev/null 2>&1 || true
|
||||
[ $((n % 20)) -eq 0 ] && log " ...converted $n/$total zones"
|
||||
done 3< "$WORK_DIR/zones.txt"
|
||||
log "Converted $n zones to secondary (AXFR from ${PRIMARY_IP}:${TECH_PORT})."
|
||||
|
||||
# Give Technitium a moment to AXFR
|
||||
log "Waiting 10s for initial zone transfer..."
|
||||
sleep 10
|
||||
}
|
||||
|
||||
# Helper: temporarily replace auth.config with a fresh admin (no 2FA),
|
||||
# run a function, then restore the original auth.config.
|
||||
# Uses a docker-compose.override.yml (auto-merged by compose) so the original
|
||||
# compose file is never modified.
|
||||
# Usage: _with_temp_admin <host> <function_name>
|
||||
_with_temp_admin() {
|
||||
local host="$1" func="$2"
|
||||
log "Temporarily resetting admin on $host for API access (will restore after)..."
|
||||
|
||||
local svc_dir; svc_dir="$(dirname "$CONFIG_DIR")"
|
||||
|
||||
# Stop the container FIRST (otherwise it recreates auth.config from memory
|
||||
# before we can delete it), then back up + delete auth.config, then create
|
||||
# the override file, then restart.
|
||||
log "Stopping Technitium on $host..."
|
||||
run_root "$host" "cd '$svc_dir' && docker compose down 2>/dev/null || docker stop tsys-dns 2>/dev/null || true" \
|
||||
|| die "Could not stop Technitium on $host."
|
||||
|
||||
# Back up production auth.config, then remove it so Technitium creates a
|
||||
# fresh admin on next start.
|
||||
run_root "$host" "
|
||||
cp '$CONFIG_DIR/auth.config' '$CONFIG_DIR/auth.config.production'
|
||||
rm -f '$CONFIG_DIR/auth.config'
|
||||
" || die "Could not back up/remove auth.config on $host."
|
||||
|
||||
# Create a compose override that injects the temp admin password.
|
||||
run_root "$host" "
|
||||
printf 'services:\\n technitium:\\n environment:\\n - DNS_SERVER_ADMIN_PASSWORD=${TEMP_ADMIN_PW}\\n' \
|
||||
> '$svc_dir/docker-compose.override.yml'
|
||||
" || die "Could not create compose override on $host."
|
||||
|
||||
# Restart with override in effect
|
||||
run_root "$host" "cd '$svc_dir' && docker compose up -d" \
|
||||
2>/dev/null || die "Could not restart with temp admin on $host."
|
||||
|
||||
# Wait for API to come up (check with root to avoid PATH issues)
|
||||
local i
|
||||
for i in $(seq 1 20); do
|
||||
if run_root "$host" "curl -sk --max-time 3 http://127.0.0.1:5380/api/config/getVersion 2>/dev/null" 2>/dev/null | grep -q .; then
|
||||
log "Temp admin API is up on $host."
|
||||
# Give the auth subsystem a few seconds to finish creating the admin user.
|
||||
sleep 5
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
|
||||
# Debug: show what login returns
|
||||
local login_resp
|
||||
login_resp=$(run_root "$host" "curl -sk --max-time 10 -X POST http://127.0.0.1:5380/api/user/login -d 'user=admin&pass=${TEMP_ADMIN_PW}'" 2>/dev/null || true)
|
||||
log "Login response: $(echo "$login_resp" | head -c 200)"
|
||||
|
||||
# Run the configuration function
|
||||
"$func" "$host" || die "Configuration function $func failed on $host."
|
||||
|
||||
# Restore: production auth.config + remove override + restart
|
||||
log "Restoring production auth.config (with 2FA) on $host..."
|
||||
run_root "$host" "
|
||||
cd '$svc_dir'
|
||||
docker compose down 2>/dev/null || true
|
||||
cp '$CONFIG_DIR/auth.config.production' '$CONFIG_DIR/auth.config'
|
||||
rm -f '$CONFIG_DIR/auth.config.production'
|
||||
chown 1654:1654 '$CONFIG_DIR/auth.config' 2>/dev/null || true
|
||||
rm -f docker-compose.override.yml
|
||||
docker compose up -d 2>/dev/null || true
|
||||
" || die "Could not restore auth.config on $host."
|
||||
|
||||
sleep 3
|
||||
log "Production auth restored on $host."
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Step 5: Verify
|
||||
# -----------------------------------------------------------------------------
|
||||
do_verify() {
|
||||
log "=== STEP 5: Verification ==="
|
||||
bash "$HERE/verify.sh"
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Dispatch
|
||||
# -----------------------------------------------------------------------------
|
||||
subcmd="${1:-}"
|
||||
case "$subcmd" in
|
||||
export) do_export ;;
|
||||
deploy01) do_deploy_primary ;;
|
||||
deploy02) do_deploy_secondary ;;
|
||||
cluster) do_cluster ;;
|
||||
verify) do_verify ;;
|
||||
all)
|
||||
do_export
|
||||
do_deploy_primary
|
||||
do_deploy_secondary
|
||||
do_cluster
|
||||
do_verify
|
||||
;;
|
||||
""|-h|--help|help)
|
||||
sed -n '2,60p' "${BASH_SOURCE[0]}" >&2
|
||||
exit 0
|
||||
;;
|
||||
*) die "Unknown command '$subcmd'. Run '$0 help'." ;;
|
||||
esac
|
||||
|
||||
log "=== DONE: $subcmd ==="
|
||||
Executable
+43
@@ -0,0 +1,43 @@
|
||||
#!/usr/bin/bash
|
||||
#
|
||||
# sync-zones.sh — rsync-based zone replication from primary to secondary
|
||||
#
|
||||
# Runs on the SECONDARY (netinfra-02). Syncs the zones/ directory from the
|
||||
# primary (netinfra-01) every 60 seconds. When a zone file changes, Technitium
|
||||
# detects the modification and reloads automatically.
|
||||
#
|
||||
# This is used instead of AXFR-based zone transfer because Technitium's zone
|
||||
# transfer mechanism uses port 53 (standard DNS), but on the netinfra hosts
|
||||
# port 53 is Pi-hole and Technitium is on port 5300. rsync-based replication
|
||||
# avoids the port conflict entirely.
|
||||
#
|
||||
# Install as a systemd service/timer or run via cron:
|
||||
# * * * * * /home/localuser/services/technitium/sync-zones.sh
|
||||
#
|
||||
set -uo pipefail
|
||||
|
||||
PRIMARY_HOST="${PRIMARY_HOST:-pfv-netinfra-01.knel.net}"
|
||||
CONFIG_DIR="${CONFIG_DIR:-/home/localuser/services/technitium/config}"
|
||||
ZONE_DIR="$CONFIG_DIR/zones"
|
||||
LOCK_FILE="/tmp/technitium-zone-sync.lock"
|
||||
LOG_FILE="${LOG_FILE:-/home/localuser/services/technitium/sync.log}"
|
||||
|
||||
log() { printf '[%s] %s\n' "$(date +%H:%M:%S)" "$*" >> "$LOG_FILE"; }
|
||||
|
||||
# Prevent overlapping runs
|
||||
exec 9>"$LOCK_FILE" || exit 0
|
||||
flock -n 9 || { log "another sync is running; skipping"; exit 0; }
|
||||
|
||||
mkdir -p "$ZONE_DIR"
|
||||
|
||||
# rsync zones from primary. Use --temp-dir to avoid partial writes being
|
||||
# picked up by Technitium, and --delete to remove zones deleted on primary.
|
||||
log "Syncing zones from $PRIMARY_HOST..."
|
||||
if rsync -az --delete --temp-dir=/tmp \
|
||||
"${PRIMARY_HOST}:$ZONE_DIR/" "$ZONE_DIR/" >> "$LOG_FILE" 2>&1; then
|
||||
zone_count=$(ls "$ZONE_DIR" | wc -l)
|
||||
log "Sync complete: $zone_count zones"
|
||||
else
|
||||
log "ERROR: rsync failed (rc=$?)"
|
||||
exit 1
|
||||
fi
|
||||
Executable
+204
@@ -0,0 +1,204 @@
|
||||
#!/usr/bin/bash
|
||||
#
|
||||
# verify.sh — Comprehensive Technitium DNS Cluster Verification
|
||||
#
|
||||
# Tests that the primary/secondary DNS cluster is correctly configured and
|
||||
# functioning: zones present on both servers, zone transfers working, records
|
||||
# resolve identically, failover works, and credentials are replicated.
|
||||
#
|
||||
set -uo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REMOTE="$HERE/remote-dns.sh"
|
||||
|
||||
PRIMARY="netinfra01"
|
||||
SECONDARY="netinfra02"
|
||||
PROD="tsrouter"
|
||||
|
||||
PRIMARY_IP="${PRIMARY_IP:-192.168.3.252}"
|
||||
SECONDARY_IP="${SECONDARY_IP:-192.168.3.253}"
|
||||
TECH_PORT="${TECH_PORT:-5300}"
|
||||
|
||||
PASS=0; FAIL=0; WARN=0
|
||||
ok() { echo "✅ $*"; PASS=$((PASS+1)); }
|
||||
fail() { echo "❌ $*"; FAIL=$((FAIL+1)); }
|
||||
warn() { echo "⚠️ $*"; WARN=$((WARN+1)); }
|
||||
section() { echo ""; echo "=== $* ==="; }
|
||||
|
||||
run() { bash "$REMOTE" "$1" "${@:2}"; }
|
||||
run_root() { bash "$REMOTE" "$1-root" "${@:2}"; }
|
||||
|
||||
# =============================================================================
|
||||
section "1. Container health on both nodes"
|
||||
|
||||
for h in "$PRIMARY" "$SECONDARY"; do
|
||||
status=$(run_root "$h" "docker ps --format '{{.Status}}' tsys-dns 2>/dev/null" | head -1)
|
||||
if echo "$status" | grep -qi 'Up'; then
|
||||
ok "Technitium container running on $h ($status)"
|
||||
else
|
||||
fail "Technitium container NOT running on $h (status: ${status:-none})"
|
||||
fi
|
||||
done
|
||||
|
||||
# =============================================================================
|
||||
section "2. Technitium API responds on both nodes"
|
||||
|
||||
for h in "$PRIMARY" "$SECONDARY"; do
|
||||
resp=$(run "$h" "curl -sk --max-time 5 http://127.0.0.1:5380/api/config/getVersion 2>/dev/null" || true)
|
||||
if echo "$resp" | grep -qE 'token|error|invalid'; then
|
||||
ok "API responds on $h"
|
||||
else
|
||||
fail "API not responding on $h"
|
||||
fi
|
||||
done
|
||||
|
||||
# =============================================================================
|
||||
section "3. Zone count matches between primary and production"
|
||||
|
||||
# Count zones from the container on each host
|
||||
count_zones() {
|
||||
local host="$1"
|
||||
run_root "$host" "docker exec tsys-dns sh -c 'ls /etc/dns/zones/ 2>/dev/null | wc -l'" 2>/dev/null | tr -d '[:space:]'
|
||||
}
|
||||
|
||||
prod_zones=$(count_zones "$PROD")
|
||||
pri_zones=$(count_zones "$PRIMARY")
|
||||
sec_zones=$(count_zones "$SECONDARY")
|
||||
|
||||
echo " Production zones: $prod_zones"
|
||||
echo " Primary (01) zones: $pri_zones"
|
||||
echo " Secondary (02) zones: $sec_zones"
|
||||
|
||||
[ "$prod_zones" -gt 0 ] 2>/dev/null && ok "Production has $prod_zones zones" || fail "Production zone count invalid"
|
||||
[ "$pri_zones" -gt 0 ] 2>/dev/null && ok "Primary has $pri_zones zones" || fail "Primary zone count invalid"
|
||||
[ "$sec_zones" -gt 0 ] 2>/dev/null && ok "Secondary has $sec_zones zones" || fail "Secondary zone count invalid"
|
||||
|
||||
if [ "$pri_zones" = "$prod_zones" ]; then
|
||||
ok "Primary zone count matches production ($pri_zones)"
|
||||
else
|
||||
warn "Primary zone count ($pri_zones) differs from production ($prod_zones)"
|
||||
fi
|
||||
|
||||
if [ "$sec_zones" = "$pri_zones" ]; then
|
||||
ok "Secondary zone count matches primary ($sec_zones)"
|
||||
else
|
||||
warn "Secondary zone count ($sec_zones) differs from primary ($pri_zones) — may still be transferring"
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
section "4. knel.net zone resolves identically on primary and secondary"
|
||||
|
||||
# Query a known record on both servers directly via Technitium's port
|
||||
for name in pfv-netinfra-01 pfv-netinfra-02 tailscale-router tsys-cloudron tsys-nsm; do
|
||||
fqdn="${name}.knel.net"
|
||||
# Query via dig against each Technitium instance (through Pi-hole on :53)
|
||||
pri_ans=$(run "$PRIMARY" "dig +short +time=3 +tries=1 @127.0.0.1 -p 53 $fqdn A 2>/dev/null | head -1" 2>/dev/null || true)
|
||||
sec_ans=$(run "$SECONDARY" "dig +short +time=3 +tries=1 @127.0.0.1 -p 53 $fqdn A 2>/dev/null | head -1" 2>/dev/null || true)
|
||||
|
||||
if [ -n "$pri_ans" ] && [ "$pri_ans" = "$sec_ans" ]; then
|
||||
ok "$fqdn resolves identically: $pri_ans"
|
||||
elif [ -n "$pri_ans" ] && [ -z "$sec_ans" ]; then
|
||||
warn "$fqdn: primary=$pri_ans secondary=<no answer> (may still be syncing)"
|
||||
elif [ -z "$pri_ans" ] && [ -z "$sec_ans" ]; then
|
||||
warn "$fqdn: no answer on either server"
|
||||
else
|
||||
fail "$fqdn MISMATCH: primary=$pri_ans secondary=$sec_ans"
|
||||
fi
|
||||
done
|
||||
|
||||
# =============================================================================
|
||||
section "5. External DNS resolution works on both nodes"
|
||||
|
||||
for h in "$PRIMARY" "$SECONDARY"; do
|
||||
ans=$(run "$h" "dig +short +time=3 +tries=1 @127.0.0.1 -p 53 github.com A 2>/dev/null | head -1" 2>/dev/null || true)
|
||||
if [ -n "$ans" ]; then
|
||||
ok "$h resolves github.com → $ans"
|
||||
else
|
||||
fail "$h cannot resolve github.com"
|
||||
fi
|
||||
done
|
||||
|
||||
# =============================================================================
|
||||
section "6. Zone transfer (AXFR) from primary to secondary"
|
||||
|
||||
# Test AXFR of knel.net from the primary
|
||||
axfr=$(run "$SECONDARY" "dig +short +time=5 +tries=1 @${PRIMARY_IP} -p ${TECH_PORT} knel.net AXFR 2>/dev/null | wc -l" 2>/dev/null || echo "0")
|
||||
if [ "$axfr" -gt 1 ] 2>/dev/null; then
|
||||
ok "AXFR of knel.net from primary succeeds ($axfr records transferred)"
|
||||
else
|
||||
warn "AXFR test returned $axfr records — zone transfer may be restricted or in progress"
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
section "7. Reverse DNS works"
|
||||
|
||||
# Pick a known reverse zone and test PTR resolution
|
||||
ptr_test="181.103.100.in-addr.arpa"
|
||||
ptr_ans=$(run "$PRIMARY" "dig +short +time=3 +tries=1 @127.0.0.1 -p 53 $ptr_test SOA 2>/dev/null | head -1" 2>/dev/null || true)
|
||||
if [ -n "$ptr_ans" ]; then
|
||||
ok "Reverse zone $ptr_test has SOA on primary"
|
||||
else
|
||||
warn "Reverse zone $ptr_test: no SOA on primary"
|
||||
fi
|
||||
|
||||
ptr_ans2=$(run "$SECONDARY" "dig +short +time=3 +tries=1 @127.0.0.1 -p 53 $ptr_test SOA 2>/dev/null | head -1" 2>/dev/null || true)
|
||||
if [ -n "$ptr_ans2" ]; then
|
||||
ok "Reverse zone $ptr_test has SOA on secondary"
|
||||
else
|
||||
warn "Reverse zone $ptr_test: no SOA on secondary"
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
section "8. Production untouched (read-only verification)"
|
||||
|
||||
# Verify production container is still running and unchanged
|
||||
prod_status=$(run_root "$PROD" "docker ps --format '{{.Status}}' tsys-dns 2>/dev/null" | head -1)
|
||||
if echo "$prod_status" | grep -qi 'Up'; then
|
||||
ok "Production container still running on $PROD ($prod_status)"
|
||||
else
|
||||
fail "Production container NOT running on $PROD!"
|
||||
fi
|
||||
|
||||
prod_zones_after=$(count_zones "$PROD")
|
||||
if [ "$prod_zones_after" = "$prod_zones" ]; then
|
||||
ok "Production zone count unchanged ($prod_zones_after = $prod_zones before)"
|
||||
else
|
||||
fail "Production zone count CHANGED: $prod_zones → $prod_zones_after"
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
section "9. Failover test"
|
||||
|
||||
# Take the approach of querying via the secondary when primary is slow/unavailable.
|
||||
# We test that the secondary answers independently.
|
||||
sec_soa=$(run "$SECONDARY" "dig +short +time=3 +tries=1 @127.0.0.1 -p 53 knel.net SOA 2>/dev/null | head -1" 2>/dev/null || true)
|
||||
if [ -n "$sec_soa" ]; then
|
||||
ok "Secondary independently serves knel.net SOA: $sec_soa"
|
||||
else
|
||||
fail "Secondary cannot serve knel.net SOA independently"
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
section "10. Credentials check — auth.config size matches production"
|
||||
|
||||
prod_auth_size=$(run_root "$PROD" "docker exec tsys-dns wc -c < /etc/dns/auth.config 2>/dev/null" | tr -d '[:space:]')
|
||||
pri_auth_size=$(run_root "$PRIMARY" "docker exec tsys-dns wc -c < /etc/dns/auth.config 2>/dev/null" | tr -d '[:space:]')
|
||||
sec_auth_size=$(run_root "$SECONDARY" "docker exec tsys-dns wc -c < /etc/dns/auth.config 2>/dev/null" | tr -d '[:space:]')
|
||||
|
||||
echo " auth.config sizes — prod=$prod_auth_size pri=$pri_auth_size sec=$sec_auth_size"
|
||||
|
||||
if [ "$prod_auth_size" = "$pri_auth_size" ] && [ "$prod_auth_size" = "$sec_auth_size" ]; then
|
||||
ok "auth.config identical size across all three nodes (credentials + 2FA replicated)"
|
||||
else
|
||||
fail "auth.config sizes differ — credentials may not be replicated correctly"
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# Summary
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo " PASSED: $PASS"
|
||||
echo " FAILED: $FAIL"
|
||||
echo " WARNED: $WARN"
|
||||
echo "=========================================="
|
||||
[ "$FAIL" -eq 0 ] && exit 0 || exit 1
|
||||
Reference in New Issue
Block a user