mrcharles
d6988bb35f
Root cause of intermittent DNS up/down alerts: Pi-hole's default
rate-limit (1000 queries/60s per client) was throttling Uptime Kuma
on the Cloudron VPS (tsys-cloudron, 100.107.35.78). Uptime Kuma sends
high-volume DNS queries to monitor dozens of hosts; when it exceeded
the limit, Pi-hole responded REFUSED, which Uptime Kuma detected as
DNS being down. This happened every 1-2 minutes.
Evidence: 40 rate-limiting events against 100.107.35.78 in recent
netinfra-01 Pi-hole logs. Also 10.53.0.1 (Docker bridge gateway) hit
4852 queries in a single 60s window.
Fix: Set dns.rateLimit.count=0 and dns.rateLimit.interval=0 on both
nodes (private tailnet, no DNS amplification risk). Also persisted in
docker-compose.yml via FTLCONF env vars.
Combined with prior IPv6 fix (4f82520), this resolves all known causes
of DNS flapping.
[#376]
62 lines
2.7 KiB
Markdown
62 lines
2.7 KiB
Markdown
# netinfra/pihole/ — Pi-hole recursive DNS (pfv-netinfra-01/02)
|
|
|
|
> **Redmine:** [#376](https://projects.knownelement.com/issues/376) (up/down alerts + commit hardening) · [#357](https://projects.knownelement.com/issues/357) (cluster build, closed)
|
|
|
|
Pi-hole v6 runs as the recursive resolver on port 53 of both DNS nodes.
|
|
Technitium (`tsys-dns`) runs as the authoritative server on port 5300; the two
|
|
share the `dnsnet` Docker network so Pi-hole can conditional-forward `knel.net`
|
|
zones to Technitium.
|
|
|
|
## Hardening (defense-in-depth against gravity.db corruption)
|
|
|
|
The operator hit a production outage when Pi-hole's `/dev/shm` was too small,
|
|
corrupting `gravity.db`. The live config on both nodes now includes:
|
|
|
|
- **`shm_size: 1024M`** — fixes the root cause (default 64M was too small).
|
|
- **`gravity-validate.sh`** — entrypoint that checks the SQLite header of
|
|
`gravity.db` before start; auto-moves a corrupt/empty DB aside so Pi-hole
|
|
can regenerate it cleanly.
|
|
- **Healthcheck** — `dig +norecurse @127.0.0.1 pi.hole` + gravity.db non-empty;
|
|
fails the container if DNS or the DB is broken.
|
|
- **`autoheal`** sidecar — restarts any container labeled `autoheal=true` that
|
|
goes unhealthy.
|
|
|
|
## Deploy
|
|
|
|
The compose reads the web UI password from a gitignored `.env`:
|
|
|
|
```bash
|
|
cd netinfra/pihole/
|
|
cp .env.example .env # then edit .env and set PIHOLE_WEB_PASSWORD
|
|
docker compose up -d
|
|
```
|
|
|
|
Files are deployed to `/home/localuser/services/pihole/` on each node. Volumes
|
|
(`./etc-pihole`, `./etc-dnsmasq.d`) hold the persistent state.
|
|
|
|
## IPv6 disabled + Rate-limiting disabled
|
|
|
|
### IPv6
|
|
Both netinfra nodes run **IPv4-only**. IPv6 is disabled at the kernel level
|
|
(`/etc/sysctl.d/99-disable-ipv6.conf`) because netinfra-01 has no IPv6 internet
|
|
route, and Pi-hole's default IPv6 upstream (Google `2001:4860:4860::8888`) was
|
|
causing continuous "Network unreachable" errors + intermittent DNS failures
|
|
detected by Uptime Kuma. The upstream is now `8.8.8.8` (IPv4 only).
|
|
|
|
### Rate-limiting
|
|
Pi-hole's default rate-limit (1000 queries / 60 seconds per client) was
|
|
**the root cause of Uptime Kuma DNS flapping**. Uptime Kuma runs on the
|
|
Cloudron VPS (`tsys-cloudron`, `100.107.35.78`) and sends high-volume DNS
|
|
queries to monitor dozens of hosts. When it exceeded 1000 queries/60s,
|
|
Pi-hole responded with REFUSED, which Uptime Kuma detected as DNS being
|
|
down. Rate-limiting is now disabled (`count=0, interval=0`) since this is a
|
|
private tailnet with no risk of DNS amplification attacks.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
dig @127.0.0.1 +short google.com # recursive
|
|
dig @127.0.0.1 +short git.knownelement.com # knel.net via Technitium forward
|
|
docker inspect pihole --format '{{.State.Health.Status}}'
|
|
```
|