# Tailscale vs. Managed DNS — Architecture Analysis

> **Status:** analysis for review. No code decisions are final. Read the
> "Known issues" section before acting on the managed-resolv.conf change.

## 1. Executive summary

Every host in this build runs the Tailscale client, and Tailscale — by default —
**manages `/etc/resolv.conf` itself**, pointing it at `100.100.100.100`
(Tailscale's MagicDNS resolver). This directly conflicts with the managed
`resolv.conf` (pointing at `192.168.3.252`/`192.168.3.253`) that
`SetupNewSystem.sh` deploys: whichever runs last wins, and Tailscale's daemon
re-wins on every `tailscale up` and on reboot.

Worse, a probe of the live network shows that **knel.net device records only
resolve through the Tailscale 100.100.100.100 path** — querying the LAN IPs of
the DNS servers directly returns NXDOMAIN for current hostnames (the Technitium
`knel.net` zone has the SOA but is stale/empty of actual records). So pointing
`resolv.conf` at the LAN IPs would break resolution of the very names this
project's modules depend on (`tsys-nsm.knel.net`, `tsys-cloudron.knel.net`,
`tsys-librenms.knel.net`).

This document lays out the options and a recommended path forward.

## 2. How name resolution actually works today (as measured)

Probed from `sectestbed-sandbox` (192.168.3.50):

| Query path | External name (`github.com`) | knel.net device name (`pfv-netinfra-01.knel.net`) |
|---|---|---|
| Via current resolver = `100.100.100.100` (Tailscale) | resolves | **resolves** → `100.70.181.72` (Tailscale CGNAT) |
| Direct `dig @192.168.3.252` (Technitium, LAN) | resolves (recurses) | **NXDOMAIN** (SOA present, no record) |
| Direct `dig @192.168.3.253` (Pi-hole, LAN) | resolves (recurses) | **NXDOMAIN** (SOA present, no record) |

Other measured facts:

- `dig @192.168.3.252 knel.net SOA` → `NOERROR`, returns
  `knel.net. 900 IN SOA dns.knel.net. hostadmin.knel.net. 2025062313 …`
  (serial dated **2025-06-23** — the zone exists but is stale).
- NTP on both `.252` and `.253` answers time queries (stratum 2/3).
- The live `/etc/resolv.conf` on a deployed host reads:
  ```
  # resolv.conf(5) file generated by tailscale
  # DO NOT EDIT THIS FILE BY HAND -- CHANGES WILL BE OVERWRITTEN
  nameserver 100.100.100.100
  nameserver fd7a:115c:a1e0::53
  search knel.net
  ```

**Interpretation:** the `knel.net` device→Tailscale-IP mappings are synthesised
by Tailscale's MagicDNS from the tailnet device registry (every device that
joins the tailnet gets `hostname.knel.net` → its `100.x.x.x` address). The
Technitium `knel.net` zone is a separate, manually-maintained zone that has
fallen out of date. The two are not the same source of truth.

## 3. The core tension

| Goal | Who provides it today |
|---|---|
| Resolve `*.knel.net` device names (→ Tailscale IPs) | Tailscale MagicDNS via `100.100.100.100` |
| Resolve external names with ad-blocking | Pi-hole (`.253`), reachable via Tailscale → Technitium → Pi-hole chain |
| Redundant, low-latency, tunnel-independent DNS | LAN resolvers `.252`/`.253` — **but these lack knel.net records** |
| Authoritative time | NTP on `.252`/`.253` (works on either path) |

The conflict: you cannot simply point `resolv.conf` at the LAN resolvers,
because they do not know about the current `knel.net` device records, and
several modules in this project resolve `knel.net` hostnames at runtime
(wazuh manager, postfix relay, syslog target). You also cannot ignore Tailscale,
because it is the only thing that resolves those names today.

## 4. Options

### Option A — Let Tailscale own DNS (status quo, `accept-dns=true`)

Leave the default. Tailscale writes `100.100.100.100` to `resolv.conf`; the
control-plane forwarding (`100.100.100.100` → Technitium → Pi-hole) handles
external names and ad-blocking; MagicDNS handles `knel.net` device names.

| Pros | Cons |
|---|---|
| Zero per-host config; new machines "just work" on `tailscale up` | **All DNS depends on the Tailscale daemon being up.** If `tailscaled` dies, every name lookup fails — including the ones you need to SSH in and fix it. |
| MagicDNS + knel.net names resolve automatically | Latency: every query goes host→tailscaled→100.100.100.100→(tunnel)→Technitium→Pi-hole→upstream |
| Ad-blocking preserved (via the Pi-hole hop) | Overwrites the managed `resolv.conf` — the `.252`/`.253` redundancy is lost |
| Centralised in the Tailscale admin console | Single resolver in `resolv.conf` (`100.100.100.100`); no glibc-level failover |
| | Boot-order risk: early-boot processes have no DNS until `tailscaled` is up |

### Option B — Pin resolv.conf to the LAN resolvers (`accept-dns=false`)

Set `--accept-dns=false` on every host and keep the managed `resolv.conf`
pointing at `.252`/`.253`.

| Pros | Cons |
|---|---|
| DNS independent of Tailscale — survives `tailscaled` outages | **`*.knel.net` device names break (NXDOMAIN)** because the LAN resolvers' knel.net zone is stale. This breaks wazuh/postfix/syslog hostname resolution. |
| Lowest latency, full glibc-level failover across two servers | MagicDNS names (`*.ts.net`) do not resolve |
| Managed `resolv.conf` wins uncontested | Requires fixing the Technitium/Pi-hole `knel.net` zone to mirror the Tailscale device records before this is viable |
| Boot-time DNS works immediately | Off-LAN hosts (laptops) can't reach `.252`/`.253` without the tunnel — back to needing Tailscale |

> **Not recommended as-is.** Only viable **after** the `knel.net` zone on
> `.252`/`.253` is repopulated with current device records (see §6).

### Option C — Tailscale Split DNS (per-domain routing)

MagicDNS `ON`, "Override local DNS" `OFF` in the admin console; only `ts.net`
(and explicitly split domains) route to `100.100.100.100`, everything else stays
on the system resolver.

| Pros | Cons |
|---|---|
| Best of both worlds: MagicDNS names resolve AND general queries go direct | Requires `systemd-resolved` (or NetworkManager `dns=dnsmasq`) for per-domain routing. These hosts use a **plain `/etc/resolv.conf`** — on which Tailscale **cannot** do per-domain split; it replaces the whole file. |
| Reduces tunnel dependency for non-Tailscale names | Migrating every host to `systemd-resolved` is a significant, cross-cutting change |
| | More moving parts to reason about and debug |

### Option D — Make Tailscale push the LAN resolvers as global nameservers

In the admin console, set global nameservers to `192.168.3.252`/`192.168.3.253`,
keep `accept-dns=true`.

| Pros | Cons |
|---|---|
| Clients get the LAN resolvers via Tailscale config (consistent) | Tailscale still overwrites `resolv.conf` |
| MagicDNS still works (100.100.100.100 added for `ts.net`/`knel.net`) | On-LAN hosts don't need Tailscale to find `.252`/`.253` — pure indirection |
| Centralised management | Still depends on `tailscaled` for DNS |
| | `knel.net` device names still only resolve via the Tailscale path, so the LAN resolvers being "global" doesn't help those names unless the zone is fixed |

## 5. Recommendation

**Short term (unblock now): Option A — let Tailscale own DNS.** Revert/disable
the managed-`resolv.conf` deployment so provisioning stops fighting Tailscale.
Today, `knel.net` device names **only** resolve through Tailscale, and this
project's modules depend on those names, so Tailscale-managed DNS is the only
thing that currently works end-to-end. Keep the NTP change (LAN IPs, no DNS
dependency) — that part is safe and beneficial regardless.

**Medium term (the real fix): populate the `knel.net` zone on the LAN
resolvers**, then choose B or C. Concretely:

1. Make Technitium (`.252`) authoritative for `knel.net` **with current records**
   (mirror the Tailscale device→IP mappings, or enable a zone-transfer/sync from
   the Tailscale device registry, or use Technitium's "Tailscale" DNS app if
   available). Confirm `dig @192.168.3.252 pfv-netinfra-01.knel.net` returns an
   answer, not NXDOMAIN.
2. Make Pi-hole (`.253`) forward `knel.net` to Technitium (or also serve the
   zone), so both resolvers in the pair can answer internal names — otherwise
   glibc failover to `.253` would silently break knel.net lookups.
3. *Then* pin `resolv.conf` to `.252`/`.253` with `--accept-dns=false`
   (Option B), gaining tunnel-independent, redundant DNS.

**Long term (optional, if per-domain routing is wanted): Option C** — adopt
`systemd-resolved` and configure Tailscale Split DNS so `ts.net`/`knel.net` go
to MagicDNS and everything else goes direct. Only worth the migration cost if
you specifically need `*.ts.net` short-name resolution alongside direct LAN DNS.

### Why not just force `.252`/`.253` today?

Because it regresses name resolution for the hostnames this project already
uses. Concretely, with `resolv.conf` pinned to the LAN resolvers the following
would fail to resolve:

- `ProjectCode/Modules/Security/secharden-wazuh.sh` → `WAZUH_MANAGER="tsys-nsm.knel.net"`
- `ProjectCode/SetupNewSystem.sh` → `postconf -e "relayhost = tsys-cloudron.knel.net"`
- `ProjectCode/ConfigFiles/Syslog/rsyslog.conf` → `*.* @tsys-librenms.knel.net:514`

All three resolve cleanly via `100.100.100.100` today and return NXDOMAIN via
`.252`/`.253`. Pinning the LAN resolvers before the zone is fixed would break
wazuh, mail relay, and syslog.

## 6. Known issues / action items

1. **Technitium `knel.net` zone is stale.** SOA serial `2025062313`
   (2025-06-23); current device names return NXDOMAIN from the LAN interface.
   Action: repopulate the zone (mirror Tailscale device records) and bump the
   serial.
2. **Pi-hole (`.253`) has no `knel.net` device records either.** For the pair
   to be truly redundant for internal names, `.253` must either serve the same
   zone or conditional-forward `knel.net` to `.252`. Action: configure Pi-hole
   to forward `knel.net` to Technitium.
3. **The managed-`resolv.conf` change (commit f010fa9) conflicts with
   Tailscale.** As written, `SetupNewSystem.sh` writes `resolv.conf` with
   `.252`/`.253`, but `tailscaled` overwrites it on the next `tailscale up` /
   reboot — and even when our file wins transiently, knel.net names break. See
   §5 for the recommended handling.
4. **NTP change is safe and good.** `ntp.conf` now uses LAN IPs
   (`192.168.3.252`/`192.168.3.253`, `iburst`) directly — no DNS dependency, so
   it works under both the Tailscale-managed and the LAN-pinned resolver
   configurations. Keep this regardless of the DNS decision.
5. **Split-horizon possibility (unconfirmed).** It is possible Technitium serves
   a richer `knel.net` zone on its Tailscale interface (`100.x`) than on its LAN
   interface (`192.168.3.252`). If so, the fix is to make the LAN view match the
   Tailscale view. Worth confirming with `dig @<technitium-tailscale-ip> knel.net host`.

## 7. Implementation guidance (once the zone is fixed)

When you are ready to move to tunnel-independent DNS (Option B):

1. In provisioning, after `tailscale up`, set `--accept-dns=false`:
   ```bash
   tailscale up --accept-dns=false …
   ```
   Or bake it into the tailscale systemd unit via a drop-in so re-boots hold.
2. *Then* deploy the managed `resolv.conf` (`.252`/`.253`). Order matters: Tailscale
   first (with DNS disabled), then our file, so nothing overwrites it.
3. Add a watchdog (timer) that restores `resolv.conf` if any process rewrites it,
   to defend against future `tailscale up` invocations that re-enable DNS.
4. Validate with `Project-Tests/validation/dns-ntp-redundancy.sh` — and extend
   its probe to assert `*.knel.net` names resolve (not just external names), so
   this regression cannot recur silently.

## 8. TL;DR

- **DNS**: don't fight Tailscale yet. Today `knel.net` names only resolve via
  Tailscale, and this project depends on them. Fix the Technitium/Pi-hole
  `knel.net` zone first, *then* pin the LAN resolvers.
- **NTP**: the LAN-IP change is correct and safe; keep it.
- **The managed `resolv.conf` (`.252`/`.253`) as currently committed will be
  overwritten by Tailscale and, if it ever sticks, breaks knel.net resolution —
  see §5/§6 before relying on it.**