Audit findings note (55/64 systems, read-only): https://projects.knownelement.com/issues/696#note-4088 Map + deploy plan: https://community.turnsys.com/t/322 Key: relayhost empty fleet-wide (direct-to-MX today); prod PMG pair already installed (VM 604/711, PMG 9.1, unclustered); VIP slot proposed 192.168.3.249; open PQs in questions-v8.md.
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
# netinfra/mail — PMG / fleet mail tooling
|
||||
|
||||
Pointers + usage. Knowledge doc (map, findings, deploy plan):
|
||||
[Discourse t/322 — Mail environment map + PMG deploy plan](https://community.turnsys.com/t/322).
|
||||
Ticket: [Redmine #696](https://projects.knownelement.com/issues/696).
|
||||
|
||||
## Files
|
||||
|
||||
- `audit-mail-env.sh` — read-only mail probe, runs ON a target
|
||||
(postfix config, alt relays, identity, :25 listeners, relay log
|
||||
evidence). Delivered via the ssh chokepoints only
|
||||
(`remote.sh prox-file|vm-file`, `remote-dns.sh <host>-file`).
|
||||
- `run-mail-audit.sh` — drives the probe across
|
||||
`audit-targets.txt`; output per host under `.crush/audit/mail/`.
|
||||
- `audit-targets.txt` — fleet target list (`mode|name|user`).
|
||||
|
||||
## Re-run the audit
|
||||
|
||||
```bash
|
||||
bash netinfra/mail/run-mail-audit.sh
|
||||
```
|
||||
@@ -0,0 +1,78 @@
|
||||
#!/usr/bin/env bash
|
||||
# audit-mail-env.sh — read-only mail environment probe.
|
||||
#
|
||||
# Runs ON the target host, delivered via the ssh chokepoints:
|
||||
# tests/remote.sh prox-file|vm-file OR remote-dns.sh <host>-file
|
||||
# Collects: installed MTAs, postfix relay config, alternative relay configs
|
||||
# (ssmtp/msmtp/nullmailer/dma), identity (mailname/aliases/forward),
|
||||
# smtp listeners, and recent relay evidence from mail logs.
|
||||
# Makes NO changes. Redacts auth material. Output is "KEY value" text.
|
||||
set -u
|
||||
|
||||
as_root() {
|
||||
if [ "$(id -u)" -eq 0 ]; then
|
||||
"$@"
|
||||
else
|
||||
if sudo -n "$@" 2>/dev/null; then
|
||||
:
|
||||
else
|
||||
"$@"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
echo "== host $(hostname -f 2>/dev/null || hostname) | $(date '+%F %T %Z')"
|
||||
|
||||
echo "== mtas"
|
||||
for b in postconf postfix sendmail exim4 msmtp ssmtp nullmailer-send dma; do
|
||||
p="$(command -v "$b" 2>/dev/null)"
|
||||
if [ -n "${p}" ]; then
|
||||
echo "bin ${b}: ${p}"
|
||||
fi
|
||||
done
|
||||
|
||||
echo "== postfix"
|
||||
if command -v postconf >/dev/null 2>&1; then
|
||||
for k in myhostname myorigin relayhost mydestination inet_interfaces; do
|
||||
v="$(as_root postconf -h "$k" 2>/dev/null)"
|
||||
echo "postconf ${k} = ${v:-<empty>}"
|
||||
done
|
||||
else
|
||||
echo "postconf: not installed"
|
||||
grep -Hs '^relayhost' /etc/postfix/main.cf 2>/dev/null
|
||||
fi
|
||||
|
||||
echo "== alt-relay"
|
||||
if [ -r /etc/ssmtp/ssmtp.conf ]; then
|
||||
sed -n -E 's/^(AuthPass|AuthUser)=.*/\1=<redacted>/p; s/^(mailhub|hostname|rewriteDomain|UseSTARTTLS|UseTLS)=/\1=/p' /etc/ssmtp/ssmtp.conf | sed 's/^/ssmtp /'
|
||||
fi
|
||||
for f in /etc/msmtp.conf /root/.msmtprc "${HOME}/.msmtprc"; do
|
||||
if [ -r "$f" ]; then
|
||||
grep -Es '^(account|host|port|from|auth |tls|syslog)' "$f" | sed "s|^|msmtp ${f} |"
|
||||
fi
|
||||
done
|
||||
for f in /etc/nullmailer/remotes /var/spool/nullmailer/remotes; do
|
||||
if [ -r "$f" ]; then
|
||||
sed -E 's/ --[^ ]*//g' "$f" | sed 's/^/nullmailer-remote /'
|
||||
fi
|
||||
done
|
||||
if [ -r /etc/dma/dma.conf ]; then
|
||||
grep -Es '^(SMARTHOST|PORT|MAILNAME)' /etc/dma/dma.conf | sed 's/^/dma /'
|
||||
fi
|
||||
|
||||
echo "== identity"
|
||||
if [ -r /etc/mailname ]; then
|
||||
echo "mailname: $(cat /etc/mailname)"
|
||||
fi
|
||||
grep -hsE '^root:' /etc/aliases /etc/mail/aliases 2>/dev/null | sed 's/^/alias /'
|
||||
if [ -r /root/.forward ]; then
|
||||
echo "root-forward: $(head -1 /root/.forward)"
|
||||
fi
|
||||
|
||||
echo "== listeners-25-465-587"
|
||||
as_root ss -ltn 2>/dev/null | awk 'NR==1 || $4 ~ /:(25|465|587|2525)$/'
|
||||
|
||||
echo "== recent-relay-evidence"
|
||||
as_root sh -c 'grep -h "relay=" /var/log/mail.log /var/log/mail.log.1 /var/log/maillog 2>/dev/null | tail -n 6'
|
||||
|
||||
echo "== done $(hostname -f 2>/dev/null || hostname)"
|
||||
@@ -0,0 +1,75 @@
|
||||
# audit-targets.txt — fleet targets for run-mail-audit.sh (#696 mail audit)
|
||||
# format: mode|name|user (user only applies to vm mode; default root)
|
||||
# prox = PVE host -> tests/remote.sh prox-file
|
||||
# vm = Linux VM / physical -> tests/remote.sh vm-file
|
||||
# dns = DNS infra alias -> remote-dns.sh <alias>-file
|
||||
# Deliberately NOT targeted (no ssh path / dark / not Linux):
|
||||
# pfv-bms (HAOS core_ssh only; HA SMTP notify lives in KNEL/pfv-bms repo),
|
||||
# tsys-umbrel, kali-tsys (sshd gated), subopi3 (#733 key-denied),
|
||||
# siter-bizoffice (Windows), stlp-3dscanner (offline 184d+)
|
||||
prox|pfv-tsys1|
|
||||
prox|pfv-tsys3|
|
||||
prox|pfv-tsys4|
|
||||
prox|pfv-tsys5|
|
||||
prox|pfv-tsys6|
|
||||
prox|pfv-tsys7|
|
||||
prox|pfv-tsys9|
|
||||
dns|netinfra01|
|
||||
dns|netinfra02|
|
||||
vm|tsys-ca|
|
||||
vm|tsys-librenms|
|
||||
vm|tsys-proxmox-datacenter|
|
||||
vm|tsys-proxmox-mailgw-01|
|
||||
vm|tsys-proxmox-mailgw-02|
|
||||
vm|tsys-ucs-01|
|
||||
vm|tsys-ucs-02|
|
||||
vm|tsys-awx|
|
||||
vm|pfv-rr-middleware-02|
|
||||
vm|rr-middleware-01|
|
||||
vm|hfnoc-uisp|
|
||||
vm|hfnoc-uisp-preprod|
|
||||
vm|tsys-siem|
|
||||
vm|tsys-voip|
|
||||
vm|devbox-cloudron|
|
||||
vm|ultix-streaming|
|
||||
vm|kali-rd|
|
||||
vm|pfv-k8s-cnode1|localuser
|
||||
vm|pfv-k8s-wnode-tsys3|localuser
|
||||
vm|pfv-k8s-cnode2|localuser
|
||||
vm|pfv-k8s-cnode3|localuser
|
||||
vm|pfv-k8s-wnode-tsys6|localuser
|
||||
vm|pfv-k8s-wnode-tsys7|localuser
|
||||
vm|pfv-k8s-wnode-tsys9|localuser
|
||||
vm|pfv-proxmox-backup-server|
|
||||
vm|pfvsvrpi|localuser
|
||||
vm|pfv-jetson-nano-1|
|
||||
vm|stlpc-artroom|localuser
|
||||
vm|stlpc-garage|localuser
|
||||
vm|subopi-dev-3|localuser
|
||||
vm|subopi-dev-4|localuser
|
||||
vm|netbird|
|
||||
vm|sectestbed-sandbox|
|
||||
vm|sectestbed-siem|
|
||||
vm|sectestbed-proxmox-pve|
|
||||
vm|sectestbed-proxmox-datacenter|
|
||||
vm|sectestbed-proxmox-pbs|
|
||||
vm|sectestbed-awx|
|
||||
vm|sectestbed-k8s-cnode|
|
||||
vm|sectestbed-k8s-wnode|
|
||||
vm|sectestbed-librenms|
|
||||
vm|sectestbed-netinfra|
|
||||
vm|sectestbed-cloudron|
|
||||
vm|sectestbed-proxmox-mailgw|
|
||||
vm|sectestbed-ca|
|
||||
vm|sectestbed-voip|
|
||||
vm|sectestbed-hfnoc-uisp|
|
||||
vm|sectestbed-rancherplatform|
|
||||
vm|preprod-awx|
|
||||
vm|preprod-siem|
|
||||
vm|preprod-rancherplatform|
|
||||
vm|preprod-proxmox-mailgw|
|
||||
vm|preprod-ca|
|
||||
vm|preprod-proxmox-datacenter|
|
||||
vm|preprod-librenms|
|
||||
vm|preprod-voip|
|
||||
vm|preprod-cloudron|
|
||||
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env bash
|
||||
# run-mail-audit.sh — drive audit-mail-env.sh across the fleet, read-only.
|
||||
#
|
||||
# All remote access goes through the sanctioned chokepoints:
|
||||
# prox targets -> tests/remote.sh prox-file (PVE hosts, root)
|
||||
# vm targets -> tests/remote.sh vm-file (VMs + physical Linux)
|
||||
# dns targets -> remote-dns.sh <alias>-file (netinfra pair, netboot)
|
||||
# Output: one text block per target under .crush/audit/mail/<name>.txt
|
||||
#
|
||||
# Usage: bash netinfra/mail/run-mail-audit.sh [targets-file]
|
||||
set -u
|
||||
|
||||
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
TARGETS="${1:-${REPO}/netinfra/mail/audit-targets.txt}"
|
||||
PAYLOAD="${REPO}/netinfra/mail/audit-mail-env.sh"
|
||||
OUTDIR="${REPO}/.crush/audit/mail"
|
||||
REMOTE_SH="${REPO}/tests/remote.sh"
|
||||
DNS_SH="${REPO}/netinfra/dns-cluster-setup/remote-dns.sh"
|
||||
TMO="${MAIL_AUDIT_TIMEOUT:-45}"
|
||||
|
||||
mkdir -p "${OUTDIR}"
|
||||
ok=0
|
||||
fail=0
|
||||
failed=""
|
||||
|
||||
while IFS='|' read -r mode name user; do
|
||||
case "${mode}" in '' | '#'*) continue ;; esac
|
||||
outfile="${OUTDIR}/${name}.txt"
|
||||
rc=0
|
||||
case "${mode}" in
|
||||
prox)
|
||||
timeout "${TMO}" env PROX_HOST="${name}" bash "${REMOTE_SH}" prox-file "${PAYLOAD}" >"${outfile}" 2>&1
|
||||
rc=$?
|
||||
;;
|
||||
vm)
|
||||
for tgt in "${name}" "${name}.knel.net"; do
|
||||
timeout "${TMO}" env VM_IP="${tgt}" VM_USER="${user:-root}" bash "${REMOTE_SH}" vm-file "${PAYLOAD}" >"${outfile}" 2>&1
|
||||
rc=$?
|
||||
if [ "${rc}" -eq 0 ]; then
|
||||
break
|
||||
fi
|
||||
case "${tgt}" in
|
||||
*.*) break ;;
|
||||
esac
|
||||
done
|
||||
;;
|
||||
dns)
|
||||
timeout "${TMO}" bash "${DNS_SH}" "${name}-file" "${PAYLOAD}" >"${outfile}" 2>&1
|
||||
rc=$?
|
||||
;;
|
||||
*)
|
||||
echo "unknown mode '${mode}' for ${name}" >&2
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
if [ "${rc}" -eq 0 ] && grep -q '^== done' "${outfile}"; then
|
||||
ok=$((ok + 1))
|
||||
echo "[ok] ${name}"
|
||||
else
|
||||
fail=$((fail + 1))
|
||||
failed="${failed} ${name}(rc=${rc})"
|
||||
echo "[FAIL] ${name} rc=${rc}"
|
||||
fi
|
||||
done <"${TARGETS}"
|
||||
|
||||
echo "=== audit run: ${ok} ok, ${fail} failed${failed}"
|
||||
@@ -0,0 +1,69 @@
|
||||
# Questions — v8 (current round)
|
||||
|
||||
v7's PMG round is partially resolved — the 2026-09-03 mail audit (55
|
||||
systems, read-only) answered several questions with ground truth and
|
||||
Charles answered others in chat. Answers recorded below; new questions
|
||||
PQ8–PQ10 need your go. Full map + deploy plan:
|
||||
[Discourse t/322](https://community.turnsys.com/t/322).
|
||||
|
||||
## PMG round — resolved
|
||||
|
||||
**PQ1 (backend).** ✅ Cloudron VPS confirmed as backend:
|
||||
`mail.knownelement.com` (152.53.37.179). Audit adds: public MX already
|
||||
points there; fleet delivers direct today; knel.net zone has no MX.
|
||||
|
||||
**PQ3 (outbound smarthost).** ✅ Charles ruled: all hosts send through
|
||||
a shared **VIP** in front of the PMG pair. (Mechanics in PQ8 below.)
|
||||
|
||||
**PQ4 (placement/sizing).** ✅ Resolved by reality: the pair already
|
||||
exists — VM 604 on tsys6 (192.168.1.11) and VM 711 on tsys7
|
||||
(192.168.1.10), both 2vCPU/4G/32G, running, onboot, PMG 9.1.0
|
||||
installed, unclustered, default config. Nothing to build; configure
|
||||
only.
|
||||
|
||||
**"Old relay in postfix?"** ✅ None. `relayhost` is empty on every
|
||||
audited system (16 postfix installs); no ssmtp/msmtp/nullmailer
|
||||
anywhere. Fleet sends direct-to-MX today.
|
||||
|
||||
## PMG round — still open
|
||||
|
||||
**PQ2 (inbound path).** Where does public :25 DNAT land — rrinfra
|
||||
Cisco? Any existing mail port-forwards? (Founder-side; only needed
|
||||
before the MX cut-over, not for outbound value.)
|
||||
|
||||
**PQ5 (native PMG cluster?).** Recommended: yes — config + quarantine
|
||||
replication between .1.11/.1.10, managed strictly serial. OK?
|
||||
|
||||
**PQ6 (TLS from fleet CA #697?).** Recommended: yes for the two admin
|
||||
UIs; SMTP stays opportunistic. OK?
|
||||
|
||||
**PQ7 (notify address).** Evidence says the pair already forwards to
|
||||
`coo@turnsys.com`. Confirm that address for quarantine digests + PMG
|
||||
alerts?
|
||||
|
||||
**PQ8 (VIP specifics).** Proposal: **VIP 192.168.3.249** (pool ends
|
||||
.3.200; .250-.254 is your infra block), keepalived VRRP VRID 10 on the
|
||||
pair, dhcpd host-declaration pin (VRRP MAC 00:00:5e:00:01:0a), DNS A
|
||||
record `smtp.knel.net` → 192.168.3.249. Confirm the number + name, or
|
||||
name your preferred slot.
|
||||
|
||||
**PQ9 (outbound cutover staging).** Recommended: serial staged — PVE
|
||||
hosts + PBS first (verify a week), then the VM postfix set, one
|
||||
blue/green hop at a time. OK, or all at once?
|
||||
|
||||
**PQ10 (side findings — sub-tickets?).** (a) tsys-ucs-02 root alias
|
||||
blackholes to `systemmail@unassigned-hostname...`; (b) Wazuh alerting
|
||||
never configured (stock example smtp). Fix as ride-alongs in #696 or
|
||||
separate sub-tickets? (Will create them either way — just tell me
|
||||
where they land.)
|
||||
|
||||
## Carried from v6 — still open
|
||||
|
||||
**Q4. Second human approver in gitea?** "Planned for very near
|
||||
future"; until then KNEL/access-roster merges are impossible by
|
||||
design. Does not block code/deploy work.
|
||||
|
||||
## Carried from v7 — superseded questions
|
||||
|
||||
PQ1/PQ3/PQ4 answers above replace the v7 wording; v7 file remains in
|
||||
git history.
|
||||
Reference in New Issue
Block a user