feat(console): manage 7 switch consoles via ser2net+conman on pfv-tsys4

Solve the long-standing USB adapter enumeration shift problem: 9 Prolific USB-to-DB9 adapters on pfv-tsys4 have no unique serial numbers and get assigned /dev/ttyUSB0-8 based on enumeration order, which changes on every reboot and breaks the old /root/conmap + manual screen workflow. Solution: udev rules pin each adapter by its ID_PATH (physical USB port topology), which is stable across reboots regardless of enumeration order. Each adapter gets a named symlink in /dev/consoles/<name>. ser2net opens these stable symlinks and exposes them on TCP ports (2001-2007) bound to the Tailscale interface only. conman connects to those TCP ports for session logging and multi-user console sharing. Architecture (layered, no port sharing): USB adapter → udev symlink → ser2net (TCP) → conman (logging + mux) Port assignments (all on Tailscale IP 100.70.77.93): 2001 = pfv-core-sw01 2002 = pfv-tor3-mgmt 2003 = pfv-tor3-stor 2004 = pfv-rrinfra-rtr 2005 = pfv-r2-tor-top 2006 = subodev-torsw 2007 = pfv-r2-sw Scripts (console/): - mapping.txt: source of truth (TCP port | name | ID_PATH | baud | comment) - generate-config.sh: generates udev rules, ser2net.yaml, conman.conf entries from mapping.txt. Idempotent (markers in conman.conf for clean regeneration). Uses | delimiter (ID_PATH values contain colons). - setup.sh: full deploy — generate configs, create symlinks (udev trigger + manual fallback for already-discovered devices), create conmand systemd unit (Debian doesn't ship one), restart services - discover.sh: read-only USB adapter and service state discovery - validate-conman.sh: verify conman→ser2net→device data path and log capture Issues fixed during development: - /dev/console is a kernel char device (major 5, minor 1) — cannot create a directory there. Changed symlink namespace to /dev/consoles/. - conman 0.3.x has no 'include' directive — CONSOLE entries written directly into /etc/conman.conf between idempotent markers. - Debian conman package has no systemd unit — created /etc/systemd/system/conmand.service with After=ser2net ordering. - conman.conf had no LOGDIR — logs weren't being written to /var/log/conman/. Fixed by adding server logdir directive. Validation: 7 symlinks resolving, 7 TCP ports on Tailscale, conmand with 7 consoles registered, 7 log files actively capturing console output, both services enabled for reboot survival. 💘 Generated with Crush Assisted-by: Crush:glm-5.2
2026-07-28 19:44:02 -05:00
parent 799d270261
commit eb99b00a70
10 changed files with 807 additions and 0 deletions
@@ -0,0 +1,244 @@
+#!/usr/bin/bash
+#
+# console/generate-config.sh — generate udev rules + ser2net.yaml + conman.conf
+#
+# Reads console/mapping.txt (the source of truth) and generates all three
+# config files. This is the fix for the USB enumeration shift problem:
+#
+#   1. udev rules pin each adapter by its STABLE ID_PATH (physical USB port)
+#      to a named symlink like /dev/consoles/pfv-core-sw01
+#   2. ser2net opens those stable symlinks and exposes them on TCP ports
+#      (2001, 2002, ...) bound to the Tailscale IP
+#   3. conman connects to those TCP ports for logging + multiplexing
+#
+# Run this script ON the target host. It writes to:
+#   /etc/udev/rules.d/99-console-ports.rules
+#   /etc/ser2net.yaml
+#   /etc/conman/console-consoles.conf  (included by /etc/conman.conf)
+#
+# Usage:
+#   PROX_HOST=pfv-tsys4 bash tests/remote.sh prox-file console/generate-config.sh
+#
+# Environment overrides:
+#   MAPPING_FILE  — path to mapping.txt (default: auto-detect next to this script)
+#   TS_IP         — Tailscale IP to bind ser2net on (default: auto-detect)
+#   CONMAN_LOGDIR — conman log directory (default: /var/log/conman)
+#
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+MAPPING_FILE="${MAPPING_FILE:-$SCRIPT_DIR/mapping.txt}"
+CONMAN_LOGDIR="${CONMAN_LOGDIR:-/var/log/conman}"
+
+UDEV_RULES="/etc/udev/rules.d/99-console-ports.rules"
+SER2NET_CONF="/etc/ser2net.yaml"
+CONMAN_CONF="/etc/conman.conf"
+CONMAN_CONSOLES="/etc/conman/console-consoles.conf"
+CONSOLE_DEV_DIR="/dev/console"
+
+echo "============================================"
+echo "  Console Config Generator"
+echo "  Host: $(hostname)   $(date)"
+echo "============================================"
+
+# --- Locate mapping file ---
+# When run via remote.sh prox-file, $0 is bash and $SCRIPT_DIR may be wrong.
+# Search common locations.
+if [ ! -f "$MAPPING_FILE" ]; then
+  for candidate in \
+    "/root/console/mapping.txt" \
+    "/tmp/mapping.txt" \
+    "$(dirname "$0")/mapping.txt"; do
+    if [ -f "$candidate" ]; then
+      MAPPING_FILE="$candidate"
+      break
+    fi
+  done
+fi
+
+if [ ! -f "$MAPPING_FILE" ]; then
+  echo "FATAL: mapping file not found. Tried: $MAPPING_FILE"
+  echo "Copy mapping.txt to the target host first."
+  exit 1
+fi
+echo "  Mapping file: $MAPPING_FILE"
+
+# --- Auto-detect Tailscale IP ---
+if [ -z "${TS_IP:-}" ]; then
+  TS_IP=$(tailscale ip -4 2>/dev/null || true)
+  if [ -z "$TS_IP" ]; then
+    echo "FATAL: could not auto-detect Tailscale IP. Set TS_IP manually."
+    exit 1
+  fi
+fi
+echo "  Tailscale IP: $TS_IP"
+echo "  ser2net will bind to: $TS_IP"
+
+# --- Parse mapping file (skip comments and blank lines) ---
+echo ""
+echo "--- Parsing mapping file ---"
+ENTRIES=()
+while IFS= read -r line; do
+  # Skip comments and blank lines
+  line="${line%%#*}"
+  line="$(echo "$line" | xargs)"  # trim whitespace
+  [ -z "$line" ] && continue
+  ENTRIES+=("$line")
+  echo "  $line"
+done < "$MAPPING_FILE"
+
+if [ "${#ENTRIES[@]}" -eq 0 ]; then
+  echo "FATAL: no entries found in mapping file."
+  exit 1
+fi
+
+echo ""
+echo "  ${#ENTRIES[@]} console ports configured."
+
+# ============================================================
+# 1. Generate udev rules
+# ============================================================
+echo ""
+echo "--- [1/3] Generating udev rules: $UDEV_RULES ---"
+
+cat > "$UDEV_RULES" <<'UDEV_HEADER'
+# Stable symlinks for USB-DB9 console adapters
+# Generated by console/generate-config.sh
+# DO NOT EDIT — edit mapping.txt and re-run generate-config.sh
+#
+# These rules pin each adapter to a named symlink based on its physical
+# USB port path (ID_PATH), which is stable across reboots regardless of
+# enumeration order. This is the fix for the "USB adapters shift on reboot"
+# problem.
+#
+# To find the ID_PATH for a device:
+#   udevadm info -q all -n /dev/ttyUSBN | grep ID_PATH
+UDEV_HEADER
+
+for entry in "${ENTRIES[@]}"; do
+  IFS='|' read -r tcp_port name id_path baud comment <<< "$entry"
+  # Build the full ID_PATH match. The mapping stores a substring like "usb-0:1.5.4.4"
+  # The actual ID_PATH is like "pci-0000:00:1a.0-usb-0:1.5.4.4:1.0"
+  # We match on the substring to be portable across PCI bus changes.
+  echo "" >> "$UDEV_RULES"
+  echo "# $name (TCP $tcp_port): $comment" >> "$UDEV_RULES"
+  echo "SUBSYSTEM==\"tty\", ENV{ID_PATH}==\"*$id_path*\", SYMLINK+=\"console/$name\"" >> "$UDEV_RULES"
+done
+
+echo "  Written: $UDEV_RULES"
+echo "  Symlinks: /dev/consoles/<name> for each device"
+
+# ============================================================
+# 2. Generate ser2net.yaml
+# ============================================================
+echo ""
+echo "--- [2/3] Generating ser2net config: $SER2NET_CONF ---"
+
+# Backup existing config if not already backed up
+if [ -f "$SER2NET_CONF" ] && [ ! -f "${SER2NET_CONF}.orig" ]; then
+  cp "$SER2NET_CONF" "${SER2NET_CONF}.orig"
+  echo "  Backed up original to ${SER2NET_CONF}.orig"
+fi
+
+{
+  echo "%YAML 1.1"
+  echo "---"
+  echo "# ser2net configuration for pfv-tsys4 console ports"
+  echo "# Generated by console/generate-config.sh on $(date)"
+  echo "#"
+  echo "# All ports bound to Tailscale IP ($TS_IP) for secure remote access."
+  echo "# Physical devices are accessed via stable udev symlinks in /dev/consoles/."
+  echo "#"
+  echo "# To connect directly:   telnet $TS_IP 2001"
+  echo "# To connect via conman: conman -f <name>"
+  echo ""
+  echo "define: &banner \\r\\nPFV console port \\p device \\d [\\B]\\r\\n\\r\\n"
+  echo ""
+
+  for entry in "${ENTRIES[@]}"; do
+    IFS='|' read -r tcp_port name id_path baud comment <<< "$entry"
+    # ser2net connection block
+    echo "connection: &con${tcp_port}"
+    echo "    accepter: tcp,${TS_IP},${tcp_port}"
+    echo "    enable: on"
+    echo "    options:"
+    echo "      banner: *banner"
+    echo "      kickolduser: true"
+    echo "      telnet-brk-on-sync: true"
+    echo "    connector: serialdev,"
+    echo "              /dev/consoles/${name},"
+    echo "              ${baud},local"
+    echo ""
+  done
+} > "$SER2NET_CONF"
+
+echo "  Written: $SER2NET_CONF"
+echo "  ${#ENTRIES[@]} TCP ports configured ($TS_IP:2001-20XX)"
+
+# ============================================================
+# 3. Write conman console entries directly into conman.conf
+# ============================================================
+# conman 0.3.x does NOT support the 'include' directive, so we write
+# CONSOLE entries directly into /etc/conman.conf between idempotent markers.
+echo ""
+echo "--- [3/3] Writing conman consoles into $CONMAN_CONF ---"
+
+# Ensure logdir exists
+mkdir -p "$CONMAN_LOGDIR" 2>/dev/null || true
+
+# Ensure LOGDIR is set in conman.conf (server-level directive for log file paths)
+if ! grep -qiE '^\s*server\s+logdir\s*=' "$CONMAN_CONF" 2>/dev/null; then
+  # Insert near the top, after the first SERVER directives
+  sed -i "1i\\server logdir = \"$CONMAN_LOGDIR\"" "$CONMAN_CONF"
+  echo "  Added server logdir = \"$CONMAN_LOGDIR\" to $CONMAN_CONF"
+fi
+
+# Remove any previous auto-generated block (between markers)
+# Then append the new block
+MARKER_BEGIN="# BEGIN PFV CONSOLE DEFINITIONS (auto-generated — do not edit between markers)"
+MARKER_END="# END PFV CONSOLE DEFINITIONS"
+
+# Strip old block if present
+if grep -q "$MARKER_BEGIN" "$CONMAN_CONF" 2>/dev/null; then
+  sed -i "/$MARKER_BEGIN/,/$MARKER_END/d" "$CONMAN_CONF"
+  echo "  Removed previous console definitions."
+fi
+
+# Append new block
+{
+  echo ""
+  echo "$MARKER_BEGIN"
+  echo "# Generated by console/generate-config.sh on $(date)"
+  echo "# Each console connects to a ser2net TCP port via telnet protocol."
+  echo "# ser2net owns the physical serial device; conman provides logging"
+  echo "# and multiplexing on top."
+  echo "# Access: conman -f <name>"
+  echo ""
+  for entry in "${ENTRIES[@]}"; do
+    IFS='|' read -r tcp_port name id_path baud comment <<< "$entry"
+    echo "CONSOLE name=\"${name}\" dev=\"${TS_IP}:${tcp_port}\" log=\"${name}.log\" logopts=\"timestamp\""
+  done
+  echo "$MARKER_END"
+} >> "$CONMAN_CONF"
+
+CONSOLE_COUNT=$(grep -c "^CONSOLE " "$CONMAN_CONF" 2>/dev/null || echo 0)
+echo "  Written $CONSOLE_COUNT CONSOLE entries to $CONMAN_CONF"
+
+# ============================================================
+# Summary
+# ============================================================
+echo ""
+echo "============================================"
+echo "  Configuration generated successfully."
+echo ""
+echo "  Files written:"
+echo "    $UDEV_RULES       ($(wc -l < "$UDEV_RULES") lines)"
+echo "    $SER2NET_CONF     ($(wc -l < "$SER2NET_CONF") lines)"
+echo "    $CONMAN_CONF      (CONSOLE entries appended between markers)"
+echo ""
+echo "  Next steps:"
+echo "    1. Reload udev:   udevadm control --reload-rules && udevadm trigger"
+echo "    2. Restart ser2net: systemctl restart ser2net"
+echo "    3. Start conman:  systemctl enable --now conmand"
+echo "    4. Or run:       bash $(basename "$0" .sh | sed 's/generate-config/setup/') .sh"
+echo "============================================"