fix(console): switch conman from ser2net-telnet to direct serial access
The ser2net layer between conman and the serial devices was causing
stair-stepping in terminal output. conman's dev="host:port" uses telnet
protocol, but ser2net's accepter was raw TCP (no telnet mode). The telnet
NVT state machine in conman was stripping bare CR characters from device
output — particularly from Dell switches that use old-style \n\r (LF+CR)
line endings instead of standard \r\n. In telnet, a bare \r not followed
by \n or NUL is non-compliant and gets dropped, leaving bare \n that
causes stair-stepping in the raw-mode terminal.
Fix: conman now opens serial devices directly via the stable udev
symlinks (/dev/consoles/<name>) with seropts, eliminating the telnet
layer entirely. ser2net is stopped and disabled but remains installed
for emergency TCP access (documented workflow: stop conmand, start
ser2net, use telnet, then reverse).
Architecture change:
Before: device → serial → ser2net (raw TCP) → conman (telnet NVT) → terminal
After: device → serial → conman (direct) → terminal
Verified: 7/7 serial devices held by conmand, 7/7 log files capturing,
all \r bytes preserved in terminal output (confirmed via PTY capture).
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
This commit is contained in:
+24
-17
@@ -6,17 +6,18 @@ switches and routers, running on **pfv-tsys4** (storage server).
|
||||
## Architecture
|
||||
|
||||
```
|
||||
USB-DB9 adapters → udev symlinks (/dev/consoles/<name>) → ser2net (TCP) → conman (logging + multiplexing)
|
||||
USB-DB9 adapters → udev symlinks (/dev/consoles/<name>) → conman (direct serial, logging + multiplexing)
|
||||
```
|
||||
|
||||
ser2net owns the physical serial devices and exposes them on TCP ports
|
||||
bound to the **Tailscale interface only** (`100.70.77.93:200X`). conman
|
||||
connects to those TCP ports for session logging, output capture, and
|
||||
multi-user console sharing.
|
||||
conman opens the serial devices directly via stable udev symlinks and
|
||||
provides session logging, output capture, and multi-user console
|
||||
sharing. Clients connect to conmand over the Tailscale interface
|
||||
(`100.70.77.93:7890`).
|
||||
|
||||
**conman and ser2net do NOT share ports** — only one process can open a
|
||||
serial device at a time. ser2net owns the physical device; conman connects
|
||||
over TCP.
|
||||
ser2net is installed but **disabled** — kept for emergency direct TCP
|
||||
access only (see [Emergency Access](#emergency-access) below). It must
|
||||
NOT run simultaneously with conmand (both would compete for the serial
|
||||
devices).
|
||||
|
||||
## The USB Enumeration Problem (SOLVED)
|
||||
|
||||
@@ -49,7 +50,8 @@ ID_PATH. On subsequent boots, udev creates them automatically.
|
||||
| 2006 | subodev-torsw | usb-0:1.5.4.1 | Suborbital device TOR switch |
|
||||
| 2007 | pfv-r2-sw | usb-0:1.6.3.2 | Rack 2 old Dell switch |
|
||||
|
||||
All ports listen on the Tailscale IP (`100.70.77.93`).
|
||||
TCP ports are used by the ser2net emergency config only. conman accesses
|
||||
serial devices directly via `/dev/consoles/<name>` symlinks.
|
||||
|
||||
## Scripts
|
||||
|
||||
@@ -74,19 +76,24 @@ conman -d pfv-tsys4:7890 -q # list all consoles
|
||||
|
||||
Escape sequence: `&.` to disconnect, `&?` for help.
|
||||
|
||||
**Direct telnet (emergency only — conflicts with conman):**
|
||||
### Emergency Access
|
||||
|
||||
If conmand is down or you need direct serial access:
|
||||
|
||||
```bash
|
||||
# Direct telnet to ser2net works ONLY when conmand is stopped, because
|
||||
# conmand maintains persistent connections to all 7 TCP ports. Use:
|
||||
# Option A — direct screen on pfv-tsys4:
|
||||
ssh pfv-tsys4 'systemctl stop conmand'
|
||||
ssh pfv-tsys4 'screen /dev/consoles/pfv-core-sw01'
|
||||
ssh pfv-tsys4 'systemctl start conmand'
|
||||
|
||||
# Option B — enable ser2net for TCP access:
|
||||
ssh pfv-tsys4 'systemctl stop conmand && systemctl start ser2net'
|
||||
telnet pfv-tsys4 2001 # pfv-core-sw01
|
||||
ssh pfv-tsys4 'systemctl start conmand' # restart when done
|
||||
ssh pfv-tsys4 'systemctl stop ser2net && systemctl start conmand'
|
||||
```
|
||||
|
||||
**Do NOT use telnet while conmand is running** — conmand will reconnect
|
||||
and kick your telnet session immediately ("Connection closed by foreign host").
|
||||
The correct workflow is conman client → conmand → ser2net → device.
|
||||
**Never run ser2net and conmand at the same time** — both compete for the
|
||||
same serial devices.
|
||||
|
||||
### Re-deploy after changing mapping.txt
|
||||
|
||||
@@ -108,7 +115,7 @@ to `mapping.txt`.
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `/etc/udev/rules.d/99-console-ports.rules` | Stable symlinks by ID_PATH |
|
||||
| `/etc/ser2net.yaml` | ser2net config (TCP ports → serial symlinks) |
|
||||
| `/etc/ser2net.yaml` | ser2net config (EMERGENCY ONLY — disabled by default) |
|
||||
| `/etc/conman.conf` | conman config (CONSOLE entries between markers) |
|
||||
| `/etc/systemd/system/conmand.service` | systemd unit for conmand |
|
||||
| `/root/console/mapping.txt` | Copy of the source-of-truth mapping |
|
||||
|
||||
Reference in New Issue
Block a user