docs: expand aug11.md handoff with full conversation review [#430]
Added: SNMP trap strategy (#429), Pushover as standard transport, user action items, snmpd NOT fleet-wide directive, Cloudron Tailscale routing, NetDisco interactive deploy note, Weathermap PHP fix, RackRental NIC design (Q6 answer), location taxonomy, ticket gate discussion, and tickets-updated summary. 💘 Generated with Crush Assisted-by: Crush:glm-5.2
This commit is contained in:
@@ -48,35 +48,102 @@ Access: `VM_IP=tsys-librenms VM_USER=root bash tests/remote.sh vm '<cmd>'`
|
||||
- UNPoller is running and polling. Need to add scrape config to Prometheus (Cloudron)
|
||||
- BLOCKED: needs ~/.creds/prometheus.env and ~/.creds/grafana.env populated by user
|
||||
- Then build Grafana dashboard for UniFi
|
||||
- UNPoller v3.4.1 exports Prometheus metrics at http://tsys-librenms.knel.net:8084/metrics
|
||||
|
||||
### 2. LibreNMS alert transports (#338)
|
||||
- Configure Pushover as alert delivery in LibreNMS
|
||||
- BLOCKED: needs ~/.creds/pushover.env (PUSHOVER_TOKEN + PUSHOVER_USER)
|
||||
- Same Pushover config needed for Beszel (#428) and Uptime Kuma updates
|
||||
- **Pushover is the STANDARD alert transport for ALL OAM tools** (user directive):
|
||||
- Uptime Kuma: already using Pushover
|
||||
- LibreNMS: #338 (this ticket)
|
||||
- Beszel: #428
|
||||
- Grafana: when wired via #430
|
||||
|
||||
### 3. TEMPer USB graphing (#341)
|
||||
- Deploy on pfv-tsys1 + stlpc-artroom
|
||||
### 3. SNMP trap strategy (#429)
|
||||
- Decide: Dell OME owns server/iDRAC traps, LibreNMS owns network traps (recommended split)
|
||||
- Awaiting Dell OME deployment (#339) before implementing
|
||||
- Switches may need trap destination configured
|
||||
|
||||
### 4. TEMPer USB graphing (#341)
|
||||
- Deploy on pfv-tsys1 (Server Room) + stlpc-artroom (Art Room)
|
||||
- USB probe driver + RRD/LibreNMS integration
|
||||
- stlpc-garage (EE Lab) may also have a probe — verify with user
|
||||
|
||||
### 4. Monitoring coverage matrix (#343)
|
||||
### 5. Monitoring coverage matrix (#343)
|
||||
- Governance doc mapping every tool to its domain (no overlap)
|
||||
- Write to Discourse, update ticket
|
||||
- Tool domains: Uptime Kuma=HTTP/service availability, Beszel=host metrics,
|
||||
LibreNMS=SNMP/network, Dell OME=Dell hardware, Smokeping=latency,
|
||||
NetDisco=L2 topology, Oxidized=config backup, phpIPAM=IP inventory,
|
||||
UNPoller=UniFi metrics
|
||||
|
||||
### 5. Uptime Kuma API + comprehensive monitors (#426/#435)
|
||||
### 6. Uptime Kuma API + comprehensive monitors (#426/#435)
|
||||
- Enable API write access
|
||||
- Add monitors for all PFV/SITER + Cloudron services
|
||||
- Add monitors for all PFV/SITER + Cloudron services (HTTPS, DNS, DB, Docker, k8s)
|
||||
|
||||
### 6. Dell OpenManage Enterprise (#339)
|
||||
- User deploying manually (VM creation + disk import)
|
||||
- Agent role: post-deployment integration only
|
||||
### 7. Dell OpenManage Enterprise (#339)
|
||||
- USER DEPLOYING MANUALLY — has KVM image downloaded, will do qm import + VM creation
|
||||
- Agent role: post-deployment integration only (SNMP traps #429, coverage matrix #343)
|
||||
- Fleet is 100% Dell (documented in Discourse #304)
|
||||
|
||||
## Blocked Items (waiting on user or hardware)
|
||||
|
||||
- #431: Cisco router fan replacement (physical, onsite) → blocks #432, #433, #434, #436
|
||||
- #427: Credential management (Vault/RADIUS/Ansible Vault) → tomorrow's IAC track
|
||||
- #430: Prometheus/Grafana creds needed
|
||||
- #338: Pushover API key needed
|
||||
- #426: Uptime Kuma API access needed
|
||||
- #430: Prometheus/Grafana creds needed (~/.creds/prometheus.env + grafana.env EMPTY)
|
||||
- #338: Pushover API key needed (~/.creds/pushover.env NOT CREATED YET)
|
||||
- #426: Uptime Kuma API access needed (read works, write not enabled)
|
||||
- #425: phpIPAM scan agent — user may need to create from web UI and trigger first scan
|
||||
|
||||
## User Action Items (things only the user can do)
|
||||
|
||||
1. **Populate ~/.creds/pushover.env** — PUSHOVER_TOKEN + PUSHOVER_USER (unblocks #338, #428)
|
||||
2. **Populate ~/.creds/prometheus.env + grafana.env** — Cloudron API keys (unblocks #430)
|
||||
3. **Deploy Dell OME manually** — VM creation + qm import + OOB config (#339)
|
||||
4. **Replace Cisco router fan** — physical onsite work (#431, blocks RackRental track)
|
||||
5. **Create phpIPAM scan agent** — from web UI, trigger first discovery scan (#425)
|
||||
6. **Enable Uptime Kuma API write access** — from web UI (#426)
|
||||
7. **Run interactive netdisco-deploy** — DONE (user already did this)
|
||||
|
||||
## Important Technical Notes
|
||||
|
||||
### snmpd is NOT fleet-wide (user directive)
|
||||
snmpd will NOT be rolled out to Linux hosts. Beszel covers host metrics.
|
||||
SNMP (community: kn3lmgmt) is ONLY for network devices (switches, routers, PDUs).
|
||||
Do NOT add snmpd deployment to any ticket scope.
|
||||
|
||||
### Cloudron Tailscale routing
|
||||
Cloudron is on Tailscale. pfv-netinfra-01/02 are Tailscale subnet routers advertising
|
||||
192.168.0.0/22. This means Cloudron-hosted services (phpIPAM, Prometheus, Grafana)
|
||||
CAN reach the LAN switches via Tailscale for SNMP discovery/scraping.
|
||||
|
||||
### NetDisco required interactive deploy
|
||||
The `netdisco-deploy` script requires a TTY (interactive yes/no prompts).
|
||||
The agent cannot run this through remote.sh. The user ran it manually.
|
||||
If NetDisco needs redeployment, the user must run this step.
|
||||
|
||||
### Weathermap PHP fix
|
||||
Weathermap plugin required `register_argc_argv = On` in /etc/php/8.4/fpm/php.ini
|
||||
on tsys-librenms. Already applied. If PHP gets updated, verify this setting persists.
|
||||
|
||||
### RackRental split-network design (Q6 answer in questions-v1.md)
|
||||
tsys6/7 have 4-6 ethernet ports. User wants:
|
||||
- 2 NICs for storage bond (keep existing bond0)
|
||||
- 1 NIC for mgmt (non-RackRental side)
|
||||
- 1 NIC trunked to RackRental VLAN 2
|
||||
- RackRental router prevents network loops
|
||||
|
||||
### Location taxonomy
|
||||
- PFV/Server-Room/R{n} for server room gear by rack
|
||||
- PFV/EE-Lab for stlpc-garage
|
||||
- PFV/Art-Room for stlpc-artroom
|
||||
- Applied to all 4 switches via SNMP. Cisco pending fan fix (#431).
|
||||
- phpIPAM sections match this taxonomy.
|
||||
|
||||
### Ticket gate discussion
|
||||
User wants to discuss relaxing the ticket gate for handoff/continuation sessions.
|
||||
For now, use a "dummy" ticket (#430 as placeholder) but don't bypass the rules.
|
||||
This is an open discussion item — see questions-v1.md for future Q&A.
|
||||
|
||||
## Credential Files (in ~/.creds/)
|
||||
|
||||
@@ -150,6 +217,18 @@ All Dell PowerConnect switches: `kn3lmgmt` (v2c)
|
||||
| 436 | Cisco router → Oxidized (after fan) | New |
|
||||
| 437 | Switch location standardization | 90% |
|
||||
|
||||
## Tickets Updated (not created) This Session
|
||||
|
||||
| # | What we did |
|
||||
|---|-------------|
|
||||
| 314 | Parent umbrella for OAM alerts — updated children list |
|
||||
| 337 | LibreNMS integration modules — 90% done (Oxidized/NetDisco/UNPoller deployed) |
|
||||
| 339 | Dell OME — confirmed 100% Dell fleet, user deploying manually |
|
||||
| 375 | Monitoring expansion — smokeping done, weathermap done, snmpd removed from scope |
|
||||
| 406 | Beszel API — cross-referenced with OAM platform |
|
||||
| 425 | phpIPAM — revised from "deploy" to "configure existing Cloudron instance" |
|
||||
| 370 | Vault umbrella — clarified parent-child relationships |
|
||||
|
||||
## Tomorrow's Goal: IAC
|
||||
|
||||
User wants to move to IAC (AWX/Ansible) tomorrow/Thursday night. Vault (#408)
|
||||
|
||||
Reference in New Issue
Block a user