chore: initialize repo with full project state
Performance optimization engagement for a 7-host Proxmox R&D cluster.
Captures the accumulated work across host tuning, network analysis,
fleet assessment, and kubernetes architecture planning.
Contents:
- Host-side tunings (scripts/): CPU governor, swappiness, BBR, NFS
nconnect, tuned profiles -- complete on 5 of 7 hosts
- Validation + benchmarking scripts: iperf matrix, bond/NFS fixes
- Collected host data (returned-logs/): check.sh output from all 7
hosts + iperf results, including newly-validated pfv-tsys9
- AGENTS.md: operating context for AI agents
- PROJECT.md: board-ready fleet assessment with VM placement and
storage redundancy analysis (40 VMs across 7 hosts)
- K8S.md: kubernetes architecture deep-dive covering cnode/wnode
distribution, StorageClass design, and ETL/HPC workload planning
💘 Generated with Crush
Assisted-by: Crush:glm-5.2
This commit is contained in:
@@ -0,0 +1,93 @@
|
||||
!###############################################################################
|
||||
!# pfv-core-sw01
|
||||
!#
|
||||
!# Dell PowerConnect 5448 (48-port 1GbE copper, 4× combo SFP / 10G via stacking)
|
||||
!# OS: DNOS 4.x / PowerConnect CLI
|
||||
!#
|
||||
!# This file is a flat list of READ-ONLY show commands.
|
||||
!# Paste them in order from enable mode. Capture the whole session to
|
||||
!# returned-logs/pfv-core-sw01.log
|
||||
!#
|
||||
!# To capture from a terminal:
|
||||
!# 1. SSH/telnet in
|
||||
!# 2. enable
|
||||
!# 3. terminal datadump (disables --More-- pause)
|
||||
!# 4. paste everything below
|
||||
!###############################################################################
|
||||
|
||||
enable
|
||||
terminal datadump
|
||||
|
||||
! --- Identify & versions ---------------------------------------------------
|
||||
show version
|
||||
show system
|
||||
show inventory
|
||||
show bootvar
|
||||
show running-config
|
||||
show stack
|
||||
show cpu utilization
|
||||
show memory
|
||||
show logging
|
||||
|
||||
! --- Interfaces ------------------------------------------------------------
|
||||
show interfaces status
|
||||
show interfaces configuration
|
||||
show interfaces description
|
||||
show interfaces counters
|
||||
show interfaces counters errors
|
||||
show interfaces counters detail
|
||||
show interfaces counters cable 1-52
|
||||
|
||||
! --- Port-channels / LAG (4-port LACP to pfv-r3-tor-stor) -----------------
|
||||
show interfaces port-channel
|
||||
show lacp 1
|
||||
show lacp 2
|
||||
show lacp 3
|
||||
show lacp 4
|
||||
show lacp 5
|
||||
show lacp 6
|
||||
|
||||
! --- VLANs -----------------------------------------------------------------
|
||||
show vlan
|
||||
show vlan detail
|
||||
show interfaces vlan 1
|
||||
show interfaces vlan 1000
|
||||
show vlan info
|
||||
|
||||
! --- Spanning tree ---------------------------------------------------------
|
||||
show spanning-tree
|
||||
show spanning-tree active
|
||||
show spanning-tree summary
|
||||
show spanning-tree detail
|
||||
|
||||
! --- IGMP / multicast (matters for storage/iSCSI-NFS sometimes) -----------
|
||||
show ip igmp snooping
|
||||
show ip igmp snooping vlan 1
|
||||
show ip igmp snooping vlan 1000
|
||||
|
||||
! --- QoS / CoS -------------------------------------------------------------
|
||||
show qos
|
||||
show qos interface
|
||||
show class-of-service
|
||||
|
||||
! --- Storm control / loop protection --------------------------------------
|
||||
show storm-control
|
||||
show ports jumbo-frame
|
||||
|
||||
! --- ARP / MAC / FDB -------------------------------------------------------
|
||||
show mac address-table
|
||||
show arp
|
||||
show ip route
|
||||
|
||||
! --- Environmental ---------------------------------------------------------
|
||||
show environment
|
||||
show fan
|
||||
show power
|
||||
show hardware
|
||||
|
||||
! --- Flow control ----------------------------------------------------------
|
||||
show flowcontrol
|
||||
show interfaces flowcontrol
|
||||
|
||||
! --- Save the capture, then exit ------------------------------------------
|
||||
exit
|
||||
@@ -0,0 +1,90 @@
|
||||
!###############################################################################
|
||||
!# pfv-r3-tor-mgmt
|
||||
!#
|
||||
!# Rack 3 management TOR (VLAN1 only for this switch).
|
||||
!# Dell switch - OS variant unknown. Commands below cover the common
|
||||
!# Dell CLI flavors (DNOS 4/6/9, OS6/OS9, N-Series). Some may not be
|
||||
!# recognized on every OS; that is fine - skip errors and keep going.
|
||||
!#
|
||||
!# Paste from enable mode after `terminal datadump` (or `terminal length 0`
|
||||
!# on DNOS 6/9). Capture session to returned-logs/pfv-r3-tor-mgmt.log
|
||||
!###############################################################################
|
||||
|
||||
enable
|
||||
terminal datadump
|
||||
terminal length 0
|
||||
|
||||
! --- Identify ---------------------------------------------------------------
|
||||
show version
|
||||
show system
|
||||
show inventory
|
||||
show running-config
|
||||
show bootvar
|
||||
show cpu
|
||||
show cpu utilization
|
||||
show memory
|
||||
show processes
|
||||
show logging
|
||||
|
||||
! --- Interfaces -------------------------------------------------------------
|
||||
show interface status
|
||||
show interface description
|
||||
show interface configuration
|
||||
show interface counters
|
||||
show interface counters errors
|
||||
show interface counters detail
|
||||
show interfaces status
|
||||
show interfaces description
|
||||
show interfaces counters
|
||||
show interfaces counters errors
|
||||
|
||||
! --- Port channels / LAG ----------------------------------------------------
|
||||
show port-channel summary
|
||||
show port-channel detail
|
||||
show interfaces port-channel
|
||||
show lacp
|
||||
show lacp neighbor
|
||||
show lag
|
||||
|
||||
! --- VLANs ------------------------------------------------------------------
|
||||
show vlan
|
||||
show vlan brief
|
||||
show vlan detail
|
||||
|
||||
! --- Spanning tree ----------------------------------------------------------
|
||||
show spanning-tree
|
||||
show spanning-tree summary
|
||||
show spanning-tree active
|
||||
|
||||
! --- Multicast / snooping ---------------------------------------------------
|
||||
show ip igmp snooping
|
||||
show ip igmp snooping vlan 1
|
||||
|
||||
! --- QoS --------------------------------------------------------------------
|
||||
show qos
|
||||
show qos interface
|
||||
|
||||
! --- Jumbo / storm control --------------------------------------------------
|
||||
show ports jumbo-frame
|
||||
show storm-control
|
||||
|
||||
! --- FDB / ARP / routes -----------------------------------------------------
|
||||
show mac address-table
|
||||
show mac-address-table
|
||||
show arp
|
||||
show ip route
|
||||
|
||||
! --- Environment / hardware -------------------------------------------------
|
||||
show environment
|
||||
show fan
|
||||
show power supplies
|
||||
show power
|
||||
show hardware
|
||||
show transceiver
|
||||
show inventory
|
||||
|
||||
! --- Flow control -----------------------------------------------------------
|
||||
show flowcontrol
|
||||
show interfaces flowcontrol
|
||||
|
||||
exit
|
||||
@@ -0,0 +1,88 @@
|
||||
!###############################################################################
|
||||
!# pfv-r3-tor-stor — TARGETED LACP hash verification
|
||||
!#
|
||||
!# Purpose: confirm whether `port-channel load-balance layer-2-3-4` actually
|
||||
!# took effect, and gather the LAG state needed to decide whether the LAG
|
||||
!# must be bounced.
|
||||
!#
|
||||
!# Context: tsys6<->tsys7 is capped at exactly single-link speed (~943 Mbps)
|
||||
!# with 90K-136K retransmits on multi-stream TCP, but a clean 928 Mbps / 0
|
||||
!# retransmits on single-stream. That = offered load above 1 Gbps is being
|
||||
!# dropped at a single egress port. We need to know if the switch hash is
|
||||
!# really layer-2-3-4, and whether tsys6 + tsys7 ports are correctly
|
||||
!# bundled into LACP groups.
|
||||
!#
|
||||
!# Paste from enable mode. Capture to:
|
||||
!# returned-logs/pfv-r3-tor-stor-lacp-verify.log
|
||||
!###############################################################################
|
||||
|
||||
enable
|
||||
terminal datadump
|
||||
terminal length 0
|
||||
|
||||
! --- 1. THE KEY COMMAND: what hash is actually active? -----------------------
|
||||
! Expected: layer-2-3-4. If it still shows source-mac / dst-mac /
|
||||
! src-dst-mac (layer 2 only), the change did NOT take effect.
|
||||
show port-channel load-balance
|
||||
show port-channel protocol
|
||||
|
||||
! --- 2. Which LAGs exist and which physical ports are members ----------------
|
||||
! Look for the port-channels carrying tsys6 and tsys7. Confirm each has
|
||||
! TWO member ports both in "bundled" / "Forwarding" state.
|
||||
show port-channel summary
|
||||
show port-channel detail
|
||||
|
||||
! --- 3. LACP actor/partner state per member port ----------------------------
|
||||
! Every member port should show: Actor State Flag = AcLg (Active, Aggr),
|
||||
! Partner Operational, and a non-zero Partner System ID.
|
||||
show lacp
|
||||
show lacp neighbor
|
||||
|
||||
! --- 4. Per-port counters during/after an iperf run -------------------------
|
||||
! THIS is the switch-side equivalent of the host RX-counter test.
|
||||
! Run an 8-stream iperf tsys7->tsys6 first, then immediately capture
|
||||
! these so the byte counters reflect the test. The TWO tsys6-facing
|
||||
! ports should show SIMILAR "RX bytes" (switch ingress from tsys6's
|
||||
! bond is not the test direction; look at TX-out bytes toward tsys6,
|
||||
! i.e. the ports' output counters). Replace Te1/0/x with the real port
|
||||
! IDs shown by "show port-channel summary" above:
|
||||
show interfaces counters
|
||||
show interfaces counters errors
|
||||
|
||||
! --- 5. Running config line that set the hash (proof it was committed) ------
|
||||
show running-config | include port-channel
|
||||
|
||||
! --- 6. Switch identity (to look up correct hash syntax for this model) -----
|
||||
show version
|
||||
show system
|
||||
|
||||
exit
|
||||
|
||||
!###############################################################################
|
||||
!# DECISION TREE after reading the output:
|
||||
!#
|
||||
!# A) show port-channel load-balance reports layer-2-3-4 AND per-port
|
||||
!# counters show traffic split across BOTH tsys6-facing ports:
|
||||
!# -> Hash is fine. Re-test iperf; if still ~943 Mbps, the cap is
|
||||
!# host-side (receiver RX bridge / IRQ / CPU).
|
||||
!#
|
||||
!# B) load-balance reports layer-2-3-4 BUT per-port counters show all traffic
|
||||
!# on ONE tsys6-facing port:
|
||||
!# -> Hash is configured but the LAG needs a bounce to pick it up.
|
||||
!# Bounce the relevant port-channel:
|
||||
!# config
|
||||
!# interface port-channel N
|
||||
!# shutdown
|
||||
!# no shutdown
|
||||
!# end
|
||||
!# WARNING: drops all traffic on that LAG for ~2-5 s. Coordinate.
|
||||
!#
|
||||
!# C) load-balance still shows a layer-2-only policy:
|
||||
!# -> The earlier change didn't commit to this context. Re-apply:
|
||||
!# config
|
||||
!# port-channel load-balance layer-2-3-4
|
||||
!# end
|
||||
!# show port-channel load-balance
|
||||
!# copy running-config startup-config
|
||||
!# Then handle as case (B) if counters still aren't split.
|
||||
!###############################################################################
|
||||
@@ -0,0 +1,108 @@
|
||||
!###############################################################################
|
||||
!# pfv-r3-tor-stor
|
||||
!#
|
||||
!# Rack 3 storage TOR (VLAN1000 storage + VLAN1).
|
||||
!# Carries the 4-port LACP to pfv-core-sw01 (Rack 5).
|
||||
!# Most important switch for cross-rack storage throughput.
|
||||
!#
|
||||
!# Paste from enable mode after `terminal datadump` (DNOS 4/6) or
|
||||
!# `terminal length 0` (DNOS 9 / OS9). Capture to returned-logs/pfv-r3-tor-stor.log
|
||||
!###############################################################################
|
||||
|
||||
enable
|
||||
terminal datadump
|
||||
terminal length 0
|
||||
|
||||
! --- Identify ---------------------------------------------------------------
|
||||
show version
|
||||
show system
|
||||
show inventory
|
||||
show running-config
|
||||
show bootvar
|
||||
show cpu utilization
|
||||
show cpu
|
||||
show memory
|
||||
show processes
|
||||
show logging
|
||||
|
||||
! --- Interfaces (most important section) ------------------------------------
|
||||
show interface status
|
||||
show interface description
|
||||
show interface counters
|
||||
show interface counters errors
|
||||
show interface counters detail
|
||||
show interfaces status
|
||||
show interfaces description
|
||||
show interfaces counters
|
||||
show interfaces counters errors
|
||||
show interfaces configuration
|
||||
|
||||
! --- LACP / port-channel (the 4-port LAG to Rack 5) ------------------------
|
||||
show port-channel summary
|
||||
show port-channel detail
|
||||
show port-channel 1
|
||||
show port-channel 2
|
||||
show port-channel 3
|
||||
show port-channel 4
|
||||
show interfaces port-channel
|
||||
show lacp
|
||||
show lacp neighbor
|
||||
show lacp 1
|
||||
show lag
|
||||
|
||||
! --- VLANs (VLAN1000 is the storage VLAN) ----------------------------------
|
||||
show vlan
|
||||
show vlan brief
|
||||
show vlan id 1000
|
||||
show vlan detail
|
||||
show interfaces vlan 1000
|
||||
show interfaces switchport
|
||||
|
||||
! --- Spanning tree ----------------------------------------------------------
|
||||
show spanning-tree
|
||||
show spanning-tree summary
|
||||
show spanning-tree active
|
||||
show spanning-tree detail
|
||||
show spanning-tree mst
|
||||
|
||||
! --- Multicast / IGMP (storage traffic) -------------------------------------
|
||||
show ip igmp snooping
|
||||
show ip igmp snooping vlan 1000
|
||||
show ip igmp snooping vlan 1
|
||||
|
||||
! --- QoS / CoS / DCB / PFC (matters for storage) ---------------------------
|
||||
show qos
|
||||
show qos interface
|
||||
show class-of-service
|
||||
show dcb
|
||||
show dcb priority
|
||||
show priority-flow-control
|
||||
show pfc
|
||||
|
||||
! --- Jumbo / storm / MTU ----------------------------------------------------
|
||||
show ports jumbo-frame
|
||||
show storm-control
|
||||
show interfaces mtu
|
||||
|
||||
! --- FDB / ARP / routing ----------------------------------------------------
|
||||
show mac address-table
|
||||
show mac-address-table
|
||||
show arp
|
||||
show ip route
|
||||
|
||||
! --- Environment / hardware / optics ---------------------------------------
|
||||
show environment
|
||||
show fan
|
||||
show power supplies
|
||||
show power
|
||||
show hardware
|
||||
show inventory
|
||||
show transceiver
|
||||
show interface transceiver
|
||||
show media
|
||||
|
||||
! --- Flow control (important for storage) ----------------------------------
|
||||
show flowcontrol
|
||||
show interfaces flowcontrol
|
||||
|
||||
exit
|
||||
Reference in New Issue
Block a user