Troubleshooting
A short decision path for the failures that actually occur, the diagnostic log, and the flow trace that answers most of the rest.
Subscribers cannot connect
Work outward from the wire. If frames are not arriving at all, the problem is VLAN tagging or physical — per-VLAN counters tell you whether anything is landing on the sub-interface the subscriber should be using. If frames arrive but authentication fails, check RADIUS reachability, the shared secret, the User-Name format, and the bad-msg-authent counter: a signature mismatch and a wrong secret look identical from the subscriber's side. The PPPoE module log carries the session-establishment detail.
show vlan stats # is anything arriving on the VLAN? show interface access0 # link, errors, drops show sessions # did a session come up at all? show radius statistics # reachability, RTT, bad-msg-authent show log pppoe last 100 # session establishment detail show log radius last 100
Subscriber reports slow speed
Compare the resolved QoS binding against the intended plan, then check whether the aggregate above the subscriber is saturated, then check per-core drops. Remember overhead compensation: a shaped pipe measured by a speed test reads below plan by design unless compensation is set.
show qos bindings show qos sub <id> show drops access0 show health
The diagnostic log
The diagnostic log is captured into an always-on in-memory ring, so you read recent history straight from memory with no file to tail. Levels are off, critical, error, warning, notice, info and debug. log level is operational and immediate, needing no commit: a named module's level applies at once and is persisted, while all is a live-only global sweep. To persist the global level use the system view command instead.
show logging # global level, sink, per-module levels show log # last 200 lines, every module show log <module> # radius, nat, pppoe, smgr, ... show log <module> last <n> show log last <n> clear log log level <module> <lvl> # immediate + persisted log level all <lvl> # live-only global sweep # persistent global level and sink, in the system view system logging level <off|critical|error|warning|notice|info|debug> system logging output memory|stdout|stderr|file <path> commit
Packet flow trace
The trace follows a packet through the fast path so you can see exactly where a flow is classified, translated, shaped or dropped. It works like tcpdump -G: one command filters, captures for a fixed window, prints the matching flow, then auto-disarms and clears. The default window is 5 seconds and the valid range is 1 to 30. Use off only to abort a long capture before its window elapses.
debug trace packet src <ip> # flow FROM this source debug trace packet dst <ip> # flow TO this destination debug trace packet src <ip> dst <ip> # a directional flow debug trace packet ip <ip> # as src OR dst debug trace packet ... drops # only DROPPED packets debug trace packet ... for <1-30> # window, default 5s debug trace packet off # abort early show debug packets # re-read the last capture show debug trace # per-stage pipeline path
Reading a capture
Each captured packet carries per-flow L4 detail, captured at RX so it reflects the real segment sizes and TCP behaviour the subscriber's flow negotiated — useful for comparing a phone's download against a laptop's, or spotting a black hole, without a span port. len is the on-wire IP total length, which is the number that matters for MTU and PMTUD. flags is the TCP flag field in tcpdump style. On a SYN the tcp column shows the receive window plus MSS and window-scale options; on data and ACK packets it shows the window plus sequence and ack numbers — a repeated ack on ACKs means duplicate ACKs and therefore downstream loss, while a repeated seq on data means retransmits. in>out is ingress to egress port, and nat reads snat, reverse, miss or none.
[Netvyn-BNG] debug trace packet src 100.64.0.2 for 10 capturing packets from src=100.64.0.2 for 10 seconds.......... filter: src=100.64.0.2 src:port dst:port prot len flags ttl tcp(...) in>out nat result 100.64.0.2:51001 142.250.4.100:443 tcp 60 ....S. 63 w64240/mss1452/ws8 1>0 snat forwarded 100.64.0.2:51001 142.250.4.100:443 tcp 52 .A.... 63 w2058 s3428... a119.. 1>0 snat forwarded 100.64.0.2:5000 1.1.1.1:53 udp 64 - 63 - 1>0 snat forwarded 3 packet(s) captured
Filter direction matters with NAT
Capture happens at RX, before NAT. Upstream packets carry the subscriber's inside address; downstream replies are addressed to the public pool address and are only un-NAT'd after capture. So to see a subscriber's download, filter on their public address rather than the inside one. This catches people out constantly — an empty capture usually means the filter was on the wrong side of the translation.
Finding a rare drop under load
Without the drops modifier a high-volume forwarded flow floods the 1024-entry ring and overwrites the rare dropped packets. With it, only dropped packets are captured, with their real source, destination and reason — which is how you find the flows hitting ttl-expired, nat-fail or no-route. On a busy box narrow it with a src or dst filter as well, because an unfiltered drops capture still stage-traces every packet. Combine the trace with show nat session when diagnosing a translation problem for one subscriber.
debug trace packet drops for 15 debug trace packet src 100.64.0.3 drops for 15 show nat session 100.64.0.3