Network Microbursts: Why a Quiet Link Can Still Drop Packets

By Editorial Team Published Updated 7 min read
Turquoise water splashes as it rushes through a narrow metal opening, with a smoother surface downstream.

The monitoring dashboard says the uplink is only 30% busy. The switch says it has dropped packets. Both can be right.

The dashboard is telling you how much traffic the link carried over a stretch of time. The switch had to decide, in a much shorter moment, whether it had room for one more packet. Network microbursts are what can happen in the gap between those two timescales.

What Are Network Microbursts?

A network microburst is a brief surge of traffic. A common case occurs when several incoming flows converge on one outgoing port and, for a short time, offer it more traffic than it can send. Packets wait in an egress queue, the waiting area for that outgoing traffic. If the queue runs out of room, packets can be discarded. [1]

The port does not transmit above its physical line rate. The excess is in the traffic offered to the output queue. A burst that fits in the available buffer can clear without loss, although some packets will wait longer than usual.

“Microburst” does not prescribe a universal duration or detection threshold. A burst that matters on one switch may be absorbed by another, depending on its size, the outgoing rate, and the buffer space available at that moment.

The 100-Microsecond Traffic Jam

Imagine two servers on separate 10 Gb/s links. They start sending at line rate at the same time, and all their traffic must leave through one 10 Gb/s port. The overlap lasts 100 microseconds, one ten-thousandth of a second.

QuantityValue
Combined rate offered to the output20 Gb/s
Rate the output can serve10 Gb/s
Data arriving in 100 microseconds250 kB
Data the output can send in that time125 kB
Buffer space needed to absorb the excess125 kB

The calculation is straightforward: excess bytes = (arrival rate − output rate) × burst duration ÷ 8, with rates in bits per second and duration in seconds. During the overlap, the queue grows at the equivalent of 10 Gb/s.

Now give the initially empty queue an effective limit of 64 kB. At that growth rate it fills after about 51.2 microseconds. The burst still has time to run, so roughly 61 kB of offered data cannot fit. Once the burst ends, the output can drain what remains in about 51.2 microseconds, assuming no further arrivals.

This is an illustrative fluid model using decimal units: 1 kB = 1,000 bytes. Real switches admit and discard whole packets, account for buffer space in hardware-specific ways, and may share memory among queues. The 64 kB represents space available to this queue under the example’s assumptions; it is not a hardware specification.

Where Did the Burst Go on the Graph?

In our example, the port forwards about 189 kB: 125 kB during the burst and 64 kB afterward. Spread that over a 30-second interval and it adds just 50.4 kb/s to the average on a 10 Gb/s link.

The graph did not lie. It answered a different question.

A byte counter sampled every 30 seconds can tell you how much traffic passed in 30 seconds. It cannot reconstruct the busiest 100 microseconds. A separate discard counter can retain evidence of lost packets, but two readings of that counter usually tell you only how many discards occurred between the samples. An empty queue when you check it tells you little about how high it rose in between. [2]

Watch for the distinction between counter deltas and instantaneous samples. A queue-depth sample taken after the burst can read zero even if a discard counter recorded its consequences. A supported queue watermark, which records a maximum, or a hardware threshold event can preserve evidence of that occupancy spike. [3, 4]

What Makes Traffic Arrive Together?

Imagine a service asking ten storage servers for parts of the same result. Each server replies promptly. The switch port toward the requester suddenly receives many replies competing for the same exit. A classic example is TCP incast, where synchronized replies in a many-to-one workload overflow switch buffers and can make the requester wait for loss recovery. [5]

Several transfers starting together or multiple fast links feeding a slower uplink can produce a similar queueing pattern. The key is the overlap at the bottleneck. Two flows that take turns may fit comfortably; the same two flows arriving together can exceed the available headroom.

Microbursts also help explain why short loss episodes and sustained high latency deserve separate diagnoses. A burst can briefly fill and then empty a queue. Bufferbloat describes excessive delay from queues that stay occupied under load. Both can occur on the same path. A standing backlog leaves less room for the next burst.

How to Confirm a Microburst

A rising discard counter is a reason to investigate. It does not establish when the losses occurred or why the device discarded the packets. Work from the affected traffic toward the queue that serves it.

  1. Find the outgoing interface and traffic class. Identify the common exit for the affected flows. Compare counter deltas during the symptom window and look for per-queue drops where the platform exposes them. Check the documented meaning of each counter: policy drops and output queue drops can have different causes. Check physical error counters separately.
  2. Look for a brief rise in queue occupancy. Use per-queue high-water marks or hardware threshold events if the switch supports them. Check what time the peak covers and when the watermark was last reset. A threshold crossing shows that the queue grew; correlate it with drop increments and the reported symptoms before assigning blame. [3, 4]
  3. Check which traffic overlapped. Look at application timestamps for synchronized replies or scheduled jobs. If you need a packet trace, choose capture points that can reveal the traffic offered toward the bottleneck, and verify capture timing and capacity. An egress trace can show what survived and when the link ran at line rate; it cannot show packets already discarded in that output queue. A saturated mirror or a capture host that drops packets can distort the result. [1]
  4. Change one likely trigger and repeat. Compare the same workload before and after pacing replies or staggering transfer starts. Look for fewer drops, lower queue peaks, and better completion times for the affected requests. Tail latency, such as the 99th percentile, can reveal an improvement that the average misses.

The bandwidth-delay product estimates how much data can be in flight at a given rate and round-trip time. It answers a different question from how much excess traffic collects at a particular output during a particular burst. You need the second calculation here.

Fix the Part of the System That Creates the Burst

What you findWhat to try
Synchronized replies or parallel jobsPace the senders, stagger starts, or limit how many replies are in flight together.
Traffic repeatedly converging on one port or link-aggregation memberVerify path placement and member utilization. Redistribute flows where possible or add capacity where the queue is forming.
Rare bursts that slightly exceed the queue’s available headroomIf the platform allows it, assess a targeted buffer adjustment and measure the effect on delay and other queues.
Loss hurts one traffic class more than othersReview queue scheduling and classification. Give important traffic suitable treatment while checking what happens to the other classes.

More output capacity helps only if it is available to the traffic causing the problem. For example, adding another member to an aggregate may leave the same flows competing on the original member, depending on how they are distributed.

A larger buffer can absorb an isolated burst, but it can also make packets wait longer or take headroom from other queues. Shaping can smooth traffic before the bottleneck, but the shaping queue needs limits of its own. Congestion feedback and queue management can help responsive senders adjust; some bursts are already in flight by the time that feedback has any effect. See how backpressure works for the role of headroom and reaction time. [6]

When the graph looks quiet and packets still disappear, ask a more precise question: Which output did they all try to use at the same time?

References and Further Reading

  1. Cisco: Understand Output Drops on Catalyst 9000 Switches — bursty output traffic, queue drops, and capture considerations.
  2. RFC 2863: The Interfaces Group MIB — interface counters, including output octets and discards.
  3. Cisco Nexus 9000 NX-OS: Micro-Burst Monitoring — an example of per-queue threshold detection.
  4. Arista EOS: Latency Analyzer (LANZ) — examples of queue congestion monitoring and reporting.
  5. Vasudevan et al.: A (In)Cast of Thousands — original research on synchronized datacenter workloads and TCP incast.
  6. RFC 7567: IETF Recommendations Regarding Active Queue Management — queue management, congestion signals, and burst absorption.

Network Encyclopedia Editorial Team

Every entry is written, reviewed and kept up to date by our editors. Spotted a mistake? We fix it in the open.

Editorial Mission · Correction Policy · Suggest a correction