Expert Reference Series of White Papers

Troubleshooting
Slow Networks with
Wireshark

1-800-COURSES www.globalknowledge.com
Troubleshooting Slow Networks with
Wireshark
Laura Chappell, Founder, Wireshark University and Chappell University

Introduction
Your phone begins ringing before you find a suitable spot to put down your first comforting cup of coffee in the
morning. Users are complaining that the network is slow – web browsing sessions are painfully sluggish and
email takes forever to download. They state that they simply can’t work this way.

The problem appears to be widespread as your coffee cools faster than the users’ tempers. A lack of error mes-
sages or network alarms makes the problem more elusive and guarantees you’ll be hunting down the problem
well through lunchtime – at least.

Could the problem be related to the infrastructure devices? Is a rogue switch dropping packets periodically?
What about the servers? Could the email server finally be giving in to the pressure of handling all those email
chain letters the users pass amongst themselves? What is the chance that the users’ systems have been compro-
mised with a virus or bot that is spreading stealthily through the shadows of the network like the plague?

In this white paper, we examine how to use Wireshark, the world’s most popular open-source network analyzer,
to troubleshoot some of the top causes of poor network performance, including
• High latency
• Packet loss
• Inefficient window sizes
• Intercepting devices
• Application dependencies

First, we’ll look at Wireshark and examine methods used to “see” network communications.

Wireshark: The Open-Source Network Savior

Wireshark, formerly Ethereal, is the world’s most popular open-source network analyzer and the ideal first-re-
sponder tool on a troubled network. Wireshark enables you to “see” the network communications and defini-
tively point to where the problem lies. Although it cannot tell you why the problem exists, Wireshark reduces
the troubleshooting time and effort drastically by providing a definitive answer to the location of the problem –
removing the guesswork that typically consumes the IT professional’s time while users impatiently wait for their
network services to be restored.

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 2

A system loaded with Wireshark is connected to the network using one of the methods defined below. Network
traffic is captured and decoded by Wireshark’s dissectors, predefined code that breaks apart the packets into
their fields and field contents. Wireshark also contains an Expert system that identifies possible problems in
network communications, thereby shortening the problem isolation process further. For more information on
Wireshark, visit www.wireshark.org.

The Naked Network

The first step in analyzing network performance is to capture the network traffic. Ideally, you’ll capture the traf-
fic to and from a complaining host system from a location as close to that user as possible. You want to experi-
ence the slow performance from their perspective and their location on the network.

There are four basic options available to capture network traffic.

• Load Wireshark directly on one of the host systems.
• Insert a network hub between a host and a switch (half-duplex).
• Insert a network tap between a host and a switch (full-duplex).
• Span the switch port of a user to an analyzer port.

Loading Wireshark on the User’s System

This option makes my skin crawl a bit. I detest the idea of being so invasive and have nightmares imagining the
users running Wireshark on their systems with little or no knowledge of network communications. This would be
my least-favorite recommendation.

Hubbing Out
This is a great option for half-duplex networks. Simply remove the cable from the user’s system and connect it
to a hub. With another cable, connect the user’s system and your analyzer to the hub as shown in the diagram
below. Hubs are stupid – they only know 1s and 0s, and forward all bits down all active ports. All traffic to or
from your user’s system will be copied to your analyzer as well.

Tapping Out
Hubs work great on half-duplex networks, but most of us have migrated to full-duplex networks. Hubs can’t
handle these full duplex communications; this is the job for a full-duplex tap. The connection process would be
the same as shown in Figure 1, provided you have an aggregating full-duplex tap. An aggregating tap combines
both transmit and receive channel information between the user and the switch into a single data stream to the
analyzer system.

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 3

 Figure 1: Use full-duplex taps to listen in on all traffic to and from the user’s system on a full-duplex network.

Spanning
Spanning requires reconfiguration of the switch that the user’s system connects to. A switch that is configured
with a spanned port sends a copy of all traffic to/from that spanned port down another port – the port that the
analyzer is connected to. This method of tapping-in is ideal for listening to traffic to/from a server as you are
unlikely to break the server’s network connection to install a hub or tap.

High Latency: Somebody’s Dragging their Feet

Latency is a measurement of travel time from one host to another or the roundtrip between hosts. Although
packets on a 100Mbps network always travel at a rate of 100Mbps, latency is introduced by distance and inter-
connecting devices that process packets.

Slow travel from one endpoint to another is defined as high latency. High latency has a tremendously negative
effect on network communications. As an example, in Figure 2, we examine the roundtrip time of a file down-
load process on a high-latency path. At times, the roundtrip latency time reaches 1 second, which is completely
unacceptable.

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 4

Figure 2: Use Wireshark’s Statistics > TCP Stream Graph > Round Trip Time Graph to determine the current roundtrip
latency for a file download.

We use Wireshark to determine the roundtrip time on a path to determine if this is the reason for poor net-
work performance for Transmission Control Protocol (TCP) communications. TCP is used for web browsing,
email receipt and transmission, file transfer protocol, and many other popular applications. In many situations,
especially when hosts are using Windows XP, the operating system can be adjusted to work more efficiently on
high-latency paths.

Packet Loss: Losing Data in Bits and Pieces

Packet loss is one of the most common problems I see on networks. When a user accesses a web site and begins
to download the elements of the site, lost packets trigger retransmissions, increasing the overhead required to
download the site elements and delaying the total download process.

In addition, when an application uses TCP, the effect of lost packets is especially detrimental. Each time a TCP
connection senses a lost packet, the throughput rate automatically throttles back dramatically to account for

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 5

network problems. Slowly, it recovers to a more acceptable rate until the next packet is lost again, causing a
drastic cut-back in data throughput. Packet loss has a tremendously negative effect on large file downloads that
should otherwise stream across a network smoothly.

What does packet loss look like? It depends. If the application is running over TCP, packet loss has two different
looks. In one case, the receiver tracks packets based on their sequence numbers and notices a packet is missing.
The client requests the missing packet three times (duplicate acknowledgments) which triggers a retransmission.
If the sender times-out when it notices the receiver has not acknowledged receipt of a data packet, the sender
retransmits the data packet.

In Figure 3, Wireshark indicates that packet loss has occurred and duplicate acknowledgments trigger the
retransmission. A high number of duplicate acknowledgments indicates that a network has experienced packet
loss and is also facing high latency.

Figure 3: Wireshark indicates that packet loss has occurred by color coding the problematic traffic.

Locating the exact point of packet loss is imperative in improving network performance. When packet loss is
experienced, we move the Wireshark along the path until we can no longer see packet loss. At this point, we are
“upstream” from the packet loss point, and we know where to concentrate our troubleshooting efforts.

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 6

Inefficient Window Sizes: Peering Through Small Windows
There are several “windows” in TCP/IP networking besides the Microsoft operating system.
• Sliding window
• Receiver window
• Congestion window

As a set, these windows define the TCP-based communication performance on the network. First, let’s define
each of these windows and their individual effect on network throughput.

The Sliding Window

The sliding window is a process used to send the next TCP segments onto the network as data is acknowledged.
The sliding window expands as the sender receives acknowledgments for previously transmitted segments.
Larger amounts of data can be sent on the network as long as there are no dropped packets on the network.
When packet loss occurs, the sliding window shrinks under the assumption that the network cannot handle the
larger amount of data on the wire.

The Receiver Window (rwin)

The receiver window is a buffer space in the TCP stack. When data is received, it is held in this buffer space
until an application picks up the data. When an application does not keep up with the receive rate, the receiver
window fills and may eventually lead to a “zero window” condition. When a receiver advertises a zero window
condition, all data transmission to the host must stop. Throughput drops to zero. A process called Window Scal-
ing (RFC 1323) enables a host to scale the receiver window to a larger value and reduce the chance of a zero
window condition occurring.

Figure 4, below, depicts a zero window condition that caused a 32-second delay in network communications.

The Congestion Window (cwnd)

The congestion window defines the actual amount of data that the network can support. It is defined by the
sender’s packet transmission rate, the network packet loss rate, and the receiver’s window size. During a healthy
network communication, the congestion window increases consistently until the transfer completes or it hits a
“ceiling” defined by the network health, the sender’s transmit capabilities, or the receiver’s window size. Each
new connection begins the process of window size negotiation anew.

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 7

 Figure 4: It took over 32 seconds to resolve the zero window condition, denoted by Wireshark’s Expert system.

Intercepting Devices: When Network Traffic Cops Go Bad

“Network Traffic Cops” are interconnecting devices, such as switches, routers, and firewalls that make forward-
ing decisions. When packet loss occurs, it is best to look at these devices as the possible cause.

These interconnecting devices can also add latency to the path. For example, if traffic prioritization is in use, we
can see additional latency injected into a stream that meets a low priority level.

Application Dependencies: Choose Your Application Friends

Carefully
Some applications have dependencies on other applications, processes or communications with other hosts. For
example, if your database application relies on connecting to numerous servers to pull database elements, slow
performance to those other servers can affect the local application’s load time.

As a simple example, we can look at a web browsing session in which the target server references numerous
other websites. In order to load the main page of the site, www.espn.com, for example, you must access 16
hosts that provide advertisements and content for the main www.espn.com page. Figure 5 shows the list of
hosts that you must contact when you load the www.espn.com home page.

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 8

 Figure 5: Wireshark’s HTTP Load Distribution window lists all servers referenced by the www.espn.com home page.

In addition, poorly-written applications can affect the performance on both the sending side and the receiving
side. No matter how healthy and free of dropped packets the network is, an application may not take advantage
of the network’s capabilities, because it has its own throttling mechanisms limiting the amount of data that it
sends. On the receiving side of the connection, an application that does not pull data out of the receive buffer in
a timely manner can lead to a limited or zero window condition. In the case of poorly performing applications,
consider researching the possibility that the application can be tuned for better performance.

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 9

First Steps to a Healthier Network
Most networks can be tuned for better performance through a series of tasks.
• L earn to use Wireshark as a first-responder task to identify the cause of poor performance quickly and
efficiently.
•D
 etermine the source of latency on the network path and, if possible, reduce the latency to an acceptable
value.
• Identify the location of packet loss and resolve the problem.
• E xamine the window size in use on data transfers and consider optimizing the receiver window size, if
possible.
• E xamine the performance of intercepting devices to determine if they are adding latency or dropping
packets.
•O
 ptimize applications to send larger amounts of data and pull data out of the receiver window, if
possible.

We’ve discussed the primary causes of network performance problems, but one cause – lack of insight into
network communications behavior – cannot be overlooked. After 20 years of analyzing network traffic and
teaching traffic interpretation and problem resolution, it is clear that network analysis is a skill that every IT
professional should possess.

Wireshark offers an insight into networks in a similar way that X-rays and CAT scans offer an insight into the
human body for accurate and timely diagnoses. And, just like those indispensable technologies in the medical
field, Wireshark has become an essential tool to locate and diagnose the cause of network problems in the most
efficient and cost-effective method possible.

Note: This white paper was developed as a follow-up to the Global Knowledge webinar by the same name.
Visit our Knowledge Center at www.globalknowledge.com/knowledgecenter to view the related webinar.

Learn More
Learn more about how you can improve productivity, enhance efficiency, and sharpen your competitive edge.
Check out the following Global Knowledge courses:
Analyzing TCP/IP Networks with Wireshark
Troubleshooting and Securing TCP/IP Networks with Wireshark
TCP/IP Networking

For more information or to register, visit www.globalknowledge.com or call 1-800-COURSES to speak with a
sales representative.

Our courses and enhanced, hands-on labs offer practical skills and tips that you can immediately put to use. Our
expert instructors draw upon their experiences to help you understand key concepts and how to apply them to

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 10

your specific work situation. Choose from our more than 700 courses, delivered through Classrooms, e-Learning,
and On-site sessions, to meet your IT and management training needs.

About the Author

Laura Chappell is the Founder of Wireshark University and Chappell University, and has been researching, writ-
ing, and lecturing on network analysis for over 20 years. Laura designed and created Wireshark University in
cooperation with Gerald Combs, the creator of Ethereal/Wireshark. In March 2009, the Wireshark Certification
Program was released to validate a candidate’s knowledge of Wireshark functionality, TCP/IP troubleshooting,
and network forensics/security.

Wireshark University (www.wiresharkU.com)

Chappell University (www.chappellU.com)

Copyright ©2009 Global Knowledge Training LLC. All rights reserved. 11