Analysing traces for performance bottlenecks

[Sam Stickland]

Hi,

Are there any packages (or Wireshark options that I've missed) that can
follow a TCP stream and determine the limiting factor on throughput. E.g
Latency, packet loss, out of sequence packets, window size, or even just
the senders rate onto the wire. I know how to analyse a trace by hand
for performance issues, but it's relatively time consuming.

Googling for variations on "Analyse TCP stream limit throughput" didn't
find anything.

Sam

[Tim Eberhard]

One potentially useful piece of software that is a work in progress is
called Pcapdiff. (http://www.eff.org/testyourisp/pcapdiff/)

Written by Seth Schoen and Steven Lucy it's a pretty useful piece of
software. While still in a relative infant stage I think it could mature
into a very useful tool to troubleshoot network connectivity.

Pcapdiff was originally written to find out if your ISP was toying with you
P2P packets (comcast) and injecting resets. I have worked with the authors a
bit and found it highly useful to take two packet captures on my network and
use it to verify A) All packets sent are recieved B) They are in their
original state.

Again the features of pcapdiff are pretty limited but I love the idea of the
program and I really think it could grow into an excellent tool to analyze
packet captures with just a few additional features (a few listed below)

-Tim Eberhard

On Tue, Jul 15, 2008 at 5:05 AM, Sam Stickland <

Post by Sam Stickland
Hi,
Are there any packages (or Wireshark options that I've missed) that can
follow a TCP stream and determine the limiting factor on throughput. E.g
Latency, packet loss, out of sequence packets, window size, or even just the
senders rate onto the wire. I know how to analyse a trace by hand for
performance issues, but it's relatively time consuming.
Googling for variations on "Analyse TCP stream limit throughput" didn't
find anything.
Sam

[Sam Stickland]

A bit more googling has found the Web100 projects NDT
(http://e2epi.internet2.edu/ndt/). I'm currently making a Linux VM that
can run it. It's useful, but I'm still really after something that can
do it's type of analysis from a packet capture.

Sam

Post by Sam Stickland
Hi,
Are there any packages (or Wireshark options that I've missed) that
can follow a TCP stream and determine the limiting factor on
throughput. E.g Latency, packet loss, out of sequence packets, window
size, or even just the senders rate onto the wire. I know how to
analyse a trace by hand for performance issues, but it's relatively
time consuming.
Googling for variations on "Analyse TCP stream limit throughput"
didn't find anything.
Sam

[r***@rochester.rr.com]

Wireshark can show the throughput on a bits/sec or pps, by IP, etc. This is under IO Graphs. You'll want to change the time display format of the main decode window to Seconds Since Beginning of Capture to sync up time with the graph.

At least that way, you can just focus on the dips in throughput in the graphs.


Rick

Date: Tue, 15 Jul 2008 11:05:34 +0100
Hi,
Are there any packages (or Wireshark options that I've missed) that can
follow a TCP stream and determine the limiting factor on throughput. E.g
Latency, packet loss, out of sequence packets, window size, or even just
the senders rate onto the wire. I know how to analyse a trace by hand
for performance issues, but it's relatively time consuming.
Googling for variations on "Analyse TCP stream limit throughput" didn't
find anything.

tcptrace is old and pretty basic, but it can provide a LOT if
information. Combined with xplot, the graphs often point to the exact
nature of a TCP problem, but you need a really good understanding of TCP
to figure anything out.
--
R. Kevin Oberman, Network Engineer
Energy Sciences Network (ESnet)
Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751

[Tim Sanderson]

What about gnuplot? Maybe it provides something more than xplot?

http://www.gnuplot.info/

--
Tim Sanderson, network administrator
***@donet.com


-----Original Message-----
From: Bulger, Tim [mailto:***@polk.com]
Sent: Thursday, July 17, 2008 12:14 PM
To: Sam Stickland; Matt Cable
Cc: ***@nanog.org
Subject: RE: Analysing traces for performance bottlenecks

There is a Java version of xplot available now called jPlot. It works in largely the same way.

http://www.tcptrace.org/jPlot/

Regards,
Tim

-----Original Message-----
From: Sam Stickland [mailto:***@spacething.org]
Sent: Thursday, July 17, 2008 11:53 AM
To: Matt Cable
Cc: ***@nanog.org
Subject: Re: Analysing traces for performance bottlenecks

Kevin Oberman <oberman <at> es.net> writes

tcptrace is old and pretty basic, but it can provide a LOT if
information. Combined with xplot, the graphs often point to the exact
nature of a TCP problem, but you need a really good understanding of

TCP

to figure anything out.

Wireshark also provides tcptrace-like graphs ("Statistics -> TCP

Stream Graph ->

Time Sequence Graph (tcptrace)"). They're not quite as pretty, but

are just as

effective at tracking down all sorts of TCP problems, provided, as

Kevin said,

you have a really good understanding of how TCP behaves

Thanks for all the replies so far. While the TCP graphs are useful they are very difficult to read in Wireshark - they really need to be displayed in xplot, but this requires an X11 setup?

I've found NDT:

http://e2epi.internet2.edu/ndt/

This uses a java applet hosted on a web100 patched linux server to record network diagnostics from connecting clients. A typical report might look like this:

Web100 reports the Round trip time = 122.15 msec; the Packet size = 1260 Bytes; and
No packet loss was observed.
C2S throughput test: Packet queuing detected: 1.09%
S2C throughput test: Packet queuing detected: 1.32%
This connection is receiver limited 84.33% of the time.
Increasing the the client's receive buffer (63.0 KB) will improve performance
This connection is sender limited 1.70% of the time.
Increasing the NDT server's send buffer (127.0 KB) will improve performance
This connection is network limited 13.96% of the time.

The theoretical network limit is 7869.69 Mbps
The NDT server has a 127.0 KByte buffer which limits the throughput to 16.37 Mbps
Your PC/Workstation has a 63.0 KByte buffer which limits the throughput to 4.09 Mbps
The network based flow control limits the throughput to 8.73 Mbps

Client Data reports link is 'OC-48', Client Acks report link is 'OC-12'
Server Data reports link is 'OC-48', Server Acks report link is 'T3'

Something that could provide a similar, automated analysis of a TCP stream capture is what I'm after, although I doubt a standard packet capture will be able to provided as many metric as web100 stack can.

Sam
*****************************************************************
This message has originated from R. L. Polk & Co.,
26955 Northwestern Highway, Southfield, MI 48033.
R. L. Polk & Co. sends various types of email communications. If this email message concerns the potential licensing of a Polk product or service, and you do not wish to receive further emails regarding Polk products, forward this email to ***@polk.com with the word "remove" in the subject line.

The email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed.

If you have received this email in error, please delete this message and notify the Polk System Administrator at ***@polk.com.
*****************************************************************