Possible explanations for a large hop in latency

Discussion:

(too old to reply)

Frank Bulk

2008-06-26 22:51:42 UTC

Our upstream provider has a connection to AT&T (12.88.71.13) where I
relatively consistently measure with a RTT of 15 msec, but the next hop
(12.122.112.22) comes in with a RTT of 85 msec. Unless AT&T is sending that
traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that 12.122.112.22
has some kind of ICMP rate-limiting.

Is this a real performance issue, or is there some logical explanation?

Frank

Tim Peiffer

2008-06-27 02:03:19 UTC

Permalink

We had a similar situation going from Minneapolis to Kansas City via
Chicago. Normal latency from Minneapolis to Chicago via Level3 MPLS
network is about 14msec RTT. When the the circuit from Minneapolis to
Chicago went out for one reason or another, our MPLS link went from
Minneapolis to Tulsa, to Dallas, and then to Chicago.. That added a
little latency in the path from Minneapolis to Chicago.. We didn't need
to exceed the SLA in order to cry foul. They didn't intentionally
create an inefficient path.. The problem was recognized and fixed the
same day.

Latency on an MPLS circuit is the cumulative latency on the Label Switch
Path, and a number of the hops are invisible. The latency per hop is
still the same... you just can't see that your traffic is travelling to
say Denver or Dallas.

Tim Peiffer
Network Support Engineer
Networking and Telecommunications Services
University of Minnesota/NorthernLights GigaPOP

Thanks for the added information.
Even if their MPLS path went from the midwest (where I'm located) to San
Francisco and then back to St. Louis (where 12.122.112.22 appears to be), I
don't think that accounts for a 70 msec jump in traffic. And I don't think
they would (intentionally) create such an inefficient MPLS path.
Someone off-list told me they tried to trace to 12.88.71.13, but once they
hit an AT&T router their ICMP traffic appeared to be blocked.
Frank
-----Original Message-----
Sent: Thursday, June 26, 2008 8:09 PM
Cc: nanog list
Subject: Re: Possible explanations for a large hop in latency
The explanation I got, was that the latency seen at the first hop was
actually a reply from the last hop in the path across their MPLS
network. Hence, all the following hops had very similar latency.
Personally, I thought it was rather strange for them to do that. And,
I've never seen that occur on any other network.
Perhaps someone from ATT would like to chime in.
--John

Did that satisfy you? I guess with MPLS they could tag the traffic and

send

it around the country twice and I wouldn't see it at L3.
Frank
-----Original Message-----
Sent: Thursday, June 26, 2008 7:04 PM
Cc: nanog list
Subject: Re: Possible explanations for a large hop in latency
When I asked ATT about the sudden latency jump I see in traceroutes,
they told me it was due to how their MPLS network is setup.
--John

that

Post by Frank Bulk
traffic over a cable modem or to Europe and back, I can't see a reason

why

Post by Frank Bulk
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that

12.122.112.22

Post by Frank Bulk
has some kind of ICMP rate-limiting.
Is this a real performance issue, or is there some logical explanation?
Frank

James R. Cutler

2008-06-26 23:58:53 UTC

Permalink

Deep Packet Inspection engine delay. <G>

Post by Frank Bulk
Our upstream provider has a connection to AT&T (12.88.71.13) where I
relatively consistently measure with a RTT of 15 msec, but the next hop
(12.122.112.22) comes in with a RTT of 85 msec. Unless AT&T is sending that
traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that
12.122.112.22
has some kind of ICMP rate-limiting.
Is this a real performance issue, or is there some logical
explanation?
Frank

James R. Cutler
***@consultant.com

John T. Yocum

2008-06-27 00:03:46 UTC

Permalink

When I asked ATT about the sudden latency jump I see in traceroutes,
they told me it was due to how their MPLS network is setup.

--John

Post by Frank Bulk
Our upstream provider has a connection to AT&T (12.88.71.13) where I
relatively consistently measure with a RTT of 15 msec, but the next hop
(12.122.112.22) comes in with a RTT of 85 msec. Unless AT&T is sending that
traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that 12.122.112.22
has some kind of ICMP rate-limiting.
Is this a real performance issue, or is there some logical explanation?
Frank

Frank Bulk - iNAME

2008-06-27 02:27:29 UTC

Permalink

Interestingly enough, when I trace from my Cisco router it seems to show
some MPLS labels after the hop of interest (12.88.71.13 to 12.122.112.78,
only 24 msec here!). I'm not sure how our Cisco box derives these from a
foreign network.

Router#traceroute 69.28.226.193

Type escape sequence to abort.

Tracing the route to 69.28.226.193

1 sxct.sxcy.mtcnet.net (167.142.156.197) 0 msec 0 msec 0 msec

2 siouxcenter.sxcy.137.netins.net (167.142.180.137) 4 msec 4 msec 4 msec

3 ins-b12-et-4-0-112.desm.netins.net (167.142.57.106) 8 msec 8 msec 8 msec

4 ins-h2-et-1-10-127.desm.netins.net (167.142.57.129) 8 msec 8 msec 8 msec

5 ins-c2-et-pc2-0.desm.netins.net (167.142.57.142) 8 msec 8 msec 8 msec

6 12.88.71.13 28 msec 24 msec 28 msec

7 tbr2.sl9mo.ip.att.net (12.122.112.78) [MPLS: Label 30663 Exp 0] 52 msec
48 msec 52 msec

8 cr2.sl9mo.ip.att.net (12.122.18.69) [MPLS: Label 17306 Exp 0] 52 msec 52
msec 52 msec

9 cr2.cgcil.ip.att.net (12.122.2.21) [MPLS: Label 16558 Exp 0] 52 msec 52
msec 52 msec

10 cr1.cgcil.ip.att.net (12.122.2.53) [MPLS: Label 17002 Exp 0] 48 msec 52
msec 52 msec

11 cr1.n54ny.ip.att.net (12.122.1.189) [MPLS: Label 17033 Exp 0] 52 msec 52
msec 48 msec

12 tbr1.n54ny.ip.att.net (12.122.16.138) [MPLS: Label 32364 Exp 0] 52 msec
52 msec 52 msec

13 12.122.86.165 48 msec 48 msec 52 msec

14 12.118.100.58 60 msec 60 msec 64 msec

15 oc48-po2-0.tor-151f7-cor-2.peer1.net (216.187.115.125) 52 msec 52 msec
68 msec

16 oc48-po7-0.tor-151f-dis-1.peer1.net (216.187.114.149) 52 msec 52 msec 48
msec

17 tor-fe3-5a.ne.peer1.net (216.187.68.6) 52 msec 52 msec *

Router#

Wondering why the RTT dropped to 24 msec for that hop, I entered both
69.28.226.192 and the IP address that my customer has been complaining about
(12.129.255.4) into PingPlotter and I see that those behave very
differently. I'm now guessing that AT&T is routing back traffic sent to
12.129.255.4 in a different way (perhaps asymmetrically) than traffic sent
to 69.28.226.192, but it doesn't show up until it hits 12.122.112.22.
Perhaps it's all those 1's and 2'. ;)

I notice that in the low RTT trace router 12.88.71.13 goes to
tbr2.sl9mo.ip.att.net (12.122.112.78), but in the high RTT trace, roouter
12.88.71.13 goes to tbr1.sl9mo.ip.att.net (12.122.112.22). Must be
something about the way AT&T gets to tbr1.sl9mo.ip.att.net (12.122.112.22).
I can't traceroute to either of those networks directly. In fact, I don't
appear to be able to traceroute to any of the 12.122.x.x or 12.129.x.x I see
in my traceroutes, perhaps because AT&T uses some of that space internally
and doesn't advertise it.

Frank

From: Robert Richardson [mailto:***@gmail.com]
Sent: Thursday, June 26, 2008 8:20 PM
To: John T. Yocum
Cc: ***@iname.com; nanog list
Subject: Re: Possible explanations for a large hop in latency

They probably don't propagate TTL w/in their MPLS core. Depending on how
they have MPLS implemented, you may only see 2 hops on the network; the
ingress and egress routers. If the ingress router was in NYC and the egress
in Seattle, you could understandably expect a large jump in RTT.

Not an ATT customer but do know other providers run their MPLS core's this
way...

-Robert

On Thu, Jun 26, 2008 at 6:09 PM, John T. Yocum <***@fluidhosting.com>
wrote:

The explanation I got, was that the latency seen at the first hop was
actually a reply from the last hop in the path across their MPLS network.
Hence, all the following hops had very similar latency.

Personally, I thought it was rather strange for them to do that. And, I've
never seen that occur on any other network.

Perhaps someone from ATT would like to chime in.

--John

Frank Bulk - iNAME wrote:

Did that satisfy you? I guess with MPLS they could tag the traffic and send
it around the country twice and I wouldn't see it at L3.

Frank

-----Original Message-----
From: John T. Yocum [mailto:***@fluidhosting.com] Sent: Thursday, June 26,
2008 7:04 PM
To: ***@iname.com
Cc: nanog list
Subject: Re: Possible explanations for a large hop in latency

When I asked ATT about the sudden latency jump I see in traceroutes,
they told me it was due to how their MPLS network is setup.

--John

Frank Bulk wrote:

Our upstream provider has a connection to AT&T (12.88.71.13
<http://12.88.71.13/> ) where I
relatively consistently measure with a RTT of 15 msec, but the next hop
(12.122.112.22 <http://12.122.112.22/> ) comes in with a RTT of 85 msec.
Unless AT&T is sending

that

traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that 12.122.112.22
<http://12.122.112.22/>
has some kind of ICMP rate-limiting.

Is this a real performance issue, or is there some logical explanation?

Frank

Frank Bulk - iNAME

2008-06-27 03:31:50 UTC

Permalink

Just google "tbr1.sl9mo.ip.att.net" and it's clear that high latency through
that point has occurred before. And guess what kind of customer complained
to me about the latency? A gamer.

Frank

-----Original Message-----
From: Frank Bulk - iNAME [mailto:***@iname.com]
Sent: Thursday, June 26, 2008 9:27 PM
To: 'Robert Richardson'; John T. Yocum
Cc: nanog list
Subject: RE: Possible explanations for a large hop in latency

<snip>

Wondering why the RTT dropped to 24 msec for that hop, I entered both
69.28.226.192 and the IP address that my customer has been complaining about
(12.129.255.4) into PingPlotter and I see that those behave very
differently. I'm now guessing that AT&T is routing back traffic sent to
12.129.255.4 in a different way (perhaps asymmetrically) than traffic sent
to 69.28.226.192, but it doesn't show up until it hits 12.122.112.22.
Perhaps it's all those 1's and 2'. ;)

I notice that in the low RTT trace router 12.88.71.13 goes to
tbr2.sl9mo.ip.att.net (12.122.112.78), but in the high RTT trace, roouter
12.88.71.13 goes to tbr1.sl9mo.ip.att.net (12.122.112.22). Must be
something about the way AT&T gets to tbr1.sl9mo.ip.att.net (12.122.112.22).
I can't traceroute to either of those networks directly. In fact, I don't
appear to be able to traceroute to any of the 12.122.x.x or 12.129.x.x I see
in my traceroutes, perhaps because AT&T uses some of that space internally
and doesn't advertise it.

Frank

From: Robert Richardson [mailto:***@gmail.com]
Sent: Thursday, June 26, 2008 8:20 PM
To: John T. Yocum
Cc: ***@iname.com; nanog list
Subject: Re: Possible explanations for a large hop in latency

They probably don't propagate TTL w/in their MPLS core. Depending on how
they have MPLS implemented, you may only see 2 hops on the network; the
ingress and egress routers. If the ingress router was in NYC and the egress
in Seattle, you could understandably expect a large jump in RTT.

Not an ATT customer but do know other providers run their MPLS core's this
way...

-Robert

On Thu, Jun 26, 2008 at 6:09 PM, John T. Yocum <***@fluidhosting.com>
wrote:

The explanation I got, was that the latency seen at the first hop was
actually a reply from the last hop in the path across their MPLS network.
Hence, all the following hops had very similar latency.

Personally, I thought it was rather strange for them to do that. And, I've
never seen that occur on any other network.

Perhaps someone from ATT would like to chime in.

--John

Frank Bulk - iNAME wrote:

Did that satisfy you? I guess with MPLS they could tag the traffic and send
it around the country twice and I wouldn't see it at L3.

Frank

-----Original Message-----
From: John T. Yocum [mailto:***@fluidhosting.com] Sent: Thursday, June 26,
2008 7:04 PM
To: ***@iname.com
Cc: nanog list
Subject: Re: Possible explanations for a large hop in latency

When I asked ATT about the sudden latency jump I see in traceroutes,
they told me it was due to how their MPLS network is setup.

--John

Frank Bulk wrote:

Our upstream provider has a connection to AT&T (12.88.71.13
<http://12.88.71.13/> ) where I
relatively consistently measure with a RTT of 15 msec, but the next hop
(12.122.112.22 <http://12.122.112.22/> ) comes in with a RTT of 85 msec.
Unless AT&T is sending

that

traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that 12.122.112.22
<http://12.122.112.22/>
has some kind of ICMP rate-limiting.

Is this a real performance issue, or is there some logical explanation?

Frank

Randy Bush

2008-06-27 03:36:14 UTC

Permalink

Post by Frank Bulk - iNAME
Just google "tbr1.sl9mo.ip.att.net" and it's clear that high latency through
that point has occurred before. And guess what kind of customer complained
to me about the latency? A gamer.

you can pay a lot of money for the net propagation anomaly detection
services that gamers give you for free.

randy

Warren Kumari

2008-06-27 14:06:20 UTC

Permalink

Post by Randy Bush

Post by Frank Bulk - iNAME
Just google "tbr1.sl9mo.ip.att.net" and it's clear that high
latency through
that point has occurred before. And guess what kind of customer complained
to me about the latency? A gamer.

you can pay a lot of money for the net propagation anomaly detection
services that gamers give you for free.

----
Many years ago I worked for a small Mom-and-Pop type ISP in New York
state (I was the only network / technical person there) -- it was a
very free wheeling place and I built the network by doing whatever
made sense at the time.

One of my "favorite" customers (Joe somebody) was somehow related to
the owner of the ISP and was a gamer. This was back in the day when
the gaming magazines would give you useful tips like "Type 'tracert
$gameserver' and make sure that there are less than N hops". Joe
would call up tech support, me, the owner, etc and complain that there
was N+3 hops and most of them were in our network. I spent much time
explaining things about packet-loss, latency, etc but couldn't shake
his belief that hop count was the only metric that mattered.

Finally, one night he called me at home well after midnight (no, I
didn't give him my home phone number, he looked me up in the
phonebook!) to complain that his gaming was suffering because it was
"too many hops to get out of your network". I finally snapped and
built a static GRE tunnel from the RAS box that he connected to all
over the network -- it was a thing of beauty, it went through almost
every device that we owned and took the most convoluted path I could
come up with. "Yay!", I figured, "now I can demonstrate that latency
is more important than hop count" and I went to bed.

The next morning I get a call from him. He is ecstatic and wildly
impressed by how well the network is working for him now and how great
his gaming performance is. "Oh well", I think, "at least he is happy
and will leave me alone now". I don't document the purpose of this GRE
anywhere and after some time forget about it.

A few months later I am doing some routine cleanup work and stumble
across a weird looking tunnel -- its bizarre, it goes all over the
place and is all kinds of crufty -- there are static routes and policy
routing and bizarre things being done on the RADIUS server to make
sure some user always gets a certain IP... I look in my pile of notes
and old configs and then decide to just yank it out.

That night I get an enraged call (at home again) from Joe *screaming*
that the network is all broken again because it is now way too many
hops to get out of the network and that people keep shooting him...

What I learnt from this:
1: Make sure you document everything (and no, the network isn't
documentation)
2: Gamers are weird.
3: Making changes to your network in anger provides short term
pleasure but long term pain.

-----

W

Post by Randy Bush
randy

--
Do not meddle in the affairs of dragons, for you are crunchy and taste
good with ketchup.

Mikael Abrahamsson

2008-06-27 06:50:52 UTC

Permalink

Post by Frank Bulk - iNAME
Interestingly enough, when I trace from my Cisco router it seems to show
some MPLS labels after the hop of interest (12.88.71.13 to 12.122.112.78,
only 24 msec here!). I'm not sure how our Cisco box derives these from a
foreign network.

The ICMP packet (TTL exceeded in transit) contains a copy of the packet
which TTL expired, including the labels, so label information is available
to traceroute.

--
Mikael Abrahamsson email: ***@swm.pp.se

Frank Bulk - iNAME

2008-06-27 00:54:42 UTC

Permalink

Did that satisfy you? I guess with MPLS they could tag the traffic and send
it around the country twice and I wouldn't see it at L3.

Frank

-----Original Message-----
From: John T. Yocum [mailto:***@fluidhosting.com]
Sent: Thursday, June 26, 2008 7:04 PM
To: ***@iname.com
Cc: nanog list
Subject: Re: Possible explanations for a large hop in latency

When I asked ATT about the sudden latency jump I see in traceroutes,
they told me it was due to how their MPLS network is setup.

--John

Post by Frank Bulk
Our upstream provider has a connection to AT&T (12.88.71.13) where I
relatively consistently measure with a RTT of 15 msec, but the next hop
(12.122.112.22) comes in with a RTT of 85 msec. Unless AT&T is sending that
traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that 12.122.112.22
has some kind of ICMP rate-limiting.
Is this a real performance issue, or is there some logical explanation?
Frank

John T. Yocum

2008-06-27 01:09:25 UTC

Permalink

The explanation I got, was that the latency seen at the first hop was
actually a reply from the last hop in the path across their MPLS
network. Hence, all the following hops had very similar latency.

Personally, I thought it was rather strange for them to do that. And,
I've never seen that occur on any other network.

Perhaps someone from ATT would like to chime in.

--John

Post by Frank Bulk - iNAME
Did that satisfy you? I guess with MPLS they could tag the traffic and send
it around the country twice and I wouldn't see it at L3.
Frank
-----Original Message-----
Sent: Thursday, June 26, 2008 7:04 PM
Cc: nanog list
Subject: Re: Possible explanations for a large hop in latency
When I asked ATT about the sudden latency jump I see in traceroutes,
they told me it was due to how their MPLS network is setup.
--John

that

Post by Frank Bulk
traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that 12.122.112.22
has some kind of ICMP rate-limiting.
Is this a real performance issue, or is there some logical explanation?
Frank

Mikael Abrahamsson

2008-06-27 01:16:09 UTC

Permalink

The explanation I got, was that the latency seen at the first hop was
actually a reply from the last hop in the path across their MPLS
network. Hence, all the following hops had very similar latency.
Personally, I thought it was rather strange for them to do that. And,
I've never seen that occur on any other network.

This is standard for MPLS, the ICMP TTL expire message is sent along the
LSP and returned via the router at the end of the LSP.

--
Mikael Abrahamsson email: ***@swm.pp.se

Robert Richardson

2008-06-27 01:19:48 UTC

Permalink

They probably don't propagate TTL w/in their MPLS core. Depending on how
they have MPLS implemented, you may only see 2 hops on the network; the
ingress and egress routers. If the ingress router was in NYC and the egress
in Seattle, you could understandably expect a large jump in RTT.

Not an ATT customer but do know other providers run their MPLS core's this
way...

-Robert

The explanation I got, was that the latency seen at the first hop was
actually a reply from the last hop in the path across their MPLS network.
Hence, all the following hops had very similar latency.
Personally, I thought it was rather strange for them to do that. And, I've
never seen that occur on any other network.
Perhaps someone from ATT would like to chime in.
--John

Post by Frank Bulk - iNAME
Did that satisfy you? I guess with MPLS they could tag the traffic and send
it around the country twice and I wouldn't see it at L3.
Frank
-----Original Message-----
26, 2008 7:04 PM
Cc: nanog list
Subject: Re: Possible explanations for a large hop in latency
When I asked ATT about the sudden latency jump I see in traceroutes,
they told me it was due to how their MPLS network is setup.
--John

that

Post by Frank Bulk
traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that
12.122.112.22
has some kind of ICMP rate-limiting.
Is this a real performance issue, or is there some logical explanation?
Frank

Sam Stickland

2008-07-02 00:07:23 UTC

Permalink

Even if they are decrementing TTL inside of their MPLS core, the TTL
expired message still has to traverse the entire MPLS LSP (tunnel), so
the latency reported for each "hop" is in fact the latency of the last
hop in the MPLS network. Always.

Sam

Post by Frank Bulk - iNAME
They probably don't propagate TTL w/in their MPLS core. Depending on how
they have MPLS implemented, you may only see 2 hops on the network; the
ingress and egress routers. If the ingress router was in NYC and the egress
in Seattle, you could understandably expect a large jump in RTT.
Not an ATT customer but do know other providers run their MPLS core's this
way...
-Robert

Bruce Pinsky

2008-07-02 03:55:00 UTC

Permalink

Sam Stickland wrote:
| Even if they are decrementing TTL inside of their MPLS core, the TTL
| expired message still has to traverse the entire MPLS LSP (tunnel), so
| the latency reported for each "hop" is in fact the latency of the last
| hop in the MPLS network. Always.
|

And who said tunneling protocols aren't fun :-)

- --
=========
bep

Frank Bulk - iNAME

2008-06-27 01:32:30 UTC

Permalink

Thanks for the added information.

Even if their MPLS path went from the midwest (where I'm located) to San
Francisco and then back to St. Louis (where 12.122.112.22 appears to be), I
don't think that accounts for a 70 msec jump in traffic. And I don't think
they would (intentionally) create such an inefficient MPLS path.

Someone off-list told me they tried to trace to 12.88.71.13, but once they
hit an AT&T router their ICMP traffic appeared to be blocked.

Frank

-----Original Message-----
From: John T. Yocum [mailto:***@fluidhosting.com]
Sent: Thursday, June 26, 2008 8:09 PM
To: ***@iname.com
Cc: nanog list
Subject: Re: Possible explanations for a large hop in latency

The explanation I got, was that the latency seen at the first hop was
actually a reply from the last hop in the path across their MPLS
network. Hence, all the following hops had very similar latency.

Personally, I thought it was rather strange for them to do that. And,
I've never seen that occur on any other network.

Perhaps someone from ATT would like to chime in.

--John

that

12.122.112.22

Post by Frank Bulk - iNAME

Post by Frank Bulk
has some kind of ICMP rate-limiting.
Is this a real performance issue, or is there some logical explanation?
Frank

Frank Bulk - iNAME

2008-06-28 01:30:15 UTC

Permalink

Just to close this issue on the list: a (top) engineer from AT&T contacted
me offline and helped us out.

Turns out that 12.88.71.13 is located in Kansas City and
tbr1.sl9mo.ip.att.net (12.122.112.22) is in St. Louis. AT&T has two L1
connections to that site for redundancy, but traffic was flowing over the
longer loop. The engineer tweaked route weights so that the traffic prefers
to flow over the shorter link to tbr2.sl9mo.ip.att.net (12.122.112.78),
shaving about 12 msec.

He also explained that the jump of ~70 msec is due to how ICMP traffic
within MPLS tunnels is handled. It wasn't until I ran a traceroute from a
Cisco router that I even saw the MPLS labels (that included in the ICMP
responses) for each of the hops within the tunnel. Apparently each ICMP
packet within an MPLS tunnel (where TTL decrementing is allowed) is sent to
the *end* of the tunnel and back again, so my next "hop" to
tbr1.sl9mo.ip.att.net (12.122.112.22) was really showing the RTT to the end
of the tunnel, Los Angeles.

Frank

-----Original Message-----
From: Frank Bulk [mailto:***@iname.com]
Sent: Thursday, June 26, 2008 5:52 PM
To: nanog list
Subject: Possible explanations for a large hop in latency

Our upstream provider has a connection to AT&T (12.88.71.13) where I
relatively consistently measure with a RTT of 15 msec, but the next hop
(12.122.112.22) comes in with a RTT of 85 msec. Unless AT&T is sending that
traffic over a cable modem or to Europe and back, I can't see a reason why
there is a consistent ~70 msec jump in RTT. Hops farther along the route
are just a few msec more each hop, so it doesn't appear that 12.122.112.22
has some kind of ICMP rate-limiting.

Is this a real performance issue, or is there some logical explanation?

Frank