nanog mailing list archives
is something weird going on with cox, level3, and/or cogent
From: Miles Fidelman <mfidelman () meetinghouse net>
Date: Sun, 12 Feb 2017 20:12:13 -0700
Hi Folks,I'm visiting AZ, and seeing some really really poor performance accessing some of our servers via Cox broadband. The folks at Cox technical support are useless - all they say is "well you're on a DOCSIS 2 modem." Meanwhile, everything I'm seeing is several hops upstream of the local segment - and all that Cox level2 tech support will say is "if there was a backbone problem, our backbone people would have dealt with it."
I'm having problems reaching both our own server, and sites like google, facebook, windows update.
Traceroutes to and from our server are illustrative - note that for most of the past week, the average ping time was 85msec. Now we're seeing this:
From 98.177.135.186 - the public IP address on Cox's local broadband service. To 107.154.13.58 (ntcorp.server - one of our servers, sitting in a Tierpoint data center, near Boston)
traceroute to ntcorp.com (207.154.13.58), 64 hops max, 52 byte packets 1 10.128.128.1 (10.128.128.1) 199.893 ms 75.319 ms 27.295 ms 2 100.127.69.178 (100.127.69.178) 38.710 ms 40.075 ms 43.598 ms 3 72.215.229.22 (72.215.229.22) 39.674 ms 201.368 ms *4 lag-157.bear2.phoenix1.level3.net (4.28.82.53) 686.499 ms 1837.141 ms 16.273 ms 5 ae-1-60.edge1.losangeles6.level3.net (4.69.144.16) 35.498 ms 964.377 ms * 6 ae-3-80.edge1.losangeles6.level3.net (4.69.144.144) 551.760 ms 525.014 ms
ae-1-60.edge1.losangeles6.level3.net (4.69.144.16) 2061.191 ms7 be3036.ccr41.lax04.atlas.cogentco.com (154.54.14.129) 847.778 ms 87.601 ms 71.504 ms 8 be2965.ccr22.lax01.atlas.cogentco.com (154.54.45.1) 79.060 ms 225.647 ms
be2964.ccr21.lax01.atlas.cogentco.com (154.54.44.77) 60.306 ms9 * be2931.ccr21.phx02.atlas.cogentco.com (154.54.44.85) 2264.071 ms 185.180 ms
10 be2929.ccr21.elp01.atlas.cogentco.com (154.54.42.66) 61.208 msbe2930.ccr21.elp01.atlas.cogentco.com (154.54.42.78) 386.149 ms 1278.868 ms
11 be2928.ccr22.iah01.atlas.cogentco.com (154.54.30.161) 384.136 msbe2927.ccr41.iah01.atlas.cogentco.com (154.54.29.221) 2339.833 ms 615.415 ms
12 be2690.ccr42.atl01.atlas.cogentco.com (154.54.28.129) 233.061 ms be2687.ccr41.atl01.atlas.cogentco.com (154.54.28.69) 87.902 ms be2690.ccr42.atl01.atlas.cogentco.com (154.54.28.129) 861.159 ms 13 be2113.ccr42.dca01.atlas.cogentco.com (154.54.24.221) 998.858 ms be2112.ccr41.dca01.atlas.cogentco.com (154.54.7.157) 249.930 ms * 14 be2807.ccr42.jfk02.atlas.cogentco.com (154.54.40.109) 768.461 ms be2806.ccr41.jfk02.atlas.cogentco.com (154.54.40.105) 136.772 ms be2807.ccr42.jfk02.atlas.cogentco.com (154.54.40.109) 288.225 ms15 be2094.ccr21.bos01.atlas.cogentco.com (154.54.30.14) 271.736 ms 166.224 ms
be2096.ccr22.bos01.atlas.cogentco.com (154.54.30.42) 565.015 ms 16 te0-4-1-7.agr22.bos01.atlas.cogentco.com (154.54.80.34) 1944.479 ms te0-4-1-6.agr22.bos01.atlas.cogentco.com (154.54.80.10) 149.803 ms te0-4-1-6.agr21.bos01.atlas.cogentco.com (154.54.47.230) 897.115 ms17 te0-0-2-0.nr11.b000254-0.bos01.atlas.cogentco.com (154.24.9.82) 107.207 ms te0-0-2-1.nr11.b000254-0.bos01.atlas.cogentco.com (154.24.9.78) 295.881 ms 185.453 ms
18 38.122.127.18 (38.122.127.18) 115.652 ms 461.168 ms 615.526 ms 19 207.154.0.57 (207.154.0.57) 1871.023 ms 1987.832 ms 2165.248 ms 20 server1.ntcorp.com (207.154.13.58) 587.560 ms 263.328 ms 333.542 ms Traceroute in the reverse direction: 1 207.154.13.47 (207.154.13.47) 0.000 ms 0.000 ms 0.000 ms 2 * * *3 h130.207.190.173.static.ip.windstream.net (173.190.207.130) 0.000 ms 0.000 ms 0.000 ms 4 xe1-2-0-0.cr01.cley01-oh.us.windstream.net (40.128.250.166) 12.000 ms 12.000 ms 12.000 ms 5 et11-0-0-0.cr01.chcg01-il.us.windstream.net (40.128.248.71) 20.000 ms 20.000 ms 20.000 ms 6 10gigabitethernet4-1.core1.chi1.he.NET (206.223.119.37) 72.001 ms 20.000 ms 16.000 ms 7 chgobbrj01pos010100.r2.ch.cox.net (68.105.30.193) 20.000 ms 20.000 ms 20.000 ms 8 chnddsrj01-ae1.0.rd.ph.cox.net (68.1.5.211) 72.001 ms 72.001 ms 72.001 ms
9 * * * 10 * * * 11 * * * <snip> 30 * * * <looks like a lot of the later hops don't respond to pings> Two things jump out at me:1. The rather large number of hops from cox to ntcorp - with high delays from several nodes in both the level3 and cogent networks. 2. That there's a rather more direct path from the datacenter to cox, that shows up in the reverse direction.
Some kind of routing or peering issue, perhaps? (And I also note the earlier string of messages regarding youtube streaming problems - that also seemed to involve cox and level3.
Thanks for any insight (and better, for any fixes!). Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
Current thread:
- is something weird going on with cox, level3, and/or cogent Miles Fidelman (Feb 12)
- Re: is something weird going on with cox, level3, and/or cogent Mel Beckman (Feb 12)
- Re: is something weird going on with cox, level3, and/or cogent Mel Beckman (Feb 12)
- Re: is something weird going on with cox, level3, and/or cogent Miles Fidelman (Feb 12)
- Re: is something weird going on with cox, level3, and/or cogent Mel Beckman (Feb 12)
- Re: is something weird going on with cox, level3, and/or cogent Mel Beckman (Feb 12)
- Re: is something weird going on with cox, level3, and/or cogent Mel Beckman (Feb 12)