nanog mailing list archives

Re: 10g residential CPE


From: Baldur Norddahl <baldur.norddahl () gmail com>
Date: Sat, 26 Dec 2020 18:19:42 +0100

On Sat, Dec 26, 2020 at 5:41 PM Mikael Abrahamsson <swmike () swm pp se> wrote:

On Sat, 26 Dec 2020, Baldur Norddahl wrote:

That is why. The RTT to the source can not be larger than the minimum
buffer size in the transport path. Otherwise the speed will start
decreasing.

This is no longer correct. There has been lots of TCP innovation since
this was true.

Please stop repeating it.


It is true there have been TCP improvements but you can very easily verify
for yourself that it is very hard to get anywhere near 1 Gbps of actual
transfer speed to destinations just 10 ms away. Try the nlnog ring network
like this:

gigabit@gigabit01:~$ iperf -c netnod01.ring.nlnog.net
------------------------------------------------------------
Client connecting to netnod01.ring.nlnog.net, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 185.24.168.23 port 50632 connected with 185.42.136.5 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   452 MBytes   379 Mbits/sec

And that is a direct peer of ours.

In general you will have trouble with any server that has a NIC > 1G. If
you find a server that has a 1G NIC this happens instead:

gigabit@gigabit01:~$ iperf -c bahnhof01.ring.nlnog.net
------------------------------------------------------------
Client connecting to bahnhof01.ring.nlnog.net, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 185.24.168.23 port 56412 connected with 195.178.185.171 port
5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.08 GBytes   930 Mbits/sec

Why? Because the 1G NIC server naturally will pace the traffic at maximum
1G and therefore not fill any buffers in the transfer path. The 10G servers
on the other hand WILL fill the buffers and experience packet loss.

Regards,

Baldur

Current thread: