Re: Sufficient Buffer Sizes

[Mike Hammett <nanog () ics-il net>]

I'm assuming that modern queuing mechanisms aren't going to be viable in switches until which time they're baked into 
the silicon. 




----- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

----- Original Message -----

From: "Dave Taht" <dave.taht () gmail com> 
To: "William Herrin" <bill () herrin us> 
Cc: "Mike Hammett" <nanog () ics-il net>, "NANOG" <nanog () nanog org> 
Sent: Tuesday, January 2, 2024 6:02:27 PM 
Subject: Re: Sufficient Buffer Sizes 

Hoo, boy. This is now such an old debate that I do not know where to 
start anymore. 

I am of the firm opinion nowadays that if you are buffering more than 
a few ms at these enormous speeds, you are doing it wrong, and 
regardless https://arxiv.org/abs/2109.11693 seems to hold as for 
highly multiplexed traffic. 

My outside number for a FIFO buffer in the modern CDN´d world is a 
mere 30ms at lower speeds which allows for good gaming and 
videoconferencing experiences, and good performance with modern paced 
transports (in general linux now does packet pacing across all 
congestion controls) and with a good head drop AQM and FQ algo, far, 
far less buffering is feasible across the board. 

https://blog.cerowrt.org/post/juniper/ 

But regrettably not available at 100Gbit yet (tho libreqos is coming close) 

On Tue, Jan 2, 2024 at 6:22 PM William Herrin <bill () herrin us> wrote: 
> On Tue, Jan 2, 2024 at 3:02 PM Mike Hammett <nanog () ics-il net> wrote: 
> > While attempting to ascertain how big of switch buffers I needed in a 100G switch, I rediscovered this article 
> > where I first learned about switch buffers. 
> >
> > https://fasterdata.es.net/network-tuning/router-switch-buffer-size-issues/#:~:text=Optimum%20Buffer%20Size&text=The%20general%20rule%20of%20thumb,1G%20host%20across%20the%20WAN.
> >  
> >
> > It suggests that 60 meg is what you need at 10G. Is that per interface? Would it be linear in that I would need 600 
> > meg at 100G? 
> >
> > Some 100G switches I was looking at only had 36 megs, so that's insufficient either way you look at it. 
>
> Hi Mike, 
>
> My thoughts: 
>
> 1. 50 ms is -way- too much buffer. A couple links like that in the 
> path and the user will suffer jitter in excess of 100ms which is 
> incredibly disruptive for interactive applications. 
>
> 2. The article discussed how much buffer to apply to the -slower- 
> interfaces, not the faster ones, the idea being that data entering 
> from the faster interfaces could otherwise overwhelm the slower ones 
> resulting in needless retransmission and head-end blocking. Are the 
> 100G interfaces on your switch the -slower- ones? 
>
>
> I don't know the best number, but I suspect the speed at which packets 
> clear an interface is probably a factor in the equation, so that the 
> reasonable buffer depth in ms when a packet clears in 1ms is probably 
> different than the reasonable buffer depth when a packet clears in 1 
> us. 
>
> Regards, 
> Bill Herrin 
>
>
>
> -- 
> William Herrin 
> bill () herrin us 
> https://bill.herrin.us/ 



-- 
40 years of net history, a couple songs: 
https://www.youtube.com/watch?v=D9RGX6QFm5E 
Dave Täht CSO, LibreQos