nanog mailing list archives

Re: 100G - Whitebox


From: Fredrik Korsbäck <hugge () nordu net>
Date: Sun, 20 Aug 2017 19:24:41 +0200

The only viable merchant silicon chip that would be useful for a IXP is from the StrataDNX-family which house the 
jericho/qumran/petra/arad chips from broadcom. No packetbuffer in the exhangepoint will shred performance 
significantly, especially when one of your bursty 100G customers starts sending data into 1/10G customers. 

To the best of my knowledge the only one that offers DNX in whitebox-fashion is Agema and Edgecore. But why whitebox? 
Except on a very few occasions whitebox is just "i like paying hardware and software on different invoices = whitebox" 
the TCO is just the same but. As an exchangepoint i also see that it can hard to reap the benefits of all the 
hipstershit going on in these NOS-startups, you want spanning-tree, port-security, something to loadbalance over links 
and perhaps a overlaying-technology if the IXP becomes to big and distributed, like vxlan. This is to easy almost. 

Whenever i see unbuffered mix-speed IXPs i ask if i can pay 25% of the portcost since that is actually how much oumpfff 
i would get through the port.

// hugge @ 2603

20 aug. 2017 kl. 18:40 skrev Bill Woodcock <woody () pch net>:

Why don't we just swap out your 40g switch for a 100g switch?  You've had the 40g one for a while, and we anticipate 
upgrades every 18-24 months. 

               -Bill


On Aug 20, 2017, at 08:46, Mike Hammett <nanog () ics-il net> wrote:

I first sent to an IX-specific mailing list, but as I have yet to see the message hit the list, I figured I would 
post it here as well. 

We've had multiple requests for 100G interfaces (instead of Nx10G) and no one seems to care about the 40G interfaces 
we have available. 

Looking at cost effective options brings us to whitebox switches. Obviously there's a wide range of hardware vendors 
and there are a few OSes available as well. Cumulus seems to be the market leader, while IPinFusion seems to be the 
most feature-rich. 

We're not doing any automation on the switches at this time, so it would still need decent manual configuration. It 
wouldn't need a Cisco-centric CLI as we're quite comfortable managing standard Linux-type config files. We're not 
going all-in on some overlay either given that we wouldn't be replacing our entire infrastructure, only 
supplementing it where we need 100G. I know that LINX has gone IPinfusion. What OS would be appropriate for our 
usage? I'm not finding many good comparisons of the OSes out there. I'm assuming any of them would work, but there 
may be gotchas that a "cheapest that meets requirements" doesn't quite unveil. 

Any particular hardware platforms to go towards or avoid? Broadcom Tomahawk seems to be quite popular with varying 
control planes. LINX went Edgecore, which was on my list given my experience with other Accton brands. Fiberstore 
has a switch where they actually publish the pricing vs. a bunch of mystery. 

Thoughts? 




----- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 




Current thread: