nanog mailing list archives
Re: scaling linux-based router hardware recommendations
From: Sudeep Khuraijam <skhuraijam () liveops com>
Date: Mon, 26 Jan 2015 17:05:29 -0800
It really depends on the application that you are interested in beyond forwarding, but not knowing that and to scale forwarding ³at a reasonable price", things have to come off cpu and become more customized for forwarding, especially for low latency forwarding. The optimization comes in minimizing packet tuple copies, off load to co-processors and network coprocessors (some of which can be in NICs) and parallel processing with some semblance of shared memory across, all of which takes customization beyond CPU and Kernel which in itself needs to be stripped down bare and embedded. Ultimately that¹s what appliance vendors do with different levels of hardware/firmware customization depending on ROI of features, speeds and price. A generic OpenSource compatible OEM product with multi-gig ports will generally be at least half to 5th the price of a high end latest server architecture server product with ability to support 10 gig interfaces in the same forwarding performance range (which are in the market for a different scale problem in compute and net I/O but exist at a price point that make them exorbitant to solve forwarding speed). Cheers, Sudeep Khuraijam On 1/26/15, 2:53 PM, "micah anderson" <micah () riseup net> wrote:
Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Current thread:
- Re: scaling linux-based router hardware recommendations, (continued)
- Re: scaling linux-based router hardware recommendations Ken Chase (Jan 26)
- Re: scaling linux-based router hardware recommendations Mike Hammett (Jan 26)
- Re: scaling linux-based router hardware recommendations Adair Winter (Jan 26)
- Re: scaling linux-based router hardware recommendations Alexander Neilson (Jan 26)
- Re: scaling linux-based router hardware recommendations Ken Chase (Jan 26)
- Re: scaling linux-based router hardware recommendations Faisal Imtiaz (Jan 26)
- Re: scaling linux-based router hardware recommendations joel jaeggli (Jan 26)
- Re: scaling linux-based router hardware recommendations Joe Holden (Jan 27)
- Re: scaling linux-based router hardware recommendations David bass (Jan 26)
- Re: scaling linux-based router hardware recommendations Paul S. (Jan 27)
- Re: scaling linux-based router hardware recommendations Pavel Odintsov (Jan 27)
- Re: scaling linux-based router hardware recommendations Baldur Norddahl (Jan 27)
- Re: scaling linux-based router hardware recommendations Phil Bedard (Jan 27)
- Re: scaling linux-based router hardware recommendations Eduardo Schoedler (Jan 27)
- Re: scaling linux-based router hardware recommendations Robert Bays (Jan 28)
- Re: scaling linux-based router hardware recommendations Paul S. (Jan 28)