Re: Thousands of hosts on a gigabit LAN, maybe not

[charles () thefnf org]

On 2015-05-08 13:53, John Levine wrote:
> Some people I know (yes really) are building a system that will have
> several thousand little computers in some racks.


How many racks?
How many computers per rack unit? How many computers per rack?
(How are you handling power?)
How big is each computer?

Do you want network cabling to be contained to each rack? Or do you want 
to run the cable to a central networking/switching rack?

Hmmmm even a 6513 fully populated with POE 48 port line cards (which 
could let you do power and network in the same cable (I think? Does POE 
work on gigabit these days)? would get you (12*48 = 576) ports.

So.... 48U rack - 15U (I think the 6513 is 15U total) leaves you 33U. 
Can you fit 576 systems in 33U?


  Each of the
> computers runs Linux and has a gigabit ethernet interface.



Copper?

  It occurs
> to me that it is unlikely that I can buy an ethernet switch with
> thousands of ports

6515?


, and even if I could, would I want a Linux system
> to have 10,000 entries or more in its ARP table.

Add more ram. That's always the answer. LOL.


> Most of the traffic will be from one node to another, with
> considerably less to the outside.  Physical distance shouldn't be a
> problem since everything's in the same room, maybe the same rack.
>
> What's the rule of thumb for number of hosts per switch, cascaded
> switches vs. routers, and whatever else one needs to design a dense
> network like this?  TIA


We need more data.