nanog mailing list archives
Re: IXP
From: Paul Vixie <vixie () isc org>
Date: Sat, 18 Apr 2009 00:08:04 +0000
Arnold Nipper <arnold () nipper de> writes:
On 18.04.2009 00:04 Paul Vixie wrote... has anybody ever run out of 1Q tags in an IXP context?Why? You only need 1 ;-)
really? 1? at PAIX we started with three, two unicast (wrongheadedness) and one multicast, then added another unicast for V6. then came the VNI's, so i'm betting there are hundreds or thousands at most PAIX nodes today. are others just using one big shared network for everything? i should expand on something i said earlier on this thread. the progression i saw at PAIX and later saw from inside MFN was that most new peerings would happen on a shared port and then as that port filled up some peerings would move to PNI. given that success in these terms looks like a PNI, i'm loathe to build in any dependencies on the long term residency of a given peering on a shared multiaccess subnet. i should answer something said earlier: yes there's only 14 bits of tag and yes 2**14 is 4096. in the sparsest and most wasteful allocation scheme, tags would be assigned 7:7 so there'd be a max of 64 peers. it's more likely that tags would be assigned by increment, but it's still nowhere near enough for 300+ peers. however, well before 300 peers, there'd be enough staff and enough money to use something other than a switch in the middle, so that the "tagspace" would be per-port rather than global to the IXP. Q in Q is not how i'd build this... cisco and juniper both have hardware tunnelling capabilities that support this stuff... it just means as the IXP fabric grows it has to become router-based. i've spent more than several late nights and long weekends dealing with the problems of shared multiaccess IXP networks. broadcast storms, poisoned ARP, pointing default, unintended third party BGP, unintended spanning tree, semitranslucent loops, unauthorized IXP LAN extension... all to watch the largest flows move off to PNI as soon as somebody's port was getting full. conventional wisdom says a shared fabric is fine. conventional wisdom also said that UNIX came only from bell labs, that computers and operating systems were bought from the same vendor on a single PO, that protocols built for T1 customers who paid $1000 MRC would scale to DSL customers who paid $30 MRC, that Well and Portal shell users should be allowed to use outbound SMTP, that the internet would only be used cooperatively, and that business applications were written in COBOL whereas scientific applications were written in FORTRAN, and that the cool people all used BSD whereas Linux was just a toy. so i think conventional wisdom isn't perfectly ageless. -- Paul Vixie
Current thread:
- Re: IXP - PNI, (continued)
- Re: IXP - PNI Antonio Querubin (Apr 17)
- Re: IXP - PNI Joe Greco (Apr 17)
- Re: IXP - PNI bmanning (Apr 17)
- Re: IXP - PNI Paul Vixie (Apr 17)
- Re: IXP Nuno Vieira - nfsi telecom (Apr 18)
- Re: IXP Mikael Abrahamsson (Apr 17)
- Re: IXP Paul Vixie (Apr 17)
- Re: IXP Arnold Nipper (Apr 17)
- Re: IXP Paul Vixie (Apr 17)
- Re: IXP Arnold Nipper (Apr 17)
- Re: IXP Paul Vixie (Apr 17)
- Re: IXP Nathan Ward (Apr 17)
- Re: IXP Paul Vixie (Apr 18)
- Re: IXP Paul Vixie (Apr 17)
- Re: IXP Nick Hilliard (Apr 18)
- Re: IXP Paul Vixie (Apr 18)
- Re: IXP Jeff Young (Apr 18)
- Re: IXP vijay gill (Apr 19)
- Re: IXP Alan Hannan (Apr 19)
- RE: IXP Deepak Jain (Apr 20)
- RE: IXP Michael K. Smith - Adhost (Apr 20)