Re: few big monolithic PEs vs many small PEs

[James Bensley <jwbensley+nanog () gmail com>]

On 27 June 2019 16:26:03 BST, Mark Tinka <mark.tinka () seacom mu> wrote:
> On 27/Jun/19 10:58, James Bensley wrote:
>
> > Hi Adam,
> >
> > Over the years I have been bitten multiple times by having fewer big
> > routers with either far too many services/customers connected to them
> > or too much traffic going through them. These days I always go for
> > more smaller/more routers than fewer/larger routers.
> >
> > One experience I have made is that when there is an outage on a large
> > PE, even when it still has spare capacity, is that the business
> impact
> > can be too much to handle (the support desk is overwhelmed, customers
> > become irate if you can't quickly tell them what all the impacted
> > services are, when service will be restored, the NMS has so many
> > alarms it’s not clear what the problem is or where it's coming from
> > etc.).
> >
> > I’ve seen networks place change freeze on devices, with the exception
> > of changes that migrate customers or services off of the PE, because
> > any outage would create too great an impact to the business, or risk
> > the customers terminating their contract. I’ve also seen changes
> > freeze be placed upon large PEs because the complexity was too great,
> > trying to work out the impact of a change on one of the original PEs
> > from when the network was first built, which is somehow linked to
> > virtually every service on the network in some obscure and
> > unforeseeable way.
>
> I would tend to agree when the edge routers are massive, e.g., boxes
> like the Cisco ASR9922 or the Juniper MX2020 are simply too large, and
> present a real risk re: that level of customer aggregation (even for
> low-revenue services such as Broadband). I don't think I'd ever justify
> buying these towers to aggregate customers, mainly due to the risk.
>
> For us, even the MX960 is too big, which is why we focus on the MX480
> (ASR9906 being the equivalent). It's a happy medium between the small
> and large end of the spectrum.
>
> And as I mentioned before, we just look at a totally different box for
> 100Gbps customers.
>
> Mark.

Yeah, if you want to name specific boxes then yes I've made similar experiences with the same boxen. Even the MX960 is 
slightly too big for a PE depending on how you load it (port combinations).

Large boxes like the MX2020, ASR9922, NCS6K, etc. these can only reasonably be used as P nodes in my opinion.

Cheers,
James.