nanog mailing list archives

Re: FYI Netflix is down


From: "steve pirk [egrep]" <steve () pirk com>
Date: Sun, 1 Jul 2012 19:38:16 -0700

On Sun, Jul 1, 2012 at 11:38 AM, Jay Ashworth <jra () baylink com> wrote:

Not entirely.  Datacenters do go down, our best efforts to the contrary
notwithstanding.  Amazon doesn't guarantee you redundancy on EC2, only
the tools to provide it yourself.  25% Amazon; 75% service provider
clients;
that's my appraisal of the blame.


From a Wired article:

That’s what was supposed to happen at Netflix Friday night. But it didn’t
work out that way. According to Twitter messages from Netflix Director of
Cloud Architecture Adrian Cockcroft and Instagram Engineer Rick Branson, it
looks like an Amazon Elastic Load Balancing service, designed to spread
Netflix’s processing loads across data centers, failed during the outage.
Without that ELB service working properly, the Netflix and Pintrest
services hosted by Amazon crashed.

http://www.wired.com/wiredenterprise/2012/06/real-clouds-crush-amazon/

The GSLB fail-over that was supposed to take place for the affected
services (that had configured their applications to fail-over) failed.

I heard about this the day after Google announced the Compute Engine
addition to the App Engine product lines they have. The demo was awesome.
I imagine Google has GSLB down pat by now, so some companies might start
looking... ;-]

--steve


Current thread: