nanog mailing list archives
Re: FYI Netflix is down
From: Tony McCrory <tony.mccrory () gmail com>
Date: Mon, 2 Jul 2012 19:53:32 +0100
On 2 July 2012 19:20, Cameron Byrne <cb.list6 () gmail com> wrote:
Make your chaos animal go after sites and regions instead of individual VMs. CB
From a previous post mortem
http://techblog.netflix.com/2011_04_01_archive.html " Create More Failures Currently, Netflix uses a service called "Chaos Monkey<http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-aws.html>" to simulate service failure. Basically, Chaos Monkey is a service that kills other services. We run this service because we want engineering teams to be used to a constant level of failure in the cloud. Services should automatically recover without any manual intervention. We don't however, simulate what happens when an entire AZ goes down and therefore we haven't engineered our systems to automatically deal with those sorts of failures. Internally we are having discussions about doing that and people are already starting to call this service "Chaos Gorilla". *"* It would seem the Gorilla hasn't quite matured. Tony
Current thread:
- Re: FYI Netflix is down Jay Ashworth (Jul 01)
- Re: FYI Netflix is down steve pirk [egrep] (Jul 01)
- <Possible follow-ups>
- RE: FYI Netflix is down Dan Golding (Jul 02)
- Re: FYI Netflix is down Todd Underwood (Jul 02)
- Re: FYI Netflix is down Leo Bicknell (Jul 02)
- Re: FYI Netflix is down david raistrick (Jul 02)
- Re: FYI Netflix is down Leo Bicknell (Jul 02)
- Re: FYI Netflix is down david raistrick (Jul 02)
- Re: FYI Netflix is down Leo Bicknell (Jul 02)
- Re: FYI Netflix is down Cameron Byrne (Jul 02)
- Re: FYI Netflix is down Tony McCrory (Jul 02)
- Re: FYI Netflix is down Paul Graydon (Jul 02)
- Re: FYI Netflix is down James Downs (Jul 02)
- Re: FYI Netflix is down david raistrick (Jul 02)
- Re: FYI Netflix is down James Downs (Jul 02)
- Re: FYI Netflix is down Rodrick Brown (Jul 02)
- Re: FYI Netflix is down James Downs (Jul 02)
- RE: FYI Netflix is down Dan Golding (Jul 03)
- Re: FYI Netflix is down James Downs (Jul 03)
- Re: FYI Netflix is down Rodrick Brown (Jul 03)
- Re: FYI Netflix is down david raistrick (Jul 03)
- Re: FYI Netflix is down Todd Underwood (Jul 02)