Amazon EC2 outage downs Reddit, Quora

[webair]

NEW YORK (CNNMoney) -- A rare and major outage of Amazon's cloud-based Web service on Thursday took down a plethora of other online sites, including Reddit, HootSuite, Foursquare and Quora.

The outages began Thursday morning just before 5 a.m. ET and were still ongoing more than 12 hours later. In a series of running updates on its Web services "Health Dashboard," Amazon described its efforts to recover from the cascade of problems kicked off by an early-morning "networking event."


Amazon (AMZN, Fortune 500) hosts many major websites on its web servers through a service called EC2. The "elastic" infrastructure model is designed to scale up automatically to handle giant traffic spikes -- the kind Amazon gets every year during its December e-commerce rush.

Thursday's crash happened at Amazon's northern Virginia data center, one of five global sites that underpin EC2. In its status log, Amazon said that the networking glitch caused many of its storage volumes to create new backups of themselves. That filled up Amazon's available storage capacity and kicked off a series of connectivity problems.

"A number of people have asked us for an ETA on when we'll be fully recovered," Amazon wrote early Thursday afternoon, nearly 10 hours after the outages began. "Our high-level ballpark right now is that the ETA is a few hours. We can assure you that all-hands are on deck to recover as quickly as possible."

Thousands of customers hitch a ride on Amazon's cloud, renting space on its servers. Some of them went down Thursday in the wake of Amazon's outage.

Who was crashed: Q&A site Quora posted on its homepage: "We'll be back shortly, we hope. Sorry - it sucks for us too. We'd point fingers, but we wouldn't be where we are today without EC2."

News aggregator Reddit had resumed some functionality by midday, but it still noted on its homepage that "Amazon is experiencing a degradation...We are still waiting on them to get to our volumes."

Foursquare suffered some glitches earlier in the day, but its site seemed to be functioning normally by early afternoon.

http://money.cnn.com/2011/04/21/tech...dex.htm?hpt=T2

[Theo]

http://hootsuite.com/ still down

[pamon]

lots of companies using amazon and EC2 storage for backend stuff... will be interesting to see how this affects Amazon's cloud options

[react]

Bad karma Webair.

[Babaganoosh]

Originally Posted by react

Bad karma Webair.

They're trying to deflect. It's been a rough day for weberror here on GFY.

[sandman!]

the bigger it is the harder it will fall when it fails this aint the first major cloud failure and it wont be the last.

[BareBacked]

Amazon should have hosted at webair

[Supz]

If you are a big website like that, or running a big company with many servers. This is nobodies fault but there own. Fell into the cloud pitch because they were to cheap to pay a CIO and a tech staff.. It is much more cost effective and beneficial to run your own private cloud, AKA virtualization infrastructure.

[Supz]

Originally Posted by sandman!

the bigger it is the harder it will fall when it fails this aint the first major cloud failure and it wont be the last.

I am suprisced these companies build some amateur level shit. I mean, the true way, with a good setup would be if there was an electric outage and the generators were out of gas, otherwise this is just a design flaw of some sorts. o

This is not a small hardware failure if this many sites are down. They do not share resources between these companies. I would think they would give them there own physical servers and storage. They have to be charging them an arm and a leg.

[webair]

Originally Posted by react

Bad karma Webair.

Hi Aaron =)

[madawgz]

im dying to see new rage comics!

[webair]

To be honest, my intention with making this post was not to 'deflect' or create 'bad karma', I think it's prudent to look at all aspects of cloud hosting, and recognize the differences as to what each provider is offering, & recognize / identify the differences.

The Amazon outage is also likely to prompt discussion of the reliability of cloud computing. Is it a fair question to raise? This outage has affected many customers, highlighting the vulnerability of a single service hosting many popular sites.

[B.Barnato]

i think also gumgum have been hit by this

[webair]

Originally Posted by B.Barnato

i think also gumgum have been hit by this

Quite a few large companies that use them for their back end have been effected. We have a project in beta that uses it, it's still down. More than 48 hrs.

[Herb Kornfield]

Machines fail, systems fail, power fails, software/hardware fail and also the usual suspect - Human error. ( under the guise " software upgrade " )

There is no fool proof way to ensure nothing will ever fail no matter how much redundancy you have in place. Failure is not an option, its a reality.

How you brace against this failure and move to restore is the key. Just like insurance on your home or car ( or whatever else ) its only utterly important when you need it.

[Due]

Originally Posted by webair

Quite a few large companies that use them for their back end have been effected. We have a project in beta that uses it, it's still down. More than 48 hrs.

That's just ridiculous, I'd be out the door if my host couldn't get get us up one way or the other within 4-5 hours of outage if it's a severe outage effecting the entire or big parts of the data center. Less if it's only effecting us

[undersoul]

Originally Posted by Due

That's just ridiculous, I'd be out the door if my host couldn't get get us up one way or the other within 4-5 hours of outage if it's a severe outage effecting the entire or big parts of the data center. Less if it's only effecting us

take a look at the news regarding this. it's pretty big outage effecting a ton of sites and companies. you'd be surprised how wide spread. hootsuite for example just came back up today.

[Due]

Originally Posted by undersoul

take a look at the news regarding this. it's pretty big outage effecting a ton of sites and companies. you'd be surprised how wide spread. hootsuite for example just came back up today.

At the end of the day it doesn't help my bank account and it doesn't help me that other people are suffering. It looks like they where selling a solution they couldn't handle 100% so I'd be out of there!

I'm also very picky about my hosts, I moved away from rackspace a couple of years ago after they told me they had no staff on duty to perform certain tasks after-hours and it would take them about 2 hours to just look at it.