Thursday, April 21, 2011

Well *that* was painful....

FRED is back from today's EC2 outage. Amazon has three of the four availability zones in their us-east-1 region (virginia) datacenter operating. Unfortunately, FRED's database server was running in the one zone that is still sick. However, I was able to snapshot the database's EBS volume and create a new volume from that snapshot in one of the other zones, then fire up a new DB server instance in that zone, attach the new volume to that instance and get things rolling again.

While riding the bus home from work.

Yay for wifi on the bus. Boo for Amazon having a full day outage.

I was getting pretty proud of FRED's 99.99% 30 day uptime. Now it's all shot to hell: 97.58%!

Oh well. At least it wasn't on friday or saturday when everyone would be trying to download their preregs.

-p

EC2 outage

Hello faithful FRED users-  Today Amazon EC2's us-east-1 region is experiencing a serious, sustained outage in EBS connectivity and creation. FRED lives in us-east-1a, so his database server (whose files live in an EBS volume) became inaccessible at about 1am PDT.

All I can do is wait for EC2 to fix the issue. Very curious or geeky folks can follow their progress here: http://status.aws.amazon.com

Please accept my apologies for the FRED outage.

-Peet