Wednesday, February 16, 2011

Query of Death Part Deux

Well, I've made some code changes in an attempt to prevent the offending "expensive" query from being executed. To be honest, only time will tell if I've actually eliminated the exact code path that has been running that query, and even if I have, we'll have to see if that stops these little 5 minute outages from happening.

Wednesday, February 9, 2011

Query of death

FRED continues to have short periods of downtime ranging from 3-6 minutes a few times per week. These appear to be caused by a request that runs a very expensive query that takes a couple minutes to run and consumes most of the database's resources during that time. This slows or blocks other users' queries, causing page requests to stack up and take all of the available memory on the web server, subsequently causing all pages to fail to load.

I'm looking at how to prevent that query from occurring. It may actually be triggered by a search engine crawler (hence FRED not being built for its behavior, and failing to handle it).

Sunday, February 6, 2011

Wow, FRED sends a lot of mail

With FRED's current SMTP (outgoing email) service, it'll cost about $550 per year to send email. Yikes! EC2 did just launch an outgoing email service that would be WAY more cost-effective (maybe even free for FRED since he might make it under their free tier), but it's not an SMTP service. Instead, it's a webservice API, which makes it harder for FRED to use it. I have hope that soon they'll add SMTP as an alternative method of using the service.