Another try at handling the accented characters in fencers' names

FRED is getting used more and more in Canada these days, which is very cool. However, it's brought a long-standing problem with FRED closer to the surface: Multibyte characters. 


FRED is written in PHP, a great language for quickly building complex web applications. However, PHP's support for multibyte encodings is less than awesome. Also, lots of the code in FRED was written in the early days of PHP4 when mb support was even worse. 


Lots of our Canadian friends and their clubs have names with accented characters represented in the UTF-8 multibyte character encoding. Their names are accepted into FRED just fine, but when they are transmitted back and forth with Fencing Time, the XML-related PHP functions FRED uses to read and write preregistration and results handle the mb characters badly. There's a bunch of info out there on the web as to how to best handle this problem, and I've tried lots of them, with mixed results. 


Today I deployed another attempted solution to write UTF-8 XML preregistration files for import into Fencing Time using the mb_convert_encoding() function to ensure that the stream output is valid UTF-8.


Given how many systems this data passes through (FRED's webserver, db server, your browser, your OS, Fencing Time, and back again...), it's hard to be sure everything works 100%, but so far this change has performed well in my tests. Hopefully the real world will behave similarly.

-P

And here I was getting all happy about the uptime...

Only to have FRED go down hard for 6 hours this morning!  What happened: Every night logrotate rotates FRED's web server logs (among others) and restarts the web server process. Last night the webserver didn't come back from the restart for some reason. All it took was a simple (re)start to get FRED back up (not of the whole server, just the webserver process). It's hard to tell why it died, and I'm looking at its logs to see if I can tell, but ultimately the more important question is "how do we prevent this in the future?" Answer: I have installed SIM (http://www.rfxn.com/projects/system-integrity-monitor/), a cron initiated script that periodically checks to make sure certain services are running and healthy and (re)starts them automatically if they are not.

So hopefully this won't happen again.

Query of Death Part Deux

Well, I've made some code changes in an attempt to prevent the offending "expensive" query from being executed. To be honest, only time will tell if I've actually eliminated the exact code path that has been running that query, and even if I have, we'll have to see if that stops these little 5 minute outages from happening.

Query of death

FRED continues to have short periods of downtime ranging from 3-6 minutes a few times per week. These appear to be caused by a request that runs a very expensive query that takes a couple minutes to run and consumes most of the database's resources during that time. This slows or blocks other users' queries, causing page requests to stack up and take all of the available memory on the web server, subsequently causing all pages to fail to load.

I'm looking at how to prevent that query from occurring. It may actually be triggered by a search engine crawler (hence FRED not being built for its behavior, and failing to handle it).

Wow, FRED sends a lot of mail

With FRED's current SMTP (outgoing email) service, it'll cost about $550 per year to send email. Yikes! EC2 did just launch an outgoing email service that would be WAY more cost-effective (maybe even free for FRED since he might make it under their free tier), but it's not an SMTP service. Instead, it's a webservice API, which makes it harder for FRED to use it. I have hope that soon they'll add SMTP as an alternative method of using the service.

So far so good

Well, it's been a week and change since the last problem related to the server move. Is it too early to declare the move a success? At this point the only thing that is still an open issue is the volume of email FRED sends. The new outgoing mail service I'm using charges a certain amount of money for a certain number of emails per month. At first I just guessed how much quota FRED would need, and as it turned out i guessed low. In just three days FRED had burned through the quota I bought for him, and I had to buy more. At this rate, FRED will be spending around $400 a year just to send email. Whew!

Ok, maybe the backup wasn't the problem...

Now it looks like FRED's PCI compliance scanner took him offline.

Background: Like all web based businesses that accept credit cards online, FRED is required by the credit card companies to have a security scan done periodically by an approved independent vendor. In FRED's case, that's done weekly. As it happens, they do it at noon PST monday, exactly the time FRED went offline today. There was also a coinciding huge spike in network traffic. I'd had problems with their scan being a little "enthusiastic" for FRED before, but the measures I put in place to deal with it were on the old server. So now that FRED moved, the problem came back.

Grrrrr......