Long time readers will know how much this blog has struggled with its various incarnations over the past 4 years. Initially I ran it from home on a server that I was using for development purposes so it ran inside a virtual machine that contained not one, but two database engines (MS-SQL for development and MySQL for the blog) all behind the tenuous 1.5Mbit upstream connection. This held up ok until I wanted to do anything fancy like put pictures on there (which would kill the connection for anything over 50kb) and it was relatively unstable, going down for days at a time since I couldn’t get a reliable remote connection to it. Since then I’ve churned my way through different virtual private servers (and all the issues they have) before landing on my current Burst.NET Ubuntu box which has been the best of the bunch so far.
Well, on the surface at least.
Since my blog as attained a steady amount of traffic it usually doesn’t take long for someone to pipe up when it goes down, especially if it happens during the day time in Australia. Since I now have remote access to the server I’m one command away from rebooting it should anything happen to it and have done so multiple times when it has come to my attention. However there’s a good 12 or so hours during the day when I’m not really paying attention to the blog due to being at home and/or sleep and downtime during this period usually goes unnoticed until I try to login during the morning. Since a good chunk of my audience is in the USA this can mean an awful amount of missed traffic which isn’t the greatest way to start the day.
Now when I first set up the blog on this host there were a couple teething issues (mostly due to my rusty Linux skills) but for probably 2 months afterwards everything ran without the slightest indication of an issue. Then every so often the blog would simply stop responding, the server would be up and everything else on it was running fine but try as I might I couldn’t get it to serve out a PHP page. Wanting to get it back up as quickly as I could I recycled the Apache service and it came back up instantly and I figured it was just some transient error and went back to my everyday blogging routine. However it kept happening, usually at the most inopportune times, and so last weekend I sat down to find the root cause of the issue.
Turns out its WordPress itself.
The above screenshot shows the error pretty quickly, essentially Apache has reached the maximum number of clients it can serve and will start to reject users after that point. Whilst the causes of this are wide and varied it can usually the culprit can usually be traced down to some WordPress plugin or script that’s opening up connections and then not closing them properly. The best way to take care of this is to fix the script in question but since I have little interest in diving into the mess that is PHP I’ve simply upped the MaxClients setting, reduced the time out period and scheduled an Apache reboot to clear out anything that gets stuck open. All of these combined seems to be an effective solution to this issue in the mean time and once I feel up to the task of delving through all the code to find the offending script I’ll nip it in the bud for good.
Apart from that little quirk though this iteration of the blog’s underlying infrastructure has been pretty fantastic with all the plugins functioning the way I expect them to without me having to fiddle with web.config settings for hours on end. It’s also significantly faster as well, reducing page load times by half for dynamic pages and becoming near instant when its served from cache. You could attribute this to the fact that it’s a lot beefier than its predecessor but neither of them showed significant load for an extended period of time. I guess where I’m going with this is that if you’re going to host your own WordPress blog it’s just plain better on Linux, especially if you’ve better things to be doing (like, you know, blogging).
I’ve spent the last 4 days lost in a technical limbo. On Saturday I was making good progress with Geon and left it to spend the night out with a bunch of my great friends and to see Inception (well worth seeing by the way). However Sunday morning saw me stuck on what seemed to be the most simple of problems, wasting at least 5 hours trying to get past this point. To make matters worse my web server decided to give up the ghost and just stop serving out PHP pages and neither upgrades, reboots or the hours of troubleshooting I threw at it would bring it back to life. Since you’re reading this now you know that I’ve managed to fix it, but it wasn’t an easy journey.
You see I was running this blog from IIS 7.0 which is Microsoft’s web server. For the most part it’s pretty good, the administration tools are top notch and it only took me a couple weeks to get my head around the whole idea of hosting a web site. Before starting up this site I’d never really tried any web stuff, preferring to stick to my cosy little offline world. Still I knew when starting this blog that I was using the least preferred web server on the market and the tutorials I found for getting it all started seemed more complicated than they needed to be. But I had been given a free copy of Windows Server 2008 and I didn’t really feel like re-educating myself with Linux, which had burned me in the past.
For a long time everything was good, the site was always up (pending my Internet connection of course) and seemed to be quite responsive. Over the course of almost 2 years though I’ve added quite a lot of things to the site like photo albums, sexy comments and all manner of behind the scenes stuff. About 6 weeks ago though I started getting PHP timeouts on this site, but all the others were running fine. The weekend just past saw all of the sites die a similar death and nothing I could do seemed to revive them. I was then faced with a choice, either move this site to one of my external providers or rebuild the server completely. After considering my options I went for the rebuild and 2 days later I’m back online serving this page to you from an Apache web server running on a freshly minted Ubuntu server.
Now I’m no stranger to the world of Linux. I spent quite a good chunk of my University years programming in Linux and my time at the National Archives of Australia had me administering a couple Linux servers that were powering the Digital Preservation Project there. Still I shyed away from it mostly because it was always a pain to administer when compared to its Window’s cousins which I always able to get functioning after 1 or 2 Google searches. To its credit Ubuntu made this process far less painful that it had been in the past and the tools have matured to a point when even a Windows administrator shouldn’t feel out of place. Last night saw this website resurrected in all its glory and hopefully I’ll figure out how to get my other hosted sites up and running sometime soon.
So you might be wondering: what happened to your old server? Well it’s still there, sitting alongside it’s new Ubuntu brother on my ESXi box and it won’t be going anywhere for quite a long time. You see, as great as Apache is, it’s still not IIS and with all my code relying on many things that IIS provides I’ll still need it in some form for quite a while to come. Additionally I had it set up as my Microsoft SQL box, again for development, and whilst I could use MySQL or PostgreSQL to achieve the same end the integration with the Microsoft tools just isn’t there yet. Plus I get it for free because of my Technet subscription so the cost factor that usually accompanies it is a non-issue.
Hopefully the move to the new server will mean the site will load faster for you, won’t cause me anymore administrative headaches and will reacquaint me with the Linux world I once knew so well. It’s a load off my mind to finally see this site back up and running again, especially after 4 days of feeling like I couldn’t make progress in anything technical. We’ll be back to our regularly scheduled programming tomorrow and I hope to see you guys back here then 🙂