Blog-Site Performance Issues

Yesterday’s blog-entries-announcement newsletter, which went out late morning EST to both HTML and Text edition subscribers, taught me something important: My current web server arrangement is not going to stand up to the first hour after the newsletter is mailed each month.

Many of you wrote to me to tell me you either got error messages or had to wait minutes for individual blog stories to load in your browsers. Some of you just assumed that this was the normal experience at Scot’s Blog, and that is just not the case. Most of the time it’s pretty snappy.

The problem boils down to this: WordPress (and all other blog packages I’ve inspected) use a database to store the stories in. Many of them, including WordPress, use MySQL. My forums also uses a separate MySQL database. The two databases are on the same server. And the arrival of the blog newsletter quickly maxed out the number of simultaneous connections that my webhost has limited my MySQL service to. So people either got “database connection” errors or just waited and waited for the server’s time to free up. The forums also experienced interruptions and slow page loads.

An hour or two later, the problem had disappeared as the initial rush for the newsletter slowed down to a steady stream.


Clearly, I have to separate the forums database from the blog database. Those two can’t co-exist on the same server with a shared limitation on concurrent connections. The only way I can do that, however, is to change the domain name of either the forums or the blog.

Then I have two choices: Either I purchase a new, separate account at the existing webhost or I move either the blog or the forums to a new webhost.

I just wanted to announce to all who experienced problems yesterday that I am actively working on this issue as a top priority. I can’t promise it will be fixed by the next issue of the newsletter — but if not, it will be explained in the next issue of the newsletter. And this will be resolved.

Update on December 8: Please see this addendum in the form of a comment to this blog post. I’ve added caching software to WordPress, which may solve or at least alleviate some of the problem.

Update on December 15: The MySQL databases for the Scot’s Newsletter Forums and Scot’s Newsletter Blog have been reconfigured so that they will have separate maximum concurrent connections. The change was easy to configure, once my Webhost explained it to me, and it didn’t cost anything.

With these two changes — the cache and separation of concurrent connections — I think we’re ready to give this another go with the next newsletter. If we continue to have performance and time-out problems during the two hours following the initial sending of the newsletter, I will look for another webhost — a solution that won’t happen as quickly as these changes did. But what’s the point of having newsletter notification if the server can’t hold up to the demand? That isn’t acceptable.

11 Responses to “Blog-Site Performance Issues”

  1. Tiktaalik Says:

    Maybe the Slashdot effect was a one time thing. For example, I’ll rarely, if ever again, access the blog content from the email announcement. 🙂 If there are enough of us like that, we’ll have already accessed all the posts between email days.

  2. Martin Says:

    I experienced the site as totally unavailable yesterday. It’s too bad WordPress doesn’t have a way to build a pure HTML image of the site pages.

  3. bk58 Says:

    I ditto Tiktaalik

  4. alexgieg Says:

    Are you using the cache mechanisms available for script-based websites? I don’t know the details, but I remember there are ways to minimize the hits on the database by caching a content that doesn’t change frequently.

    I also remember that, when I tried TypePad (it’s been 1.5 year), it allowed you to set a mixed mode in which some pages would be generated as purely static content while others would stay dynamic. I don’t know whether WordPress has this feature too, but it’s worth checking, since then you’d also hit the database less frequently.

  5. Scot Says:

    Good point, Alexgieg. There is third-party cache add-on that I should look at again. When I looked at it originally, it hadn’t been updated for my WP version. But maybe it has now.

    I discovered from my webhost, though, that I’m limited to 60 simultaneous MySQL connections, and that apparently combines both the forums and the blog, even though they’re in separate databases.

    I’m definitely going to try the cache option, if possible. But if anyone knows of a webhost with robust MySQL support, please post the name.

  6. ExF15AIS Says:

    I am relieved to hear that the delay appears technical in nature. I one of those who work in the city but choose to reside in the rural areas (housing is a killer); and with that choice there is a price to pay. There is no broadband, no DSL, no fixed-based wireless, and too many trees for satellite feeds, so I am relegated to only having dial-up or a couple of tin cans and string.

    Needless to say, it was painful that the text-based went away in lieu of HTML, scripts, data mining, etc. But that is how technology is I guess. No discredit to you; but it is a little reminiscent of your blog on “Windows XP or Vista?” where more is added to the system with little to no improvement; but that’s just a dial-uppers rant.

    Hope all resolved soon.

  7. Scot Says:

    The problems only occur during the first hour or two after the newsletter mails, when the blog experiences peak demand from hundreds and possibly thousands of concurrent users. The same thing happened with the last mailing of the newsletter in October — although to a far less degree. I think that one may have gone out at night, which probably minimized the number of concurrent sessions.

    I’ve just installed and enabled a WordPress add-on called WP-Cache 2. It has already cached the main pages, rss feeds, searches, and all the most recent blog posts and is now working its way back through the whole site.

    It’s not clear to me that this will completely eliminate the problem. I guess we won’t know until the next newsletter goes out. But I think this software solution is worth a try. It’s supposed to serve a lot more concurrent users and also improve the performance of the site by serving key parts of the site out of cache folder instead of making calls to the database.

    WordPress does have basic caching functionality, although it’s turned off by default and is fully manual in how it’s configured. WP-Cache also gives me manual options for including or excluding specific pages. One way or another, i think the site will be improved by this code and judicious management of it.

    Meanwhile I’m exploring other webhosts. A friend of mine recommended a low cost host with “semi-dedicated” (shared hosting with far fewer other customers on the same box). But the issue is really the concurrent MySQL connections, something that most hosts don’t even list among their specs. So now I’m asking that question of prospective hosts.

    — Scot

  8. Frank Says:

    Scot, a simpler, first step would be to stage the delivery of the announcement email. Send it out in four, five, six different groups separated by an hour or two. I don’t think many people would be ticked to find out someone else read it a few hours before they had the chance. Well, the “first post” newbies maybe. 😉

  9. Scot Says:

    Frank, everyone seems to think that everything about sending a newsletter is easy, but it’s not. “Staging” the delivery of the newsletter is a big deal that would cost me a lot of time and aggravation. While I understand that it sounds like a logical step, it’s not even a realistic option.

  10. DF Says:

    Any chance that the date of the entry can be displayed with the title of the entry.

  11. Scot Says:

    DF: Done!

    Be advised, customizations that I make to the WordPress Theme are sometimes lost later. I’ve already experienced this before.

    — Scot

Leave a Reply

You must be logged in to post a comment.