David J. Pryke

Husband | Father | Techie

Browsing Posts published in November, 2008

In a previous post, WordPress woes, I went through the procedure I used to solve some performance problems related to how the IP address on this server is NATed.  Well, there was another problem somewhere that caused the WordPress admin to be incredibly slow most of the time, caused Akismet to not work at all, and a variety of other symptoms.

I went through a bit of diagnosis, and I came across this in my /var/log/php-errors.log (I enabled this logging in /etc/php.ini to track what was happening):

php_network_getaddresses: getaddrinfo failed: Temporary failure in name resolution in /var/www/sitename/scriptname.php on line 20

I checked my /etc/hosts and /etc/resolve.conf and /etc/host.conf files, and everything was fine there.  In addition, DNS resolution worked fine from the command-line via dig and ping.  I tried running my simple test script through the php command-line client, and it worked fine from there.  I checked the script in a browser, and no luck.  It was loading in the browser until the DNS lookup in the script timed out, then it finished loading.

So, I did that again, but this time, I ran

netstat -an

on the command line while the script was attempting its DNS lookup.  It turns out that Apache was looking at an old DNS server, which is not accessible via this network.  Oiy!

This makes a little bit of sense to me, as this server was originally setup in a convenient location on a different network, and then physically moved the server to the colocation data center.

Well, I don’t know why Apache still thought it should look at the old DNS server (the one from the other network) but I made sure there was no reference to it in old /var/lib/dhclient.leases files and then ran

apachectl stop
apachectl start

And, viola! The problem went away!  All the other times, I just ran

apachectl graceful

which apparently wasn’t clearing out it’s memory of the old DNS server, or it was still looking at the (not currently relevant) dhclient lease file, which is silly, since I now have the server setup on networks with dedicated IPs, and I am not using dhcp on any interface.

Weird.  But, solved!

Well, as part of my WordPress install, I am using the Crawl Rate Tracker plugin. It shows the hits on your blog from various spiders and so on. In it’s dashboard (in your WordPress admin) there is a nifty chart that visualizes the info. Or, at least, there should be. In my case, I got a text link that just output the URL of the php script that should have generated the chart.

By looking into sbtracking-chart-data.php, I found that as it incremented a value to track the date, it hit the number 1225684800, which is apparently the max integer value in PHP 4.3.9, which is the version that was included in CentOS 4.x, with security patches backported. This value corresponds to some point during November 2, 2008. Well, as you well know, we are past that point in history, causing this PHP script to loop infinitely, since it never errored out upon incrementing that variable, it just stuck at the value 1225684800, and it was using the date as a means of breaking the loop.

To resolve this, I upgraded to PHP 5.1.6, which also required a MySQL upgrade. (The way I did it, anyway, which was the fast & easy, CentOS “semi-supported” way to do it.) I edited the file /etc/yum.repos.d/CentOS-Base.repo to make the centosplus repository available by changing enabled=0 to enabled=1 and then adding this line under the same repository:

includepkgs=php* mysql*

which restricts the upgrades and installs from this repository to the php and mysql packages.

I ran yum update and let it download and install all the necessary packages. This put me at PHP 5.1.6 with some extra security patches and MySQL 5.0.68 with the same.

Upon an Apache restart (apachectl graceful) I tested the Crawl Rate Tracker plugin, and AHA! The chart is there, and as nifty as ever.
Goodnight.

During the journey of installing WordPress, choosing a theme, setting up plugins, and so on, I ran into a problem that oh-so-many out there have hit: terrible performance in the WordPress admin (downloadable version of WordPress, not at wordpress.com)

Well, to make a long story short this time: Do as everyone recommends, despite believing that the problem occurred before a certain point, or isn’t related…just do it: Disable all plugins which should get your performance back to something more acceptable. Once you are that point, enable your plugins one at a time until you hit your performance snag. I won’t list the problematic one for me, as yours will probably be different.

Just do it. You’ll be glad you did, even if for the fact that then you can be certain that it isn’t your plugins.

Just do it.

WordPress woes

1 comment

Well, as my first post, let me touch on some wordpress performance problems, as I just ran into said problems while getting this blog running.

So, I setup this virtual server a couple weeks ago.  On it, I installed a variety of things, including some other PHP-heavy sites.  All of them work great; faster than I expected, in fact.

Today, I installed WordPress in a subfolder on an Apache VirtualHost (specifically, david.pryke.us, which, since you are here and reading this, you already know.)  I performed the basic setup of wp-config.php and then navigated to the site to complete the setup.  This took a while to run, but I figured that perhaps the MySQL instance on this server was taxed, and it just took a while to setup the basic database.

Once I clicked on the “Log In” button at the last screen, I realized that something was terribly wrong.  It took 40 seconds or so to load the basic login page for me to enter my username and password.  Once I entered them, it took another 40 seconds or so to get to the admin.  I tried just loading the basic blog, without loggin into the admin…40 seconds.  I started looking for WordPress performance problems on Google.  Many of the results pointed me to resort to the default “Kubrick” theme and start from there (no problem, it was a default install, I’m already using that theme.) After that, it indicated to disable all the plugins and start enabling them one at a time, taking note of when big slowdowns start to happen.

Well, since this was a fresh, default install, I figured that was’t the major problem.  While I am a SysAdmin, and have no problem going through things one at a time, I also hit on a post by Paul Spoerry about how to Diagnose slow WordPress performance using FireBug.  I found that to be a great idea, and one that I should have thought of by this point, as a lot of my coworkers use FireBug to look at problems during website development.

So, I installed FireBug and inspected my site with it.  Wow.  40.532 seconds for the basic index.php page, and then a few hundred milliseconds at worst for the rest of the jpg’s and such combined.  So, I start looking for WordPress performance diagnosis, and I come across a WordPress forum topic regarding a similar problem.  In there, I found to insert code like:

<!-- <?php echo get_num_queries(); ?> queries. <?php timer_stop(1); ?> seconds. -->

which, I found out later, was already in the Kubrick theme.  One other thing I found and enabled in my footer.php was this:

You can see how long each query is taking with a few modifications.

In your wp-config.php file, add this line of code to the top:
define('SAVEQUERIES', true);

Then, in the theme’s footer.php file, you can do this to dump all the queries and how long they took:

if (SAVEQUERIES) {
global $wpdb;
echo '<!--\n';
print_r($wpdb->queries);
echo '\n--!>';
}

Then looking at the source of the page will show all the queries and a number showing how long they took.

After you do this, turn the SAVEQUERIES back off by setting it to false in the wp-config.php file. You don’t want it to spit out the queries all the time.

The key there, which I knew, but some other readers on that forum topic didn’t, was to put “<?php” before the code block, and “?>” after the code block in footer.php.  I looked at the source of the page after I added those pieces of code, and it told me two things.  The first line output this:

<!-- 21 queries. 40.195 seconds. -->

Which told me that it believed the database queries were taking over 40 seconds to perform.  However, the second piece of output (from $wpdb->queries) told me something totally different.  This command lists the SQL for each query, as well as how long it took to run that query.  Each on was along the lines of 0.0001130104064941 or 0.00027084350585938, which, when added together, was still much less than one second.  Something isn’t “adding up” here…

After reading the rest of that forum topic, someone mentioned a problem which went away when he ran the internal “wp-cron.php” script by hand, but came back every time he created a new post.  Well, there are two important pieces of information here.  One is that this internal cron script is scheduled to run again when certain actions are taken, such as creating a new post.  The second thing, and important in my case, is that this is run from within the web server itself…specifically, from within the PHP parser.

Now, a key piece of info for my problem is that I am hosted on a virtual machine that lives on a non-routable IP, an RFC 1918 IP of 10.53.22.13; this is important in that the public IP of this site is 66.179.100.13 (at least, as of this writing, on November 3rd, 2008.)  The PHP parser tried to connect to david.pryke.us/blog/….. and could not get there because the firewall & NAT machine “in front” of the server would not redirect traffic back down this network link to the 10.53.22.13 address when one of the machines on the same link asked for the public, routable IP of 66.179.100.13;

To resolve this, and make the long story short, I had to add a line in /etc/hosts that read:

10.53.22.13     david.pryke.us

Which allowed the server to see the “correct” IP for that domain name (david.pryke.us) and vioala! the server loaded the first default post in less than a second!

Problem solved.  (This should never have been a problem, as I usually setup the hosts file on these servers right away…but I forgot in this instance. Oops!)