Code optimization with Xdebug and KCachegrind

We have been working on a migration script to move a custom CMS into Drupal, and one of the problems we have been dealing with is the script's performance. The server that it will run on kills PHP scripts that run for more than 5 minutes, and our script was running for about 20 minutes. By reducing the number of queries, making small code optimizations, and properly indexing and keying the legacy database we managed to cut the time to 10 minutes—still we were way off target. To get a better look at what was going on, we thought it would be great to install Xdebug and take a look at it with KCacheGrind. This combination of tools gives a very granular look at where time is spent in the code and should give some hints at where the bottlenecks are. Here are my steps to get a LAMP stack, Xdebug, and KCacheGrind installed on a fresh copy of Ubuntu. This was done in a virtual machine hosted on my Mac but could be done on any Ubuntu install. There are number of virtual machines that you could use. Around the office, for Mac, we use VMWare Fusion, Parallels, and Virtualbox. If you'd like to run a virtual machine and have never done it before, here are some guides to getting one setup:

Once you have Ubuntu running, the first step was to install a LAMP stack to host my files. Luckily Ubuntu has a handy shortcut to get this set up: sudo tasksel install lamp-server This sets up PHP, MySQL, and Apache with a web root at /var/www. I copied in my code and database, configured them (mostly changing database settings in settings.php), and browsed around a couple pages to make sure the site was working properly. Next I installed some requirements for Xdebug: sudo apt-get install php-pear php5-dev And then Xdebug itself: sudo pecl install xdebug This installed the Xdebug extension to /usr/lib/php5/20060613+lfs/xdebug.so. I then added some settings to the end of my php.ini (found at /etc/php5/apache2/php.ini):

zend_extension="/usr/lib/php5/20060613+lfs/xdebug.so" 
xdebug.remote_enable=1 
xdebug.remote_handler=dbgp 
xdebug.remote_mode=req 
xdebug.remote_host=127.0.0.1 
xdebug.remote_port=9000 
; profiler settings 
xdebug.profiler_append=1 
;xdebug.profiler_append=0 
xdebug.profiler_enable=0 
xdebug.profiler_enable_trigger=1 
xdebug.profiler_output_name = cachegrind.out.%s 
xdebug.profiler_output_dir=/xdebug/

Then I created the directory 'xdebug' in the root and set the permissions to 777 just to be sure Apache can write here: sudo mkdir /xdebug sudo chmod 777 /xdebug At this point I restarted Ubuntu to make sure all the changes would take effect. To test that Xdebug is installed correctly, you can create a small test script with the following contents and run it from the web root:

<?php phpinfo();?> 

If you see anything related to Xdebug in the output, everything should be working fine. Next I installed KCachegrind by going to Applications > Add/Remove and searching for KCachegrind. Lastly, I installed XDebug helper in Firefox; this nifty extension allows you to turn Xdebug on and off for any page. Once everything was installed I turned on the Xdebug session and profiler buttons in the bottom right corner of Firefox and browsed to my migration script (it's a page callback). This created a file at /xdebug/cachegrind.out._var_www_index_php which I opened with KCachegrind (found under Applications > Programming > KCachegrind). Here is a block diagram of the code's flow generated by KCachegrind (ignoring drupal_render):

 

Here is a list of functions with information about their execution:

As you can see, pathauto is eating up a lot of time. The next time we ran the script we disabled pathauto and this cut the run time to just under 2 minutes, nicely under our target time. Of course there are further optimizations that can be made and now we have a great way to see where to focus our energy.

Resources

Here are some resources that were used to get this working:

Comments

Excellent write-up

Tylor,

excellent write up. I've been trying to set something like this up to see what's going on with some custom work I've been doing that causing load problems etc on a server.

I was wondering, you said you implemented this on your local machine on an Ubuntu Virtualbox instance, how does this replicate or act as a benchmark for your actual server? Would the results not vary, or was it sufficient to get an idea of where the bottleneck was?

Col.

Hi Colin, We haven't had a

Hi Colin,

We haven't had a chance to run the migration on the actual server yet but I'm confident these changes will translate there. I'm assuming that if it can run fine on my Mac it will be faster on something tuned and dedicated. The script I actually ran through Xdebug was greatly simplified, migrating only 10 nodes instead of 2500. This small sample was enough to find where the bottleneck was.

Remember, you're building a profile of the flow of the code. Benchmarking for time will be different, but a profile of the code should give similar results in different environments.

That's what I thought

Tylor,

"Remember, you're building a profile of the flow of the code. " - that's what I was thinking, benchmarking will be a separate issue. I think I've found a new personal project to get this up and running on Ubuntu. Thanks for the great post.

Col.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options