Benchmarking Drupal with PHP op-code caches: APC, eAccelerator and XCache compared

Published Sun, 2008/04/20 - 17:33, Updated Thu, 2008/08/28 - 15:45

Almost every one knows that running high traffic sites powered with PHP applications is impossible without a PHP op-code cache / accelerators. Op-code caches speed up PHP applications by parsing and tokenizing PHP scripts once, and executing them faster for every subsequent request.

There are several accelerators available, many of them are free. The ones that are maintained are:

Sometime ago, we conducted a benchmark of Drupal using APC vs eAccelerator where the latter was found to be a better performer than the former in terms of speed and memory consumption.

It is now time to update this benchmark and include XCache as well.

Configuration

The tests were run on the following configuration:

Hardware

AMD Athlon 64 X2 Dual Core Processor 4400+ @ 2.2GHz, 1MB cache and 2GB RAM. 160GB SATA 7200RPM hard disk.

Software

GNU/Linux Ubuntu server Gutsy 7.10, Apache 2.2.4, MySQL 5.0.45 and PHP 5.2.3.

Drupal

The tests were run on Drupal 6.3-dev, checked out from CVS. Views and CCK are installed and enabled, but there are no views nor CCK types defined.

Using the devel generate module, we created 2,000 users, 5 vocabularies, and 50 terms. We then created 2,000 nodes of type page and story, with 5 comments each, and assigned terms to them.

All Drupal caching is disabled.

PHP op-code caches

We used the following versions of each op-code cache, we also note how it was installed, and what the configuration details we used:

XCache

XCache is a spinoff of the lighttpd web server. It is currently maintained.

We used version: 1.2.1, which is available from the APT repositories for Ubuntu Gutsy, as php5-xcache.

Being available in the Ubuntu/Debian repositories means that installing Xcache was the easiest of the bunch using aptitude.

Configuring XCache requires some parameters to change though if you want the cache to be in memory though. We used the following configuration to change the file which resides in /etc/php5/conf.d/xcache.ini.

[xcache]
xcache.shm_scheme =        "mmap"
xcache.size  =                48M
xcache.count =                 2
xcache.slots =                8K
xcache.ttl   =                 0
xcache.gc_interval =           0
xcache.readonly_protection = Off
xcache.mmap_path =           "/var/cache/xcache.mmap"
xcache.coredump_directory =  ""
xcache.cacher =              On
xcache.stat   =              On
xcache.optimizer =           Off
xcache.var_size  =            0M
xcache.var_count =             1
xcache.var_slots =            8K
xcache.var_ttl   =             0
xcache.var_maxttl   =          0
xcache.var_gc_interval =     300
xcache.test =                Off 

eAccelerator

eAccelerator is a fork of an older op-code cache called Turck MMCache that has been abandoned. eAccelerator development seems to have been slow of late, and is not keeping up with the latest versions of PHP 5. We found that eAccelerators can be unstable with newer versions of PHP and Drupal, and Apache will die with segmentation faults often. We had to use the log watcher script for restarting Apache automatically when this happens, but it was too frequent in some cases to be a real nuisance.

We used version v0.9.5.2, which is the latest stable release.

It was installed from source using the following commands:

phpize
./configure
make
make install 

We used the following configuration which resides in /etc/php5/conf.d/eaccelerator.ini.

zend_extension                  = /usr/lib/php5/20060613/eaccelerator.so
eaccelerator.shm_size           = 48
eaccelerator.cache_dir          = /var/cache/eaccelerator
eaccelerator.enable             = 1
eaccelerator.optimizer          = 1
eaccelerator.check_mtime        = 0
eaccelerator.debug              = 0
eaccelerator.filter             = ""
eaccelerator.shm_max            = 0
eaccelerator.shm_ttl            = 0
eaccelerator.shm_prune_period   = 0
eaccelerator.shm_only           = 1
eaccelerator.compress           = 1
eaccelerator.compress_level     = 9  

APC

APC is a PECL package that is maintained by the core PHP developers, including Gopal and Rasmus. It has several advantages including a very simple configuration, and close tracking of PHP versions. Being actively maintained is a big plus for APC.

We used version 3.0.16 of APC. It was installed using PECL, via the following command:

pecl install apc 

The configuration for APC is very minimalistic. We used the following configuration file in /etc/php5/conf.d/apc.ini.

extension                 = apc.so
apc.shm_size              = 48

Benchmarking methodology

We used the Apache Benchmark (ab) command, with a concurrency of 5, and 3,000 requests, like so:

ab -c5 -n3000 http://example.com/ 

Test 1: No PHP op-code cache

In this test, we established a baseline of how Drupal 6 would perform without any op-code cache.

Document Path:          /
Document Length:        21757 bytes
Concurrency Level:      5
Time taken for tests:   288.255212 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      66777000 bytes
HTML transferred:       65271000 bytes
Requests per second:    10.41 [#/sec] (mean)
Time per request:       480.425 [ms] (mean)
Time per request:       96.085 [ms] (mean, across all concurrent requests)
Transfer rate:          226.23 [Kbytes/sec] received
Connection Times (ms)
min  mean[+/-sd] median   max
Connect:        0    0   0.5      0      19
Processing:   181  479 186.0    444    1822
Waiting:      166  461 184.7    427    1708
Total:        181  479 186.0    444    1822
Percentage of the requests served within a certain time (ms)
50%    444
66%    525
75%    577
80%    619
90%    732
95%    819
98%    946
99%   1012
100%   1822 (longest request) 

Using the devel module, we see that the page generation time for PHP is around 200 ms.

Page execution time was 209.58 ms. Executed 101 queries in 9.6 milliseconds. 

Memory utilization is consistent at 24MB per Apache process. Note that the sixth colum (Resident Set Size) is what matters.

                         Virt Res
13616 www-data  16   0  213M 24660  3744 S  0.0  1.2  0:51.37 /usr/sbin/apache2 -k start
13619 www-data  15   0  213M 24484  3824 S  0.0  1.2  0:51.37 /usr/sbin/apache2 -k start
13627 www-data  15   0  213M 24484  3824 S  0.0  1.2  0:50.95 /usr/sbin/apache2 -k start
13617 www-data  15   0  213M 24484  3824 S  0.0  1.2  0:50.14 /usr/sbin/apache2 -k start
13624 www-data  15   0  213M 24408  3748 S  0.0  1.2  0:48.99 /usr/sbin/apache2 -k start
13626 www-data  16   0  213M 24408  3748 S  0.0  1.2  0:50.07 /usr/sbin/apache2 -k start
13615 www-data  16   0  213M 24404  3744 S  0.0  1.2  0:51.03 /usr/sbin/apache2 -k start
13623 www-data  15   0  213M 24404  3744 S  0.0  1.2  0:49.39 /usr/sbin/apache2 -k start
13625 www-data  15   0  213M 24404  3744 S  0.0  1.2  0:52.95 /usr/sbin/apache2 -k start
13618 www-data  16   0  213M 24404  3744 S  0.0  1.2  0:49.93 /usr/sbin/apache2 -k start 

Test 2: eAccelerator

Document Path:          /
Document Length:        21757 bytes
Concurrency Level:      5
Time taken for tests:   95.983986 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      66777000 bytes
HTML transferred:       65271000 bytes
Requests per second:    31.26 [#/sec] (mean)
Time per request:       159.973 [ms] (mean)
Time per request:       31.995 [ms] (mean, across all concurrent requests)
Transfer rate:          679.39 [Kbytes/sec] received
Connection Times (ms)
min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       3
Processing:    57  159  91.3    148    3830
Waiting:       50  152  89.8    142    3704
Total:         57  159  91.3    148    3830
Percentage of the requests served within a certain time (ms)
50%    148
66%    174
75%    193
80%    205
90%    239
95%    263
98%    289
99%    309
100%   3830 (longest request)

Using devel, the PHP time is around 47 ms. A significant improvement over not using an op-code cache.

Page execution time was 57.88 ms. Executed 101 queries in 9.01 milliseconds. 

Memory utilization for Apache is as follows. The 30MB process size is from the first run where the PHP scripts were loaded, parsed and tokenized. The other processes are from the subsequent requests and they range from 23MB to 18MB.

                         Virt Res
9801 www-data  16   0  261M 30688 12644 S  0.0  1.5  0:14.96 /usr/sbin/apache2 -k start
9799 www-data  16   0  254M 23848 12588 S  0.0  1.2  0:15.89 /usr/sbin/apache2 -k start
9797 www-data  16   0  253M 22536 12272 S  0.0  1.1  0:15.74 /usr/sbin/apache2 -k start
9800 www-data  15   0  251M 20028 11832 S  0.0  1.0  0:14.08 /usr/sbin/apache2 -k start
9806 www-data  16   0  251M 18536 10396 S  0.0  0.9  0:14.74 /usr/sbin/apache2 -k start
9808 www-data  16   0  251M 18536 10396 S  0.0  0.9  0:14.83 /usr/sbin/apache2 -k start
9807 www-data  15   0  251M 18536 10396 S  0.0  0.9  0:14.67 /usr/sbin/apache2 -k start
9803 www-data  15   0  251M 18536 10396 S  0.0  0.9  0:15.62 /usr/sbin/apache2 -k start
9809 www-data  16   0  251M 18536 10396 S  0.0  0.9  0:14.07 /usr/sbin/apache2 -k start
9805 www-data  16   0  251M 18536 10396 S  0.0  0.9  0:14.16 /usr/sbin/apache2 -k start

 

Test 3: XCache

Document Path:          /
Document Length:        21757 bytes
Concurrency Level:      5
Time taken for tests:   99.76300 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      66777000 bytes
HTML transferred:       65271000 bytes
Requests per second:    30.28 [#/sec] (mean)
Time per request:       165.127 [ms] (mean)
Time per request:       33.025 [ms] (mean, across all concurrent requests)
Transfer rate:          658.19 [Kbytes/sec] received
Connection Times (ms)
min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       2
Processing:    59  164  83.4    155    3367
Waiting:       52  156  66.4    148    1802
Total:         59  164  83.4    155    3367
Percentage of the requests served within a certain time (ms)
50%    155
66%    178
75%    196
80%    206
90%    237
95%    263
98%    287
99%    305
100%   3367 (longest request) 

Using devel, the page generation is around 50 ms.

 Page execution time was 59.37 ms. Executed 101 queries in 9.27 milliseconds.

For memory utilization, you can see that it ranges from 29MB to 19MB, a bit more than eAccelerator.

                         Virt Res
10316 www-data  16   0  263M 32316 11832 S  0.0  1.6  0:17.03 /usr/sbin/apache2 -k start
10319 www-data  15   0  259M 29976 13440 S  0.0  1.5  0:17.05 /usr/sbin/apache2 -k start
10318 www-data  15   0  261M 29724 11628 S  0.0  1.4  0:16.83 /usr/sbin/apache2 -k start
10317 www-data  16   0  254M 23576 12128 S  0.0  1.1  0:15.33 /usr/sbin/apache2 -k start
10322 www-data  15   0  251M 19624 11524 S  0.0  1.0  0:16.85 /usr/sbin/apache2 -k start
10328 www-data  15   0  251M 19624 11524 S  0.0  1.0  0:14.95 /usr/sbin/apache2 -k start
10324 www-data  15   0  251M 19624 11524 S  0.0  1.0  0:15.32 /usr/sbin/apache2 -k start
10325 www-data  15   0  251M 19624 11524 S  0.0  1.0  0:14.42 /usr/sbin/apache2 -k start
10327 www-data  16   0  251M 19624 11524 S  0.0  1.0  0:15.05 /usr/sbin/apache2 -k start

Test 4: APC

Document Path:          /
Document Length:        21757 bytes
Concurrency Level:      5
Time taken for tests:   98.530068 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      66777000 bytes
HTML transferred:       65271000 bytes
Requests per second:    30.45 [#/sec] (mean)
Time per request:       164.217 [ms] (mean)
Time per request:       32.843 [ms] (mean, across all concurrent requests)
Transfer rate:          661.84 [Kbytes/sec] received
Connection Times (ms)
min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       2
Processing:    58  163  71.2    155    2452
Waiting:       53  158  69.6    150    2329
Total:         58  163  71.2    155    2452
Percentage of the requests served within a certain time (ms)
50%    155
66%    178
75%    193
80%    204
90%    235
95%    258
98%    285
99%    302
100%   2452 (longest request) 

Using devel, the page generation time is around 50 ms as well.

Page execution time was 59.8 ms. Executed 101 queries in 9.1 milliseconds. 

Memory utilization is noticeably consistent at 21MB per process, more than the other two op-caches, but suprisingly consistent.

                        Virt Res
9263 www-data  16   0  263M 38172 18036 S  0.0  1.9  0:15.31 /usr/sbin/apache2 -k start
9266 www-data  15   0  252M 21704 13020 S  0.0  1.1  0:18.14 /usr/sbin/apache2 -k start
9270 www-data  15   0  252M 21604 12768 S  0.0  1.0  0:18.30 /usr/sbin/apache2 -k start
9274 www-data  15   0  252M 21604 12768 S  0.0  1.0  0:15.75 /usr/sbin/apache2 -k start
9264 www-data  16   0  252M 21520 12692 S  0.0  1.0  0:16.55 /usr/sbin/apache2 -k start
9267 www-data  15   0  252M 21520 12688 S  0.0  1.0  0:18.10 /usr/sbin/apache2 -k start
9268 www-data  16   0  252M 21520 12688 S  0.0  1.0  0:16.89 /usr/sbin/apache2 -k start
9269 www-data  15   0  252M 21520 12688 S  0.0  1.0  0:16.51 /usr/sbin/apache2 -k start
9273 www-data  16   0  252M 21520 12688 S  0.0  1.0  0:17.32 /usr/sbin/apache2 -k start
9275 www-data  16   0  252M 21520 12688 S  0.0  1.0  0:16.03 /usr/sbin/apache2 -k start 

Summary

The following table summarizes the above results.

  Requests per Second Single Request (milliseconds) Memory (Maximum, MB) Memory (Minimum, MB)
None 10.41 96.08 24 24
eAccelerator 31.26 31.99 23 18
XCache 30.28 33.02 29 19
APC 30.45 32.84 21 21

Conclusions

From the above results, one can come to the following conclusions:

  • All op-code caches provide a noticable improvement for Drupal over a default PHP installation.
  • The speed gain is about 3X.
  • eAccelerator is marginally better than the XCache or APC both in terms of speed and memory utilization.
  • Installation of each op-code cache is different: one has a Debian package, the other is installed from source and the third is via PECL.
  • The configuration for each is also different. Some work well with a default install, others require more tweaking.

Update:

We have noticed that in production, Xcache suffers from the same instability that eAccelerator exhibits: segmentation fault after a day or so. This happened with Xcache 1.2.1-3 which ships with Ubuntu 8.04.1, and PHP 5.2.4. The latest stable release is 1.2.2 from Xcache's web site. So, try compiling that from source, or install the logwatcher script if a minute of downtime is acceptable.

Another cache alternatives?

Hello,
Thanks for this very informative article.
May be a new post about Memcache and Varnish?

Regards.

you have check time enabled

you have check time enabled on all except eAccelerator. That why eAccelerator did the best. Tune APC and xcache and they are just as fast/faster

APC memory limit

Hi,

We have a private website set up for our distance learning students and have run into some problems using apc with cachrouter. I've upped the apc.shm memory, but the shared memory always reads: 1 Segment(s) with 30.0 MBytes (mmap memory, pthread mutex locking), and apc seems to be sticking to this limit, even though shm is set much higher. Could anyone suggest where the 30M value might be coming from, as this seems to be overriding the apc value.

Many thanks,
Paul

It is not enough to increase

It is not enough to increase the shm setting in the APC config. You need to make sure your server has that amount of shm available. Not knowing what OS you are using I cannot tell you where to change the OS settings, but you can search that info on the web.

eAccelerator reaches memory limit without a warning!

eAccelerator support enabled
Version 0.9.6.1
Caching Enabled true
Optimizer Enabled true
Check mtime Enabled true
Memory Size 33,554,392 Bytes
Memory Available 2,856 Bytes
Memory Allocated 33,551,536 Bytes
Cached Scripts 400
Removed Scripts 0

What you see is plain phpinfo.php output. The cache of eAccelerator (32MB) fills up and most of your modules won't be treated by eAccelerator. Increase it's cache size and make sure LINUX will also allow to do so:

echo 37748736 > /proc/sys/kernel/shmmax
==> used to be 32MB, now 36MB to fit eaccelerator memory size

I have a complete realtime monitor with special DRUPAL features available. Cannot attach any graphs here so ask me using my email-id to get more documentation on it.

Another benchmark of Drupal: PHP vs APC vs HipHop for PHP

It can be found here: http://php.webtutor.pl/en/2011/05/17/drupal-hiphop-for-php-vs-apc-benchmark/

During our tests, we've found

During our tests, we've found that APC uses about twice as much memory as eAccelerator to cache about the same number of php files. This might not show on this benchmark because only the cached front page is requested.

Optimizing for Windows

Can any of this be made in a WIMP (windows, IIS, mySQL, php) stack? Yes, I know it is not a common step but the client will never change it... so what can do about it to make drupal run fast? Any advices or similar experiences?

Great article

Very interesting article.
Would love to see similar tests but with Views and CCK in use.
Also would love to see a comparison of Drupal vs Joomla and also accelerated PHP vs .NET test ?
Also why was the Zend PHP Accelerator not included in this current test ?

Views, CCK and Zend

Views and CCK are a more complicated mix, because not only do they execute more code, but also do more SQL queries. PHP op-code caches do not help with the database load at all, but may free enough CPU for the database (if the db server is on the same machine as PHP).

The Zend Accelerator is proprietary, and our test was for free op-code caches for PHP. Since then, Zend released the Zend Community Server, and we wrote an article at: benchmarking Zend server community edition with Drupal.

As for PHP vs. .NET, it is a moot point for most people, since speed is not the only decision point, but rather other things like platform (Windows vs. LAMP), in house knowledge, tool familiarity, integration with existing applications, ...etc.

Drupal vs. Joomla is an interesting one, but it would be hard to come with a scenario where the sites have identical feature/functionality so the benchamrk is fair.

If you have specific sites that need to be benchmarked, you are always welcome to contact us for consulting via the Contact form on this site.

Dates

Excellent review, thank you for quality information.

However, my pet internet peeve applies: Please provide 'last updated' date at top of page and 'original article' date.

Keep up the good work, I rely on good people like you to make my important decisions. Well, except for where to take 'her' for a good date. ;-)

Re: Dates

Excellent article. Very helpful to get me ramped up on these concepts, try them out on my server, and see what's out there to pick for accelerators..

Regarding dates -- my peeve, too :-P so I agree with Dennis.. These 2 dates would be helpful on each article page. Especially when following the links back to your earlier articles on accelerators: i want to know how dated they are.

Thanks again for your work put into this, and your clear presentation. A big time-saver, thanks.

APC problem: white screen of death

Hello,

I would like to use Drupal on apache+mod_php with APC to test with the APC upload progress feature (rfc1867) but every time I submit a form with APC enabled I get a 'white screen of death' (WSOD)... Any ideas why?

Recent versions?

Are you trying recent versions of PHP and APC or old ones?
--
2bits -- Drupal consulting

up-to-date gentoo

Yes, all my packages are up-to-date (gentoo distribution) I found out that I only get the WSOD when I submit forms with file fields and when apc.rfc1867 is enabled. I'm investigating more.

Thank you for this report! I

Thank you for this report!

I can confirm stability problems with eAccelerator (0.8.5.3). On a Debian Etch system with Apache2 and PHP5 even D5 installations may cause segementation faults. I haven't checked XCache and instead directly went to APC and hope things will work out well.

Re: Thank your for this report!

Also, thank you for this report!

We linked this to our Drupal-Blog.

Only APC is stable

With several servers running Ubuntu 8.04.01 LTS, Apache 2.2.8 and PHP 5.2.4 I have found that neither eAccelerator nor XCache are stable. In a matter of a day or so, you will get segfaults.

On the contrary, APC -- currently using 3.0.19 -- is very stable and never experiences segmentation faults.
--
2bits -- Drupal consulting

Good comparisons

Thanks for a more comprehensive and updated comparison (I read your APC vs EA post earlier) and the concurrency usage (although it would be great to show multiple levels).

Really nice

Thanks for sharing those results.

Really nice test! Thanks

Really nice test!

Thanks

Tool for statistics ?

This article is very interesting and the tests are well done and presented.
What tool did you use for the statistics ? Is it the devel module ?

I am talking about the following statistics :

Document Path: /
Document Length: 21757 bytes
Concurrency Level: 5
Time taken for tests: 98.530068 seconds
Complete requests: 3000
[...]

Apache Benchmark

I mentioned that in the article, but perhaps it is not prominent enough. It is called Apache Benchmark (ab).
--
2bits -- Drupal consulting

Thanks !

Thank you very much for your answer.
In fact I thought that Apache Benchmark was used to generate the traffic only, not to get the stats.

xcache

Why did you disable the xcache variable cache?

"xcache.var_size = 0M"

Also, xcache.var_count should be the same as xcache.count.

Code caching, not object caching

All three op-caches were used in this benchmark solely as op-code caches, that is as accelerators for PHP code itself.

There was no content caching of any kind in this benchmark, whether on the Drupal level (database tables) or via object caches (memcached, xcache, apc, ...etc.)

So, this is why the above was done.
--
2bits -- Drupal consulting

Concurrent usage of APC & memcached possible?

Hello,

can you recommend a concurrent usage of APC for opcode caching and memcached for content and PHP session caching?

I'm asking because I noticed that using APC for content caching will heavily fragment the APC cache due to the short expire time of the content objects and the regular garbage collection in APC.

I'm hoping separating the opcode cache from the content cache will improve performance and solve the problem with APC fragmentation.

BTW, APC fragmentation is very low and available memory is high if I do not use it for content caching.

Thank you for all the great and helpful articles on your site.

Definitely

Most definitely: yes.

I do recommend using APC for code cache, and memcache for content cache (and session cache, if needed).

This is how I configure all client sites. No worries about fragmentation, and you have the ability to move memcache to a separate server, and not be limited by one machine for everything.

I never use APC for content caching, only code caching. This also allows me to use FastCGI for PHP, where each process has its own copy of the APC code cache. If I were to use APC for content caching in that mode, then the content cache would be inconsistent.

Great. Thank you. Especially

Great. Thank you. Especially for pointing out that using APC for content caching in FastCGI mode will lead to inconsistent caches. I use Apache and mod_fcgid for running php. I'm happy the fragmentation forced me to turn off user/content caching in APC.

Seems it wouldn't help anyway.

Nevermind, I did some research, and it looks like the xcache variable cache wouldn't work with Drupal anyway.

I am using eaccelerate for a

I am using eaccelerate for a while but didn't know that it's best choice.
Thanks for test. Now i'm feel with confidence.

Is your Drupal or WordPress site slow?
Is it suffering from server resources shortages?
Is it experiencing outages?
Contact us for Drupal and WordPress Performance Optimization and Tuning Consulting