At the last Waterloo Region Drupal Users Group meeting, Lana mentioned a benchmark of Drupal 5.8 vs. Drupal 6.3 by Vision Media in Victoria.
The benchmark concludes that Drupal 5 is faster than Drupal 6.
Intrigued by this benchmark, we proceeded to replicate it in a rigorous fashion.
We created a Drupal 5.10 site with the following data set:
Users 4,950
Nodes 5,000
Comments 20,000
Vocabularies 10
Terms 1,000
We then upgraded the site to Drupal 6.4, so that the data set is exactly the same, and then ran benchmarks simulating 10 concurrent users hammering the site for 2 minutes. The URLs to visit were the front page, plus the 30 nodes on the front page.
We benchmarked without APC, and with APC, as well as with page cache and no page cache, so we ended up with the following results.
Drupal 5 is consistently faster than Drupal 6.
Description, Trans, Elap Time, Resp Time, Trans Rate, Concurrent, OKAY, Failed
No APC; Drupal 5; no cache, 1960, 119.75, 0.61, 16.37, 9.97, 1960, 0
No APC; Drupal 6; no cache, 1850, 119.72, 0.64, 15.45, 9.97, 1850, 0
No APC; Drupal 5; w/ cache, 22605, 119.69, 0.05, 188.86, 9.97, 22605, 0
No APC; Drupal 6; w/ cache, 18896, 119.93, 0.06, 157.56, 9.98, 18896, 0
APC; Drupal 5; no cache, 6998, 120.24, 0.17, 58.20, 9.97, 6998, 0
APC; Drupal 6; no cache, 4972, 119.74, 0.24, 41.52, 9.99, 4972, 0
APC; Drupal 5; w/ cache, 52190, 119.76, 0.02, 435.79, 9.97, 52190, 0
APC; Drupal 6; w/ cache, 48778, 119.91, 0.02, 406.79, 9.96, 48778, 0
The data can be summarize as follows, using the transaction rate:
- No APC, no page cache: Drupal 5 faster by 5.6%
- No APC, with page cache: Drupal 5 faster by 16.5%
- APC, no page cache: Drupal 5 faster by 28.6%
- APC, with page cache: Drupal 5 faster by 6.6%
The results in a spreadsheet format are attached to this article, if you want to analyze it further.
The exact specifics for your site may vary depending on what modules you have enabled, system configuration and other factors.
Once Drupal 7.x has a code freeze, and we are able to run update.php again, I plan to compare Drupal 7.x against Drupal 6.x to see who is the faster one.
| Attachment | Size |
|---|---|
 drupal-5.10-vs-drupal-6.4-benchmark.xls | 106 KB |
Comments
Visitor (not verified)
Yeesh
Sun, 2008/10/12 - 20:12OK, seriously, with all the effort that went into making Drupal 6 faster, where is the performance going? I thought we narrowed it down back in January to bootstrap and the theme system. And bootstrap got 20% faster with the page split. What got so much slower?
Visitor (not verified)
It could be interesting to
Mon, 2008/10/13 - 09:02It could be interesting to see such tests, but with some contributed modules (for example, with views, cck, panels ...)
Visitor (not verified)
This is weird. Shouldn't a
Wed, 2008/10/15 - 05:37This is weird. Shouldn't a newer version be faster compared to an older version of a product? What went wrong?
And it would definitely be interesting to see how Drupal 5 and Drupal 6 compare with several modules installed. I think CCK, Viewsand also Imagecache should be in the installation since almost everybody uses them.
Visitor (not verified)
Interesting
Sat, 2008/11/22 - 11:37Interesting. Was this with any modules installed, or just plain, out of the box Drupal with dummy data?
I have run similar benchmarks privately, the figure where I have seen a lot of variation is APC w/ no page cache. Generally it stays around 25%, but I have seen it go as high as 40% depending on what modules are installed.
For anonymous users, seeing this kind of performance drop off is not a major deal. By the time a site reaches the point where anonymous page generation times become critical, there is typically a CDN in place helping to offload much of that traffic from the origin servers. A more interesting set of metrics is to look at authenticated usage performance, where D6 generally outperforms D5 by several orders of magnitude.
M
Khalid
More current article
Sat, 2008/11/22 - 14:30Re: APC and no page cache, the figures you quote 25% or 40% are percentage of what?
This was with dummy data and core Drupal only.
A more current article with better controlled conditions and a larger data set can be found in benchmarking Drupal 5.12 Drupal 6.6 and Drupal 7.x: we are getting slower.
Eric Kennedy (not verified)
We left Drupal for greater speed and easy of maintainability
Mon, 2010/11/22 - 15:22Our site ran Drupal 5 for 3 years and then we decided to rewrite using a PHP framework rather than upgrade to Drupal 6 or 7 because of the every-increasing bloat in Drupal. Khalid helped us with the site in 2007, and I've followed his performance benchmarks ever since.
Our new site is twice as fast (Google Webmaster Tools say our average page download time is 74 ms versus 160 ms on Drupal). Even better, we don't use a caching service (such as memcached), which simplifies site administration.
I prefer working on databases, and Drupal + CCK makes for a frighteningly complex database system. We now have 1/3 the number of tables that we did on Drupal, and the naming conventions make a lot more sense.