Using ApacheBench for benchmarking logged in users - an automated approach

A recent blog post by Ezra Gildesgame on benchmarking Drupal got me rolling with some code laying around for ages. The instructions provided in the aforementioned article do indeed work, but there has to be a better way for automating benchmarking of Drupal removing the repetitive or manual steps needed.

Ensure proper cookie name

In the most recent Drupal 5.x releases, the cookie name has been changed to contain a has value, so that it is unique. Because benchmarking is mostly conducted on test servers not on live sites, we need to make sure that the cookie value is valid. More often than not, you will be taking a copy of the site and conducting the test on it.

The code to generate the cookie name is a bit complex, but the most straightforward way on a test copy, is to make sure that the cookie name is valid by adding a host name on your test copy of the site.

To do so, add the following line in settings.php.

ini_set('session.cookie_domain', 'test1.example.com'); 

Replace the host name with the name of the host you are using (e.g. localhost ...etc).

Use a script to extract the sessions

Now that we have a cookie that will work, we can extract the sessions from the database, and create the ApacheBench commands we need, without mucking around in the browser checking our cookie.

Moreover, we can generate multiple user streams, and not just one.

Copy the following code somewhere outside of your document root, and name it something like ab-logged-in.php.

<?php
// Script to generate ab tests for logged in users using sessions
// from the database
if ($argc != 4) {
$prog = basename($argv[0]);
print "Usage: $prog url concurrency num_requests\n";
exit(1);
}
$url = $argv[1];
$num_curr = $argv[2];
$num_reqs = $argv[3];
require_once('./includes/bootstrap.inc');
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
$result = db_query('SELECT sid FROM {sessions} WHERE uid > 1 LIMIT %d', $num_curr);
while($row = db_fetch_object($result)) {
$cookie = session_name() .'='. $row->sid;
print "ab -c 1 -n $num_reqs -C $cookie $url &\n";
} 

Now, you can run it like this:

$ cd /your/drupal/directory
$ php /dev/ab-logged-in.php http://test1.example.com/ 2 200

You are asking it to generate two users, with 200 requests each.

The commands you need to run are displayed for you so you can copy and paste:

ab -c 1 -n 200 -C SESSd26f42dd28b1e33d9ab70260052d538b=4497ec4d30f64b7c5213580ce259eb74 http://test1.example.com/ &
ab -c 1 -n 200 -C SESSd26f42dd28b1e33d9ab70260052d538b=d857649ac81db7678bfb8f12ae856708 http://test1.example.com/ & 

But there is more ...

Making it automated

You can take this a step forward, and not having to cut and paste anything: the script can run the commands for you!

So, you can do this:

$ php /dev/ab-logged-in.php http://test1.example.com/ 2 200 | sh > logged-in.txt

Which pipes the output from the php script to the shell, and captures the output in a file.

If you check the file via grep:

$ grep Req logged-in.txt  

You can see the number of requests per second for each user.

Requests per second:    5.40 [#/sec] (mean)
Requests per second:    5.37 [#/sec] (mean) 

Comparing against anonymous users

If you want to compare that against anonymous users, you can run the commands above without the -C option, and log the output to a file and compare.

The following commands will do this for you:

cat > anon.sh
ab -c 1 -n 200 http://test1.example.com/ &
ab -c 1 -n 200 http://test1.example.com/ & 
Ctrl-D
sh anon.sh > anon.txt
grep Req anon.txt

And the output is something like:

Requests per second:    187.54 [#/sec] (mean)
Requests per second:    173.29 [#/sec] (mean)

So there you have it, fully automated, with comparison too.

Enjoy ...