Installing Varnish 3.x on Ubuntu Server 12.04 LTS and using Munin to monitor it

Ubuntu Server 12.04 LTS finally provides a stable long term support server distro that has a recent version of Varnish in its repositories.

Trouble is, the repository provided package of Varnish has some issues. Specifically, the command line tools, such as varnishhist, varnishstat, ...etc. do not report anything. Therefore one cannot know the hit/miss rates, hits per second, or other useful information. Moreover, monitoring Varnish using Munin for such statistics does not work either.

There are two ways you can overcome this, both are described below.

Use the Varnish provided packages from their repository

The Varnish project provides an Ubuntu repository that contains Ubuntu packages. This is the recommended way, because you get updates for security if and when they come out. As well, you will get startup scripts installed and configured automatically, instead of having to create them from scratch, as in the second option.

First, install curl, if it is not already installed on your server:

# aptitude install curl 

Then, add the repository key to your server:

# curl http://repo.varnish-cache.org/debian/GPG-key.txt |
    sudo apt-key add -

And then add the repository itself.

# echo "deb http://repo.varnish-cache.org/ubuntu/ lucid varnish-3.0" | 
    tee /etc/apt/sources.list.d/varnish.list

Now, update the list of packages from the new repository:

# aptitude update

And then install Varnish

# aptitude install varnish 

Configuring Varnish listening port and memory

Then, you need to configure Varnish to front end your web server, so that it runs on port 80 (regular HTTP traffic) and 443 (SSL secure traffic, if your site uses it).

Also, increase, or decrease the amount of memory allocated to Varnish if your server has memory to spare.

We also change the name of the .vcl file, so when upgrading, Ubuntu will not ask about two files changed, and rather one file only (/etc/default/varnish).

So, go ahead and edit the /etc/default/varnish file, and this to the end, if you have a small server (e.g. 1G RAM).

DAEMON_OPTS="-a :80,:443 \
             -T localhost:6082 \
             -f /etc/varnish/main.vcl \
             -S /etc/varnish/secret \
             -s malloc,128m"

If you have a larger server, e.g. 8GB, you would allocate 2G of RAM for varnish.

DAEMON_OPTS="-a :80,:443 \
             -T localhost:6082 \
             -f /etc/varnish/main.vcl \
             -S /etc/varnish/secret \
             -s malloc,2048m"

Then restart Varnish:

service varnish restart

In an upcoming article, we will discuss the details of configuring Varnish's VCL.

Compiling from source

The second option is to download the source code, and compile it yourself.

This means that you are responsible for upgrading if there is a security fix.

You first need to install the C compiler, and some other libraries like curses development, PCRE, as well as the make utility.

You do this via the command:

# aptitude install libncurses5-dev libpcre3-dev pkg-config make 

Which will pull the C compiler if it is not already installed.

Then, download the source:

# wget http://repo.varnish-cache.org/source/varnish-3.0.3.tar.gz

Extract the source from the archive

# tar xzf varnish-3.0.3.tar.gz

Change the directory to what you just extracted

# cd varnish-3.0.3

Run the configure tool. Make sure there are no errors.

# ./configure

Build the binaries

# make

And install the source.

# make install

Varnish will be installed in /usr/local.

You will need to create start scripts for Varnish.
You should use a different name other than what would have been installed by the repository packages, so that it would not clash with the same file names, e.g. /etc/default/varnish-local. This would hold the DAEMON_OPTS mentioned above. You also need to create an /etc/init.d/varnish-local script for startup. I then use the following command to make Varnish run at run level 2.

update-rc.d varnish-local start 80 2 .

Monitoring Varnish with Munin

We assume that you have Munin installed and configured and is already monitoring other things on your server.

We need to install a perl library that will be able to pull the statistics info from Varnish

aptitude install libnet-telnet-perl

Then, we need get the monitoring scripts

cd /usr/share/munin/plugins/

git clone git://github.com/basiszwo/munin-varnish.git

chmod a+x ./munin-varnish/varnish_*

Then create symbolic links for the monitoring scripts.

cd /etc/munin/plugins

ln -s /usr/share/munin/plugins/munin-varnish/varnish_allocated
ln -s /usr/share/munin/plugins/munin-varnish/varnish_cachehitratio
ln -s /usr/share/munin/plugins/munin-varnish/varnish_hitrate
ln -s /usr/share/munin/plugins/munin-varnish/varnish_total_objects

Then edit the file /etc/munin/plugin-conf.d/munin-node, and add the following to the end.

[varnish*]
user root

And restart Munin for the changes to take effect.

service munin-node restart

Configuring Varnish for Drupal

As mentioned above, in an upcoming article, we will discuss the details of configuring varnish for Drupal.