Configuring Drupal with multiple bins for memcached

In a recent article, we explained how to build memcached from source and  PHP memcache using PECL on Ubuntu Gutsy 7.10.

This article is a followup on how to configure memcache for Drupal, and how multiple bins help with performance.

Each bin in memcached correspond to one or more cache table in Drupal.

To do this, we first setup a start script for memcache that would start each bin with the correct size. This would go into /usr/local/bin/memcache.sh:

#!/bin/sh
case "$1" in
  start) /usr/local/bin/memcached -d -u root -m 64  -p 11211
    /usr/local/bin/memcached -d -u root -m 320 -p 11212
    /usr/local/bin/memcached -d -u root -m 128 -p 11213
  ;;
  stop)  killall memcached
  ;;
esac 

Then we need to change Drupal's settings.php to match that. You see that the page cache and filter cache each have their own bin, running on a separate instance of memcache. Everything else shares a "default" cache, which includes the menu cache and variables.

$conf = array(
  'cache_inc' => './sites/all/modules/memcache/memcache.inc',
  'memcache_servers' => array(
    'localhost:11211' => 'default',
    'localhost:11212' => 'page',
    'localhost:11213' => 'filter',
  ),
  'memcache_bins' => array(
    'cache'        => 'default',
    'cache_menu'   => 'default',
    'cache_page'   => 'page',
    'cache_filter' => 'filter',
  ),
);

Here are some statistics for each bin, after running for three weeks.

This one is for the default bin, which is used for the "cache_menu" and "cache" tables.

localhost:11211
Property Value
pid 7504
uptime 1898048
time 1209915881
version 1.2.5
pointer_size 64
rusage_user 255.863990
rusage_system 1472.936052
curr_items 2974
total_items 113719
bytes 42995914
curr_connections 147
total_connections 239905
connection_structures 426
cmd_get 26226172
cmd_set 113719
get_hits 26110949
get_misses 115223
evictions 0
bytes_read 2155610782
bytes_written 333186343217
limit_maxbytes 67108864
threads 1

The following bin is "page", and it is used for the "cache_page" table.

localhost:11212
Property Value
pid 7506
uptime 1898053
time 1209915886
version 1.2.5
pointer_size 64
rusage_user 440.247513
rusage_system 970.756668
curr_items 30107
total_items 6182196
bytes 240481330
curr_connections 146
total_connections 239471
connection_structures 423
cmd_get 16400465
cmd_set 6182196
get_hits 9604323
get_misses 6796142
evictions 192963
bytes_read 61544930270
bytes_written 92989069966
limit_maxbytes 335544320
threads 1

The filter bin is for the "cache_filter" table.

localhost:11213
Property Value
pid 7508
uptime 1898056
time 1209915889
version 1.2.5
pointer_size 64
rusage_user 904.692539
rusage_system 4203.874725
curr_items 185337
total_items 3453867
bytes 68096819
curr_connections 145
total_connections 230625
connection_structures 424
cmd_get 182240932
cmd_set 3453867
get_hits 128639599
get_misses 53601333
evictions 0
bytes_read 11149900389
bytes_written 51268296346
limit_maxbytes 134217728
threads 1

The page bin is the one that has the worst hits/misses ratio. The other two are very good. By separating the bins, we get better performance.

Update September 2010

Note that as of the 6.x-1.5 release of the memcache module, you do not not need to configure multiple bins. Only one bin is needed and memcache will handle the details internally.

This simplifies the configuration for memcached and Drupal considerably.

See #493448 for more details.

Contents: 

Comments

How does it help?

How is it better to run 3 processes totaling 512Mb rather than 1 big process at 512Mb? I dont understand how the hit/miss ratio could effect performance...

Contention not memory

Memory size is not the issue here. Contention is.

When cached items are inserted/deleted independently for each bin, without being affected by what the other bins are doing (locking, flushing, ...etc.)
--
2bits -- Drupal consulting

Multiple sites

What if i want to introduce multiple caches for multiple sites - and have each portnumber reserved for each site..

i have setup such and i am not convinced it does any good - maybe i dont understand the way i should setup the scripts - so each site gets his share.

$conf = array(
  'memcache_servers' => array('127.0.0.1:11202' => 'default'),
  'memcache_bins' => array('cache_page' => 'pages'),
);
$conf = array(
   // The path to wherever memcache.inc is. The easiest is to simply point it
   // to the copy in your module's directory.
   'cache_inc' => 'sites/all/modules/memcache/memcache.db.inc',
);

One of my site uses this instance
the next site uses : 127.0.0.1:11211

i do miss something, what is it?

Khalid, informative and

Khalid, informative and helpful as usual! I can't help but note that evictions for the first two are pretty high - you might need more memory for those bins, or is there a different reason?

--Martijn

cannot find - memcache.sh

I cannot find the file memcache.sh in my memcache installation on Centos. Would this file go by a different name now?

Please help.

Regards

Adrian

Create the script

It is not a standard file from your Linux distribution. It is a file that you have to create yourself, make it executable and place in /usr/local/bin.

However, things have changed since this article was written. You no longer have to configure multiple bins for memcache with the latest 6.x memcache module. It works well with a single bin.