Drupal Performance

We had a site for a client that was stable for close to two years, then suddenly started to experience switches from the master to the geographically separate slave server as frequently as twice a week.

The site is an entertainment news site, and its articles get to Google News on occasions.

The symptoms was increased load on the server, a sudden influx of traffic causing over 800 simultaneous connections all in the ESTABLISHED state.

Khalid of 2bits.com, Inc, was interviewed by Modules Unraveled, on Drupal Performance.

Together with Alan Dixon of Black Fly Solutions, Khalid Baheyeldin of 2bits.com, Inc. gave a presentation on Web Site Performance, Optimization and Scalability at Drupal Camp 2011.

The slides from the presentation are attached below.

Earlier today, I presented at DrupalCamp Toronto about 3.4 million page views a day, 92 million per month, one server and Drupal.

As promised, the slides are attached, as a PDF for everyone's reference.

A client site was facing intermittent outages, specially at peak hours.

We investigated the issue over a few days, and narrowed down the cause to certain slow queries, described in more details below.

They had tagadelic on every page, displaying a tag cloud, and from that, crawlers hit every term to paths like taxonomy/term/1581.

Slow queries causing server outage

One great feature that Drupal has is the ability to make modules run certain tasks, often heavy ones, in the background at preset intervals. This can be achieved by a module implementing hook_cron.

Core uses this feature to index new content for the search module, ping module to notify remote sites of new content, fetch new release information from drupal.org, poll other sites for RSS feeds, and more.

Various contributed modules use this for various purposes, such as mailing out newsletters, cleaning up logs, synchronizing content with other servers/sites, and much more ...

With the "Cloud" being in vogue currently, we see a lot of clients asking for cloud solutions, mostly Amazon AWS. Sadly, this is normally done without really doing a proper evaluation into whether the cost is reasonable, or the technology is suitable for their specific needs.

Amazon AWS provides some unique and compelling features. Among those are: instant provisioning of virtual servers, billing for used resources only, ability to provision more instances on demand, a wide variety of instance types, and much more.

Apache, and all other web servers, have a mechanism to write an "access log" recording every HTTP access to the server. The information that is logged is valuable, and includes things like the IP address of the user making the request, the date and time, the size of the request in bytes, the return code from the HTTP protocol, the request's URI, the referer, and the browser/operating system that the user is using.

One of the things that a default Drupal installation does is that it handles 404s for static files within Drupal itself. In other words, a 404 for a .css or .jpg file causes a full Drupal bootstrap. This is not necessary, and wastes resources on a site that gets lots of 404s: lots of code is executed, many database queries are performed, only to return a 404 for a static file.