Update May 2013: We no longer recommend fcgid ever since Ubuntu Server 12.04 was released. This is because that version has PHP-FPM, which provides every benefit that fcgid has, with the added advantage of a shared opcode cache for all processes. We will be writing a full article on PHP-FPM with Drupal in the near future (soon to appear at High Performance Drupal with Apache MPM Worker Threaded Server and PHP-FPM).
Most sites that serve large Drupal instances use Apache with mod_php. Although this is often the easiest and fastest option, it may not be the ideal situation in some cases. This article explains fcgid, a way to run PHP other than mod_php, how to get it configured, and what are the advantages and disadvantages of it.
Apache's mod_php is the most widely used mode for PHP with Apache. mod_php itself is the entire PHP interpreter embedded in each Apache process that gets spawned. This provides performance and stability benefits, e.g.
- No need to call an external process (e.g. CGI).
- No need to communicate with another process via sockets (e.g. Fast CGI).
- The APC cache is shared by all Apache processes.
It also has some disadvantages
- The memory footprint per Apache process is large, specially when sites indulge in contributed modules.
- If Apache is serving static content, e.g. images and CSS files, it still has to spawn large processes because of the embedded PHP interpreter.
CGI (Common Gateway Interface) is the legacy way of runing applications on the web from the mid 1990s or so. It was too inefficient for anything but small sites. CGI spawns a new process for every incoming request to execute a PHP script, a very resource intensive and inefficient way of doing things. No wonder it faded away over time as web applications became more complex.
FastCGI was introduced to avoid some of the issues with running languages, including PHP, inside the Apache process, as well as avoiding the inefficiency of CGI.
A FastCGI application is executed outside of the web server (Apache or other wise), and waits for requests from the web server using a socket. The web server and the FastCGI application can even be on separate physical machines and communicate over the network.
Because the web server adn the application processes are separate better isolation is possible.
In reality, running PHP as mod_fastcgi with Apache has proved to be problematic. Mainly with stability. Even on Drupal.org we tried it for a while, but switched back to mod_php after some time.
mod_fcgid was introduced to be binary compatible with FastCGI, but with better control over spawning processes. The benefits of process isolation are still there.
The article assumes you are using Ubuntu/Debian. For Red Hat or Centos, use the equivalent packages and yum to achieve the same results.
First we install the required Apache components, with Apache threaded server (MPM Worker), to save memory.
aptitude install apache2-mpm-worker libapache2-mod-fcgid
Then we enable the Apache fcgid module.
And install PHP CGI, and the a few other PHP components, if they are not already on your system.
aptitude install php5-cgi php5-curl php5-gd php5-mysql
To configure fcgid, you have to do two things:
1. Create a new file in /etc/apache2/conf.d/php-fcgid.conf and put the following in it:
AddHandler fcgid-script .fcgi .php # Where to look for the php.ini file? DefaultInitEnv PHPRC "/etc/php5/cgi" # Maximum requests a process handles before it is terminated MaxRequestsPerProcess 1000 # Maximum number of PHP processes MaxProcessCount 10 # Number of seconds of idle time before a process is terminated IPCCommTimeout 240 IdleTimeout 240 #Or use this if you use the file above FCGIWrapper /usr/bin/php-cgi .php ServerLimit 500 StartServers 3 MinSpareThreads 3 MaxSpareThreads 10 ThreadsPerChild 10 MaxClients 300 MaxRequestsPerChild 1000
For a large site with a server with more memory and CPUs we can use this:
AddHandler fcgid-script .fcgi .php # Where to look for the php.ini file? DefaultInitEnv PHPRC "/etc/php5/cgi" # Where is the PHP executable FCGIWrapper /usr/bin/php-cgi .php # Maximum requests a process handles before it is terminated MaxRequestsPerProcess 1500 # Maximum number of PHP processes. MaxProcessCount 45 # Number of seconds of idle time before a process is terminated IPCCommTimeout 240 IdleTimeout 240# Large site ServerLimit 2048 ThreadLimit 100 StartServers 10 MinSpareThreads 30 MaxSpareThreads 100 ThreadsPerChild 64 MaxClients 2048 MaxRequestsPerChild 5000
2. Add ExecCGI to the Options line in the vhosts you want to use PHP and or fcgid on, for example.
Order Allow,Deny Allow From All Allow Override All Options MultiViews Indexes Includes FollowSymLinks ExecCGI
Now you should restart Apache, and fcgid should be active.
The first thing you observe is that the memory size of the fcgi process is considerably larger (40MB) than when PHP is running as a mod_php within Apache (31MB). However, when using fcgi, Apache (pre-fork) is only 3.3 to 2.4 MB per process so you can have more of them to serve static files. The savings are even more if you use the threaded server. For example, an Apache process on a large site would be 20 to 25 MB, and the total number needed for thousands of threads would be 10 to 15 processes total. You also normally have less CGI processes in total than Apache.
fcgi 15 processes * (40 MB - 21 MB shared) + 21 MB shared + (20 Apache procs * 6 MB) = 426 MB
mod_php 26 processes * (26 MB - 13 MB shared) + 13 MB shared = 351 MB
Apache threaded MPM Worker
fcgi 15 processes * (40 MB - 21 MB shared) + 21 MB shared + (10 Apache procs * 25 MB) = 290 MB
mod_php 26 processes * (26 MB - 13 MB shared) + 13 MB shared = 351 MB
The exact figures will vary for your site, depending on how you configured Apache and what modules you have enabled on your Drupal site.
Maximum number of processes
Setting the upper limit for the number php-cgi processes is possible but tricky. Normally MaxProcessCount should be the parameter to set to prevent creation of PHP processes above the number set for that parameter. I had to dig in the mod_fcgid source to find that parameter.
Make sure that you set this figure to a reasonable one. Setting it too low will cause an error to be logged in your web servers error log, for example:
[Sun Jul 25 17:06:23 2010] [notice] mod_fcgid: /home/example.com/www/index.php total process count 25 >= 25, skip the spawn request
On a 512MB VPS for example, a value of 7 to 10 seems to be adequate. For an 8GB dedicated server, a value of 45 or even higher will do. The exact figures will depend on how many modules you have, and how memory they consume.
Also, make sure that you do not set PHP_FCGI_CHILDREN at all. What happens is that fcgid uses it as a multiplier, not an upper maximum, and PHP will kept on breeding like rabbits, and the server will thrash and swap very quickly.
We conducted performance benchmarks comparing mod_php and mod_fcgid. The conclusion is that mod_php is a bit faster than fcgid, but not by much.
Our benchmarks how the following at different concurrency levels:
With 4 concurrent users fcgid Transactions: 826 hits Response time: 0.58 secs Transaction rate: 6.90 trans/sec Successful transactions: 826 Longest transaction: 2.13 Shortest transaction: 0.26 Apache mod_php Transactions: 853 hits Response time: 0.56 secs Transaction rate: 7.12 trans/sec Successful transactions: 853 Longest transaction: 0.95 Shortest transaction: 0.26 With 15 concurrent users: fcgid Transactions: 674 hits Elapsed time: 120.35 secs Response time: 2.65 secs Transaction rate: 5.60 trans/sec Concurrency: 14.83 Successful transactions: 674 Longest transaction: 7.28 Shortest transaction: 2.23 Apache mod_php Transactions: 748 hits Elapsed time: 120.42 secs Response time: 2.40 secs Transaction rate: 6.21 trans/sec Concurrency: 14.90 Successful transactions: 748 Longest transaction: 4.19 Shortest transaction: 1.80
Hunter Scott Newman confirmed the benchmarks independently, and sent us the following results. Note that he is using the threded Apache server with fcgid.
Apache Worker + fcgid 4 users Requests per second: 391.59 [#/sec] (mean) Time per request: 10.215 [ms] (mean) Time per request: 2.554 [ms] (mean, across all concurrent requests) 10 users Requests per second: 585.91 [#/sec] (mean) Time per request: 17.067 [ms] (mean) Time per request: 1.707 [ms] (mean, across all concurrent requests) 50 users Requests per second: 647.78 [#/sec] (mean) Time per request: 77.187 [ms] (mean) Time per request: 1.544 [ms] (mean, across all concurrent requests) Apache Prefork + mod_php 4 users Requests per second: 445.88 [#/sec] (mean) Time per request: 8.971 [ms] (mean) Time per request: 2.243 [ms] (mean, across all concurrent requests) 10 users Requests per second: 655.13 [#/sec] (mean) Time per request: 15.264 [ms] (mean) Time per request: 1.526 [ms] (mean, across all concurrent requests) 50 users Requests per second: 670.00 [#/sec] (mean) Time per request: 74.627 [ms] (mean) Time per request: 1.493 [ms] (mean, across all concurrent requests)
Benefits and advantages
There are several advantages for using fcgid. Performance is not one of them, but the penalty may be well worth it for the following advantages:
- Less memory consumption. Apache processes that serve static files are very small (3MB or less). So we can have more of those in memory.
- Less network connections active, since there are less PHP process
- Less database connections active, since there are less PHP processes. This is a very important point, since MySQL performance drops as the number of connections increase.
Case study: Large site with fcgid
In order to show what fcgid can help with, consider the following graphs. See the before and after comparison.
Memory usage by day, before and after.
The same was true over a week too.
And the number of MySQL threads went down dramatically, because only PHP processes will connect to it, and not every Apache process.
And also the number of network connections inside the server.
And just to make our lives easier, we wrote a monitoring script that will plot the number of php-cgi processes on a graph. We can know what is going on and what was going on too.
Caveats and precautions
Not everything is rosy though. There has to be some drawback.
APC still stores the op-code caches for Drupal and its modules. It has to do the parsing more often though, unlike on mod_php.
Because of that, you may on occasions see errors such as "Fatal error: Allowed memory size of nnnnnn bytes exhausted (tried to allocate nnnn bytes)". Reloading the page causes it to load normally.
Although we have not dug too deep into it, we have found is that APC is not shared among the processes. And because the php-cgi processes are killed and new ones spawned, the persistence of APC's user/data cache cannot be relied upon. This is not important for most users, but if you are using things like the Drupal cache router module or the performance logging module, this will impact you.
If pure speed is what you are after, then stay with mod_php.
However, for better resource usage and efficiency, consider moving to fcgid.
Here are some other links on fcgid. Note that some of the configurations mentioned recommend the use the PHP_FCGI_CHILDREN and other settings that proved disastrous. They also do not mention the MaxProcessCount parameter which we found from the source code.
- Typo3: Using PHP with mod_fcgid.
- How to setup Apache2 with mod_fcgid on PHP5 and Debian Etch.
- FastCGI with PHP opcode cache.
- Installing Apache 2 and PHP5 using mod_fcgid (currently offline, Google Cache here)
- Memory usage Apache + PHP as module versus FastCGI (on Windows), and comments on it.
Update: Article updated to use Apache threaded server (MPM Worker) instead of the pre-fork server. This provides lots of memory savings.
Benji (not verified)
Cautiously switching from mod_php to mod_fcgidWed, 2010/12/29 - 22:25
First of all, thank you for this article, and for your many helpful posts on Drupal.org. I decided to run Drupal on my own server, and it helps to get advice from someone who (unlike me) knows what he is doing.
I run a few low-traffic sites on a small virtual server (512 MB of RAM). I probably enable more Drupal modules than I need, and so I set memory_limit to 96 MB in php.ini. As a result, I started running into trouble: too many apache processes eating up my RAM, swapping, ugh. I set MaxClients to 17 (apache prefork) and decided to switch to the worker MPM and fast CGI as a better solution.
I followed your instructions, and I think I made some improvements. I would love to know what you think. One of my goals was to make the switch cautiously, and with as little down time as possible.
First, a question. What is the meaning of the comment
#Or use this if you use the file above
in your php-fcgid.conf ?
My first step was to keep mpm-prefork but switch to mod_fcgid. I created php-fcgid.conf by copying from this article and added "Options +ExecCGI" to all my vhost files. I wrapped the apache directives for the worker mpm in <IfModule mpm_worker_module>...</IfModule> (see below) so that they would not take effect during this step. Then
(Did I mention that I am running Ubuntu? I upgraded to 10.04 LTS yesterday.) It all went smoothly, with almost no down time. I could have skipped the a2enmod step, because apt-get took care of that for me.
The second step was easy: switch from prefork to worker.
Again, apt-get took care of things (stopping apache and then starting it again) for me.
Now for the changes I made. First, in php-fcgid.conf, I made separate sections for the fcgid, mpm_prefork, and mpm_worker directives. Next, I checked the Apache docs and found that the fcgid directives all have new names: the ones you use still work, but they are deprecated. As a bonus, I found the documentation for the FcgidMaxProcesses directive (formerly MaxProcessCount). As a result, my php-fcgid.conf file (including some comments and an extra ServerLimit directive for the worker mpm cribbed from the main apache2.conf file) looks like this:
I know, more comments than I need. You will notice that I commented out the PHPRC directive and that my PHP wrapper is actually a wrapper and not the cgi executable. Not a big deal, but I thought that I might also use fcgid for Perl or some other language, and they do not need the PHP environment variables.
I remembered to make the PHP wrapper executable.
One last change. Further down the page in the Apache docs, it says that one should set the environment variable PHP_FCGI_MAX_REQUESTS to be at least as large as FcgidMaxRequestsPerProcess. Since PHP_FCGI_MAX_REQUESTS defaults to 500, this might explain some of the "Fatal error" messages you mention at the end of your article. If you test this, please let us know! Again, I could have used another FcgidInitialEnv directive to set the environement variable, but since I decided to use a wrapper, I put it in there. Here is my wrapper, mostly cribbed from the URL in its first comment:
Chris Herberte (not verified)
Awesome postWed, 2011/01/19 - 20:43
I have recently implemented fcgid+worker to alleviate excessive memory usage which was causing server stability issues on a cPanel VPS. All sites are Drupal, as multi-site (shared core and contributed module code base). I have no stats but seems quicker than DSO. So far, so good - swapping no more.
Question: currently I have disabled SUEXEC and wondering if by enabling it there will be any negative impact to the benefits of shared code base. eg; are additional PHP processes spawned due to running SUEXEC (compiled per user) or does it work the same as without?
Hope this makes sense.
Akshar (not verified)
Fcgid some instalation problemsMon, 2011/05/30 - 03:21
I am trying to install Fcgid but, after installing if i try to run any application it gives me "Internal Server Error", also iam unable to find the file where i should add "Order Allow,Deny
Allow From All
Allow Override All
Options MultiViews Indexes Includes FollowSymLinks ExecCGI". Also, when i run "aptitude install apache2-mpm-worker libapache2-mod-fcgid" i get this message "The following actions will resolve these dependencies:
Remove the following packages:
". Please let me know if iam doing anything wrong. Thanks in advance
JK (not verified)
From my understanding, your'Wed, 2011/10/05 - 17:09
From my understanding, your' saying that is not possible to use APC with fcgid because the opcode wont work ?
Yes, and noWed, 2011/10/05 - 17:21
Yes, and no.
It will not work with one opcode shared memory segment for all the fcgid processes, but it will work with each process having its own code.
So total memory usage is more, but it does work out well in the end in most cases.
JK (not verified)
Thank you for yourWed, 2011/10/05 - 17:27
Thank you for your response!
Do you have an article or example of how to do that ?
I just found this http://www.brandonturner.net/blog/2009/07/fastcgi_with_php_opcode_cache/ but is using mod_fastcgi instead mod_fcgid .
thank you for your time.
fcgid and apcWed, 2011/10/05 - 17:37
Configuring fcgid itself and Apache are covered in detail above in the article we are comment on right now.
For APC, you need the following, e.g. in /etc/php5/conf.d/apc.ini for Debian/Ubuntu:
extension = apc.so
apc.shm_size = 96
apc.rfc1867 = 1
That is all.
JK (not verified)
Thank you! I have fcgidWed, 2011/10/05 - 17:42
Thank you! I have fcgid installed, I just needed the APC part, I thought it was more dificult hehe.
wipeout (not verified)
I am not finding this to be a great setup with multiple sites..Tue, 2011/12/13 - 09:41
Thanks for the article...
I have switched from php-cgi to using mod_fcgid to get better performance but its using far more memory because I am not running a single site on the server..
Each site is spawning its own php processes and these processes aren't shared between sites even though they are all running running as the "www-data" user.. So if I go to site1.com a new php process is started and uses about 30MB of memory.. Then I go to site2.com and another 30MB of RAM allocated to a second process.. So by the time I have visited 5 sites on the server I have chewed up ~150MB of memory in php processes..
I have set FcgidMinProcessesPerClass 1 so additional processes per site are killed off but I am still sitting with 1 running per site rather than the php processes being shared between sites and spawning and killing additional as needed.. Setting FcgidMinProcessesPerClass 0 seems pointless because it would effectively then be similar to running php-cgi when php processes are spawned and killed as needed anyway and still wouldn't solve the high memory usage issue when all sites are being accessed because its not "pooling" the php processes..
If you have any thoughts on how this can be resolved so that running php processes are shared between sites please let me know..
Yes, and No ...Tue, 2011/12/13 - 13:54
This setup is used on mainly dedicated servers (4GB or more), or VPS's with sites with a low number of modules (we do use it on 512MB VPS's with no issues.
We do use Ubuntu Server, and the setup we described works well without specifying per class and all that.
I don't think that a PHP process cares which site it is serving per se. But perhaps not spawning a new process is depending on keep alive?
For fcgid (or any FastCGI), the main issue is that the APC code cache is not shared across processes. So it has to replicate this cache for each process, unlike mod_php, which shares the cache. The size of each process is dependent on the number of modules that are used, whether they are true multisite (i.e. sites/all/modules is the same for all sites or not), and the APC cache.
Using less modules will reduce the memory footprint of each process.
You can find out how much APC is using if you use the apc.php script and tune it accordingly.
You need to find the sweet spot where FcgidMaxProcesses is set to a value that does not overflow memory, yet have enough processes to serve the site efficiently. For large sites on dedicated machines, we use 35 or 40. Less for smaller servers with constrained memory.