This how-to will show you how to configure:

• An Apache 2 web server using virtual hosts
• The ITK MPM allows each virtual host to serve requests as its own user/group
• mod_ssl to serve pages over the secure HTTP (HTTPS) protocol
• mod_security to help prevents everything from SQL injections to data leaks
• mod_php for PHP scripts along with mod_suhosin to help protect mitigate risks from known and unknown flaws in PHP scripts

Rebuilding httpd for ITK

About privilege separation

By default, the Apache web server runs as the 'apache' user. This is good because a successful attack on the web server will only cause limited damage to the system, as they do not have root access. However on a shared servers which hosts multiple websites, an attack can still be very dangerous because the 'apache' user is used by all of the websites. Thus, an attack on one website can potentially grant access to any other websites that are being hosted on the system!

Privilege separation is a technique that can be used to mitigate the risk of an attack against a shared hosting server. By allowing webpages to be served by different users, each host can be assigned its own user and so the damage from a site hack will be limited to just that account and not all of the accounts hosted on the system. We will be using the ITK Apache MPM to achieve this.

Rebuild process

Unfortunately, the ITK MPM is not included in the stock httpd distribution. Fortunately, it is relatively easy to add it. Run this as your regular user to install the httpd source RPM and download the ITK patch sources:

With that done, let's make a few quick modifications to the RPM spec file located at ~/rpmbuild/SPECS/httpd.spec.

We will first need to add the ITK patches. Find the last patch line in the RPM spec file, for example, Patch202: httpd-2.2.3-deflate2215.patch, then add after it:

Now in the %pre section - after the Red Hat patches are applied - add the following lines:

This will copy the (fully Red Hat-patched) prefork MPM code into a new folder and then apply the ITK patches.

In the %build section of the spec file, you will see these lines:

Following the same format, add a line for the ITK MPM:

Similarly, we find in the %install section:

Add a line for the ITK MPM:

Last of all, in the %check section there is another section were we need to add in the ITK MPM:

Change the middle line to read:

We are now ready to rebuild our ITK-enabled httpd package.

Install the dependencies listed and then re-run the rpmbuild again if necessary. After the build has finished, install your new RPMs:

Remember to replace [arch] in the second to last command with the appropriate value (most probably i686 for 32-bit machines or x86_64 for 64-bit machines). The last step is to set the httpd worker to the ITK binary by editing /etc/sysconfig/httpd and adding after the line #HTTPD=/usr/sbin/httpd.worker:

Installing add-on modules and additional software

Let's install PHP, AWStats and a few other useful add-ons:

Keep in mind that by default, mod_security's rules are very restrictive and you will almost certainly need to tweak them. Be sure to test thoroughly before enabling mod_security on your live server. You can modify the mod_security settings by editing the rule files in /etc/httpd/modsecurity.d.

Configuring the web server

You will find in a stock installation, no indexes are permitted at all and that visiting localhost displays the standard CentOS test page. This can be changed by editing /etc/httpd/conf.d/welcome.conf and comment out the entire LocationMatch section:

Next, let's add a small configuration file with some custom settings. Below we will try and prevent information leaks by restricting access to backup files (files ending with a "~"), .sql and .inc plaintext files which could potentially reveal critical information about how a site functions. We will also add a one-line include directive so that all files in the virtual hosts configuration directory get included as well.

The following will initialize a template VirtualHost configuration that can be used by a script to automate the installation of new VirtualHosts:

Similarly, we will setup a awstats template that can create configuration via a script:

(For those wondering, this template was derived by removing all comments from the stock awstats model configuration at /etc/awstats/awstats.model.conf and then substituting important values such as SiteDomain with variables so that can be easily changed via sed)

Because the web server will be using the ITK MPM, each site's requests will be run under its respective owner and group. This causes problems with the standard PHP session setup, which uses one directory owned by "apache" for all session information. To solve this, a session directory will need to be created for each user. Let's create the default one for the apache user now:

You are now ready to add VirtualHosts for your domains. See Adding a new web hosting account section of Administering the server below for more information on how to do so.

The last step is to add the firewall port exceptions for http/https and start Apache:

Suhosin PHP extension

We will now be configuring /etc/php.d/suhosin.ini. Because suhosin can interfere with some site's functionality, it is best to enable it in simulation mode so that errors are logged and you can configure suhosin accordingly:

Limit the maximum PHP memory_limit that scripts can set:

Lastly, disable session encryption (breaks roundcubemail if enabled):

Optional add-on: roundcubemail

Let's rebuild roundcubemail 0.3 for CentOS 5:

Edit the spec file roundcubemail.specand comment the line Requires: php-pear-Mail-mimeDecode so that it reads #Requires: php-pear-Mail-mimeDecode. This is the only change required for CentOS 5, so let's build the package now:

Now it is time to configure roundcubemail. In the file /etc/httpd/conf.d/roundcubemail.conf, add after the line :

As well, by default roundcubemail denies access to any non-localhost clients. To change this, comment out Deny from all and add Allow from all:

We will also need to create the roundcubemail database. If you have not setup your database server, follow the database tutorial now.

Then configure /etc/roundcubemail accordingly. You should now be able to access the webmail portal at http://www.yourdomain.com/roundcubemail.

But I want PHP 5.2!

PHP 5.2 is available in the CentOS 5 Testing repository. Certain PHP add-ons, such as php-suhosin, have not been rebuilt so you will need to do that manually.

Now rebuild and reinstall php-suhosin:

Now install the resulting php-suhosin binary RPM and then move the configuration file (/etc/php.d/suhosin.ini.rpmsave) back into place.

But I want PHP 5.3!

PHP 5.3 is available as of CentOS 5.6 the php53 package. Certain PHP add-ons, such as php-suhosin and php-mcrypt have not been rebuilt so you will need to do that manually. This is a bit tricky as you will need to rebuild the php-extras SRPM among others; I'll leave you to figure out the details. See the instructions above for PHP 5.2 for a rough workflow using php-suhosin as an example.

Some final words

Two 3rd party MPM modules satisfied my need privilege separation in this setup, the ITK MPM and the Peruser MPM. Each has its own advantages and disadvantages.

ITK's approach is to accept a request, determine which virtual host the request belongs to, fork, set the UID and GID of the fork, serve the request and then to terminate the fork after the request has been completed allowing it to fork again later. Because ITK is very simple in nature and functions similarly to the traditional prefork MPM, it is more compatible with modules like mod_ssl, mod_php and mod_python. That said, it is also slower than because of the fork/kill overhead associated with each request.

Peruser, on the other hand, starts a pool of processes that fork only on startup. There are multiplexers than accept requests, determine which virtual host the request belongs to, and then forwards the request to the appropriate process from the pool. This approach incurs practically no overhead as the process from the pool has already been forked and had its UID/GID changed. Peruser is therefore much faster than ITK; its performance is nearly almost on par with that of the traditional prefork MPM! The downfall of Peruser is that because of this connection passing from the multiplexer to a preforked process, it is more complex and has had a long history of bugs and incompatibilities. Experimental support for mod_ssl was only recently added, and upstream development still seems to have a number of bugs to work out. In addition, because Peruser requires at least one process per virtual host for the pool and then requires multiplexers to handle incoming requests, it is much more resource intensive that the ITK MPM, where a process can dynamically handle a request from any virtual host. For these reasons, I favoured ITK over Peruser.

Keep in mind that ITK does have one security flaw. The request headers must be processed in order to determine which virtual host that request belongs to, and therefore which UID/GID ITK would need to switch to. Because of this, all of the header processing is performed as root (albeit with restricted POSIX capabilities). Should someone be able to take advantage of a flaw in Apache or another module (e.g. mod_ssl), the server could theoretically be rooted.

There is an emphasis on theoretically because an attacker would need to be extremely clever (or lucky) to successfully root the server via a flaw in the request processing given the restricted POSIX capabilities. In most cases, that specific apache process would just crash and life goes on. All in all, I still consider the ITK MPM to be more secure than the standard prefork MPM because a much more likely attack on the server is a hacker breaking in through a flaw in one of the hosted website's scripts (e.g. an outdated CMS installation). In this scenario the ITK MPM protects you completely; the potential damage from the attacker is essentially limited to that user's home directory, since each user's scripts run as their own UID/GID. They do not get access to any other user's scripts, nor to any other user's emails.

Administering the server

Setting up the scripts

The following code will setup the "web_domain_add" script which can be used to create the initial website configurations for a hosting users on your server:

Adding a new web hosting account

In order to create the new website configurations, you must also create a new system user that the VirtualHost will be mapped to via ITK. See the SSH+SFTP tutorial for details on how to add a new restricted system user. Once you have done so, you can run the web_domain_add script:

This will initialize a new configuration for the domain primarydomain.tld mapped to system_username. You may optionally specify as many domain aliases as needed, for example:

www.example.com will be used as the primary domain, and the script will automatically alias example.com as well as www.example.net, example.net, www.example.org and example.org as domain aliases for www.example.com.
Note: The script does not create any DNS entries for these domains! That task must be performed separately with your DNS provider.

Resources and further reading

• The Apache Wiki page on privilege separation
• The peruser MPM homepage
• The ITK MPM homepage
• Stuart Herber's post on securing an Apache shared hosting server using the ITK MPM
• Roman Barczyński's itk vs peruser performance comparison
  
• My own quick performance comparison of prefork/suphp/ITK
• PHP manual on (the now deprecated) Safe Mode