This iterates over the directories in /var/www and runs Apache for each one, each with it’s own config file.

One interesting thing you can do here is make the user’s Apache process “nice” to fellow processes:

“nice” changes priority for tasks waiting on CPU, while “ionice” changes the priority for tasks waiting on IO (for example disk reads and writes).

This can keep one busy or misbehaving server from interrupting other sites or important background tasks; alternatively, you could set one site to be highest-priority.

All the work is in my local repo, but I will be pushing to github as milestones are hit.

I want to get the basics right, and not worry right now about competing feature-for-feature with the big guys:

• basic file management
• configure apache, stop/start server
• allow (and enthusiastically support!) plugins. I am using django/python, should be no problem.

That’s for the user-facing side, the backend takes care of scaling across multiple hosts (on-demand scaling), billing, activating/deactivating accounts, you get the idea.

What do you want to see in a free, open-source web hosting control panel? Leave a comment or feel free to email me – rhelmer@anyhosting.com – Thanks!

• I’ve been using it for a long time; familiarity
• very simple/powerful plugin system
• tons of users, so lots of examples and plugins already available

Nagios version 3 is provided in the Ubuntu repositories, and is quite simple to install:

The default config comes set up to monitor a set of services on localhost; I don’t really like the default Ubuntu/Debian setup of having one config file per host/service/etc, so on the master I’ve replaced the config file structure:

groups.conf contains the set of server types that I care about:

Note that the “http-servers” can define “members” (localhost in this case), however in general I do not add members in this file but instead in the hosts.cfg:

Note the “hostgroups” line; anyhosting1 is the physical server (this monitor is really checking on the reverse proxy), and example.com is a vhost (which is really proxying to a user running Apache for the “example.com” domain). These two checks make sure that the whole system is working and proxying correctly.

Finally, services.cfg brings it all together by defining which groups should run which services:

The Ubuntu nagios-plugins package (which by default is installed along with the nagios3 package) contains plugins that can intelligently check MySQL databases, disk space, load average, etc. By default these only work on the local machine, but these can be made to run on remote machines by installing the nagios-nrpe-server package. I will cover this further in a future blog post.

Our original goal was to provide websites for multiple users on one host, keeping them compartmentalized from each other, and to restrict break-ins. We chose to do reverse proxying using Apache, with separate Apache instances for each user, using mod_chroot.

Pros:

• user cannot see or read/write other users files, even on the same shared server, or use any system resources not explicitly provided in the chroot.
• mod_chroot is way less work than building a full chroot jail, and is potentially safer since you don’t have to provide system libraries or binaries, reducing your attack surface

Cons:

• multiple Apache instances consume more memory, and need special configuration and startup scripts
• it is possible to break out of a chroot. It is especially easy if the attacker can somehow get elevated to root (e.g. local security exploit)

Further investigation:

• apache-mpm-itk – This runs Apache child processes as a separate user per vhost. Cannot be combined with mod_chroot, unfortunately, and has some serious drawbacks (see the section on Quirks and Warnings).

• solaris containers, freebsd jail, linux vserver – These are real jails; less overhead than Virtual Machines, but provide more protection than a chroot. These tend to be significantly more complex to set up than something like mod_chroot, but are by design a lot harder to escape than a chroot.

There are some other interesting security-related modules such as mod_evasive that I am checking into further, which may mitigate some of the “cons” above.

Hopefully this will be useful to someone, however security is a process, not a product, and you should not rely solely on something like mod_chroot, but it can be a useful tool as part of an overall strategy that includes monitoring, intrusion detection, and regular security updates.

Ubuntu 8.10 Server

For each domain, create a virtual host config like /etc/apache2/sites-available/example.com:

This will be used by the Apache proxy, which is the normal system apache2 running as the default “www-data” user. This proxy handles name-based virtual hosts, and proxies the requests to a second process, running at localhost on port 8080.

Note that the logs are configured here and not in the user’s Apache process, for two reasons:

1. keep logs pristine in the event of a break-in on a user site (for example via a buggy or malicious PHP script)
2. single system-wide log file analysis process instead of per-user

Next, create a user account for each domain:

Create the following in /var/www/example.com/conf/apache2.conf:

Start up the user Apache process:

Everything should now work, but you may notice some problems with PHP or CGI scripts. Generally this means that you will need to make parts of the system available in the chroot; however making copies takes up a lot of space and can quickly get out of date, which is a security risk.

One workaround is to provide needed directories using the “mount -o bind” option, which will remount an existing, mounted part of the file hierarchy somewhere else. For example, this will provide all of “/usr” inside the chroot:

PHP sessions require /tmp to exist inside the chroot; there is no benefit to sharing this one, an empty directory is fine.

That’s pretty much it! Testing, getting startup right etc. is left as an exercise for the reader, since we’re venturing a bit out of the standard Ubuntu Apache setup. Entries in /etc/fstab and creating an init script to handle user Apache processes is probably the best way to go.

In part 3 we’ll cover take a more general look at the pros and cons to this setup, as well as possible future directions.

UPDATE link to secure shared hosting on ubuntu server part 3

EDIT 2009-Oct-03 2:25 PM Pacific – create dir before cding into it; add read-only (ro) option to usr mount; override “host” header in user Apache process

EDIT 2009-Oct-05 4:41 PM Pacific – load resolver library and install caching proxy dns server, needed by popular wordpress anti-spam plugin Akismet

At AnyHosting we want to have multiple users on one shared server, but protect them from reading or writing each other’s files (whether accidental or intentional) and also limit damage caused by a break-in via a poorly configured PHP script or CGI, or even an Apache exploit.

The solution here is to have each customer run their own Apache process, and do name-based virtual hosting and forwarding via a reverse proxy. The proxy is currently Apache but we are also considering more scalable alternatives such as nginx. An important thing to note here is that the proxy could be on a separate machine and also combine load-balancing into the mix, so it provides a lot of flexibility.

The classic way to chroot Apache (or any server process) is to copy the server process and all of it’s dependencies into the chroot jail, which makes keeping the files up to date not just a depressing chore but also a serious security hazard. Also it’s a ton of work to get right, especially if you want to support server-side scripting like PHP, since it depends on lots of system files and libraries.

Enter mod_chroot. This Apache module runs in the user’s process, and does the chroot system call after opening all the files it needs. The only caveats I’ve found (besides the ones on the mod_chroot caveats page) are some files that the Ubuntu PHP install assumes it can reach (such as the MySQL server socket, timezone info, and random number generator). I will cover this in more detail in part 2.

UPDATE link to secure shared hosting on ubuntu server part 2

If you’re looking for simple, secure and affordable web hosting, check it out!

I’m also working on a series of blog posts and articles about the new setup (reverse proxy, mod_chroot, on ubuntu server).

They seem to fall into three broad categories:

1. expensive proprietary packages
2. free open-source packages
3. custom software developed by hosting companies

The leader in #1 seems to be cPanel. I am not at all impressed with their demo, it looks very cluttered to me and I really want something simple, easy and aesthetically pleasing.

I have checked out all of the free/open-source web hosting control panels that Wikipedia lists, and I am also very displeased with the UI, and the code doesn’t look very easy to jump into to me. There seems to be a lot of custom code (I’d be happiest with something based on Django, but the PHP ones could at least use Cake). This makes me a little worried on the security aspect (XSS, SQL injection, etc.). Most of these code bases seem to be very old and not necessarily very active.

So is #3 the way to go? I’ve seen and heard about lots of great hosting control panels that only exist behind close doors, is this the big differentiator for modern hosting companies?

All of the major control panels like cPanel, Plesk, etc. are licensed at substantial fees, which cut into what you as a web host must charge each user. There are open-source alternatives such as ISPConfig and  GNUPanel, but this of course means that you will be taking on a lot more of the support burden, although you are of course free to make and keep any customizations or enhancements that you like, unlike cPanel or Plesk.

Besides licensing fees, the other snare to recognize is switching costs. As the LAMP (Linux, Apache, MySQL, PHP) stack has grown in popularity, it has become quite easy to move from one web host to another. Don’t like DreamHost? Move to Linnode. Don’t like them? Try RimuHosting.

However with the slick new cloud computing services like Amazon’s EC2 and Google’s App Engine, there is a ton of opportunity and also some things to watch out for:

• EC2 can run a regular Linux VM, but the management tools and other services like S3 (storage), queueing, billing etc. will not work out-of-the-box elsewhere
• App Engine lets you use the free and open-source Python programming language and the popular Django web framework, but you must use Google’s storage service. It can be used in a very SQL-like way, or hidden behind Django’s ORM however

I am going to continue using both Amazon and Google’s services, but I am being very careful about putting all of my eggs in one basket. There are some impressive updates in the pipeline, but you might want to think twice about letting any one company collect the tolls on your users.

You can read more about vendor lock-in and switching costs at Wikipedia.

I have heard and read a lot about App Engine, so I knew roughly what to expect, but I am still impressed with it. It is a very simple model, it’s basically CGI with a 10-second limit. Only the Python programming language is supported right now (although they plan to add more), and the Django web framework is pre-installed. There is a nice little SDK for running the environment locally, which I just noticed is open-source as well (Apache license).

The really incredible thing about this is that it runs on and takes advantage of Google’s massive server infrastructure. In-memory or persistent storage is super fast and easy to use, and no need to worry about redundancy of individual servers (this is probably why they use the CGI+shared storage model, way simpler to distribute applications on-demand).

Today the roadmap was updated to include a few very cool features coming later this year:

• Support for running scheduled tasks
• Task queues for performing background processing
• Ability to receive and process incoming email
• Support for sending and receiving XMPP (Jabber) messages

This environment being so easy to use and the cost being low due, which is likely because the price of hosting so marginal to Google (I imagine that they are effectively outsourcing spare capacity) plus these new features pretty much replace the need for a traditional shared or dedicated server.

They haven’t yet started charging for the service, but proposed pricing is available, and they plan to start charging this year. The price is quite low considering the feature set, is pay-per-use, and is comparable with the popular cloud computing service Amazon Web Services (AWS).

The difference between this and something like AWS is that while it is much easier to get from start to finish on Google App Engine, one must (likely) re-write your application in Python, using Google’s libraries. You’ve got less flexibility than a shared PHP host, for example; you can’t easily take your code elsewhere. AWS is on the other end of the spectrum, more like dedicated servers where you can install anything you want: Linux or Windows, PHP or .Net, etc.

In any case I highly recommend checking out Google App Engine, especially if you’re doing any new development. If you’re looking to move your existing servers to the cloud, then I think Amazon Web Services still has the edge here.