If you run most any type of services in a production environment, chances are, you want some sort of redundancy and load balancing. As an example, if you're running a web site, you want that to be up as much as possible, so generally a load balancer is used to send the load between different web servers. A common load balancer used is a big-ip F5. These have a great configuration tool and work very well, but they're also very expensive.

With RHEL 5, RedHat has included the IP clustering tools from the Linux Virtual Server project. Earlier this week I was evaluating using this for a project I was working on, and decided to write it up here a bit. While it didn't work the way I envisioned it to work for my use case, it's still pretty cool.

The RedHat configuration documentation can be found on the CentOS website, however I found this documentation to be kind of awkward. It may be because I'm hyper-impatient, or maybe it's something else, but here's what you need to do to get it working.

0) Install RHEL 5 with as few packages as possible. You just need the basics here, so the fewer, the better.

1) Install ipvsadm & piranha

yum install ipvsadm piranha

2) Run pirahna-passwd to create a password for the GUI

3) If you're not running selinux, you can skip this step, but I found I had to set some things up in selinux for pirahna to work properly.

First, the pirahna process needs to be allowed access to port 3636.

semanage port -a -p tcp -t httpd_t 3636

Next, run this command

setsebool -P httpd_disable_trans=1

Note: selinux will still complain when changing interfaces, but in my testing, it still worked fine.

4) Next, go to the web-gui and go to 'global settings', enter your main IP address. I didn't fully research the differences in the type of connection pool here, I just used direct for my testing. If you're deploying this in production, you'll want to do that research.

5) Click on 'redundancy' and enter the IP address of your second box. You may want to tweak the 'heartbeat'/'assume dead' intervals on this form, since an 18 second outage is a really long one.

6) Next, click on 'virtual servers'. This is where you do the main configuration for the virtual servers. Click the 'add' button at the bottom of the screen to add a new one.

a) Give the virtual server a name, this can be whatever label you want.

b) Enter the port that will be balanced.

c) Enter the IP address that will be the Virtual IP shared by both of the load balancers. This is NOT the IP of your web servers.

d) Enter the Linux device to bind the VIP to, if you're running more than one service, you'll need to manually change this.

e) Select your scheduling methods. You'll defiantly want to spend some time researching this since not all methods are created equal. These methods are also dependent on how your application behaves. As an example, a simple website doesn't care about state and users can be blindly sent to any node in the cluster. A website that has users logging in generally does care about state and if a user bounces between servers, the webapp doesn't like it very much.

7) Click the 'real server' tag and add the servers in your cluster. If you have more powerful servers in your cluster, you may want to give them more weight. As a backwards-thinking button, click the '(de)activate' button to activate the nodes.

8) Click on 'Monitoring Scripts'. This is a really important part of doing load balancing. More than likely, you'll want to write your own scripts. Anymore, simply doing a 'GET / HTTP/1.1' on a webserver simply isn't enough to determine if a host is up. You'd also want to check the database it connects to, or any other middle-ware, such as Tomcat or Weblogic, to see if there's problems. I didn't research this aspect of this solution that much, but, since it's just running a shell process, you could script most anything.

9) Finally, click on 'virtual servers' again, and click the '(de)activate' button to activate the virtual server you just created.

10) Copy the configuration in /etc/sysconfig/ha/lvs.cf to your second node. Depending on the method of your balancing, you may also want to copy your /etc/sysconfig/iptables rules over as well. The RedHat document goes into this further.

11) Start up the cluster software 'service pulse start', and monitor /var/log/messages for any errors. Once you see the message 'gratuitious lvs arps finished'.

12) Run ipvsadm -l to show the stats:

 IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.227.20:3128 dh
TCP 192.168.227.30:80 wlc

This method seems like a good inexpensive way to do load balancing. I found the tools worked well, but lacked some of the finesse of commercial tools offered by F5. I didn't spend a huge amount of time researching it, but I also found that some of the tracking tools available are lacking in this solution. But, at 1/10 the cost, it's certainly worth evaluating.