High-Availability Load Balancer setup

heartbeat and ldirectord are not session-aware, so you need HAProxy.

Yes.

 

Thanks Falko. I'll be building it this week. on my two servers.

 

Hey Falko

Is there any hope that "High-Availability Load Balancer (With Failover and Session Support) With HAProxy/Heartbeat" on Ubuntu 8.04 server could be done WITHOUT using two virtual nodes, but two physical servers?
I've been working all week with mine setting up two virtual servers using Xen, and it's not working well. I have them set up, and running, but I don't feel very confident in them yet. I would much rather use "real" servers, but I only have two boxes.

 

It doesn't matter if you use virtual or physical systems.

 

I realize it doesn't matter, I was just wondering if there was a way with just two computers to have the load balancer, and the web servers? Like I said, I don't feel very confident with my virtual servers I set up.

 

Should be possible with just two nodes, although I didn't try it.

 

I've gone the Xen route with 2 virtual servers on one machine. I had to combine three different how-to's to install Xen with a load balancer. I changed the load balancer to HAProxy from your how-to, then installed the server using the Perfect server 8.10 with ISPConfig 3. I had to change that to 8.04 as well. It's kind of working.

I was able to log into http://192.168.31.100/haproxy?stats (that's the shared local address) before, but now I can't. I'm messing up config files somewhere.

So far I only have one machine built with the load balancer/web server setup. I can access a test website from the Internet, and I can log into ISPConfig from both local and Internet. Not being able to pull up HAProxy stats page now make me wonder if I really messed up the installation. If I stop both HAProxy, and Heartbeat, shouldn't I NOT be able to access the test website and ISPConfig?

 

That's right. Are you using the correct IP to connect to HAProxy, or are you using the IP that is used by ISPConfig and the test web site?

 

I'm not sure actually. I feel as though I didn't set this up correctly. I was setting this up using several different how-to's for guidance. I'm using Xen, heartbeat, HAProxy, on the load balancer node. ISPConfig 3 and your how-to for 8.04 on the web server node. I didn't end up using the public IP address in any part of the setups that I can remember. I can make my own local IP's fit whatever is needed, but I am assigned 5 public IP's from my ISP. One of them was used by my old server setup as Bind the nameserver. Currently I have my router set providing netmask 255.255.255.0, network 192.168.31.0, and gateway 192.168.31.1. I set up the Xen master, and the two virtual servers like this:
server1.tlthost.net : 192.168.31.200 (Xen DOM0)
lb1.tlthost.net : 192.168.31.201
web1.tlthost.net : 192.168.31.202

Second server will be:
server1.tlthost.net : 192.168.31.210
lb2.tlthost.net : 192.168.31.211
web2.tlthost.net : 192.168.31.212

www.tlthost.net: 192.168.31.100 (as the shared IP)
In your how-to, you say: "The shared (virtual) IP address is no problem as long as you're in your own LAN where you can assign IP addresses as you like. However, if you want to use this setup with public IP addresses, you need to find a hoster where you can rent two servers (the load balancer nodes) in the same subnet; you can then use a free IP address in this subnet for the virtual IP address." Is this where I'm going wrong? Should the shared IP address be the public one I use for my websites?

Now after going over my setting for my test site in ISPConfig 3, I don't see anything wrong there. I can't get to my test site as of today through the Internet, just locally. I checked, and HAProxy/heartbeat are both running, so I know somewhere something is set wrong.

 

Can you access the web site and the HAProxy stats page on 192.168.31.100?

 

No. I tried last night switching the IP in /etc/haproxy.cfg to my public IP. That didn't work either.

I can get to the website when I connect to the local net, and type in the URL. I also can access ISPConfig on the web server by http://192.168.31.100:8080/

Now I went back to setting the common IP as a local address of 192.168.31.100. If I look at ip addr sh eth0
I get:

Code:

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:16:3e:78:98:46 brd ff:ff:ff:ff:ff:ff
    inet 192.168.31.201/24 brd 192.168.31.255 scope global eth0
    inet 192.168.31.100/24 brd 192.168.31.255 scope global secondary eth0:0
    inet6 fe80::216:3eff:fe78:9846/64 scope link
       valid_lft forever preferred_lft forever
root@lb1:~# ip addr sh eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:16:3e:78:98:46 brd ff:ff:ff:ff:ff:ff
    inet 192.168.31.201/24 brd 192.168.31.255 scope global eth0
    inet 192.168.31.100/24 brd 192.168.31.255 scope global secondary eth0:0
    inet6 fe80::216:3eff:fe78:9846/64 scope link
       valid_lft forever preferred_lft forever

As you see the server address of .201 is there, and the common .100. as well. My /etc/haproxy.cfg looks like:

Code:

global
        log 127.0.0.1   local0
        log 127.0.0.1   local1 notice
        #log loghost    local0 info
        maxconn 4096
        #debug
        #quiet
        user haproxy
        group haproxy
defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        retries 3
        redispatch
        maxconn 2000
        contimeout      5000
        clitimeout      50000
        srvtimeout      50000
listen webfarm 192.168.31.100:80
       mode http
       stats enable
       stats auth someuser:somepassword
       balance roundrobin
       cookie JSESSIONID prefix
       option httpclose
       option forwardfor       
       option httpchk HEAD /check.txt HTTP/1.0
       server web1 192.168.31.202:80 cookie web1 check
       server web2 192.168.31.212:80 cookie web2 check
       #server backup 127.0.0.1:80 backup source 0.0.0.0

/etc/network/interfaces for LB1:

Code:

auto lo
iface lo inet loopback
auto eth1
iface eth1 inet static
        address 192.168.31.200
        netmask 255.255.255.0
        network 192.168.31.0
        broadcast 192.168.31.255
        gateway 192.168.31.1
        dns-nameservers 71.243.0.12 71.250.0.12
        dns-search tlthost.net

I've stopped/started both HAProxy, and Heartbeat. I've also rebooted the server, but nothing helps.

 

Sorry to keep going on about this, but I'm determined to make it work; which it currently isn't.
I have installed ISPConfig 3 on both web servers now as master/slave. I can now access the test web page, and ISPC interface from the Internet, and local. I ran the tests in the how-to for seeing if the fail safe is working, and they passed.

I still can't see the HAProxy stats interface, or Squirrelmail from either local, or Internet. You can see from the previous post what IP's I'm using. I don't know if it matters, but another thing that isn't working is ISPC master isn't getting any info from the slave server. I'm assuming the installation of ISPConfig has no effect on HAProxy, or Squirrelmail? After I tried to get to Squirrelmail by using 192.168.31.202/webmail, I checked my apache error log and saw:

Code:

[error] [client 192.168.30.101] File does not exist: /var/www/apache2-default/webmail

My router/firewall has my public IP pointing to the common IP from the how-to. I'm using 192.168.31.100 for that one. I don't know what other info to add. All I know is I don't dare to start adding sites when things are not "talking" to each other properly, and ISPConfig isn't passing data to the slave. If I can get these working, then I'll add the master/master replication, and rsync.

 

How should ISPConfig pass data to the slave if you haven't set up rsync and MySQL replication?

What's in your haproxy.cfg?

 

That's been one of my questions all along. Does ISPC 3 handle syncing itself with it's slave automatically, or do we still need to set up master/master replication?

Code:

global
        log 127.0.0.1   local0
        log 127.0.0.1   local1 notice
        #log loghost    local0 info
        maxconn 4096
        #debug
        #quiet
        user haproxy
        group haproxy
defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        retries 3
        redispatch
        maxconn 2000
        contimeout      5000
        clitimeout      50000
        srvtimeout      50000
listen webfarm 192.168.31.100:80
       mode http
       stats enable
       stats auth someuser:somepassword
       balance roundrobin
       cookie JSESSIONID prefix
       option httpclose
       option forwardfor       
       option httpchk HEAD /check.txt HTTP/1.0
       server web1 192.168.31.202:80 cookie web1 check
       server web2 192.168.31.212:80 cookie web2 check
       server backup 127.0.0.1:80 backup source 0.0.0.0

 

You must set this up manually.

Looks ok - you should see the stats on http://192.168.31.100/haproxy?stats

 

That helps.

Some of the problem seem to be clearing up. It seems that during the setting up of the virtual DOMU's, the swap files weren't be made. I noticed after installing Webmin on the web1 drive it showed that there was no swap file. I ran Xen's console and saw an error when it trid to set up the swap. It seems a step was missing during the setup I used. Now I can get into the HAProxy ststs at the expected address.

Now the only thing on the install I can't get at is Squirrelmail using http://192.168.31.202/webmail.

UPDATE: I noticed that in my apache error log it kept showing:

Code:

[error] [client 192.168.30.101] File does not exist: /var/www/apache2-default/webmail

so it tried this little modification to the how-to:

Code:

ln -s /usr/share/squirrelmail/ /var/www/apache2-default/webmail

now I can get to Squirrelmail.

 

There was a problem with the swap files that seems to be cleared up. As I said, I was able to see the HAProxy stats page finally, but it didn't last.

It seems that it only comes up when the web2 server is down. What's worse is I seem to have to reboot LB1 as well when something changes. It's driving me mad because I can't see any pattern to what makes it work. I do see that as soon as the web server is off on web2, I can get into HAProxy.

I also noticed when web1, web2, lb1, and lb2 are up and running, if I access VIA the Internet using my public IP to get to ISPConfig, it goes to web2 not web1. i would think I still have something wrong in HAProxy, but all the config files look correct to me.

Now what I tried is uninstalling ISPConfig on both servers. I think HAProxy is working normally now. I can get to the stats. I can shut off either Apache server and the other shows up when accessed from my public IP. Also something was rewriting my networks file every time I rebooted. That also stopped when I uninstalled ISPConfig 3.

 

I am trying to setup haproxy/heartbeat as per the instructions on http://howtoforge.com/setting-up-a-...lancer-with-haproxy-heartbeat-on-debian-lenny

I have only set up one load balancer to make sure I can get that working.

I am having a problem in that the virtual IP address does not seem to get bound to eth0.

I have added the virtual ip address (192.0.25.70 below but in reality a public ip address) in haproxy.cfg

listen webfarm 192.0.25.70:80

I have configured the virtual IP address in /etc/heartbeat/haresources

IS-08713 192.0.25.70

where IS-08713 is the output from uname -n. Do I need this to be a FQDN?

I have added the line to /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind=1

I have done sysctl -p and heartbeat and haproxy are both running. the virtual IP address does not appear when I run ip addr show and I cannot ping the virtual IP address or access the IP address in a URL.

Which part of the above binds the IP address so that ip addr show displays it? Do I need to issue an ip addr add?? Any ideas?

 

Not sure why this was not working before, but it started working. Maybe my host enabled something in their network to allow ARP spoofing.