Skip to main content

Configuring Failover and Load Balancing with HAproxy using Keepalived

Network Scenario:

LB1: 192.168.10.10
LB2: 192.168.10.11
Virtual IP: 192.168.10.12
APP_Server1: 192.168.10.20
APP_Server2: 192.168.10.21

Load Balancing:

STEP 1 - Install HAProxy:

HAProxy package is available under default yum repository for CentOS, Redhat systems. Use the following yum package manager command to install HAProxy on your system.
 
# yum install haproxy
 

STEP 2 - Configure HAProxy :

Update your HAProxy configuration file /etc/haproxy/haproxy.cfg as per your requirement, You may also use below given configuration file as an example of setup and modify it.
Keep the config file same of both servers i.e. LB1 and LB2.
 
-----------------------------------------------------------------------------------------------------------
global
        log /dev/log    local0
        log /dev/log    local1 notice
        maxconn  4000
        user demo
        group demo
        daemon

defaults
        log     global
       
        timeout connect 5000
        timeout client  50000
        timeout server  50000
       
frontend http_frontend
    bind *:80 v4v6
    mode http
    redirect scheme https if !{ ssl_fc }       #Use to redirect http to https

frontend https_frontend
  bind *:443
   bind *:80
  mode tcp
  default_backend app_server

backend app_server
  mode tcp
  balance roundrobin                              #Protocol for load balancing,
  stick-table type ip size 200k expire 30m
  stick on src
  server app1_server 192.168.10.20:443 check
  server app2_server 192.168.10.21:443 check
 
listen webfarm 192.168.10.10:1936      #change the IP for LB2
    mode http
    stats enable
    stats uri /haproxy?stats
------------------------------------------------------------------------------------------------------------
 
NOTE: Make sure that you have updated the /etc/hosts files on all the servers.
 
Change the IPs in configuration file as per your network setup. Webfarm IP can be used to view the stats of HAProxy server. URL: 192.168.10.10/11:1936/haproxy?stats shows the live stats of HAproxy server.
 
For learning the concepts and protocols of HAProxy, visit https://bachalor.blogspot.com/2017/04/haproxy-automatic-failover-haproxy-is.html
A point worth noting is the use of check against each server. This is automatically check the status of  the server and then redirect the user to the server. If a server is down, HAProxy will automatically transfer the request to the other server.

Step 3 - Service:

Start HAProxy service using following command, also configure it to auto start on system boot.
 
# service haproxy start
# chkconfig haproxy on

Step 4 - Important things to Note for HAProxy:

Firewall:

In case your load balancer does not reply, check that HTTP connections are not getting blocked by the firewall. Use the following commands to allow the access through firewall.
 
sudo firewall-cmd --permanent --zone=public --add-service=http
sudo firewall-cmd --permanent --zone=public --add-port=0000/tcp
sudo firewall-cmd --reload
 
NOTE:  Use your port in place of 0000, if it is other than 443 or 80.
 

SELinux:

Many times SElinux doesn't allow to connect at non-conventional ports. For allowing the ports in SElinux use the following commands,
 
# semanage port --add --type http_port_t --proto tcp PORT
 
For installing semanage use following command:
 
# yum install policycoreutils-python
 

Socket Bind Issue:

Many a times starting HAProxy service gives error for like
 
For resolving this issue check for another process, likely another instance of HAProxy, is already bound to those ports. Check with netstat -apn.
Kill the other instance of HAProxy.
 
If this didn't resolve your problem or there is no other process running. You may run the following command. This is resolve the issue.
 
setsebool -P haproxy_connect_any=1

Failover with Keepalived:

Step 5 - Assign Virtual IP to HAProxy servers:

The concept of creating or configuring multiple IP addresses on a single network interface is called IP aliasing. IP aliasing is very useful for setting up multiple virtual sites on Web servers using one single network interface with different IP addresses on a single subnet network.
Here we will use this concept to implement Failover for load balancers.
 
The device network files are located in “/etc/sysconfig/network-scripts/” directory. We will navigate to this dir.
 
# cd /etc/sysconfig/network-scripts/
# ls
 
Sample Output:
---------------------------------------------------------------------------------------------------------
ifcfg-eth0   ifdown-isdn    ifup-aliases  ifup-plusb     init.ipv6-global
ifcfg-lo     ifdown-post    ifup-bnep     ifup-post      net.hotplug
ifdown       ifdown-ppp     ifup-eth      ifup-ppp       network-functions
ifdown-bnep  ifdown-routes  ifup-ippp     ifup-routes    network-functions-ipv6
ifdown-eth   ifdown-sit     ifup-ipv6     ifup-sit
ifdown-ippp  ifdown-tunnel  ifup-isdn     ifup-tunnel
ifdown-ipv6  ifup           ifup-plip     ifup-wireless
---------------------------------------------------------------------------------------------------------
 
We want the following Virtual IP:
 
Server                        Adaptor                        IP address                       Type____
LB1                              eth0                         192.168.10.10                  Primary
LB2                              eth0                         192.168.10.11                  Primary
Both                             eth0:0                      192.168.10.12                  Alias
 
Currently we are in Network-script directory. Now make a copy of the eth0 to eth0:0
# cp ifcfg-eth0 ifcfg-eth0:0
 
Change the content of eth0:0 to the following:
--------------------------------------------------------------------------------------------------------
DEVICE="eth0:0"
BOOTPROTO=static
ONPARENT=yes
TYPE="Ethernet"
IPADDR=192.168.10.12
NETMASK=255.255.255.0
GATEWAY=192.168.10.1
DNS1=192.168.10.1
HWADDR=00:0C:29:28:FD:4C
---------------------------------------------------------------------------------------------------------
 
Save the file and restart the network service with following command.
 
#service network restart
 
NOTE: Change ONBOOT=YES to ONPARENT=YES in the eth0:0 because the interface will only come up when the parent interface comes up, while ONBOOT=YES would pull up the parent interface even if that is configured to not come up on boot.
 

Check Virtual IP:

Use the following command to show assigned virtual IP on an interface.
 
# ip addr show eth0

Sample Output:
-----------------------------------------------------------------------------------------------------------
eth0: mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000    
link/ether 00:0c:29:6f:ed:60 brd ff:ff:ff:ff:ff:ff
inet 192.168.10.11/24 brd 192.168.1.255 scope global eth0
inet 192.168.10.12/32 scope global eth0
inet6 fe80::20c:29ff:fe6f:ed60/64 scope link
  valid_lft forever preferred_lft forever
------------------------------------------------------------------------------------------------------------

Step 6 - Installation of required packages:

Use following command to install required packages to configure Keepalived on server. Keepalived is available in centos base repository. Install it using yum command line tool.
 
# yum install gcc kernel-headers kernel-devel
# yum install keepalived

Step 7 - Configure Keepalived:

Now we will configure the keepalived service. Keepalived configuration File can be located at /etc/keepalived/keepalived.conf
Edit Keepalived configuration file on LB1 and add following configuration.
------------------------------------------------------------------------------------------------------------
! Configuration File for keepalived

global_defs {
   notification_email {
         admin@domain.com
   }
   notification_email_from lb1@domain.com
   smtp_server localhost
   smtp_connect_timeout 30
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 101
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.10.12
    }
}

---------------------------------------------------------------------------------------------------------
 
Now configure LB2 with the following configuration.
---------------------------------------------------------------------------------------------------------
! Configuration File for keepalived

global_defs {
   notification_email {
         admin@domain.com
   }
   notification_email_from lb2@domain.com
   smtp_server localhost
   smtp_connect_timeout 30
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.10.12
    }
}
---------------------------------------------------------------------------------------------------------
 

Important Things to note:

  • Priority value will be higher on Master server, It doesn’t matter what you used in state.
  • virtual_router_id should be same on both LB1 and LB2 servers.
  • By default single vrrp_instance support up to 20 virtual_ipaddress. In order to add more addresses you need to add more vrrp_instance.

Step 8 - Service:

Start KeepAlived service using following command and also configure to auto start on system boot.
 
# service keepalived start
# chkconfig keepalived on

Step 9 - Verify Failover:

Now shutdown LB1 and check if the IP is automatically assigned to LB2.

# ip addr show eth0

Repeat same with LB2 and check if the IP is shifted to LB1. Also logs can be viewed to verify if the keepalived and failover is working fine.

# tailf /var/log/messages

This command will give you the logs showing HAproxy and keepalived running.


 

Comments

Post a Comment

Popular posts from this blog

ElasticSearch Clustering and Backups

ElasticSearch Installation: The version we want to install is 2.x (latest sub version of 2). First we have to install java on centOS machine with following command: sudo yum install java-1.8.0-openjdk.x86_64 Install Public Signing Key:   rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch Create new repo in your /etc/yum.repos.d/ directory. For example I have created elasticsearch.repo here. /etc/yum.repos.d/elasticsearch.repo -------------------------------------------------------------------------------------------- [elasticsearch-2.x] name=Elasticsearch repository for 2.x packages baseurl=http://packages.elastic.co/elasticsearch/2.x/centos gpgcheck=1 gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch enabled=1 --------------------------------------------------------------------------------------------- Now Elasticsearch 2.x will avilable for installation using yum   yum install elasticsearch After installation enable the service a...

How to configure CentOS Firewalld

Introduction Firewalld is a complete firewall solution available by default on CentOS 7 servers. In this guide, we will cover how to set up a firewall for your server and show you the basics of managing the firewall with the firewall-cmd administrative tool. Basic Concepts in Firewalld Before we begin talking about how to actually use the firewall-cmd utility to manage your firewall configuration, we should get familiar with a few basic concepts that the tool introduces. Zones The firewalld daemon manages groups of rules using entities called "zones". Zones are basically sets of rules dictating what traffic should be allowed depending on the level of trust you have in the networks your computer is connected to. Network interfaces are assigned a zone to dictate the behavior that the firewall should allow. For computers that might move between networks frequently (like laptops), this kind of flexibility provides a good method of changing your ru...