2 ISPs Auto Switching when 1 is down.

From: Manuel (news_at_buyee.com.sg)
Date: 03/20/04 

Date: Sat, 20 Mar 2004 18:23:14 +0800

RFC: Request for Comments

My company has two static ADSLs from different ISP serve as load balancing
and backup. Any better ideas, suggestions, corrections are most welcome. I
am not an expert in Linux or Networking, so please feel free to correct me
if any part is not right.

My plan was almost perfect, except for the DNS zone transfer.

I have searched the forum and there is no concreate answers. Maybe those who
are interest in this topic can join in as well. Please read on, a little
long post.

My current setup:
Internet -> ISP1 Router -> Switch(Ext) -> FW -> Switch(Int) -> Local LAN
Internet -> ISP2 Router -> Switch(Ext) -> FW -> Switch(Int) -> Local LAN

Switch (Ext) -> (DMZ) Few Public Servers (Mail & WWW)

All public servers have 2 NICs (ISP1, ISP2), FW had 3 NICs (Local, ISP1,
ISP2).

My logic for load balance and backup.
Incoming traffic from ISP1, Outgoing traffic to ISP2.
Public servers will ping ISP1 and ISP2 remote WAN at a fixed interval.
If ISP1 down - change default route to ISP2
If ISP2 down - change default route to ISP1

my link-chk script
***************
isp1_packet_count=`ping ISP1_WAN_IP -c 5 |grep trans | awk '{ print
substr($4,1,1) }'`
isp2_packet_count=`ping ISP2_WAN_IP -c 5 |grep trans | awk '{ print
substr($4,1,1) }'`

# If packet drop more than 20%, change link
if [ "$isp1_packet_count" -ge "4" ]
then
echo "***************************************** "
echo " ** NO CHANGES NEEDED ** "
echo "***************************************** "
else
echo "***************************************** "
echo " ** NOTICE : Link quality drop. ** "
echo "***************************************** "
/usr/local/bin/dns-isp2
killall named;named -u nobody
route add default gw ISP2_Router_IP metric 0 eth2
fi
# If packet drop more than 20%, change link
if [ "$isp2_packet_count" -ge "4" ]
then
echo "***************************************** "
echo " ** NO CHANGES NEEDED ** "
echo "***************************************** "
else
echo "***************************************** "
echo " ** NOTICE : Link quality drop. ** "
echo "***************************************** "
/usr/local/bin/dns-isp1
killall named;named -u nobody
route del default gw ISP1_Router_IP metric 0 eth2
fi

** END SCRIPT **

My public servers' IPs are with ISP1. (eg. www.domain.com -> ISP1.x.x.x)
When ISP1 down my DNS will copy the zone files with ISP2 IPs and restart.
(dns-isp1 and dns-isp2) (eg. www.domain.com -> ISP2.x.x.x)

All is well here except for my secondary DNS (two of them).
My script will run at the fixed interval and remove the zonefiles and
restart the DNS server.

I have the similar link-chk script above with the exception of the commands
to be run when failed.

If ISP1 failed
rm -f /var/named/*.hosts
cp -p /etc/named.conf.isp2 /etc/named.conf
killall named;named -u nobody

If ISP2 failed
rm -f /var/named/*.hosts
cp -p /etc/named.conf.isp1 /etc/named.conf
killall named;named -u nobody

Sorry if this post is a little long winded.... Hope to have a good
discussion on this topic.

Relevant Pages

  • 2 ISPs Auto Switching when 1 is down.
    ... Public servers will ping ISP1 and ISP2 remote WAN at a fixed interval. ... killall named;named -u nobody ...
    (comp.os.linux.networking)
  • Linux route question
    ... (ISP1 and ISP2). ... then three more interface connected to one server zone and two different ... I am trying that all download traffic should be from ISP2. ... Redhat Channel Partner and Training Partner ...
    (RedHat)
  • Re: policy based routing
    ... clients, one is to ISP1 and the other is to ISP2. ... based routing on protocol used would be cool too. ...
    (microsoft.public.isa)
  • Re: triple NIC route challenge
    ... ISP1 is Vendor X. ... I would like all inbound to be on ISP2. ... But I would like to split outbound between ISP1 and ISP2. ...
    (freebsd-questions)