RE: [fw-wiz] IPSEC over load-shared T1s (per packet) (NxT1 and ML P)

TSimons_at_Delphi-Tech.com
Date: 09/24/03

  • Next message: Wes Noonan: "RE: [fw-wiz] Wat Does PI RUNS IOS?????????" 
    To: firewall-wizards@honor.icsalabs.com
Date: Wed, 24 Sep 2003 08:46:39 -0400

    Hello All

    Just a status....
    -The problem was ESP packets coming in out of sequence
    -A Mikael O said I should try to strong arm our firewall vendor into fixing
    this by recognizing and reordering the ESP packets, but I didn't get far.
    -Cisco recommended Multilink PPP, which aggregates the 2 T1s into one 3mbit
    virtual interface, our ISP welcomed this change -and we put it through.
    I'll be testing and monitoring today.

    Links on Cisco's Site:
    -Alternatives for High Bandwidth Connections Using Parallel T1/E1 Links
      http://www.cisco.com/warp/public/cc/pd/ifaa/pa/much/tech/althb_wp.pdf
    -Bundling NxT1 Links With a Multilink Interface
      http://www.cisco.com/warp/public/793/access_dial/ppp_11044.pdf

    Changes to our Cisco Router:
    1.) Create INT MULTILINK 1 (virtual interface)
      interface Multilink1
      ip address ?.?.?.? 255.255.255.252
      ip verify unicast source reachable-via any allow-self-ping 108
      no ip directed-broadcast
      no ip route-cache cef
      no ip route-cache
      load-interval 30
      no cdp enable
      ppp multilink
      no ppp multilink fragmentation
      multilink load-threshold 1 either
      multilink-group 1
    2.) Strip back IP addresses from member Serial Interfaces and add the
    following to each Interface:
      encap ppp
      no ip route-cache distributed
      ip unnumbered Multilink1
      ppp multilink
      multilink-group 1

    To view the status of the new multi1 interface use "sh ppp multilink":
    SL-Gateway#sh ppp multilink

    Multilink1, bundle name is [remote router name]
      Bundle up for 10:57:52
      1 lost fragments, 16609 reordered, 0 unassigned
      0 discarded, 0 lost received, 6/255 load
      0x828E9 received sequence, 0x61758 sent sequence
      Member links: 2 active, 0 inactive (max not set, min not set)
        Serial0/0, since 10:57:52, last rcvd seq 0828E5
        Serial0/1, since 10:57:50, last rcvd seq 0828E8
    SL-Gateway#

    Hope this helps!!
    ~Todd

    -----Original Message-----
    From: Pano Xinos [mailto:pano.xinos@ca.mci.com]
    Sent: Monday, September 22, 2003 10:21 AM
    To: Jan Bervar; Ben Nagy
    Cc: firewall-wizards@honor.icsalabs.com
    Subject: RE: [fw-wiz] IPSEC over load-shared T1s (per packet)

    Hi All,

    My experience has been that load-sharing per-destination on Cisco routers
    is nowhere near evenly balanced. Typically I've seen anything between
    90%:10% and 60%:40% traffic ratios/ulitization (it has never been 50%:50%
    when doing per-destination routing). The main issue is the sequence of
    packets and the anti replay feature of IPSec (don't remember who discussed
    it i na previous email...). AFAIC, you may be better off doing some QoS to
    shunt IPSec packets down a single link and run regular traffic over the
    other link. If redundancy is not an issue, simply get a bigger pipe to
    handle all traffic.

    Cheers!

    Pano

    At 09:17 AM 9/22/03 +0200, Jan Bervar wrote:
    >Just my 0.02 EUR... MPPP can be performance intensive on routers, and your
    >ISP may not be willing to implement it at all.
    >
    >Cisco routers can also load-balance on a source-destination hash, which
    >means that ideally, L3 sessions are evenly balanced across a number of
    >links. In a VPN scenario, this works much better compared to
    >per-destination balancing, especially if the number of your VPN peers is
    >large and dynamically addressed. Both sides of the link(s) need to enable
    >Cisco Express Forwarding, and there is no significant perfomance hit
    >involved (provided their and your routers have the memory to handle CEF
    >tables).
    >
    >Cheers,
    >Jan
    >
    >
    >firewall-wizards-admin@honor.icsalabs.com wrote on 20.09.2003 05:51:54:
    >
    > > I think this is pretty much solved now, but just for the sake of the
    > > archives:
    > >
    > > The problem was pretty much as I guessed (just lucky ;).
    > >
    > > The packets were being sent over alternating links in strict
    >round-robin,
    > > which meant that the ESP packets sometimes arrived out of sequence. The
    > > IPSec implementation was dropping all the ones with seq < currentseq,
    >which
    > > was causing retransmits in the tunneled TCP sessions.
    > >
    > > One fix is to use "per destination" load balancing - but that is bad
    >because
    > > if all the traffic is VPN then only one link will get used (only one
    > > destination).
    > >
    > > What I suggested offlist is to look at either ppp-multilink, or
    >MUX/DE-MUX -
    > > both of those will make the link look like one big layer2 pipe, which
    >will
    > > fix the problem and preserve sequencing. PPP Multilink is software, and
    > > simple. MUX stuff is more complicated but faster and can be more
    >flexible.
    > >
    > > I also got queries offlist about the E1/T1 RJ connectors. Yes, I did,
    >OK? I
    > > was curious. Ow.
    > >
    >
    >_______________________________________________
    >firewall-wizards mailing list
    >firewall-wizards@honor.icsalabs.com
    >http://honor.icsalabs.com/mailman/listinfo/firewall-wizards

    _______________________________________________
    firewall-wizards mailing list
    firewall-wizards@honor.icsalabs.com
    http://honor.icsalabs.com/mailman/listinfo/firewall-wizards
    _______________________________________________
    firewall-wizards mailing list
    firewall-wizards@honor.icsalabs.com
    http://honor.icsalabs.com/mailman/listinfo/firewall-wizards

  • Next message: Wes Noonan: "RE: [fw-wiz] Wat Does PI RUNS IOS?????????"