IBGP Load Balancing with BGP Link Bandwidth

In the previous lab exercise, you used the BGP Link Bandwidth extended community to implement unequal-cost load balancing across multiple links connected to the same router. In this exercise, you’ll use the same approach but extend it across your autonomous system — routers receiving external routes over IBGP should perform unequal-cost multipathing (UCMP) toward external destinations based on the BGP Link Bandwidth extended community attached to BGP paths.

Existing Router Configuration

The routers in your lab use the following BGP AS numbers. X1 and X2 advertise a shared IPv4 prefix.

Node/ASN	Router ID	Advertised prefixes
AS65000
core	10.0.0.1
we1	10.0.0.2
we2	10.0.0.3
AS65100
x1	10.0.0.4	10.1.3.0/24
x2	10.0.0.5	10.1.3.0/24

Your routers are running OSPF in the backbone area. They have these BGP neighbors:

Node	Router ID / Neighbor	Router AS/ Neighbor AS	Neighbor IPv4
core	10.0.0.1	65000
	we1	65000	10.0.0.2
	we2	65000	10.0.0.3
we1	10.0.0.2	65000
	core	65000	10.0.0.1
	we2	65000	10.0.0.3
	x1	65100	10.1.0.10
	x1	65100	10.1.0.14
we2	10.0.0.3	65000
	core	65000	10.0.0.1
	we1	65000	10.0.0.2
	x2	65100	10.1.0.18

netlab automatically configures IP addresses and routing protocols; if you’re using some other lab infrastructure, you’ll have to configure your devices manually.

Start the Lab

Assuming you already set up your lab infrastructure:

• Change directory to lb/3-ibgp
• Execute netlab up (device requirements)
• Log into your routers and verify that the IP addresses and the EBGP sessions are properly configured.

The Problem

Log into the Core router and check its BGP table. It should have two paths for the 10.1.3.0/24 prefix.

core#show ip bgp
BGP routing table information for VRF default
Router identifier 10.0.0.1, local AS number 65000
Route status codes: s - suppressed contributor, * - valid, > - active, E - ECMP head, e - ECMP
                    S - Stale, c - Contributing to ECMP, b - backup, L - labeled-unicast
                    % - Pending best path selection
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI Origin Validation codes: V - valid, I - invalid, U - unknown
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  AIGP       LocPref Weight  Path
 * >Ec    10.1.3.0/24            10.0.0.2              0       -          100     0       65100 i
 *  ec    10.1.3.0/24            10.0.0.3              0       -          100     0       65100 i

Configuring IBGP load balancing on the Core router will send half of the outgoing traffic for the 10.1.3.0/24 prefix each way. That’s undesirable, as the WE1-X1 path has significantly more bandwidth than the WE2-X2 path.

Configuration Tasks

The simplest way to change the load balancing ratio on the Core router is to use the BGP Link Bandwidth extended community:

• Attach the BGP Link Bandwidth community to EBGP paths on WE1 and WE2¹. The bandwidth on the WE1-X1 links is 10Gbps, and the WE2-X2 link has 4 Gbps.
• Configure UCMP load balancing on the Core router
• Configure propagation of extended communities between WE1, WE2, and Core routers². On some devices, you’ll also have to configure the propagation of the BGP Link Bandwidth community.
• If possible, aggregate the bandwidth on WE1 – it should advertise a single prefix with BGP Link Bandwidth set to 20 Gbps³.

Verification

Log into the Core router and inspect the BGP prefix 10.1.3.0. The paths advertised by WE1 and WE2 should have the BGP Link Bandwidth community. The value advertised by WE1 should be set to 10 or 20 Gbps⁴ (try to get it to 20 Gbps); the value advertised by WE2 should be 4 Gbps.

core>show ip bgp 10.1.3.0/24
BGP routing table information for VRF default
Router identifier 10.0.0.1, local AS number 65000
BGP routing table entry for 10.1.3.0/24
 Paths: 2 available
  65100
    10.0.0.3 from 10.0.0.3 (10.0.0.3)
      Origin IGP, metric 0, localpref 100, IGP metric 20, weight 0, tag 0
      Received 00:01:26 ago, valid, internal, ECMP head, ECMP, UCMP, best, ECMP contributor
      Extended Community: Link-Bandwidth-AS:65000:500.0 MBps
      Rx SAFI: Unicast
  65100
    10.0.0.2 from 10.0.0.2 (10.0.0.2)
      Origin IGP, metric 0, localpref 100, IGP metric 20, weight 0, tag 0
      Received 00:00:27 ago, valid, internal, ECMP, UCMP, ECMP contributor
      Extended Community: Link-Bandwidth-AS:65000:2.5 GBps
      Rx SAFI: Unicast

Check the prefix 10.1.3.0/24 in the IP routing table. The load balancing ratio displayed by the Core router depends on its implementation details, but you should observe an unequal traffic distribution. This is what you could get on an Arista cEOS container:

core#show ip route 10.1.3.0
...

 B I      10.1.3.0/24 [200/0]
           via 10.1.0.2, Ethernet1, weight 3/4
           via 10.1.0.6, Ethernet2, weight 1/4

Reference Information

Device Requirements

• Use any device supported by the netlab BGP configuration module for the customer- and provider routers.
• Git repository contains external router initial device configurations for Cumulus Linux.

Lab Wiring

Origin Device	Origin Port	Destination Device	Destination Port
core	Ethernet1	we1	Ethernet1
core	Ethernet2	we2	Ethernet1
we1	Ethernet2	x1	swp1
we1	Ethernet3	x1	swp2
we2	Ethernet2	x2	swp1
x1	swp3	x2	swp2

Lab Addressing

Node/Interface	IPv4 Address	IPv6 Address	Description
core	10.0.0.1/32		Loopback
Ethernet1	10.1.0.1/30		core -> we1
Ethernet2	10.1.0.5/30		core -> we2
we1	10.0.0.2/32		Loopback
Ethernet1	10.1.0.2/30		we1 -> core
Ethernet2	10.1.0.9/30		we1 -> x1
Ethernet3	10.1.0.13/30		we1 -> x1
we2	10.0.0.3/32		Loopback
Ethernet1	10.1.0.6/30		we2 -> core
Ethernet2	10.1.0.17/30		we2 -> x2
x1	10.0.0.4/32		Loopback
swp1	10.1.0.10/30		x1 -> we1
swp2	10.1.0.14/30		x1 -> we1
swp3	10.1.0.21/30		x1 -> x2
x2	10.0.0.5/32		Loopback
swp1	10.1.0.18/30		x2 -> we2
swp2	10.1.0.22/30		x2 -> x1