Establish an IBGP Session Between WAN Edge Routers
The Setting Up BGP lab exercises covered the simplest possible scenario: your site has a single WAN edge router running BGP with one or more upstream Internet Service Providers (ISPs). That scenario is implausible – an organization investing in its own IP address space and AS number usually does that to achieve higher resilience of its Internet connectivity, and having a single WAN edge router is not good enough for that.
In this lab exercise, you’ll build a more realistic solution: your organization uses two WAN edge routers running BGP with two upstream ISPs.
Most organizations want to optimize the use of their (still relatively expensive) WAN links – you’ll have to ensure that all your routers reach the destinations in ISP-1 via the R1-X1 uplink (and similarly for ISP-2).
Existing Lab Configuration
When starting the lab with netlab you’ll get a preconfigured lab with EBGP sessions between the three autonomous systems and OSPF running between R1 and R2.
The routers in your lab use the following BGP AS numbers. Each upstream router advertises an IPv4 prefix; your routers advertise the IPv4 prefix of the LAN subnet connecting them.
|Node/ASN||Router ID||Advertised prefixes|
netlab configures these EBGP sessions when starting the lab; if you’re using some other lab infrastructure, you’ll have to configure EBGP neighbors and advertised prefixes manually.
|Node||Neighbor||Neighbor AS||Neighbor IPv4|
OSPF running in backbone area is configured on the following routers:
Start the Lab
Assuming you already set up your lab infrastructure:
- Change directory to
- Execute netlab up (other options)
- Log into your devices (R1, R2) with netlab connect and verify that netlab correctly configured their IP addresses, OSPF routing, and EBGP sessions.
If you’re not using netlab you’ll have to configure R1 and R2 yourself.
Why Do We Need IBGP?
Inspect the BGP tables on R1 and R2. They contain the routes received from upstream ISPs, but not those received by the other WAN router – R1 cannot use the R2-X2 uplink to reach ISP-2 (and vice versa for R2). The following printouts contain BGP tables on R1 and R2 (all printouts in this lab exercise were generated by Arista vEOS):
r1>show ip bgp | begin Network Network Next Hop Metric AIGP LocPref Weight Path * > 10.0.100.0/24 - - - - 0 i * > 172.16.101.0/24 10.1.0.2 0 - 100 0 65100 65101 i * > 192.168.100.0/24 10.1.0.2 0 - 100 0 65100 i
r2>show ip bgp | begin Network Network Next Hop Metric AIGP LocPref Weight Path * > 10.0.100.0/24 - - - - 0 i * > 172.16.101.0/24 10.1.0.10 0 - 100 0 65101 i * > 192.168.100.0/24 10.1.0.10 0 - 100 0 65101 65100 i
The content of the BGP tables on R1 and R2 shouldn’t surprise you; R1 and R2 are exchanging internal routes (using OSPF) but not external routes. We could “solve” the challenge by redistributing external routes into OSPF (hint: don’t do that), but then we’d lose the BGP information like the AS path the routers need to compare the routes.
The only sane way forward is to establish a BGP session between R1 and R2, and because that session is set up between two routers in the same autonomous system, we call it an internal BGP (IBGP) session.
Establish IBGP Session
- Configure an IBGP session between the loopback interfaces of R1 and R2 (10.0.0.1 and 10.0.0.2) using a BGP configuration command similar to neighbor address remote-as 65000.
You’ll find more details in the Configure a Single EBGP Session lab exercise.
- On some devices you’ll have to activate the IBGP session within the IPv4 address family
Check the status of the IBGP session with a command similar to show ip bgp summary or show ip bgp neighbors
The following printout contains the BGP summary information on R1 after the IBGP session has been configured. As you can see, the router tries to establish the IBGP session but does not succeed.
r1#show ip bgp summary BGP summary information for VRF default Router identifier 10.0.0.1, local AS number 65000 Neighbor Status Codes: m - Under maintenance Description Neighbor V AS MsgRcvd MsgSent InQ OutQ Up/Down State PfxRcd PfxAcc r2 10.0.0.2 4 65000 0 0 0 0 00:06:03 Active x1 10.1.0.2 4 65100 2616 3068 0 0 02:10:34 Estab 2 2
Fix the Source IP Address of the IBGP Session
BGP uses TCP as the transport protocol, and without further configuration, the TCP session’s source IP address becomes the outgoing interface’s IP address. Attempts to establish an IBGP session using the source IP address of the LAN interface are rejected by the IBGP neighbor as the source IP address in the TCP SYN packet does not match the neighbor IP address configured on the remote router.
- Configure the source address of the IBGP TCP session with a BGP configuration command similar to neighbor update-source.
You could also configure the IBGP session between LAN IP addresses to make the lab work but never do that in a real-life network. You want the IBGP sessions to be stable, and using loopbacks as the endpoints of the TCP sessions (and relying on IGP to figure out how to reach remote loopbacks) is the best way to achieve that goal.
After configuring the source IP address of the IBGP session on R1 and R2 the routers should be able to establish the IBGP session, as illustrated by the following printout:
r1#show ip bgp summary BGP summary information for VRF default Router identifier 10.0.0.1, local AS number 65000 Neighbor Status Codes: m - Under maintenance Description Neighbor V AS MsgRcvd MsgSent InQ OutQ Up/Down State PfxRcd PfxAcc r2 10.0.0.2 4 65000 8 8 0 0 00:00:07 Estab 2 2 x1 10.1.0.2 4 65100 2795 3276 0 0 02:19:28 Estab 2 2
After the IBGP session has been established, R1 and R2 exchange BGP prefixes received from X1 and X2, but the prefixes advertised by R2 are not selected as the best routes by R1 (and vice versa):
r1#show ip bgp | begin Network Network Next Hop Metric AIGP LocPref Weight Path * > 10.0.100.0/24 - - - - 0 i * 10.0.100.0/24 10.0.0.2 0 - 100 0 i * > 172.16.101.0/24 10.1.0.2 0 - 100 0 65100 65101 i 172.16.101.0/24 10.1.0.10 0 - 100 0 65101 i * > 192.168.100.0/24 10.1.0.2 0 - 100 0 65100 i 192.168.100.0/24 10.1.0.10 0 - 100 0 65101 65100 i
Further investigation shows that the IBGP prefixes are not used because they are considered invalid.
r1#show ip bgp 172.16.101.0 BGP routing table information for VRF default Router identifier 10.0.0.1, local AS number 65000 BGP routing table entry for 172.16.101.0/24 Paths: 2 available 65100 65101 10.1.0.2 from 10.1.0.2 (192.168.100.1) Origin IGP, metric 0, localpref 100, IGP metric 0, weight 0, tag 0 Received 00:07:37 ago, valid, external, best Rx SAFI: Unicast 65101 10.1.0.10 from 10.0.0.2 (10.0.0.2) Origin IGP, metric 0, localpref 100, IGP metric -, weight 0, tag 0 Received 00:05:29 ago, invalid, internal Rx SAFI: Unicast
Arista EOS is not exactly helpful in this scenario; you have to guess that the underlying root cause is that the BGP next hop is not reachable1.
Fix the BGP Next Hop of IBGP Prefixes
BGP routers do not change the BGP next hop of EBGP routes when advertising them over IBGP – the BGP next hop of routes in AS 65101 as advertised by R2 to R1 is thus the IP address of X2 on the R2-X1 link (more details). R1 does not have a route to that IP subnet in its IP routing table, so it considers the IBGP prefix using that BGP next hop invalid.
There are two approaches to make the BGP next hop of IBGP prefixes valid:
- You can include the external subnets in the IGP process (for example, you can make them part of the OSPF area).
When using this approach, you must make the external subnets passive – you don’t want to run IGP routing with another autonomous system.
- You can change the BGP next hop of the prefixes advertised over IBGP to the IP address of the advertising router. We’ll use this approach.
- Change the BGP next hop of prefixes advertised over IBGP sessions with a BGP configuration command similar to neighbor next-hop-self.
Inspect the BGP tables and IP routing tables on R1 and R2 and verify that
- R1 and R2 use BGP prefixes with the shortest AS path as the best BGP routes
- BGP-derived IP prefixes in the IP routing tables point to the WAN uplinks or LAN link between R1 and R2.
You should get printouts similar to the ones generated by Arista EOS on R1:
r1#show ip bgp | begin Network Network Next Hop Metric AIGP LocPref Weight Path * > 10.0.100.0/24 - - - - 0 i * 10.0.100.0/24 10.0.0.2 0 - 100 0 i * > 172.16.101.0/24 10.0.0.2 0 - 100 0 65101 i * 172.16.101.0/24 10.1.0.2 0 - 100 0 65100 65101 i * > 192.168.100.0/24 10.1.0.2 0 - 100 0 65100 i
r1#show ip bgp 172.16.101.0 BGP routing table information for VRF default Router identifier 10.0.0.1, local AS number 65000 BGP routing table entry for 172.16.101.0/24 Paths: 2 available 65101 10.0.0.2 from 10.0.0.2 (10.0.0.2) Origin IGP, metric 0, localpref 100, IGP metric 20, weight 0, tag 0 Received 00:00:29 ago, valid, internal, best Rx SAFI: Unicast 65100 65101 10.1.0.2 from 10.1.0.2 (192.168.100.1) Origin IGP, metric 0, localpref 100, IGP metric 0, weight 0, tag 0 Received 00:51:03 ago, valid, external Rx SAFI: Unicast
r1#show ip route | begin Gateway Gateway of last resort is not set C 10.0.0.1/32 is directly connected, Loopback0 O 10.0.0.2/32 [110/20] via 10.0.100.2, Ethernet3 C 10.0.100.0/24 is directly connected, Ethernet3 C 10.1.0.0/30 is directly connected, Ethernet1 B I 172.16.101.0/24 [200/0] via 10.0.100.2, Ethernet3 B E 192.168.100.0/24 [200/0] via 10.1.0.2, Ethernet1
You might find the following information useful if you’re not using netlab to build the lab:
This lab uses the 4-router lab topology:
|Link Name||Origin Device||Origin Port||Destination Device||Destination Port|
|WAN uplink R1-to-X1||r1||Ethernet1||x1||swp1|
|WAN uplink R2-to-X2||r2||Ethernet2||x2||swp3|
|Inter-ISP link X1-to-X2||x1||swp2||x2||swp2|
|Intra-site LAN C1-C2||r1||Ethernet3||r2||Ethernet3|
Note: Some interfaces are not used to conform with the predefined 4-router lab topology.
|Node/Interface||IPv4 Address||IPv6 Address||Description|
|Ethernet1||10.1.0.1/30||WAN uplink R1-to-X1|
|Ethernet3||10.0.100.1/24||Intra-site LAN C1-C2|
|Ethernet2||10.1.0.9/30||WAN uplink R2-to-X2|
|Ethernet3||10.0.100.2/24||Intra-site LAN C1-C2|
|swp1||10.1.0.2/30||WAN uplink R1-to-X1|
|swp2||10.1.0.5/30||Inter-ISP link X1-to-X2|
|swp2||10.1.0.6/30||Inter-ISP link X1-to-X2|
|swp3||10.1.0.10/30||WAN uplink R2-to-X2|
Note: Some interfaces are not configured with IP addresses to conform with the predefined 4-router lab topology.
Some network devices consider any BGP next hop reachable if they have a default route. That can happen in a virtual lab if the lab device does not use a management VRF – most lab environments use DHCP to add the default route pointing to the management interface. When that default route appears in the global IP routing table, the IBGP prefix could be considered valid, but the resulting route would point to the management interface. ↩