MPLS Lab 019 Internet Access Backup via Remote CE Using Static Route

Image requirements:
VIRL: IOSv 15.7
EVE-NG: Cisco vIOS Router vios-15.6
GNS3: vios-adventerprisek9-m.vmdk.SPA.156-2.T



Description:
In the previous lab, CE routers got their access to the Internet via secondary ISP, each of them has its own dedicated connection but what if the second ISP link fails then one of the offices got cut off from the outside networks, installation of the third ISP will not be economically reasonable, fortunately for us, we can use the connection over MPLS VPN to the remote CE routers and even have not just one but two backup paths to Internet access. Each of CE routers will be configured with additional floating static routes to the other two CE routers if the main connection fails then the second will come up and in case of the failure of the second backup path, there still, be the last option over the third path.



Topology:


Download Lab: EVE-NG | GNS3





Scenario:
To accomplish this project the network engineer at the customer site needs to configure the three static routes on each of the CE routers, begin with CE1-A, one route will be pointing directly to the secondary ISP, and two other floating static routes will be using IP addresses of routers CE1-C and CE1-B assigned to the interfaces G0/1 as next-hop. Then continue to configure the other two routers CE1-C and CE1-B with static routes the same way as router CE1-A configured. Additionally, CE routers have to advertise default-routes to their PE partners, then with manipulation of route-map CE nodes depend on the state of the links between CE nodes and secondary ISP will influence PE routers' choice of preference for the default route. Finally, NAT ACLs have to be modified to allow the networks of other CE routers to be translated.



Lab tasks:
1. Verify the current implementation of NAT on each of the CE routers.
2. Configure BGP to redistribute the IP subnets of the links between CE routers and secondary ISP.
3. Configure IP SLA to monitor the status of local connection to the secondary ISP.
4. Configure track object for IP SLA that monitors local connection.
5. Configure route-map to match track object, and set metric.
6. Configure BGP to advertise the default route to PE routers.
7. Configure CE routers' interfaces g0/1 as NAT inside.
8. Add additional prefixes to the NAT ACLs. 
9. Configure IP SLA to monitor the status of the remote connection to the secondary ISP.
10. Configure the secondary track object for IP SLA that monitors remote connection. 
11. Configure floating static routes.
12. Verify current NAT operation after backup configurations have been applied.
13. Check the BGP table on the PE routers for VRF CE1 before you simulate the failure of the Internet link.  
14. Simulate the failure of the main Internet connection, then verify the results.
15. Simulate the failure of the first backup Internet connection, then verify the results.
16. Restore the normal state of Internet connections.
17. Repeat tasks 14 - 16 on every CE router.



Lab procedure:

Task1: Check the current NAT configurations on the CE nodes.
CE1-A#show ip route | section GigabitEthernet0/2
C 50.0.0.0/30 is directly connected, GigabitEthernet0/2
L 50.0.0.2/32 is directly connected, GigabitEthernet0/2

CE1-A#ping 8.8.8.8 source lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of 10.155.0.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

CE1-A#show ip nat translations
Pro Inside global Inside local Outside local Outside global
icmp 50.0.0.2:1 10.155.0.1:1 8.8.8.8:1 8.8.8.8:1
CE1-A#
CE1-A#show ip nat statistics
Total active translations: 0 (0 static, 0 dynamic; 0 extended)
Peak translations: 2, occurred 00:02:09 ago
Outside interfaces:
GigabitEthernet0/2
Inside interfaces:
Loopback0
Hits: 20 Misses: 0
CEF Translated packets: 10, CEF Punted packets: 0
Expired translations: 2
Dynamic mappings:
-- Inside Source
[Id: 1] access-list NAT interface GigabitEthernet0/2 refcount 0
Total doors: 0
Appl doors: 0
Normal doors: 0
Queued Packets: 0
CE1-A#
Complete verification on each of CE routers.



Task2: Configure BGP to redistribute the IP subnets of the links between CE routers and secondary ISP.
Configuring router CE1-A:

Step1. To prevent BGP from advertising every other connected subnet, configure prefix-list to identify the only /30 subnet of the link to the second ISP:
CE1-A(config)#ip prefix-list PUBLIC_IP permit 50.0.0.0/30


Step2. Configure the route-map:
CE1-A(config)#route-map RDR_PUBLIC_IP permit 10
CE1-A(config-route-map)#match ip address prefix-list PUBLIC_IP


Step3. Redistribute connected with the route-map:
CE1-A(config)#router bgp 65000
CE1-A(config-router)#redistribute connected route-map RDR_PUBLIC_IP



Step4. Perform steps above on the routers CE1-C and CE1-B.


Step5. Verify redistribution:
CE1-A#show bgp ipv4 unicast
Network Next Hop Metric LocPrf Weight Path
*> 10.150.0.0/30 0.0.0.0 0 32768 i
*> 10.150.0.4/30 10.150.0.2 0 64500 65002 i
*> 10.150.0.8/30 10.150.0.2 0 64500 65001 i
*> 10.155.0.0/24 0.0.0.0 0 32768 i
*> 10.160.0.0/24 10.150.0.2 0 64500 65002 i
*> 10.165.0.0/24 10.150.0.2 0 64500 65001 i
*> 50.0.0.0/30 0.0.0.0 0 32768 ?
CE1-A#



Task3: Configure IP SLA to monitor the status of local connection to the secondary ISP.
Configuring router CE1-A:
CE1-A(config)#ip sla 1
CE1-A(config-ip-sla)#icmp-echo 50.0.0.1 source-interface g0/2
CE1-A(config-ip-sla-echo)#threshold 500
CE1-A(config-ip-sla-echo)#timeout 900
CE1-A(config-ip-sla-echo)#frequency 1
CE1-A(config-ip-sla-echo)#exit
CE1-A(config)#ip sla schedule 1 start-time now life forever
Perform similar configurations on the routers CE1-C and CE-B.

Verification:
show ip sla summary



Task4: Configure track object for IP SLA that monitors local connection.
Configuring router CE1-A:
CE1-A(config)#track 1 ip sla 1
CE1-A(config-track)#delay down 2
CE1-A(config-track)#delay up 2
Perform similar configurations on the routers CE1-C and CE-B.

Verification:
show track



Task5: Configure route-map to match track object, and set metric.
This route-map is to control the default route propagation to the PE routers if the state of track object is changed then BGP will not advertise the DF route. Also, every PE router will choose the DF of its own local CE router over any other next-hop because of the local-preference, only if the local connection to the second ISP fails then PE router will move to MED attribute for prefix determination.
Configuring router CE1-A:
CE1-A(config)#route-map BGP_DF_ROUTE permit 10
CE1-A(config-route-map)#match track 1
CE1-A(config-route-map)#set metric 10
CE1-A(config-route-map)#exit
Perform similar configurations on the routers CE1-C and CE-B.
Metric:
CE1-C is 20
CE1-B is 30

Verification:
CE1-A#show route-map



Task6: Configure BGP to advertise the default route to PE routers.
CE1-A(config)#router bgp 65000
CE1-A(config-router)#network 0.0.0.0 route-map BGP_DF_ROUTE
CE1-A(config-router)#end

Verification:
CE1-A#show bgp ipv4 unicast
Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0 50.0.0.1 0 500 i



Task7: Configure CE routers' interfaces g0/1 as NAT inside.
CE1-A(config)#interface g0/1
CE1-A(config-if)#ip nat inside
CE1-A(config-if)# end
Perform similar configurations on the routers CE1-C and CE-B.

Verification:
show ip nat statistics



Task8: Add additional prefixes to the NAT ACLs. 
Configuring router CE1-A:
Allow LAN networks of CE1-C and CE1-B to be translated:
CE1-A#show ip access-lists NAT
Standard IP access list NAT
10 permit 10.155.0.0, wildcard bits 0.0.0.255 (2 matches)
CE1-A#
CE1-A(config)#ip access-list standard NAT
CE1-A(config-std-nacl)#permit 10.160.0.0 0.0.0.255
CE1-A(config-std-nacl)#permit 10.165.0.0 0.0.0.255
CE1-A(config-std-nacl)#end
CE1-A#show ip access-lists
Standard IP access list NAT
10 permit 10.155.0.0, wildcard bits 0.0.0.255 (2 matches)
20 permit 10.160.0.0, wildcard bits 0.0.0.255
30 permit 10.165.0.0, wildcard bits 0.0.0.255
 Perform similar configurations on the routers CE1-C and CE-B.



Task9: Configure IP SLA to monitor the status of the remote connection to the secondary ISP.
Configuring router CE1-A:
CE1-A(config)#ip sla 2
CE1-A(config-ip-sla)#icmp-echo 50.0.0.5 source-interface lo0
CE1-A(config-ip-sla-echo)#threshold 500
CE1-A(config-ip-sla-echo)#timeout 900
CE1-A(config-ip-sla-echo)#frequency 1
CE1-A(config-ip-sla-echo)#exit
CE1-A(config)#ip sla schedule 2 start-time now life forever

CE1-A(config)#ip sla 3
CE1-A(config-ip-sla)#icmp-echo 50.0.0.9 source-interface lo0
CE1-A(config-ip-sla-echo)#threshold 500
CE1-A(config-ip-sla-echo)#timeout 900
CE1-A(config-ip-sla-echo)#frequency 1
CE1-A(config-ip-sla-echo)#exit
CE1-A(config)#ip sla schedule 3 start-time now life forever
Perform similar configurations on the routers CE1-C and CE-B.

Verification:
show ip sla summary
Also good to know that as soon as IP SLAs to the remote locations are up, then on both remote CE routers NAT tables will get populated so you can use this as additional confirmation for proper operation.



Task10: Configure the secondary track object for IP SLA that monitors remote connection. 
Configuring router CE1-A:
CE1-A(config)#track 2 ip sla 2
CE1-A(config-track)#delay down 2
CE1-A(config-track)#delay up 2
CE1-A(config-track)#exit

CE1-A(config)#track 3 ip sla 3
CE1-A(config-track)#delay down 2
CE1-A(config-track)#delay up 2
CE1-A(config-track)#exit
Perform similar configurations on the routers CE1-C and CE-B.



Task11: Configure floating static routes. 
Table of Preference:
Devices First Backup Path   Second Backup Path
 CE1-A  CE1-C   CE1-B
  CE1-C   CE1-A   CE1-B
  CE1-B   CE1-A   CE1-C

Configuring router CE1-A:
CE1-A(config)#ip route 0.0.0.0 0.0.0.0 10.150.0.9 5 track 2
CE1-A(config)#ip route 0.0.0.0 0.0.0.0 10.150.0.5 10 track 3
The first static route will be installed in the routing table of CE1-A if the main connection to the Internet fails, the second static route will be installed if the main and backup via CE1-C connections both failed. Similar behavior will be observed on the other CE routers.
Perform similar configurations on the routers CE1-C and CE-B. 



Task12: Verify current NAT operation after backup configurations have been applied.
CE1-A#traceroute 8.8.8.8 source lo0
Type escape sequence to abort.
Tracing the route to 8.8.8.8
VRF info: (vrf in name/id, vrf out name/id)
1 50.0.0.1 9 msec * 5 msec
CE1-A#
CE1-C#traceroute 8.8.8.8 source lo0
Type escape sequence to abort.
Tracing the route to 8.8.8.8
VRF info: (vrf in name/id, vrf out name/id)
1 50.0.0.5 6 msec * 3 msec
CE1-C#
CE1-B#traceroute 8.8.8.8 source lo0
Type escape sequence to abort.
Tracing the route to 8.8.8.8
VRF info: (vrf in name/id, vrf out name/id)
1 50.0.0.9 4 msec * 3 msec
CE1-B#
Everything is still working as supposed to be. 



Task13: Check the BGP table on the PE routers for VRF CE1 before you simulate the failure of the Internet link.  
PE2#show bgp vrf CE1
BGP table version is 31, local router ID is 10.100.0.12
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 2000:2000 (default for vrf CE1)
* i 0.0.0.0 10.100.0.14 30 100 0 65002 i
* i 10.100.0.13 20 100 0 65001 i
*> 10.150.0.1 10 0 65000 i
r> 10.150.0.0/30 10.150.0.1 0 0 65000 i
*>i 10.150.0.4/30 10.100.0.14 0 100 0 65002 i
*>i 10.150.0.8/30 10.100.0.13 0 100 0 65001 i
*> 10.155.0.0/24 10.150.0.1 0 0 65000 i
*>i 10.160.0.0/24 10.100.0.14 0 100 0 65002 i
*>i 10.165.0.0/24 10.100.0.13 0 100 0 65001 i
*> 50.0.0.0/30 10.150.0.1 0 0 65000 ?
*>i 50.0.0.4/30 10.100.0.13 0 100 0 65001 ?
*>i 50.0.0.8/30 10.100.0.14 0 100 0 65002 ?
PE2#
If packets to the unknown destination will arrive at the PE2 router for VRF CE1 it will send them to the CE1-A.
You can continue to verify the BGP table for CE1 VRF on the rest of the CE routers.



Task14: Simulate the failure of the main Internet connection, then verify the results.
 Step1. Test backup Internet path for the router CE1-A, shutdown its g0/2 interface:
CE1-A(config)#int g0/2
CE1-A(config-if)#shutdown
CE1-A(config-if)#end

Step2. Verify routing table:
CE1-A#show ip route
Gateway of last resort is 10.150.0.9 to network 0.0.0.0
S* 0.0.0.0/0 [5/0] via 10.150.0.9
10.0.0.0/8 is variably subnetted, 8 subnets, 3 masks
C 10.150.0.0/30 is directly connected, GigabitEthernet0/1
L 10.150.0.1/32 is directly connected, GigabitEthernet0/1
B 10.150.0.4/30 [20/0] via 10.150.0.2, 03:03:08
B 10.150.0.8/30 [20/0] via 10.150.0.2, 03:03:07
C 10.155.0.0/24 is directly connected, Loopback0
L 10.155.0.1/32 is directly connected, Loopback0
B 10.160.0.0/24 [20/0] via 10.150.0.2, 03:03:08
B 10.165.0.0/24 [20/0] via 10.150.0.2, 03:03:07
50.0.0.0/30 is subnetted, 2 subnets
B 50.0.0.4 [20/0] via 10.150.0.2, 02:58:58
B 50.0.0.8 [20/0] via 10.150.0.2, 02:55:33
CE1-A#
If you notice the static default route has different next-hop, its the IP addrss of router CE1-C inteface G0/1.

Step3. Confirm connectivity to the internet and its path:
 CE1-A#ping 8.8.8.8 source lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of 10.155.0.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 5/6/7 ms
CE1-A#
CE1-A#traceroute 8.8.8.8 source lo0
Type escape sequence to abort.
Tracing the route to 8.8.8.8
VRF info: (vrf in name/id, vrf out name/id)
1 10.150.0.2 3 msec 3 msec 4 msec
2 10.0.3.5 [AS 65001] [MPLS: Labels 314/1319 Exp 0] 6 msec 7 msec 6 msec
3 10.150.0.10 [AS 65001] [MPLS: Label 1319 Exp 0] 7 msec 4 msec 5 msec
4 10.150.0.9 [AS 65001] 6 msec 5 msec 4 msec
5 50.0.0.5 [AS 65001] 8 msec * 8 msec
CE1-A#
As you can observe the traffic to the Internet goes via a backup Internet path which is the CE1-C router. Special note about IOS handling track object with IP SLA, it acts abnormal, I notice this with EVE-NG when g0/2 interface goes down the track-object does not disable default route advertisement towards PE2 and loop occurs, the best way to test is to shut down the interface then save running-config and reboot device. 



Task15: Simulate the failure of the first backup Internet connection, then verify the results.
Step1. Shutdown the interface g0/2 on the router CE1-C:
CE1-C(config)#interface g0/2
CE1-C(config-if)#shutdown
CE1-C(config-if)#

Step2. Back on the router CE1-A check the IP routing table, look for changes:
CE1-A#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
Gateway of last resort is 10.150.0.9 to network 0.0.0.0
S* 0.0.0.0/0 [5/0] via 10.150.0.9
10.0.0.0/8 is variably subnetted, 8 subnets, 3 masks
C 10.150.0.0/30 is directly connected, GigabitEthernet0/1
L 10.150.0.1/32 is directly connected, GigabitEthernet0/1
B 10.150.0.4/30 [20/0] via 10.150.0.2, 00:19:11
B 10.150.0.8/30 [20/0] via 10.150.0.2, 00:19:11
C 10.155.0.0/24 is directly connected, Loopback0
L 10.155.0.1/32 is directly connected, Loopback0
B 10.160.0.0/24 [20/0] via 10.150.0.2, 00:19:11
B 10.165.0.0/24 [20/0] via 10.150.0.2, 00:19:11
50.0.0.0/30 is subnetted, 1 subnets
B 50.0.0.8 [20/0] via 10.150.0.2, 00:19:11
Well! Nothing changed the next-hop IP address is still router's CE1-C. Why? Or something may be changed! Let's look deep in this problem!

Step3. Confirm the connectivity to the Internet and the path it takes:
CE1-A#ping 8.8.8.8 source lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of 10.155.0.1
!!!!!
CE1-A#traceroute 8.8.8.8 source lo0
Type escape sequence to abort.
Tracing the route to 8.8.8.8
VRF info: (vrf in name/id, vrf out name/id)
1 10.150.0.2 5 msec 3 msec 3 msec
2 10.0.3.5 [AS 65002] [MPLS: Labels 315/1419 Exp 0] 8 msec 6 msec 7 msec
3 10.0.0.22 [AS 65002] [MPLS: Labels 415/1419 Exp 0] 6 msec 5 msec 7 msec
4 10.150.0.6 [AS 65002] [MPLS: Label 1419 Exp 0] 4 msec 5 msec 6 msec
5 10.150.0.5 [AS 65002] 7 msec 7 msec 7 msec
6 50.0.0.9 [AS 65002] 7 msec * 9 msec
CE1-A#
The connectivity exist, but it goes backwards via the CE1-C the CE1-B because of the IP connectivity to the 50.0.0.5, the IP SLA for the first backup path is still up and that is why CE1-A sends traffic to the CE1-C. 
CE1-A#show ip sla summary
IPSLAs Latest Operation Summary
Codes: * active, ^ inactive, ~ pending
ID Type Destination Stats Return Last
(ms) Code Run
-----------------------------------------------------------------------
*1 icmp-echo 50.0.0.1 - Timeout 33 minutes, 1
6 seconds ago
*2 icmp-echo 50.0.0.5 RTT=10 OK 0 seconds ago
*3 icmp-echo 50.0.0.9 RTT=11 OK 0 seconds ago

But nevertheless, at least the second backup path is working, and offices able to perform work, you can also look at the BGP table for CE1 VRF on the PE routers to see how the configurations on the CE routers affect the BGP tables on the PE routers, this is the great situation to learn about BGP behavior when there are changes in the topology.

A solution to the problem with the ISP side of the first backup connection accessible over the second backup connection is to configure another IP SLA for the IP address of the CE1-C router's g0/2 interface and track-object then use as "AND" operation within the same sequence of the route-map, either failure will cause the default static route pointing to the CE1-C disappear from the routing table.



Summary:
This lab is outside of the MPLS realm but it teaches you a lot. It shows you how to integrate your CCNP skills with the service provider level project. This lab really can be the object of iteration, you can build on this topology practically multiple scenarios and practice an unlimited amount of troubleshooting situations. Define new ways of configurations and actually, I encourage to replace floating static routes with Policy Based Routing.

Note: Task 15 might be a little bit offset with GNS3 since I was performing in EVE-NG as I wrote this lab. I hope you will not find this disappointed while working on the lab using GNS3! Because I am still confused with both software platforms regarding their capabilities. I at least canceled a couple of troubleshooting labs with PE-CE Routing using OSPF and its Sham-Link configuration, due to the fact that both EVE-NG and GNS3 will not bring sham-links in the up state after routers boot up, you would have to remove them and configure again even though all parameters for sham-link proper function is fine.

Comments

Popular Posts