SLB using DNAT Solution Lab 4

Server load balancing using Dynamic NAT Solution Lab 4

Previous Next

Download Lab: GNS3

Prerequisites:
 Cisco IOSv        (vios-adventerprisek9-m.vmdk.SPA.156-2.T)
 Cisco IOSvL2    (vios_l2-adventerprisek9-m.03.2017.qcow2)
Topology for this lab created in the GNS3 VM server running on the VMware ESXi 6.5.0


Introduction: 
In this lab using structural troubleshooting approach, step-by-step you will tackle those challenges imposed by the previous lab, understanding how to start the resolution of a trouble ticket and proceed fast is very imperative because the time you spend on the solution is very valuable that is why this lab will attempt to improve your skills using methods taught in troubleshooting curriculum for CCNP studies.


Topology:



Solution for Challenge 1:

Step1: Gathering Information.  From the report made by a network security engineer and conversation with the network designer, you draw the conclusion that there might be flaws with the original design of the network. You open ticket indicating that router R1 is unable to communicate with servers located in the HQ building using domain names.


Step2: Identify the problem.  
You log in to the terminal of R1 and attempt to ping cd1.corporate.loc. The output confirms that there is a problem.
R1#ping cd1.corporate.loc
Translating "cd1.corporate.loc"...domain server (255.255.255.255)
% Unrecognized host or address, or protocol not running.


Step3: Analyze the problem.
From the output of the ping result, there might be an issue with DNS configuration locally on the R1.


Step4: Propose a hypothesis.
When pinging the server CD1.corp, router R1 attempted to translate the domain name to the broadcast IP address of 255.255.255.255 meaning that R1 does not have ip name-server command in running-config, this could be the issue preventing R1 from having connectivity to the HQ office. Verification of DNS configuration in the running-config could potentially identify the resolution to the problem.
R1#show  running-config | section  ip name-server

Nothing returns back, meaning that the hypothesis is confirmed.


Step5: Implement the solution.
Using network documentation identify proper IP address for DNS server then configure R1 using ip name-server command in the global configuration mode. According to the layer 3 topology and original documentation, DNS server is located behind NAT enabled router and to use its services router LB1 has virtual IP address and NAT port forwarding configured. The virtual IP address is identified by ACL 1 and is 10.4.4.4. Use this IP as DNS server address.
R1(config)#ip name-server 10.4.4.4


Step6: Verify the solution.
After the solution has been implemented the next step is to determine if the original problem has been solved, using ping command, from R1 again attempt to access the servers in the HQ office.
R1#ping cd1.corporate.loc
Translating "cd1.corporate.loc"...domain server (10.4.4.4) [OK]

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)


Step7: Document the results of the implemented solution.
According to the output of the ping command from step 6 verify the solution, domain name translated to to the IP address of 10.1.1.1 which corresponds to CD1.corp server, in the network documentation, despite the fact that router R1 is able to resolve corporate domains, connectivity to HQ office still remains in the same condition which is broken but you can conclude that at least partially the problem has been resolved, next step is to get additional information to understand the cause of the problem. The good news is that since the DNS problem has been eliminated you can use IP address of CD1.corp to continue troubleshooting.


Step8: Back to gathering information.
The obvious place to get some necessary info is to go back to the original suspect router R1 and ask some questions, why is router R1 unable to reach server CD1.corp and how far it can go, to do so you can use traceroute.
R1#traceroute 10.1.1.1
Type escape sequence to abort.
Tracing the route to cd1.corporate.loc (10.1.1.1)
VRF info: (vrf in name/id, vrf out name/id)
  1 10.0.11.1 2 msec 2 msec 3 msec
  2  *  *  *
  3  *  *  *
  4  *  *  *
  5  *  *  *

Step9: Identify the problem.
From the output of traceroute, you can conclude that IP packets do not travel beyond the first hop 10.0.11.1 which according to the layer 3 topology belong to the router LB1 subinterface g0/1.11. Now your troubleshooting effort can switch to the router LB1.


Step10: Analyze the problem.
If LB1 is the place where packets originating from router R1 is dropped when LB1 cannot ping 10.1.1.1, verify this proposal.
LB1#ping 10.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds:
.....

Yes, the ping is failed but before jumping to conclusions and go the wrong way, you have to determine that LB1 is really unable to communicate with HQ office. Next step it is to understand the means of how communication occurs between LB1 and HQ office, according to the network diagram LB1 and router CD1 using EIGRP routing protocol to exchange routes, and it is a good idea to verify EIGRP neighbor relationship.
LB1#show ip eigrp  neighbors
EIGRP-IPv4 Neighbors for AS(10)
LB1#

There are no EIGRP neighbors, now you are curious and the next step will be to see how EIGRP protocol configured you run command show run | section eigrp.
LB1#show run | section eigrprouter eigrp 10
!
address-family ipv4 vrf all-traffic autonomous-system 10
redistribute static route-map RM_RDR_STATIC
network 10.0.0.0 0.0.0.3
eigrp router-id 0.0.0.2
exit-address-family


Here you see the output and realized that you forgot that there is VRF all-traffic configured. Now that you are aware of VRF you decide to ping again using VRF.
LB1#ping vrf all-traffic 10.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds:
.!!!!

LB1 successfully ping HQ office server CD1.corp meaning that connectivity is fully established. A good idea would be is to ping 10.1.1.1 from the source interface which connected to router R1.
LB1#ping vrf all-traffic 10.1.1.1 source g0/1.11
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds:
Packet sent with a source address of 10.0.11.1
.....
Success rate is 0 percent (0/5)
LB1#

The result of output indicates that there might be an issue with route redistribution since LB1 is connected to multiple domains and you know from the output above that it already redistributes static routes into EIGRP. Checking EIGPR topology table would be the best shot to determine the cause of connectivity problem.
LB1#show  ip eigrp  vrf all-traffic topology
EIGRP-IPv4 Topology Table for AS(10)/ID(0.0.0.2) VRF(all-traffic)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status
P 192.168.10.0/24, 1 successors, FD is 7680
        via Rstatic (7680/0)
P 10.0.0.0/30, 1 successors, FD is 2816
        via Connected, GigabitEthernet0/2
P 10.1.1.0/24, 1 successors, FD is 3072
        via 10.0.0.1 (3072/2816), GigabitEthernet0/2
P 192.168.20.0/24, 1 successors, FD is 7680
        via Rstatic (7680/0)

Absence of route 10.0.11.0/30  in the EIGRP topology table indicates the reason why R1 and LB1 unable to ping HQ office servers from this subnet.


Step11: Propose a hypothesis.
On the router LB1 redistribution of connected network 10.0.11.0/30 into EIGRP has to be enabled using route-map, seed metric has to be specified also.


Step12: Implement the solution.
Create route-map then in the router eigrp configuration mode use redistribute connected with route-map option.
LB1(config)#route-map CONNECTED permit  10
LB1(config-route-map)#match  interface g0/1.11
LB1(config-route-map)#set metric 500000 10 255 1 1500
LB1(config-route-map)#exit
LB1(config)#router eigrp 10
LB1(config-router)# address-family ipv4 vrf all-traffic autonomous-system 10LB1(config-router-af)#redistribute connected route-map CONNECTED
LB1(config-router-af)#exit
LB1(config-router)#exit
LB1(config)#exit
LB1#cop r s
*Aug 13 18:43:11.580: %SYS-5-CONFIG_I: Configured from console by console
LB1#cop r s
Destination filename [startup-config]?
Building configuration...
[OK]


Step13: Verify the solution.
First check the eigrp topology table on the LB1 for vrf all-traffic then if subnet 10.0.11.0/30 is present in the table, attempt to ping 10.1.1.1 and cd1.corporate.loc from R1.
LB1#show  ip eigrp  vrf  all-traffic  topology
EIGRP-IPv4 Topology Table for AS(10)/ID(0.0.0.2) VRF(all-traffic)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status
P 192.168.10.0/24, 1 successors, FD is 7680
        via Rstatic (7680/0)
P 10.0.0.0/30, 1 successors, FD is 2816
        via Connected, GigabitEthernet0/2
P 10.0.11.0/30, 1 successors, FD is 7680
        via Rconnected (7680/0)
P 10.1.1.0/24, 1 successors, FD is 3072
        via 10.0.0.1 (3072/2816), GigabitEthernet0/2
P 192.168.20.0/24, 1 successors, FD is 7680
        via Rstatic (7680/0)

R1#ping 10.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 3/254/1007 ms

R1#ping  cd1.corporate.loc
Translating "cd1.corporate.loc"...domain server (10.4.4.4) [OK]
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 3/254/1006 ms
R1#

Step14: Document the solution.
Challenge1: The problem has been solved. Missing DNS configuration on the router R1 and Redistribution configuration on the router LB1 are the two main causes for the connectivity problem between router R1 and servers in the HQ office. All necessary documentation has been updated and the troubleshooting effort was archived for future references.



Solution for Challenge 2:

Note:
The first solution treated as troubleshooting effort that there are problems that you have to fix and the issues are unknown, for the second challenge the network team aware of problems and solution will be presented as step-by-step instructions.


Step1: Verify that the problem is still occurring.
From CD2.corp server in the HQ office initiate TCP connection for HTTP and HTTPS protocols.

VPCS> ping http-balancer.loc -P 6 -p 80
Cannot resolve http-balancer.loc

VPCS> ping http-balancer.loc -P 6 -p 443
Cannot resolve http-balancer.loc


Step2: Verify connectivity to DNS1 server with traceroute from CD2.corp

VPCS> trace 10.4.4.4
trace to 10.4.4.4, 8 hops max, press Ctrl+C to stop
 1   10.1.1.100   1.606 ms  1.443 ms  1.409 ms
 2   *10.1.1.100   1.880 ms (ICMP type:3, code:1, Destination host unreachable)


Step3: Verify routing table on the router CD1. The route to the host 10.4.4.4 is missing.
 CD1#show  ip route   
10.0.0.0/8 is variably subnetted, 5 subnets, 3 masks
C        10.0.0.0/30 is directly connected, GigabitEthernet0/0
L        10.0.0.1/32 is directly connected, GigabitEthernet0/0
D EX     10.0.11.0/30 [170/7936] via 10.0.0.2, 01:19:18, GigabitEthernet0/0
C        10.1.1.0/24 is directly connected, GigabitEthernet0/1
L        10.1.1.100/32 is directly connected, GigabitEthernet0/1
D EX  192.168.10.0/24 [170/7936] via 10.0.0.2, 05:04:59, GigabitEthernet0/0
D EX  192.168.20.0/24 [170/7936] via 10.0.0.2, 05:04:59, GigabitEthernet0/0


Step4: Router LB1 has to advertise DNS1 server IP address to HQ location.
LB1#show  ip route  10.4.4.4
% Subnet not in table

LB1#show  ip route vrf all-traffic 10.4.4.4
Routing Table: all-traffic
% Subnet not in table

Since the IP address of DNS server is virtual IP simulated by ACL statement it cannot be replicated into VRF all-traffic from the global routing table, as you can see from the output above. Create host static route for DNS server IP address pointing towards R1 then identify this route with prefix-list and match prefix-list in the route-map RM_RDR_STATIC under the sequence 10, it will redistribute the route into EIGRP domain since this route-map already in use with the redistribute command.
LB1(config)#ip route vrf all-traffic 10.4.4.4 255.255.255.255 10.0.11.2
LB1(config)#ip prefix-list DNS_IP permit 10.4.4.4/32
LB1(config)#route-map RM_RDR_STATIC permit  10
LB1(config-route-map)#match  ip address prefix-list DNS_IP
LB1(config)#end


Step5: Verify redistribution.
LB1#show  route-map
route-map RM_RDR_STATIC, permit, sequence 10
  Match clauses:
    ip address prefix-lists: LAN_SUBNETS DNS_IP
  Set clauses:
    metric 500000 10 255 1 1500
  Policy routing matches: 0 packets, 0 bytes
!
LB1#show  ip eigrp vrf all-traffic topology 10.4.4.4/32
EIGRP-IPv4 Topology Entry for AS(10)/ID(0.0.0.2) VRF(all-traffic)
EIGRP-IPv4(10): Topology base(0) entry for 10.4.4.4/32
  State is Passive, Query origin flag is 1, 1 Successor(s), FD is 7680
  Descriptor Blocks:
  10.0.11.2, from Rstatic, Send flag is 0x0
      Composite metric is (7680/0), route is External
      Vector metric:
        Minimum bandwidth is 500000 Kbit
        Total delay is 100 microseconds
        Reliability is 255/255
        Load is 1/255
        Minimum MTU is 1500
        Hop count is 0
        Originating router is 0.0.0.2
      External data:
        AS number of route is 0
        External protocol is Static, external metric is 0
        Administrator tag is 0 (0x00000000)


Step6: Check again if servers in the HQ office are able to communicate with Web servers in DC.
 VPCS> ping http-balancer.loc -P 6 -p 443
Cannot resolve http-balancer.loc

VPCS> trace 10.4.4.4                  
trace to 10.4.4.4, 8 hops max, press Ctrl+C to stop
 1   10.1.1.100   1.475 ms  1.169 ms  1.255 ms
 2   10.0.0.2   2.297 ms  2.426 ms  3.386 ms
 3   10.0.11.2   3.636 ms  3.100 ms  3.738 ms
 4     *  *  *
 5     *  *  *
^C 6     *  *
The problem remains the same but the only difference is that this time when CD2.corp traceroutes, router R1 is the last hop, not the router CD1 meaning DNS requests are reaching the virtual IP address.


Step7: Determine why LB1 unable to respond back to HQ server from virtual IP address.
LB1#show  ip route
 10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.0.12.0/30 is directly connected, GigabitEthernet0/1.12
L        10.0.12.1/32 is directly connected, GigabitEthernet0/1.12
      172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C        172.16.0.0/24 is directly connected, GigabitEthernet0/0
L        172.16.0.1/32 is directly connected, GigabitEthernet0/0
S     192.168.10.0/24 [1/0] via 10.0.12.2
S     192.168.20.0/24 [1/0] via 10.0.12.2

Router LB1 does not have routes in its global RIB for HQ office. The simple solution for this would be to create a static route for HQ office pointing to R1 and then since router R1 already has default route installed, it will direct packets towards HQ office.


Step8: Create a static route for the HQ network in the global RIB and verify the routing table on the router LB1.
LB1(config)#ip route 10.1.1.0 255.255.255.0 10.0.12.2
!
LB1#show  ip route
    10.0.0.0/8 is variably subnetted, 3 subnets, 3 masks
C        10.0.12.0/30 is directly connected, GigabitEthernet0/1.12
L        10.0.12.1/32 is directly connected, GigabitEthernet0/1.12
S        10.1.1.0/24 [1/0] via 10.0.12.2
      172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C        172.16.0.0/24 is directly connected, GigabitEthernet0/0
L        172.16.0.1/32 is directly connected, GigabitEthernet0/0
S     192.168.10.0/24 [1/0] via 10.0.12.2
S     192.168.20.0/24 [1/0] via 10.0.12.2


Step9: Verify if the issue still exists.
VPCS> ping http-balancer.loc -P 6 -p 443
http-balancer.loc resolved to 10.4.4.4
Connect   443@http-balancer.loc seq=1 ttl=251 time=1016.742 ms
SendData  443@http-balancer.loc seq=1 ttl=251 time=33.746 ms
Close     443@http-balancer.loc timeout
Connect   443@http-balancer.loc seq=2 ttl=251 time=14.860 ms
SendData  443@http-balancer.loc seq=2 ttl=251 time=5.338 ms
Close     443@http-balancer.loc seq=2 ttl=251 time=13.776 ms
Connect   443@http-balancer.loc seq=3 ttl=251 time=14.855 ms
SendData  443@http-balancer.loc seq=3 ttl=251 time=6.305 ms
Close     443@http-balancer.loc seq=3 ttl=251 time=12.723 ms
^CConnect   443@http-balancer.loc timeout

VPCS> ping http-balancer.loc -P 6 -p 80   
http-balancer.loc resolved to 10.4.4.4
Connect   80@http-balancer.loc seq=1 ttl=251 time=1015.956 ms
SendData  80@http-balancer.loc seq=1 ttl=251 time=206.112 ms
Close     80@http-balancer.loc seq=1 ttl=251 time=7.404 ms
Connect   80@http-balancer.loc seq=2 ttl=251 time=7.421 ms
SendData  80@http-balancer.loc seq=2 ttl=251 time=6.384 ms
Close     80@http-balancer.loc seq=2 ttl=251 time=7.468 ms
Connect   80@http-balancer.loc seq=3 ttl=251 time=8.539 ms
SendData  80@http-balancer.loc seq=3 ttl=251 time=6.345 ms
Close     80@http-balancer.loc seq=3 ttl=251 time=9.539 ms
^CConnect   80@http-balancer.loc timeout

Second solution conclusion:
Challenge 2 has been solved. Servers in the HQ building now able to communicate with web servers in the Datacenter over HTTP and HTTPS protocols.

Comments

Popular Posts