Professional Documents
Culture Documents
• Requirements
• Topology
• Session Users
• Get Started
• VNI / VNID – VXLAN Network Identifier, or VXLAN ID. This replaces VLAN ID.
• VTEP – VXLAN Tunnel End Point, the end point where the box performs VXLAN encap / decap. This could be physical HW
(Nexus9k) or virtual (Nexus 1000v, Nexus 9000v).
• VXLAN Gateway – Device that forwards traffic between VXLANS. The VXLAN Gateway can be both L2 and L3 forwarding.
• NVE – Network Virtualization Edge, is the tunnel interface, and represents VTEP
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 1 of 24
Cisco dCloud
Requirements
The table below outlines the requirements for this preconfigured demonstration.
Table 1. Requirements
Required Optional
● Laptop ● Cisco AnyConnect®
Only VMs within the same VXLAN segment can communicate with each other. Each VXLAN segment is identified by a 24 bit
segment ID called VXLAN Network Identifier (VNI). This helps to overcome the 4094 VLAN scale limitation and enables
extension to 224 segments.
VXLAN uses BGP as its control plane for Overlay. It makes its forwarding decisions at VTEPs (Virtual tunnel end points) for L2 and
L3. Forwarding happens based on MAC or IP learned via the control plane (MP-BGP EVPN) . VXLAN uses IGP, PIM and BGP as
its underlay in the fabric.
Topology
This content includes preconfigured users and components to illustrate the scripted scenarios and features of the solution. Most
components are fully configurable with predefined administrative user accounts. You can see the IP address and user account
credentials to use to access a component by clicking the component icon in the Topology menu of your active session and in the
scenario steps that require their use.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 2 of 24
Cisco dCloud
Session Users
Device IP Address Login Credentials
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 3 of 24
Cisco dCloud
Get Started
BEFORE PRESENTING
Cisco dCloud strongly recommends that you perform the tasks in this document with an active session before presenting in front
of a live audience. This will allow you to become familiar with the structure of the document and content.
It may be necessary to schedule a new session after following this guide in order to reset the environment to its original
configuration.
Follow the steps to schedule a session of the content and configure your presentation environment.
2. Connect to the demonstration workstation using one of the following connection methods:
• Cisco AnyConnect VPN [Show Me How] and the local RDP client on your laptop [Show Me How] (Workstation 1:
198.18.133.36, Username: DCLOUD\demouser, Password: C1sco12345 – click Yes on the pop-up)
4. Open the console connection to all devices, by double-clicking each device in the PuTTY sessions folder.
IMPORTANT: If a connection loss message occurs for any tab, click Reconnect and return to the proper tab before continuing.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 4 of 24
Cisco dCloud
6. If any session does not display a log in prompt, perform the Troubleshooting procedure.
NOTE: It is best practice to have separate loopback interface for VXLAN VTEP. The point-to-point links and loopback interfaces
are shown in the table below.
Leaf Switch Loopback0 Loopback1 Spine-1 Spine-2 WAN
Leaf-1 192.168.0.8 192.168.0.18 10.0.0.22/30 10.0.128.6/30
Leaf-2 192.168.0.9 192.168.0.19 10.0.0.26/30 10.0.128.10/30
Leaf-3 192.168.0.10 192.168.0.110 10.0.0.30/30 10.0.128.14/30
Leaf-4 192.168.0.11 192.168.0.111 10.0.128.2/30 10.0.128.18/30 10.0.0.18/30
7. The basic infrastructure connectivity is pre-configured. Check the ospf adjacency by entering show ip ospf neighbor on
Spine-1 and Spine-2. The output shows four OSPF neighbors across four point-to-point links.
NOTE: It is best practice to use a point-to-point OSPF network to avoid DR/BDR election
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 5 of 24
Cisco dCloud
The purpose of this scenario is to establish iBGP between the spine and leaf switches. The spine switches will act as route-
reflectors to the leaf switches in the network. This will help move the routes between leaf switches through the spines and eliminate
the need for a full mesh topology design. All the leaf switches will be dual homed to both spine switches
• from Spine 1 to all the leaf switches (leaf-1, leaf-2, leaf-3, leaf-4)
• from Spine 2 to all the leaf switches (leaf-1, leaf-2, leaf-3, leaf-4)
Steps
1. Enter the following series of commands on Spine-1 to configure the iBGP neighbor template on with the spine switch as a
route reflector, and apply the template to all the leaf switches as iBGP neighbors.
config t
feature bgp
Wait a moment for the prompt to return after the feature bgp command.
router bgp 65000
router-id 192.168.0.6
address-family ipv4 unicast
template peer LEAF-PEER
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
route-reflector-client
neighbor 192.168.0.8
inherit peer LEAF-PEER
neighbor 192.168.0.9
inherit peer LEAF-PEER
neighbor 192.168.0.10
inherit peer LEAF-PEER
neighbor 192.168.0.11
inherit peer LEAF-PEER
3. Enter the following commands on Spine-2 to configure iBGP between Spine-2 and all the leaf switches (leaf-1, leaf-2, leaf-3,
and leaf-4).
config t
feature bgp
Wait a moment for the prompt to return after the feature bgp command.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 6 of 24
Cisco dCloud
5. Enter the following commands to configure BGP on Leaf-1, Leaf-2, Leaf-3 and Leaf-4, which will establish the iBGP neighbor
relationship with Spine-1 and Spine-2 on each leaf. Note that the copy run start command at the end of each group saves
the configuration.
Leaf 1:
config t
feature bgp
Wait a moment for the prompt to return after the feature bgp command.
router bgp 65000
router-id 192.168.0.8
address-family ipv4 unicast
neighbor 192.168.0.6
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
neighbor 192.168.0.7
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
copy run start
Leaf-2:
config t
feature bgp
Wait a moment for the prompt to return after the feature bgp command.
router bgp 65000
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 7 of 24
Cisco dCloud
router-id 192.168.0.9
address-family ipv4 unicast
neighbor 192.168.0.6
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
neighbor 192.168.0.7
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
copy run start
Leaf-3:
config t
feature bgp
Wait a moment for the prompt to return after the feature bgp command.
router bgp 65000
router-id 192.168.0.10
address-family ipv4 unicast
neighbor 192.168.0.6
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
neighbor 192.168.0.7
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
copy run start
Leaf-4:
config t
feature bgp
Wait a moment for the prompt to return after the feature bgp command.
router bgp 65000
router-id 192.168.0.11
address-family ipv4 unicast
neighbor 192.168.0.6
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
neighbor 192.168.0.7
remote-as 65000
update-source loopback0
address-family ipv4 unicast
send-community both
copy run start
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 8 of 24
Cisco dCloud
6. Enter show ip bgp sum on Spine-1 to verify the iBGP neighbors on the spine switches. Optionally, also run the command on
Spine-2. Messages sent and received may vary slightly.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 9 of 24
Cisco dCloud
RP configuration is not supported on leaf switches. It is only supported on spine switches. It is recommended to configure RP only
on the spine node and use the anycast RP (Rendezvous Point) model for load balancing and redundancy.
• Enable PIM
Steps
1. Enter the following commands on Spine-1 and Spine-2 to configure PIM and OSPF on a new loopback interface. Since this is
Anycast, the configuration commands are the same on both servers because they are rendezvous points (RP). Note that the
copy run start command saves the configuration.
config t
feature pim
Wait a moment for the prompt to return after the feature pim command.
interface loopback1
ip address 192.168.0.100/32
ip pim sparse-mode
ip router ospf 1 area 0.0.0.0
copy run start
2. Enter the following commands on Spine-1 to configure PIM Anycast RP for RP redundancy.
ip pim rp-address 192.168.0.100
ip pim anycast-rp 192.168.0.100 192.168.0.6
ip pim anycast-rp 192.168.0.100 192.168.0.7
interface E1/1
ip pim sparse-mode
interface E1/2
ip pim sparse-mode
interface E1/3
ip pim sparse-mode
interface E1/4
ip pim sparse-mode
interface loopback0
ip pim sparse-mode
copy run start
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 10 of 24
Cisco dCloud
interface E1/1
ip pim sparse-mode
interface E1/2
ip pim sparse-mode
interface E1/3
ip pim sparse-mode
interface E1/4
ip pim sparse-mode
interface loopback0
ip pim sparse-mode
copy run start
Leaf-1:
config t
feature pim
Wait a moment for the prompt to return after the feature pim command.
ip pim rp-address 192.168.0.100
interface E1/1
ip pim sparse-mode
interface E1/2
ip pim sparse-mode
interface loopback0
ip pim sparse-mode
interface loopback1
ip pim sparse-mode
copy run start
Leaf-2:
config t
feature pim
Wait a moment for the prompt to return after the feature pim command.
ip pim rp-address 192.168.0.100
interface E1/1
ip pim sparse-mode
interface E1/2
ip pim sparse-mode
interface loopback0
ip pim sparse-mode
interface loopback1
ip pim sparse-mode
copy run start
Leaf-3:
config t
feature pim
Wait a moment for the prompt to return after the feature pim command.
ip pim rp-address 192.168.0.100
interface E1/1
ip pim sparse-mode
interface E1/2
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 11 of 24
Cisco dCloud
ip pim sparse-mode
interface loopback0
ip pim sparse-mode
interface loopback1
ip pim sparse-mode
copy run start
Leaf-4:
config t
feature pim
Wait a moment for the prompt to return after the feature pim command.
ip pim rp-address 192.168.0.100
interface E1/1
ip pim sparse-mode
interface E1/2
ip pim sparse-mode
interface loopback0
ip pim sparse-mode
interface loopback1
ip pim sparse-mode
copy run start
5. Enter show ip pim neighbor on both Spine-1 and Spine-2 to verify PIM neighbors.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 12 of 24
Cisco dCloud
The purpose of this scenario is to enable VXLAN features and configure a VLAN/VNI/Multicast Group based on the table below.
The VLAN ID to VNI segment ID should be one to one mapping. One unique multicast group is mapped to one VNI for BUM traffic
inside this L2 VNI. One L3 VNI is created for VXLAN routing.
The completion of this scenario will enable VXLAN in the entire topology.
NOTE: For VXLAN routing, one L3 VNI is required for each Tenant/VRF.
Steps
1. Enter the following commands on all leaf and spine switches to enable VXLAN.
config t
feature nv overlay
feature vn-segment-vlan-based
nv overlay evpn
copy run start
2. Enter the following commands on Leaf-1, Leaf-2, Leaf-3, and Leaf-4 to configure VLAN 140, VLAN 141 and VLAN 999.
spanning-tree vlan 1,140,141,999 priority 4096
vlan 140
name L2-VNI-140-Tenant1
vn-segment 50140
vlan 141
name L2-VNI-141-Tenant1
vn-segment 50141
vlan 999
vn-segment 50999
copy run start
NOTE: There will be a warning message after adding VLAN 999: Warning: Enable double-wide arp-ether tcam carving if igmp
snooping is enabled. Ignore if tcam carving is already configured. This message can be ignored.
3. Enter the following commands on Leaf-1, Leaf-2, Leaf-3, and Leaf-4 to configure VRF for Tenant-1 and SVI for VLAN/VNI.
vrf context Tenant-1
vni 50999
rd auto
address-family ipv4 unicast
route-target both auto
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 13 of 24
Cisco dCloud
NOTE: There will be a warning message after each vrf member Tenant-1 command: Warning: Deleted all L3 config on interface
Vlanxxx. This message can be ignored.
4. Enter the following commands on Leaf-1, Leaf-2, Leaf-3, and Leaf-4 to configure the VXLAN tunnel interface (NVE).
interface nve1
no shutdown
source-interface loopback1
host-reachability protocol bgp
member vni 50140
mcast-group 239.0.0.140
member vni 50141
mcast-group 239.0.0.141
member vni 50999 associate-vrf
interface nve1
no shutdown
source-interface loopback1
host-reachability protocol bgp
member vni 50140
mcast-group 239.0.0.140
member vni 50141
mcast-group 239.0.0.141
member vni 50999 associate-vrf
5. Enter show nve vni on Leaf-1, Leaf-2, Leaf-3, and Leaf-4 to verify the VXLAN configuration.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 14 of 24
Cisco dCloud
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 15 of 24
Cisco dCloud
Steps
1. Enter the following commands on Spine-1 and Spine-2 to add the l2vpn evpn address family under the existing BGP AS.
config t
router bgp 65000
address-family l2vpn evpn
retain route-target all
template peer LEAF-PEER
address-family l2vpn evpn
send-community both
route-reflector-client
2. Enter the following commands on Leaf-1, Leaf-2, Leaf-3, and Leaf-4 to add the l2vpn evpn address family under the existing
BGP AS.
config t
router bgp 65000
address-family l2vpn evpn
retain route-target all
neighbor 192.168.0.6
remote-as 65000
address-family l2vpn evpn
send-community both
neighbor 192.168.0.7
remote-as 65000
address-family l2vpn evpn
send-community both
3. Enter the following commands on Leaf-1, Leaf-2, Leaf-3, and Leaf-4 to configure RD and RT for VXLAN bridging.
evpn
vni 50140 l2
rd auto
route-target import auto
route-target export auto
vni 50141 l2
rd auto
route-target import auto
route-target export auto
NOTE: For VXLAN fabric with all Cisco Nexus switches, it is recommended to generate RD/RT automatically. For VXLAN fabric
with multi-vendor VTEP devices, it is recommended to manually configure the RT/RD values.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 16 of 24
Cisco dCloud
• Verify the node mac/IP host route being updated via BGP EVPN control plane.
As per the topology, server-1 is connected to port e1/3 on Leaf1 so the interface must be configured as access port and assigned
a VLAN.
Steps
1. Enter the following commands on Leaf-1 to configure Leaf-1 interface E1/3 to be part of VLAN 140.
int e1/3
description to Server-1
switchport mode access
switchport access vlan 140
2. Click the server-1 tab tab in MTPuTTY. If the [root@server-1 ~]# prompt is not showing, log in to Server-1
(root/C1sco12345).
3. Enter sudo ifconfig eth1 172.21.140.10 netmask 255.255.255.0 up to configure interface eth1 be part of VLAN140.
4. Type (do not copy/paste) sudo route add –net 172.21.0.0/16 gw 172.21.140.1 dev eth1 to add the static route for subnet
172.21.0.0/16 to interface Eth1 on Server-1.
NOTE: As per the topology, server-2 is connected to port e1/3 on Leaf1, so it has to be configured as access port and assigned to
a VLAN.
6. Enter the following commands on Leaf-3 to configure Leaf-3 interface E1/3 to be part of VLAN 141.
int e1/3
description to Server-2
switchport mode access
switchport access vlan 141
7. Click the server-2 tab tab in MTPuTTY. If the [root@server-2 ~]# prompt is not showing, log in to Server-2
(root/C1sco12345).
8. Enter sudo ifconfig eth1 172.21.141.10 netmask 255.255.255.0 up to configure interface eth1 to be part of VLAN141.
9. Type (do not copy paste) sudo route add –net 172.21.0.0/16 gw 172.21.141.1 dev eth1 to add the static route for subnet
172.21.0.0/16 to interface Eth1 on Server-2.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 17 of 24
Cisco dCloud
10. Enter ping 172.21.141.1 to ping default gateway 172.21.141.1 from Linux VM Server-2
11. On Server-1, enter the command ping 172.21.141.10 to perform a ping test between Server-1 and Server-2, which will trigger
an EVPN control plane update.
12. Enter the following commands on Leaf-4 to verify the VXLAN EVPN control plane update.
b. Enter show nve peers to verify the dynamic tunnel between VTEPs. If there was a configuration error during the
process, there will be no output.
c. Enter sh ip route vrf Tenant-1 to verify the routing table with the /32 host route learned from BGP.
d. Enter show bgp l2vpn evpn to verify the EVPN control plane update.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 18 of 24
Cisco dCloud
e. Enter show l2route evpn mac-ip all to verify the forwarding table.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 19 of 24
Cisco dCloud
Leaf-4 in this lab is a border leaf that connects to the external network outside the fabric. There are a few options to redistribute the
fabric routes to external entity – this scenario explores the option to use IGP between the border leaf and the WAN, and
redistribute the fabric routes in each Tennant.
Steps
1. If Leaf-4 is not in config mode, showing the Leaf-4 (config) # prompt, enter config t.
2. Enter the following commands on Leaf-4 to configure IGP under VRF Tenant-1.
interface Ethernet1/3
mtu 9216
vrf member Tenant-1
ip address 10.0.0.18/30
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 20 of 24
Cisco dCloud
no shutdown
router ospf 1
router-id 192.168.0.11
vrf Tenant-1
do copy run start
NOTE: After the vrf Tenant-1 command, an error will be generated: Warning: Deleted all L3 config on interface Ethernet1/3.
This error can be ignored.
5. Enter show ip ospf neighbor to verify the OSPF neighbor between WAN and Leaf-4.
6. Enter the following commands on Leaf-4 to configure the border leaf to redistribute Fabric routes into OSPF.
router bgp 65000
vrf Tenant-1
address-family ipv4 unicast
advertise l2vpn evpn
redistribute ospf 1 route-map permit-ospf-bgp
router ospf 1
vrf Tenant-1
redistribute bgp 65000 route-map permit-bgp-ospf
redistribute direct route-map permit-bgp-ospf
route-map permit-bgp-ospf permit 10
route-map permit-ospf-bgp permit 10
8. Enter sudo ifconfig eth1 172.21.1.10 netmask 255.255.255.0 up to configure interface eth1.
9. Type, do not copy/paste, sudo route add –net 172.21.0.0/16 gw 172.21.1.1 dev eth1 to configure the Linux VM Server-3
static route for 172.21.0.0/16 to interface Eth1.
10. Enter ping 172.21.141.10, then ping 172.21.140.10 to verify connectivity from Linux VM Server-3 to Linux VM Server-1 and
Server-2.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 21 of 24
Cisco dCloud
11. In the WAN window, show ip route ospf to verify the fabric host routes on the WAN router.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 22 of 24
Cisco dCloud
4. Right-click the affected server and select Power > Power ON from the resulting menu.
5. Wait until the MTPuTTY session shows a login prompt before proceeding with the demonstration.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 23 of 24
Cisco dCloud
Steps
2. Enter 5 at the prompt. Do not close the command window until the task fully completes.
© 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 24 of 24