Guide - Chap 3 - Cisco NX-OS Software Virtual PortChannel Fundamental Concepts With NXOS 5 PDF

Cisco N
C
NX-OS Softwaare Virttual PoortChan
nnel: Fu
undam
mental
Con
ncepts with
w NX
XOS 5..0
C
Contents
V
Virtual
PortC
Channel Tech
hnology ........................................................................................................................3
vPC Basicss ......................................................................................................................................................4
vPC Peer Link............................................................................................................................................................ 6
vPC Peer--Keepalive or Fault-Toleran
nt Link .............................................................................................................. 7
vPC Portss, and Orphan
ned Ports ............................................................................................................................... 7
vPC Topology with Fab
bric Extenderss ........................................................................................................................ 8
Traffic Flows ...................................................................................................................................................9

Dual-Contrrol Plane witth Single Laayer 2 Node Behavior.............................................................................10
Link Aggreegation Group Identifier ......................................................................................................................... 10
System ID
D in a vPC Systtem ..................................................................................................................................... 12
Primary and Secondaryy vPC Roles ......................................................................................................................... 13
Spanning Tree ........................................................................................................................................................ 13
Cisco Disccovery Protoccol ........................................................................................................................................ 14
Cisco Fabric Services over
o
Ethernet Synchronizattion Protocol ........................................................................... 14
vPC Configguration Chaanges Whenn the Peer Linnk Fails ...............................................................................15

Peer Conffiguration Che
eck Bypass (fo
or Cisco Nexu
us 5000 Series running NXO
OS version inferior to NXO
OS
5.0(2)N1(1))............................................................................................................................................................ 18
vPC Reloaad Restore ................................................................................................................................................ 19
vPC Configguration Con

nsistency ....................................................................................................................20
vPC Consiistency Checkks ......................................................................................................................................... 20
vPC Configuration Syncchronization ....................................................................................................................... 22
Duplicate Frames
F
Prev
vention in vP
PC ............................................................................................................23
vPC and Object
O
Trackiing ..............................................................................................................................25
In-Service Software Up
pgrade and vPC
v .........................................................................................................26
Innteractions Between
B
vPC
C and Routinng ............................................................................................................26
HSRP Gateeway Consid
derations ....................................................................................................................26
HSRP Con
nfiguration an
nd Best Practices for vPC ..................................................................................................... 26
ARP Synch
hronization .............................................................................................................................................. 27
Peer Gateeway ......................................................................................................................................................... 27
Layer 3 Linnk Between vPC Peers .................................................................................................................28
2 - Nexus Data Cennter Design with vPC

v vPC Technnology and Design Considerations DRAFT VERSION Cisco Interrnal Use Only
Layer 3 Linnk to the Co

ore ...............................................................................................................................30
Innteractions with
w Multicaast................................................................................................................................31
IGMP Snooping and vP
PC ..............................................................................................................................31
Protocol Inndependent Multicast
M
andd vPC......................................................................................................32
vP
PC Failure Scenarios
S
.......................................................................................................................................33
vPC Membber Port Failure .............................................................................................................................33
vPC Compplete Dual-A
Active Failuree (Double Faailure) .................................................................................33
vPC Peer-L
Link Failure...................................................................................................................................33
vPC Peer-K
Keepalive Faailure ..........................................................................................................................34
Examples .......................................................................................................................................................34
vP
PC with Fabbric Extenderr Active-Acttive Design ....................
.
........................................................................36
vP
PC Configurration Best Practices
P
.....................................................................................................................38
vPC Domaain Configuration ..........................................................................................................................38
vPC Role and
a Priority ............................................................................................................................................. 38
Reload Reestore ....................................................................................................................................................... 38
Peer Gateeway ......................................................................................................................................................... 39
vPC Peer Link

L ...............................................................................................................................................39
vPC Peer Keepalive....
K
...................................................................................................................................39
vPC Ports.......................................................................................................................................................40
LACP ........................................................................................................................................................................ 41
For More Infoormation .......................................................................................................................................42
V
Virtual
PorttChannel Technology
T
y
Virtual PorrtChannels (vvPCs) allow links that aree physically connected too two different Cisco
switches to appear to a third
t
downstrream device too be coming from a singlee device and as
a part of a
Channel. Thee third device can be a switch, a server, or any other networking device
d
that
single PortC
.
supports IE
EEE 802.3ad PortChannels
P
Cisco NX-O
OS Software vPCs and Cisco
C
Catalystt Virtual Sw
witching Systtems (VSS) are
a similar
(MCEC)
technologiees. For Cisco EtherChanneel technologyy, the term m
multichassis EtherChannel
E
refers to eitther technologgy interchanggeably.
vPC allowss the creationn of Layer 2 PortChanneels that span two switchees. At the tim
me of this
writing, vP
PC is implem
mented on thhe Cisco Nexxus 7000 and
a 5000 Serries platforms (with or
without Cissco Nexus 2000 Series Fabbric Extenderss).

v vPC Technnology and Design Considerations DRAFT Versioon Cisco Internaal Use Only
vPC Basicss
The fundam
mental conceppts of vPC aree described att
http://www
w.cisco.com/enn/US/prod/coollateral/switcches/ps9441/pps9402/white__paper_c11516396.htm
ml.
vPCs consiist of two vPC
C peer switchhes connectedd by a peer linnk. Of the vPC peers, one is primary
and one is secondary.
s
Thhe system formed by the sw
witches is refferred to as a vPC
v domain.
Following is
i a list of som
me possible Cisco
C
Nexus vPC
v topologiees:
vPC on
n the Cisco Nexus
N
7000 Series (topollogy A): Thiis topology consists
c
of acccess layer
switchees dual-homed to the Ciscco Nexus 70000 Series withh a switch PoortChannel wiith Gigabit
Etherneet or 10 Gigaabit Ethernet links. This toopology can also
a
consist of
o hosts connnected with
virtual PortChannelss to each Ciscco Nexus 7000 Series Swittch.
n Cisco Nexxus 5000 Serries (topologyy B): This toopology consists of switcches dualvPC on
connected to the Cissco Nexus 50000 Series witth a switch PoortChannel with
w 10 Gigabit Ethernet
links, with
w
one or more links to
t each Ciscoo Nexus 50000 Series Switch. Like toopology A,
topolog
gy B can connsist of serveers connectedd to each Ciisco Nexus 5000
5
Series Switch
S
via
virtual PortChannells.
vPC on
n the Cisco Nexus
N
5000 Series with a Cisco
C
Nexus 2000 Series Fabric
F
Extender singlehomed (also called straight-through mode) (topology
(
C):: This topoloogy consists of
o a Cisco
Nexus 2000 Series Fabric
F
Extender single-hoomed with onne to eight 100 Gigabit Ethernet links
(depend
ding on the fabric
f
extendeer model) to a single Ciscoo Nexus 50000 Series Swittch, and of
Gigabitt Ethernet or 10 Gigabit Ethernetconn
E
nected serverrs that form virtual
v
PortC
Channels to
the fabric extender devices. Notee that each fabric
fa
extendeer connects too a single Cisco Nexus
5000 Series
can be formeed only by
S
Switch and not to both,
b
and thatt the virtual PortChannel
P
connecting the servver network interface cards (NICs) to two fabric extenders,
e
whhere fabric
extendeer 1 depends on Cisco Neexus 5000 Seeries Switch 1 and fabric extender 2 depends
d
on
Cisco Nexus
N
5000 Series
S
Switchh 2. If both faabric extenderr 1 and fabricc extender 2 depend on
switch 1 or both of them
t
depend on switch 2, the
t PortChannnel cannot bee formed.
Dual-ho
oming of the Cisco Nexuss 2000 Seriess Fabric Exteender (topologgy D): This topology
t
is
also caalled Cisco Nexus
N
2000 Series
S
Fabric Extender (FE
EX for brief)) Active/Active. In this
topolog
gy each FEX
X is connecteed to each Cisco
C
Nexus 5000 Series device withh a virtual
PortChannel. With this topologgy, the serverr cannot creatte a PortChaannel split bettween two
fabric extenders. Thhe servers caan still be dual-homed
d
w
with
active-sttandby or acctive-active
transmiit-load-balanccing (TLB) teeaming.
Note
Topologies B,
B C, and D are not mutually exclusive.
e
Youu can have an architecture
a
thaat uses these
three topologgies concurrently.
Figure 1 illu
ustrates topollogies A and B.
B Figure 2 illlustrates topoologies C andd D.
Figure 1 vPC
v
Topologies A and B

Cisco Ne
exus 7000 Series vPC
C (Topology A)
C
Cisco
Nexus 5000 Se
eries vPC (Topology B)
Peer Keepallive Link

Peer Keepallive Link
VRF for Peer Ke
eepalive
VRF for P
Peer Keepalive
VRF for Peer Keepalive
Peer Link
vPC Member Ports
Host
PortChannel
Sw
witch
PortC
Channel
Switch
PortChannel
Sw
witch
PortC
Channel
2009 Cisco Systems, Inc. All

A rights reserved.
Host
PortChannel
Host
PortChannel
Host
nel
PortChann
h
Switch
PortChan
nnel
Cisco Confidential
Figure 2 vPC
v
Topologies C and D
Cisco Nexus 2000 Series Straight-Through Mode vPC

C (C)
Cisco
Nexus
2148T
2 Gigabit Etthernet Ports

Host PortCh
hannel

A rights reserved.
Cisco Nexus
N
2000 Series Acttive-Active Mode vPC (D)
Cisco
Nexus
2148T
4 Gigabit Ethernet Ports

Host PortChannel
Cisco Confidential
Active-Stan
ndby or
Transmit Load Balancing

Figure 3 illlustrates the main

m
vPC coomponents. Switches 1 annd 2 are the vPC
v
peer swiitches. The
vPC peer switches
s
are connected
c
thrrough a link called a peerr link, also knnown as a multichassis
m
EtherChann
nel trunk (MC
CT).
Figure 3 sh
hows devicess (switch 3, switch
s
4, andd server 2) thhat are conneected to the vPC peers
(which cou
uld be Cisco Nexus
N
7000 or 5000 Seriess Switches). Switches
S
3 annd 4 are configgured with
a normal Po
ortChannel coonfiguration, switches 1 annd 2 are confiigured with a virtual PortC
Channel
Figure 3 vPC
v
Compon
nents
PortC
Channel
Peer Keepalive
K
Link
vPC Me
ember Ports
Peer Link
L
vPC
Primary
vPC
Sec
condary
vPC Domain
Switch
1
Switch
2
Switch
h3

A rights reserved.
Cisco Confidential
Switch4
vPC Peer Link

L
The vPC peeer link is thee most importtant connectivvity element in the vPC syystem. This liink is used
to create the illusion of a single contrrol plane by forwarding
fo
Brridge Protocol data units (B
BPDUs) or
Link Aggreegation Contrrol Protocol (L
LACP) packeets to the prim
mary vPC swiitch from the secondary
vPC switch
h.
The peer link is used to
t synchronizze MAC adddresses betweeen aggregatiion groups 1 and 2, to
synchronizee IGMP entriies for the puurpose of IGM
MP snoopingg, it provides the necessaryy transport
for multicasst traffic and for the comm
munication of orphaned porrts. The term orphaned poorts refers
to switch ports connecteed to single-aattached hostss, or vPC portts whose mem
mbers are all connected
to a single vPC
v peer.
In the case of a vPC deevice that is also
a
a Layer 3 switch, thee peer link allso carries Hoot Standby
Router Prottocol (HSRP)) frames.
For a vPC to
t forward a VLAN,
V
that VLAN
V
must exist
e
on the peer
p
link and on both vPC peers, and
it must app
pear in the allowed list of
o the switch port trunk for
fo the vPC itself. If eitheer of these

conditions is not met, thhe VLAN is not displayedd when you enter
e
the com
mmand show vpc brief,
v VLAN.
nor is it a vPC
When a Po
ortChannel is defined as a vPC peer linnk, Bridge Asssurance is auutomatically configured
c
on the peer link.
vPC Peer-K
Keepalive orr Fault-Tolerrant Link
A routed llink (it is moore accurate to say path)) is used to reesolve dual-acctive scenarioos in which
the peer lin
nk connectivitty is lost. Thhis link is refeerred to as a vPC peer-keeepalive or fauult-tolerant
link. The peer-keepaliv
p
ve traffic is often
o
transporrted over thee managemennt network thhrough the
managemen
nt 0 port of the
t Cisco Neexus 5000 Seeries Switch or
o the managgement 0 porrts on each
Cisco Nexu
us 7000 Seriees supervisor. The peer-keeepalive trafficc is typically routed over a dedicated
VRF, for
Virtual Ro
outing and Foorwarding (V
VRF) instancce (which coould be the management
m
example).
The keepalive can be caarried over a routed infrasstructure; it dooes not need to be a direcct point-topoint link, and, in fact, it is desirabble to carry the
t peer-keeppalive traffic on a differennt network
instead of on
o a straight point-to-point
p
t link.
vPC Ports, and Orphan

ned Ports
A vPC porrt is a port thhat is assigned to a vPC channel grooup. The portts that form the virtual
PortChanneel are split bettween the vPC
C peers and are
a referred too as vPC mem
mber ports.
A non-vPC
C port, also knnown as an orpphaned port, is a port that is not part off a vPC.
Figure 4 sh
hows differennt types ports connected too a vPC systeem. Switch1 and Host 3 connect
c
via
vPCs. Thee ports conneecting devicees in a non-vvPC mode to a vPC topology are refeerred to as
orphaned ports.
p
Switch 2 connectss to the Ciscco Nexus Switch
S
with a regular spaanning-tree
configuratio
on: thus, onee link is forw
warding, and one link is blocking.
b
Theese links connnect to the
Cisco Nexu
us Switch withh orphaned ports.
Figure 4 vPC
v
Ports an
nd Orphan Poorts

Peer Link: VLANs 10, 20, 30, 40, and

d 50
vPC Mem
mber Ports
Forwarding
Blocking
Switch1
Switch2
6
Acttive-Standby
teaming
Host
nel
PortChann
4
Switch
PortChannel

A rights reserved.
Cisco Confidential
Server 6 co
onnects to a Cisco
C
Nexus Switch
S
with ann active-standdby teaming configuration
c
. The ports
that server 6 connects too on the Ciscoo Nexus Switcchare orphaneed ports.
vPC Topology with Fa

abric Extendeers
Figure 5 illlustrates another vPC toppology consissting of Cisco Nexus 50000 Series Sw
witches and
Cisco Nexu
us 2000 Seriees Fabric Exteenders (in strraight-throughh mode: that is, each fabriic extender
is single-atttached to a Ciisco Nexus 50000 Series Sw
witch).
Figure 5 sh
hows devicess that are connnected to thhe vPC peer (Cisco
(
Nexuss 5000 Seriess Switches
5k01 and 5k02)
5
with a PortChannel (a vPC); for example, serrver 2, whichh is configureed for NIC
teaming with the IEEE 802.3ad
8
option.
nd 3 connect to orphan ports.
Servers 1 an
Figure 5 vPC
v
Compon
nents with the Fabric Exttender (FEX))

Peer Keepalive
K
Link
PortC
Channel
vPC Me
ember Ports
vPC
Primary
Peer Link
L
vPC
Sec
condary
Switch1
Switc
ch2
vPC Domain
FEX
FE
EX

A rights reserved.
FEX
FEX
Cisco Confidential
wing componnents:
To summarrize, a vPC syystem consistss of the follow
Two peeer devices: thhe vPC peerss, of which onne is primaryy and one is secondary;
s
booth are part
of a vPC domain
A Layeer 3 Gigabit Ethernet

E
link called
c
a peer-kkeepalive linkk to resolve dual-active
d
sceenarios
A redundant 10 Giggabit Ethernett PortChannel called a peeer link whichh is used to caarry traffic
from on
ne system to the
t other wheen needed andd to synchronize forwardinng tables
vPC meember ports forming

f
the viirtual PortChaannel
Traffic Flows
system is
vPC config
gurations are optimized to help ensurre that trafficc through a vPC-capable
v
symmetric. In Figure 6, for example,, the flow on the left (in blue)
b
reachingg a Cisco Nexxus switch
(Agg1 in th
he figure) from
m the core is forwarded tooward the acccess layer swiitch (Acc1 inn the figure
without traaversing the peer
p
Cisco Neexus switch device
d
(Agg22). Similarly, traffic from the server
directed to the core reacches a Cisco Nexus Switchh (Agg1), annd the receivinng Cisco Nexxus Switch
routes this traffic directtly to the corre without unnnecessarily passing it too the peer Cisco Nexus
device. Thiis process occcurs regardless of which Cisco
C
Nexus device is thee primary HS
SRP device
for a given VLAN.
Figure 6 Traffic
T
Flowss with vPC

Core1
C
Core2
Core1
Co
ore2
Layer 3
Layer 3
1
Agg1
Agg2
Layer 2
Agg2
Agg1
Layer 2
Acc1
Acc2
Acc3
Acc1
1
A B C D
Acc2
E F
A B C D

A rights reserved.
A
Acc3
Cisco Confidential
E F
6
Dual-Conttrol Plane wiith Single Laayer 2 Node Behavior

While still operating with
w
two sepaarate control planes, vPC
C helps ensurre that the neeighboring
devices con
nnected in vP
PC mode see the
t vPC peerrs as a single spanning-treee and LACP entity. For
this to hap
ppen, the systtem has to perform
p
IEEE
E 802.3ad coontrol-plane operations
o
in a slightly
modified way
w (which is not noticeablle to the neighhbor switch).
Link Aggreegation Group Identifierr

IEEE 802.3
3ad specifies the standardd implementattion of PortC
Channels. PorrtChannel speecifications
provide LA
ACP as a standdard protocol, which enablles negotiation of port bunndling.
LACP mak
kes misconfigguration less likely, becauuse if ports arre mismatched, they will not
n form a
PortChanneel.
Consider example A inn Figure 8, inn which swittch 1 conneccts to switch 2. Port 1 onn switch 1
connects to
o port 4 on sw
witch 2, and poort 2 on switcch 1 connects to port 6 on switch
s
2.
Now imagiine that the addministrator configured
c
a PortChannel on switch 1 between portts 1 and 2,
while on sw
witch 2 the PortChannel iss configured between portts 5 and 3. Without
W
LACP
P, the ports
could be configured in
i channel-group mode, and you would
w
not diiscover that this is a
misconfigu
uration until you notice thatt traffic has dropped.
d
LACP discerns that the only ports thhat can be bunndled are portt 1 going to port
p 4. Accordding to the
IEEE specifications, to allow
a
LACP to
t determine whether a sett of links connnect to the saame system
and to deteermine whethher those linkks are compaatible with aggregation, you
y need to be able to
establish tw
wo types of iddentification:
100 - Nexus Data Ceenter Design withh vPC vPC Techhnology and Design Considerations DRAFT VERS
SION Cisco Inteernal Use Only
You neeed a globallyy unique identtifier for eachh system that participates in
i link aggreggation (that
is, the switch itselff needs to be unique. Thiss number is referred
r
to ass the system ID and is
compossed of a prioority and a MAC
M
addresss that uniquuely identifiess the switch.. Figure 7
illustrattes the system
m ID.
You neeed a means of

o identifying a link aggreggation group.
For more information,

i
p
please
refer to
t the IEEE 802.3ad stanndard, Amenddment to Carrrier Sense
Multiple Access
A
with Collision Detection (CS
SMA/CD) Acccess Method and Physiical Layer
Specificatio
onsAggregation of Multtiple Link Seggments.
Figure 7 Components
C
of the System
m ID
In Figure 8, switch 1 annnounces ports 1 and 2 as part

p of the same aggregatiion group, andd similarly
switch 2 an
nnounces portts 4 and 5 as part
p of the saame aggregatiion group. Beecause ports 3 and 6 are
not part of the
t group, theey cannot be bundled
b
with the PortChannnel.
Example A in Figure 8 shows an exttreme case inn which the PortChannel consists of an individual
port only. If
I the negotiaation had failed between switches
s
1 annd 2, the linkss would still operate as
normal, ind
dividual IEEE
E 802.3 links.
Switches 1 and 2 decidee which portss can be bunddled together based on the link aggregaation group
identifier (L
LAGID). Thhis number inncludes the system
s
identiffier (in otherr words, an ID
I for the
physical sw
witch) and a key
k that identifies the aggrregation groupp itself (that is,
i the equivaalent of the
channel gro
oup number).
As a first approximatioon, the LAGIID is compoosed of the system
s
ID off both system
ms and the
channel gro
oup number used
u
in both syystems.
Example B in Figure 8 shows the caase in which the ports aree correctly wiired. Assuminng that the
ports on sw
witch 1 (systtem ID 1) arre bundled ass channel grooup 100 and the ports onn switch 2
(system ID
D 2) are bundled as channeel group 200, the LAGID
D would appear to be as foollows: [1,
100, 2, 200].
Example C in Figure 8 illustrates how
h
the PortC
Channel is reequested betw
ween switch 2 and two
separate up
pstream switchhes, switches 1 and 3, wheere switch 1 and
a 3 form a vPC
v system.
The system
m ID for switcch 1 differs frrom the system
m ID for swittch 3 becausee the MAC adddresses of
the two swiitches are diffferent.
With vPC, the system ID
Ds of switches 1 and 3 are identical, so switch 2 belieeves it is connnected to a
single upstrream device.
Figure 8 LACP
L
Behavvior with Varrious Wiring Configuratiions
111 - Nexus Data Ceenter Design withh vPC vPC Techhnology and Design Considerations DRAFT Versiion Cisco Internnal Use Only
PortChann
nels
vPC Domain
D
Switch 1
Switch 3
Switch 1
C
S it h 1
Switch
vP
PC
Switch 2

A rights reserved.
Switch 2
Cisco Confidential
Switch 2
System ID in a vPC Sysstem

Spanning trree and LACP use the swiitch MAC adddress for, resspectively, thhe bridge ID field
f
in the
spanning-trree BPDU annd as part of LACP LAGIID. In a singgle chassis, thhey use the syystemwide
MAC addreess for this purpose.
p
For systems that use vPCs, usse of the systemwide MA
AC address
would not work
w
becausee the vPC peeers needs to apppear as a sinngle entity as shown in exaample C in
Figure 8. To
T meet this requirement, vPC offers both an automatic configuration andd a manual
configuratio
on of the systtem ID for thee vPC peers.
The automaatic solution implemented by vPC conssists of generration of a system ID com
mposed of a
priority and
d a MAC adddress, with the MAC deerived from a reserved poool of MAC addresses
combined with
w the domaain ID specifiied in the vPC
C configuratioon. The domaain ID is encooded in the
last octet an
nd the trailingg 2 bits of the previous octet of the MAC
C address.
By configu
uring domain IDs to be diifferent on addjacent vPCss complexes (and to be iddentical on
each vPC peer compleex), you willl help ensuree the uniqueeness of the system ID for LACP
negotiation
n purposes. Yoou also help ensure that thhe spanning-ttree BPDUs use
u a MAC adddress that
is representtative of the vPC
v complex..
You can ov
verride the auutomatic geneeration of thee system ID by
b using the command-lin
c
e interface
(CLI) and configuring
c
thhe system ID on both vPC peers manuallly, as follow
ws:
(config-vpc
c-domain)#syst
tem-mac <mac>
Primary an
nd Secondarry vPC Roless
In a vPC sy
ystem, one vP
PC switch is defined as prrimary and onne is defined as secondaryy, based on
defined priiorities. The lower numbber has highher priority, so it wins. Also, these roles are
nonpreemptive, so a deevice may bee operationallly primary, but
b secondaryy from a connfiguration
perspectivee.
To understaand the operaational role of a vPC mem
mber, you neeed to considerr the status of the peerkeepalive liink and the peeer link.
When the two
t
vPC systeems are joineed to form a vPC
v
domain, the priority decides
d
whichh device is
the vPC priimary and whhich is the vP
PC secondaryy. If the primary device were
w
to reload, when the
system com
mes coming online
o
and coonnectivity to the vPC seccondary devicce (now the operational
o
primary) iss restored, thee operationall role of the secondary deevice (operatiional primaryy) will not
change, to avoid unneccessary disrupptions. This behavior is achieved witth a sticky-bit method,
whereby th
he sticky infoormation is noot saved in thhe startup configuration, thus
t
making the device
that is up and
a running win
w over the reloaded devvice. Hence, the vPC prim
mary becomees the vPC
operationall secondary.
If the peerr link is discconnected buut the vPC peers
p
are stilll connected through the vPC peer
keepalive liink, the vPC operational
o
rooles stay unchhanged.
If both the peer link andd peer-keepalive link are disconnected,
d
both vPC peeers become operational
o
primary, bu
ut upon reconnnection of the
t peer-keeppalive link annd the peer liink, the vPC secondary
device (opeerational prim
mary) keeps thhe primary roole, and the vP
PC primary becomes
b
the operational
o
secondary device.
d
Spanning Tree
vPC modifiies the way inn which spannning tree workks on the switch to help ennsure that the vPC peers
entity on vPC
in a vPC do
omain appearr as a single spanning-tree
s
C ports. Alsoo, vPC helps ensure
e
that
devices can
n connect to a vPC domaiin in a non-vvPC fashion with
w classic spanning-tree
s
e topology.
vPC is desiigned to suppport hybrid toopologies. Deepending on the Cisco NX
X-OS Softwaare release,
this can be achieved in slightly
s
different ways.
o NX-OS releases, the peerr link is alwayys forwardingg because of the
t need to maintain
m
the
In all Cisco
MAC addreess tables andd Internet Grooup Managem
ment Protocol (IGMP) entriies synchronizzed.
vPC by deffault ensures that only thee primary swiitch forwards BPDUs on vPCs.
v
This moodification
is strictly limited
l
to vP
PC member ports.
p
As a result,
r
the BP
PDUs that may
m be receivved by the
secondary vPC
v peer on a vPC port arre forwarded to the primarry vPC peer thhrough the peeer link for
processing.
Note
Non-vPC porrts operate likee regular spanning-tree ports. The special beehavior of the primary
p
vPC memberr applies uniqueely to ports thaat are part of a vPC.
Starting fro
om Cisco NX
X-OS Releasees 4.2(6) and 5.0(2), vPC allows the user
u
to choosee the peerswitch optio
on. This optioon optimizes the behavior of spanning tree
t with vPC
C as follows:
The vPC primary annd secondary are both root devices and both
b
originatee BPDUs.
The BP
PDUs originated by bothh the vPC primary and the
t vPC secondary have the same
designaated bridge ID
D on vPC portts.
The BP
PDUs originatted by the vP
PC primary annd secondary on non-vPC ports
p
maintaiin the local
bridge ID instead off the vPC briidge ID and advertise
a
the Bridge ID off the vPC sysstem as the
root.
witch option has

h the follow
wing advantagges:
The peer-sw
It reducces the trafficc loss upon resstoration of thhe peer link after
a
a failure.
It reduces the disruuption associaated with a dual-active

d
faailure (whereeby both vPC
C members
becomee primary). Both
B
devices keep
k
sending BPDUs withh the same brridge ID inforrmation on
vPC meember ports, which
w
prevennts errdisable from potentiaally disablingg the PortChannnel for an
attached device.
PDUs if the prrimary and seecondary rolees change.

It reducces the potenttial loss of BP
Cisco Disco
overy Protoccol
From the perspective
p
off the Cisco Discovery Prottocol, the preesence of vPC
C does not hide the fact
that the two
o Cisco Nexus Switches arre two distinctt devices, as illustrated
i
by the followingg output:
tc-nexus5k0
01# show cdp neigh
n
Capability Codes: R - Ro
outer, T - Tra
ans-Bridge, B - Source-Rou
ute-Bridge
S - Sw
witch, H - Hos
st, I - IGMP, r - Repeater
r,
V - Vo
oIP-Phone, D - Remotely-Ma
anaged-Device,
s - Su
upports-STP-Di
ispute
Device-ID
Lo
ocal Intrfce Hldtme
H
Capability Platfor
rm
tc-nexus7k0
01-vdc2(TBM121
162254)Eth2/1
tc-nexus7k0
02-vdc2(TBM121
193229)Eth2/2
158
158
R S I s
R S I s
N7K-C
C7010
N7K-C
C7010
Port ID
I
Eth2
2/9
Eth2
2/9
Cisco Fabrric Services over

o
Etherneet Synchroniization Proto
ocol
The vPC peeers use the Cisco
C
Fabric Services
S
protoocol to synchrronize forwarrding-plane innformation
and implem
ment necessaryy configuratioon checks.
vPC peers must syncrhoonize the Layyer 2 forwardding tablethhat is, the MA
AC address innformation
between thee vPC peers. This way, if one
o vPC peerr learns a new
w MAC address, that MAC
C address is
also prograammed on the Layer 2 forw
warding table of the other peer
p device.
The Cisco Fabric
F
Servicces protocol travels
t
on the peer link andd does not require any connfiguration
by the user..
To help en
nsure that thee peer link communicatio
c
on for the Cisco
C
Fabric Services oveer Ethernet
protocol is always availlable, spanninng tree has been
b
modifiedd to keep the peer-link poorts always
forwarding.
c
y checks to
The Cisco Fabric Servicces over Etheernet protocoll is also usedd to perform compatibility
m
ports to form the channel, to synchronize the IGMP
validate thee compatibiliity of vPC member
snooping status,
s
to moonitor the staatus of the vPC
v
member ports, to syynchronize thhe Address
Resolution Protocol (A
ARP) table (sstarting from
m Cisco NX--OS 4.2(6) and
a
future Release
R
5.0
releases).
PC peers is
If the peer link is disconnnected betweeen two vPC peers, the syynchronizationn between vP
l
to trafficc drop for mullticast traffic and to floodinng for unicastt traffic.
interrupted,, which may lead
vPC Configguration Chaanges When

n the Peer Liink Fails
The correctt sequence for setting up vPCs
v
requiress that the two participatingg vPC switchees see each
other over the
t peer link and
a that they can communnicate over thee vPC peer keeepalive link.
Figure 9 illlustrates this fundamentall concept: thee configuratioon depicted inn Figure 9 (22) requires
starting fro
om Figure 9 (1). If you try
t to configgure a vPC liike in Figuree 9 (2) withoout having
established vPC peer-llink and vPC
C peer keeppalive conneectivity, vPC
C ports wont go into
forwarding state.
a in the statte depicted inn Figure 9 (1) (which is a fully formedd vpc domainn) the peer
Once you are
keepalive connectivity
c
i not strictlyy required in order to creaate or modifyy vPCs. In othher words,
is
you can con
nfigure the vP
PC in Figure 9 (2) even iff theres a losss of vPC peerr keepalive coonnectivity
after the configuration depicted
d
in Figgure 9 (1).
mmended to have
h
functional vPC peer keepalive
k
connnectivity for the correct
It is necessaary and recom
behavior off vPC in pressence of failuures, but from
m a traffic forrwarding and configuration purpose,
the temporaary failure of the peer keeppalive link dooesnt have anny impact.
The vPC peer-link connnectivity is more
m
importannt for the corrrect traffic foorwarding opeerations as
well as for the ability to create or moddify vPCs.
Figure 9 Set
S Up for vP
PC Domain, Peer
P
Link, an
nd vPC Mem
mber Ports
(1)
(2)
Peer Keep
palive Link
Peer Link

A rights reserved.
Cisco Confidential
15
The initial implementattion of vPC did

d not allow
w configuratioon changes when
w
the peer link was
disconnecteed to avoid, upon
u
reconnecction of the peer
p
switch, innconsistenciees that could bring
b
these
links down. Thus, if thee peer-link waas lost, the intterfaces on thhe vPC primarry could not be
b flapping
(i.e. if they
y went down, they stayed down
d
up untiil the peer-linnk was restoreed), and while the peer
link was do
own, a new vP
PC member port
p could nott be activated..
Example A in Figure 100 shows the case of a single vPC peerr. The user caannot activatee any vPC
member po
ort until a vPC
C peer switch is present (exxample B in Figure
F
10).
Figure 10 vPC Peers Must
M
Be Connected for In
nterfaces to Be
B Activated
d
Peer Keepa
alive Link
vPC
Primary
vPC
Primary
Switch
h
1
Switch
1
M
Missing vPC Peer
vPC
Secondary
Switch
2
Switch3
Switch4
Switc
ch3

A rights reserved.
Switch4
10
Cisco Confidential
Because off this behaviorr, if the peer-llink connectioon is lost, by default the user cannot addd any vPC
ports and activate
a
them, nor can an innterface flap.. If a vPC intterface flaps, the port will stay down
after flappin
ng.
For examplle, imagine a vPC setup wiith PortChannnel 8 configurred as vPC 8:
vPC status
-------------------------------------------------------------------------------id
Port
Status Consistency Reason
Active vlans
------ ------------ ------ ------------ --------------------------- -----------8
Po8
up
success
success
23,50
After the peeer-link failurre only the prrimary keeps the vPC interrfaces up. If the interface associated
with PortCh
hannel 8 flapss, it never goees up again.
vPC status
-------------------------------------------------------------------------------id
Port
Status Consistency Reason
Active vlans
------ ------------ ------ ------------ --------------------------- -----------8
Po8
down failed
Peer-link is doown
To restore connectivity, you need to

t first restorre the peer liink, and thenn enter a shu
ut/no shut
u
the interface on the primary
p
devicce.
command under
Similarly, if
i you want to
t create a new vPC interrface and to activate it, a vPC peer needs to be
present and
d connected through
t
the peer
p
link. While a peer link is not coonnected, vPC
C prevents
activation of
o any new vP
PC member port.
This vPC behavior may manifest itsellf in several scenarios:
s
A vPC pair in whicch the peer link is lost butt the peer-keepalive is still connected (shown as
case 1 in
i Figure 11)
A vPC pair in whichh peer link and peer-keepallive links are lost (split braain)
p of a vPC
C but has beenn reloaded; upon
u
coming online, the vPC peer is
A switcch that was part
unavaillable (shown as case 2 in Figure
F
11)
A vPC switch that has

h never beeen part of a vPC
v
domain because
b
it has just been powered up
and con
nfigured for vPC,
v
but no peer-link
p
and peer-keepalivve connectiviity has been established
e
yet
narios: Peer Link

L
Loss (11) and Reload
d (2)
Figure 11 Failure Scen
(1)
(2)
Peer Keep
palive Link
Peer Link

A rights reserved.
Cisco Confidential
16
Case 1 haas been adddressed by the

t
Cisco Nexus
N
7000 Series with Cisco NX-O
OS 4.2(3)
(CSCsz674
416). This sollution allows the vPC deviice to shut/noo shut existingg vPC membber ports as
long as the vPC secondary device caan still be reaached throughh the peer-keeepalive link. This same
behavior on the Cisco Nexus 50000 Series requuires configurration of the keyword peeer-checkconfig-byp
pass under thhe vPC domaain configurattion, but this process is superseded
s
w NXOS
with
5.0(2)N1(1) which integgrates CSCsz667416.
For case 1,, to add a neew vPCs you need the reload restore command off the Cisco Nexus
N
7000
Series and on
o the Cisco Nexus
N
5000 Series
S
running NXOS 5.0((2)N1(1) or higher. You can achieve
the same reesults with peer-check-co
p
onfig-bypass for the Ciscco Nexus 50000 Series if running
r
an
earlier version of code.
b
the vPC
v primary and
a secondaryy devices (spllit brain) is
The case off complete dissconnection between
addressed in
i the Cisco Nexus
N
7000 Series
S
or Cisco Nexus 50000 Series runnning NXOS 5.0(2)N1(1)
or higher by both CSCsz67416 (for existing
e
vPC member portts) and reload
d restore (forr new vPC
member po
orts). The equuivalent comm
mand on the Cisco Nexus 5000 Series for earlier releases is
peer-check
k-config-bypaass.
The third caase, reload off a vPC device, is addresseed with the vP
PC reload resstore commannd.
The resoluttion of the fouurth case is not
n currently contemplatedd in any Ciscoo NX-OS releease. For a
vPC port to
t be activatted, the user is expected to first creaate a functioonal vPC connfiguration
composed of
o two vPC peeer switches as
a it is depicteed in Figure 9 (1).
In summary
y starting from
m NXOS 5.0((2)N1(1) youu should confiigure the reload restore opption in the
vpc domain
n on both thhe Cisco Nexxus 7000 Serries Switchess and the Cissco Nexus 50000 Series
Switches.
Peer Confiiguration Ch
heck Bypass (for Cisco Nexus
N
5000 Series
S
runnin
ng NXOS verrsion
inferior to
o NXOS 5.0(2
2)N1(1))
To overridee the default vPC behavioor, which prevvents activatiion of new vP
PC member ports
p
when
the peer lin
nk is down, yoou can use the command peer-config-c
p
check-bypasss under the vP
PC domain
configuratio
on (on the Cisco Nexus 50000 Series onlly).
As an exam
mple:
vpc domain 2
role prio
ority 100
peer-keep
palive destin
nation 10.51.35.18
peer-conf
fig-check-byp
pass
The peer-config-check--bypass comm

mand was intrroduced in Cisco NX-OS 4.1(3)N2(1), and it will
be supersed
ded by the vPC reload resttore commannd when that command
c
is available.
a
With this co
ommand in place, even if the
t Cisco Nexxus 5000 Seriies peer link is
i down, a vP
PC member
port can flaap, and you caan also create new vPC meember ports annd activate thhem.
Figure 12 illustrates the peer-config--check-bypasss feature. Yoou need to staart from a connfiguration
(1) in which
h the vPC peers are conneected with thee peer link. Iff the peer linkk fails, the vP
PC member
ports stay active
a
on the primary deviice (2). If a port
p flaps or if
i you add a new
n vPC member port,
these ports can be activaated. (3).
Figure 12 The peer-config-check-b
bypass Featurre
vPC
Primary
Switch1
vP
PC
Secondary
Switch2
Switch3

A rights reserved.
Switch
h1
Swiitch3
Cisco Confidential
Swiitch2
Switch1
1
Switch2
Swiitch3
12
w the peerr link down, you

y need to be cautious whhen adding
With this configuration in place and with
new vPCs. The configgurations need to be reeplicated on both devicees to avoid a Type-1
ncy at peer-linnk restoration.
inconsisten
If the primaary switch relloads, the maachine restartss as if the vPC domain nevver existed: that
t
is, you
cannot actiivate existingg vPC member ports or create
c
and acctivate new vPCs
v
until a functional
redundant vPC
v
configurration is put inn place (whicch means untiil the primaryy device is coonnected to
a vPC peer that is up andd running).
vPC Reload Restore

The vPC reload
r
restorre feature was introducedd with Ciscoo NX-OS 5.00(2). With vP
PC reload
restore, wh
hen the peer link
l
is down,, or when thee both peer linnk and peer-kkeepalive linkks are lost,
the vPC priimary can acttivate new vPCs ports.
If the vPC
C peer switchh is brought online or connected to the existing vPC peer switch,
s
the
configuratio
on checks aree performed; if
i inconsistenncies are found, the vPC poorts are shut down.
d
In addition,, with reload
d restore a vP
PC peer can reload, and affter a reload, if
i no vPC peeer exists (a
condition th
hat is detectedd by a timer), the user can create vPCs on
o the standalone vPC peeer.
Upon reloaad, Cisco NX--OS starts a user-configur
u
rable timer (w
with a default of 240 seconnds). If the
peer-link port comes upp physically or
o if the peer keepalive is functional, thhe timer is sttopped and
the device waits for the peer adjacenncy to form. If
I at timer exxpiration no peer-keepalive
p
e- or peerlink-up pacckets have been
b
receivedd, Cisco NX
X-OS assumees the primaary role. Thee software
reinitializess the vPCs, acctivating its loocal ports. Beecause there are
a no peers, the
t consistenccy check is
bypassed fo
or the local vP
PC ports.
The timer is
i user configgurable and defines
d
how long
l
the standdalone vPC device
d
waits to
t detect a
vPC peer. If
I at the timerr expiration no
n peer-keepaalive- or peer--link-up packkets have beenn received,
the softwarre reinitializees the vPCs, activating itts local ports. Because there
t
are no peers, the
consistency
y check is byppassed for thee local vPC poorts.
The follow
wing output shhows the stattus of a virtuual PortChannnel configured on a standdalone vPC
system with
h restore reloaad:
------------------------------------------------------------------------id Port Status Consistency Reason
A
Active
vlans
-- ---- ------ ----------- --------------------------- ------------51 Po51 up
successs
Type checcks were bypassed 10-14, 21-24
for thhe vPC
,50,60
vPC Configguration Con

nsistency
The Ethern
net PortChannnel capabilityy allows linkss to be bundlled to form a single entityy if certain
compatibiliity conditionss are met. Thee following iss a list of connditions that are
a verified beefore ports
can form a regular PorttChannel (thiis list refers to regular PoortChannels, not vPCs specifically).
Members must:
m
Have th
he same port mode
m
configuured
Have the
t same speeed configureed; if they arre configuredd with speedd AUTO, theey have to
negotiaate the same speed when they becomee active, and if a memberr negotiates a different
speed, it
i will be susppended
Have th
he same maxiimum transmiission unit (M
MTU) value coonfigured
Have th
he same dupleex mode conffigured
Have th
he same Etherrnet layer (sw
witchport or no switchport)) configured
Not be SPAN ports
Have th
he same storm
m control configured
Have th
he same flow control confiigured
Have common capabbilities
(
2)
Be swittching ports (Layer
Have th
he same port access VLAN
N
Have th
he same port native
n
VLAN
N
Have th
he same port--allowed VLA
AN list
vPC Consisstency Checkks

Similar to regular PorttChannels, virtual
v
PortChhannels are subject to consistency checks and
compatibiliity checks. During a coompatibility check, one vPC peer conveys connfiguration
information
n to the othher vPC peeer to verify that vPC member
m
portss can actuallly form a
PortChanneel. For exampple, if two poorts that are going

g
to join the channel carry a diffeerent set of
VLANs, this is a misconnfiguration.
m either warn
w
the useer (Type-2
Depending on the seveerity of the misconfiguraation, vPC may
uration) or susspend the PorrtChannel (Tyype-1 misconnfiguration). In
I the specificc case of a
misconfigu
VLAN mismatch, only the
t VLAN thaat differs betw
ween the vPC
C member porrts will be susspended on
all the vPC PortChannells.
You can veerify the conssistency between vPC peerrs by using thhe command show vpc coonsistencyparameterr:
tc-nexus5k0
02# show vpc consistency-pa
c
arameter
Inconsisten
ncies can be global
g
or interface specific::
Global inconsistenciies: Type-1 global

g
inconsiistencies affecct all vPC meember ports (but
(
do not
affect non-vPC
n
portss).
Interfacce-specific innconsistencies: Type-1 innterface-speciific inconsisttencies affectt only the

interfacce itself.
Examples of
o areas wheree Type-1 incoonsistencies may
m occur incclude:
Multiplle Spanning Tree

T (MST) reegion definitiion (VLAN-too-instance maapping)
MTU value
v
Spannin
ng-tree globaal settings (Brridge Assurannce, loop guarrd, and root guuard)
Configu
uration changges to the folllowing (these affect onlyy individual vPCs
v
for all VLANs
V
on
the vPC
C):
o
PortChannell mode
Trunk modee
Spanning-tree interface settings
matched qualitty-of-service (QoS) definiitions were orriginally Type-1 inconsisteencies, but

Note Mism
in newer reeleases are Tyype-2 inconsisstencies. For the Cisco Nexus 5000 Serries, starting from
f
Cisco
NX-OS 5.0
0(2)N1(1) QoS inconsistenncies are categgorized as Tyype 2, and so they do not bring
b
down
vPC member ports if thee configuratioons differ betw
ween vPC peeers.
The main in
nconsistenciees that you neeed to be awaare of are listeed in Table 1.
1 This table also
a shows
which inconsistencies arre global (thaat is, which bring down alll vPCs) and indicates
i
recoommended
operations to
t avoid disruuptions.
Table 1 vP
PC Consisten
ncy Checks
Type-1 Inco
onsistency
Impact
Recommendation
VLAN to MST Reg

gion mapping
mismatch
Global
Pre-provision and MAP all

M
region
VLANs on the MST
System MTU
Global
Operate change during

e window
maintenance
Rapid-PVST+ Asy
ymmetrically
Disabled
Global
Disabling STP is NOT a Best

ce
Practic
gs (BA, Loop
STP global setting
Guard, Root Guarrd)
Global
Use perinterrface STP

configura
ations
STP Mode misma

atch
Global
None (Network
misconfiguration)
Port-channel mod
de (active/on)
vPC

e window
maintenance
Port MTU/Link Sp
peed/Duplex
mode/QoS
vPC

e window
maintenance
N
VLAN
Trunk mode and Native
vPC

e window
maintenance
STP interface setttings
vPC

e window
maintenance
VLAN on vPC
Acceptable impact
Asymmetric VLAN
Ns on the trunk

A rights reserved.
Cisco Confidential
vPC Config
guration Syn
nchronizatio
on
A vPC allo
ows two linkss that are phyysically conneected to two Cisco
C
Nexus switches to appear
a
as a
single PortC
Channel. Som
me configurattions must bee identical onn both switchhes for vPCs to
t forward
traffic. Such
h configuratioons include port
p mode, chaannel mode, speed,
s
and duuplex.
The config-sync com
mmand simpllifies the management
m
of vPCs byy synchroniizing vPC
configuratio
ons between primary
p
and secondary
s
vPC peers.
vPC config
g-sync is currently availablle on the Ciscco Nexus 50000 Series startting from Cisco NX-OS
5.0(2)N1(1).
g-sync featuree uses the cooncept of the configuratioon profile. Thhe switch proofile is the
The config
construct th
hat allows coonfigurations to be appliedd both locallyy and on thee config-sync peer. The
config-syncc peer definitiion is indepenndent of the vPC
v
peer deffinition and iss specified in the switch
profile conffiguration moode as followss:
Nexus5000(co
onfig-sp)# syn
nc-peers desti
ination {desti
ination IPs}+ [source <source IP> |
vrf
f <vrf>]
Note: Even if the connfig-sync peer is the saame as the vPC peer deevice, the config-sync
infrastructu
ure has been designed
d
so thhat it can be decoupled
d
from vPC. Thuss, you need too define the
config-syncc peer even inn presence of a vPC configguration.
After the co
onfig-sync peeer has been defined, the configurationn that uses vP
PC config-syn
nc appears
as follows:
Switch# conf
fig sync
Switch(confi
ig-sync)# swit
tch-profile profiledefiniti
ion
Switch(confi
ig-sp)# interf
face Port-chan
nnel100
Switch(confi
ig-sp-if)# int
terface Ethern
net1/1
Switch(confi
ig-sp-if)# cha
annel-group 10
00
Switch(confi
ig-sp-if)# exi
it
Switch(confi
ig-sp)# commit
t
Configuratiions are appllied only afteer the user ennters a comm

mit commandd. The configguration is
synchronizeed with the remote peerr through thee mgmt0 inteerface using routable Cissco Fabric
Service pro
otocol over IP
P. If the remoote peer cannnot be reacheed, the configguration is appplied only
locally.
fo
the twoo-phase comm
mit approach: If the configg-sync peer is reachable,
All committ operations follow
either the configuration
c
is fully comm
mitted on bothh peers or it is
i rolled backk on both. If the
t configsync peer is not reachabble, then the configuration
c
n is applied onnly locally. When
W
the peeer becomes
reachable, the
t configurattions are merged.
Duplicate Frames Prevvention in vPC

v
One of the most importaant forwardinng rules for vP
PC is that a frrame that enteers the vPC peer
p
switch
from the peeer link cannoot exit the swiitch from a vP
PC member port.
p
Figure 13 shows
s
switchees 3 and 4 connnected to 5kk01 and 5k02 with vPCs Poo51 and Po522. If one of
the hosts co
onnected to sw
witch 4 sendss either an unknown unicasst or a broadccast, this trafffic may get
hashed to port
p eth2/2 onn PortChannell 52. 5k02 recceives the brooadcast and neeeds to forwaard it to the
peer link fo
or the potentiaal orphan portts on 5k01 to receive it.
Upon receiiving the brooadcast, 5k011 detects thatt this frame is
i coming from the vPC peer link.
Therefore, it does not foorward it to port
p 2/9 or 2//10; if it did, a duplicate fframe on swittch 3 or 4,
respectively
y, would be created.
c
If a host on
n switch 4 sennds a broadcasst, 5k02 will correctly forw
ward it to Po551 on port 2/99 and place
it on the peer
p
link. 5k001 will preveent this broadcast frame from exiting onto port 2/9 or 2/10
because this frame enterred 5k01 from
m a vPC peer link.
l
Should eth2/2
e
on swiitch 3 go dow
wn, port 2/9
on 5k01 would become an orphan port
p and as a result will reeceive traffic that traversees the peer
link.
Figure 13 vPC Does Not Introducee Duplicate Frames
F
5k01
5k022
2/9
2
2/10
2/9
eth2/1
eth2/1
Po51
Switch3
2/10
eth2
2/2
eth2/2
Po52
Switc
ch4
Broadcast o
or Multicast Floo
oding on Local vP
PC Ports
Broadcast o
or Multicast Floo
oding from Peer Link
L
A rights reserved.
Cisco Confidential
14
mportant to realize that a toopology basedd on PortChaannels does noot introduce loops, even
It is also im
if the peer link
l
is lost and all the portss are forwardiing. Figure 144 shows why..
Figure 14 shows
s
the woorst-case scennario of a vPC
C dual-activee failure in whhich both peeer-link and
peer-keepallive-link connnectivity are lost.
l
In this paarticular case, one switch is
i running spaanning tree
(switch 4) with links thhat are not in PortChannell mode, and the
t other switches are connfigured in
PortChanneel mode.
With all lin
nks forwardinng, a broadcaast frame or an unknownn unicast geneerated on sw
witch 4, for
example, iss forwarded on
o both links directed
d
to sw
witches 1 andd 2. When these two frames arrive on
switch 3, th
hey are not sent back to thhe PortChannnel because thhat breaks thhe basic rule of
o Layer 2
forwarding: a frame cannnot return to the
t port from
m which it origginated.
Figure 14 Worst Case of Dual-Actiive Failure
S
Switch1
2/9
eth2/1
Switch2
2/10 2/9
etth2/1
eth2/2
S
Switch3

A rights reserved.
2/10
2
eth2/2
Switch4
Cisco Confidential
vPC and Object

O
Trackiing
The object tracking featture availablee in NXOS can be used too associate thhe vPC statuss of a vPC
device with
h the status of the interfacces that are trracked. The following
fo
expplample clariffies. At the
time of thiss writing Objeect Tracking is
i only availabble on the Cissco Nexus 70000 Series.
You can use
u a single 10
1 Gigabit Ethernet
E
card on the Ciscoo Nexus 70000 Series for both core
connectivity
y and the peeer link, but thhis is not the best option. If you lose the
t 10 Gigabit Ethernet
card on thee vPC primaryy device, youu lose not onnly core conneectivity, but also
a
the peer link. As a
result, portss will be shut down on the peer vPC devvice, isolatingg the servers completely.
c
You can ad
ddress this speecific configuuration requirrement with a tracking connfiguration. The
T objects
being track
ked are the upplinks to the core
c
and the peer
p
link. If thhese links aree lost, vPCs local to the
switch are shut
s
down so that traffic caan continue on
o the vPC peer.
To configurre this featuree, use the following comm
mand syntax:
! Track the
e vpc peer lin
nk
track 1 int
terface port-c
channel110 lin
ne-protocol
! Track the
e uplinks to the
t core
track 2 int
terface Ethern
net7/9 line-pr
rotocol
track 3 int
terface Ethern
net7/10 line-p
protocol
a tracked ob
bjects into on
ne.
! Combine all
! OR mean
ns if ALL obje
ect are down, this object will
w
go down
! --> we ha
ave lost all connectivity
c
t the core and
to
a the peer link
l
track 10 li
ist boolean OR
R
object 1
object 2
object 3
! If object
t 10 goes down
n on the prima
ary vPC peer,
! system wi
ill switch ove
er to other vP
PC peer and disable
d
all lo
ocal vPCs
vpc domain 1
track 10
In-Service Software Upgrade

U
and vPC
In presencee of vPC, youu can upgrade a device such as a Cisco Nexus
N
7000 Series
S
Switchh using InService Sofftware Upgraade (ISSU) wiith no disrupttion to the traaffic. Howeveer, if someonne modifies
the vPC co
onfiguration during
d
the uppgrade, it willl cause an innconsistency between thee vPC peer
devices (thee one being upgraded
u
and the other devvice).
To avoid this
t
undesirabble situation,, vPC can loock the confi
figuration on the device that
t
is not
undergoing
g the upgrade and release itt when the uppgrade is com
mplete.
Starting fro
om Cisco NX--OS 4.2(1)N11(1), you can upgrade the Cisco
C
Nexus 5000 Series with
w ISSU.
In this case, the controol plane can be unavailabble for up too 80 secondss, while the data plane
continues to
t forward trraffic. Becausse of this behavior, for Cisco
C
Nexus 5000 Series Switch to
undergo ISSU, it cannott be the root switch
s
of a Layer
L
2 topoloogy, nor can it
i have designnated ports
(except edg
ge ports and thhe vPC peer-llink).
To use ISSU
U on a Ciscoo Nexus 5000 Series vPC topology,
t
makke sure that thhe peer keepaalive is not
a Layer 2 liink between the
t Cisco Nexxus 5000 Series Switch(thiis would be deesignated porrt).
ISSU on th
he Cisco Nexuus 5000 Seriees requires Brridge Assurannce to be disaabled on all linnks except
the vPC peer link. You can
c then use ISSU with thhe vPC peer liink configureed for Bridge Assurance
(which is th
he default connfiguration).
In
nteractions Between
n vPC and Routing
vPC and ro
outing concuurrently coexiist without prroblems on the
t same swiitch. A Layerr 3 switch
configured for vPC provvides an aggreegation layer that is Layer 3 connected to the core annd Layer 2
connected to
t the access layer
l
with vP
PCs.
Be sure to distinguish
d
beetween a desiign where thee vPC switch routing on Layer
L
3 or 2 liinks, and a
design wheere the vPC switch is speecifically excchanging routting updates over the Layyer 2 vPC
links. This latter scenarrio is typicallly relevant onnly to data center
c
intercoonnect (DCI) designs, a
topic that iss not discusseed in this guidde.
HSRP Gate
eway Consid
derations
The use off HSRP in thhe context off vPC does not
n require anny special configuration. The
T active
HSRP interrface answerrs ARP requeests like norm
mal HSRP deployments
d
d but with vPC both
do,
HSRP interrfaces (active and standby)) can forward traffic.
HSRP Conffiguration and

a Best Practices for vP
PC
The configu
uration on thee HSRP primaary device loooks like this:
interface vLAN50
v
no shutdo
own
ip addres
ss 10.50.0.251
1/24
hsrp 50
preempt
t delay minimu
um 180
priorit
ty 150
timers 1 3
ip 10.5
50.0.1
The configu
uration on thee HSRP seconndary device looks like thiis:
interface vLAN50
v
no shutdo
own
ip addres
ss 10.50.0.252
2/24
hsrp 50
preempt
t delay minimu
um 180
priorit
ty 130
timers 1 3
ip 10.5
50.0.1
fference betweeen the HSRP

P implementaation of a nonn-vPC configuuration and
The most siignificant diff
a vPC conffiguration is thhat the HSRP
P MAC addressses of a vPC
C configuratioon are program
mmed with
the G (gateeway) flag onn both system
ms, compared with a non-vvPC configuraation, in whicch only the
active HSR
RP interface caan program thhe MAC addrress with the G flag.
Given this fact, routablee traffic can be forwarded by
b both the vP
PC primary device
d
(with HSRP)
H
and
the vPC seecondary deviice (with HS
SRP), with noo need to sennd this trafficc to the HSR
RP primary
device.
M
address would not bee routed.
Without thiis flag, trafficc sent to the MAC
The follow
wing code shoows the MAC
C address tabble programm
ming on the vPC peer withh HSRP in
active mod
de for a giveen VLAN andd the vPC peer with HSR
RP in standbby mode for that same
VLAN.
vPC HSRP on
n active:
0000.0c07.ac0
G
01
static
vPC HSRP on
n standby:
G
01
static
0000.0c07.ac0
In a non-vP
PC environment
t, the HSRP MA
AC looks as follows:
f
On Active: G
00
000.0c07.ac01
static
00
000.0c07.ac01
On Standby: *
static
ARP Synch
hronization
Starting fro
om Cisco NX--OS 5.0(2) annd 4.2(6), Layyer 3 vPC peeers synchronizze their respeective ARP
tables. Thiss feature is trransparently enabled
e
and helps
h
ensure faster
f
converggence time uppon reload
of a vPC sw
witch. When two
t switches are reconneccted after a faiilure, they use Cisco Fabriic Services
protocol ov
ver Ethernet too perform bullk synchronizzation of the ARP
A table.
Peer Gateway
If a host orr a switch forrwards a fram
me to the Layeer 3 gateway and this Layyer 3 gatewayy is present
on a vPC pair of swittches, so long as the fram
me ID is deestined to thee HSRP MA
AC address
everything works as exppected.
u
the MAC
C burned-in adddress (BIA) instead of
If the framee that is sent to the Layer 3 gateway uses
the HSRP MAC
M
addresss, the PortChhannel hashing of the fram
me may forwaard it to the wrong
w
vPC
peer, which
h would then just
j bridge thhe frame to thee other vPC peer.
p
This scenarrio can be problematic because if the vPC
v
peer thaat owns the MAC
M
address routes the
frame to a vPC membeer port, this frame
f
will noot be able to leave the sw
witch, becausse the vPC
duplicate prevention rule would applly: no frame that

t
comes froom a peer linnk is allowed to exit the
switch on a vPC membeer port.
Figure 15 shows
s
the casse in which deevice A sendss traffic to rem
mote MAC (R
RMAC) address A with
a PortChan
nnel hash thatt forwards thee traffic to sw
witch B. The result
r
is that the frame cannnot get to
server B beecause of the duplicate
d
prevvention rule.
Figure 15 The problem
m addressed by the peer-ggateway featture
RMAC A
RMA
AC B
v
vPC
PL
Layer 3
Layer 2
vPC PKL
L
RMAC A
vPC1
vPC2

A rights reserved.
Cisco Confidential
16
To address this forwardding scenario, you should configure thee peer-gatew

way commandd under the
vPC domaiin. This comm
mand enables the vPC peers to exchangge informationn about their respective
BIA MAC addresses so they can forw
ward the trafffic locally without having to send it over the peer
link.
Layer 3 Lin
nk Between vPC Peers
In vPC dessigns, you should make suure to includee a Layer 3 link
l
or VLAN
N between thhe Layer 3
switching vPC
v
peers so that the routiing areas are adjacent. Alsso, you can consider
c
HSR
RP tracking
in non-vPC
C designs, but not in vPC designs.
d
HSRP track
king is not reecommendedd for the reasons illustrateed in Figure 16.
1 Imagine that
t
traffic
from n5k on
o VLAN60 needs
n
to be routed
r
to n5k on VLAN 50. As a resullt of a core liink failure,
HSRP track
king shuts doown switch viirtual interfacce (SVI) 60 on
o Agg2 and forces the VL
LAN60-toVLAN50 trraffic to Agg1. Agg1 routees from SVI 60 to SVI 500 and then forrwards to Po552 to reach
n5k. vPC prevents this forwarding
f
beehavior as preeviously explaained.
o the peer linnk between the
t routing
Because off this behavioor, you shouldd create a Laayer 3 path on
engines on Agg2 and Aggg1 instead off using HSRP
P tracking.
Figure 16 HSRP Track
king Is Not Needed
N
or Su
uitable for vP
PC Designs
Core1
Core2
ECMP
P
SpanningTree Root
50 60
Agg1
100
110
e
eth2/10
eth2/1
Spann
ning-Tree
Sec
condary
50 60
Agg2
Po52
eth2/10
0
eth2/2 eth
h2/1
eth2/2
n
n5k
VLAN 50

A rights reserved.
VLAN 60
Cisco Confidential
17
The followiing code show

ws how to creeate a Layer 3 link to connnect the aggregation layer switches to
reroute the traffic to Aggg1 if the routeed uplinks of Agg2 fail:
tc-nexus7k0
01-vdc2(config
g)# vlan 3
tc-nexus7k0
01-vdc2(config
g-vLAN)# name l3_vlan
tc-nexus7k0
01-vdc2(config
g-vLAN)# exit
tc-nexus7k0
02-vdc2(config
g)# int vlan 3
tc-nexus7k0
02-vdc2(config
g-if)# ip addr
ress 10.3.0.2 255.255.255.252
tc-nexus7k0
02-vdc2(config
g-if)# ip rout
ter ospf 1 ar
rea 0.0.0.0
tc-nexus7k0
02-vdc2(config
g-if)# no shut
t
01-vdc2(config
g)# int Port channel
c
10
tc-nexus7k0
tc-nexus7k0
01-vdc2(config
g-if)# switchp
port trunk allowed vLAN ad
dd 3
tc-nexus7k0
01-vdc2# show ip ospf neigh
h
OSPF Proce
ess ID 1 VRF default
d
Total numb
ber of neighbo
ors: 3
Up Time Ad
Neighbor ID
I
Pri Sta
ate
ddress
128.0.0.3
1 FUL
LL/DR
01:03:05 10.51.35.126
Interface
vLAN10
Layer 3 Lin
nk to the Co
ore
At the timee of this writinng, we recom
mmend the usee of Layer 3 links
l
to conneect the vPC aggregation
layer with the
t Layer 3 coore instead off the use of vP
PC PortChannnels for Layerr 3 connectiviity.
Figure 17 shows
s
why. The
T design onn the left shoows of a routeer connected with a Layerr 3 vPC to
Cisco Nexu
us Switches Switch1
S
and Switch2.
S
At thhe time of thiis writing thiss design doess not work.
Imagine thaat client 1 sennds traffic to server
s
1. Rouuter 1 has Swiitch1 and Swiitch2 as neighhbors, so it
load-balancces the routedd traffic to booth BIA MAC
C addresses of
o routers 1 and
a 2. The PoortChannel
hashing is independent
i
a may forw
and
ward the routed frame withh the BIA MAC address of
o Switch2
to Switch1 (and Switch11 to Switch2). In this case,, the frame woould traverse the peer linkk to be then
routed to th
he PortChanneel Po2. At thiis point, the duplicate
d
prevvention rule would
w
intervenne, and the
frame woulld be droppedd.
Thus, at thee time of this writing the connectivity
c
b
between
the core and the aggregation laayers needs
to follow th
he topology depicted
d
on the right side of Figure 17.
Figure 17 Interactionss Between vP
PC and Routiing
Server 1
Serrver 1
Switch
Switch
Po2
Po2
Switch1
S
Switch2
Po1
Ro
outing Protocol Peer
Laye
er 3
ECMP
Dy
ynamic Peering Relattionship
P
Router 1
Router 1
Client1
Client 1

A rights reserved.
Cisco Confidential
18
In
nteractions with Mu
ulticast
This section
n discusses thhe most imporrtant interactiions between multicast andd vPC.
IGMP Snooping and vPC

v
Layer 2 forrwarding of multicast
m
trafffic with vPC is based on a modified IG
GMP snoopingg behavior
that consists mostly off synchronizing the IGMP
P entries bettween vPC primary
p
and secondary
devices.
mplementationn, IGMP trafffic entering a Cisco NX-OS device throough a vPC PoortChannel
In a vPC im
triggers haardware proggramming forr the multicast entry onn both vPC member devvices. The
synchronizaation of the IGMP
I
inform
mation is perfformed over the
t peer link (the M1-to-M
M2 link in
Figure 18) using
u
Cisco Fabric
F
Servicees over Ethernnet.
Figure 18 IGMP Snooping with vP
PC
You can veerify the vPC operations wiith IGMP by using this com
mmand:
switch# sho
ow ip igmp sno
ooping statist
tics vlan 10
..
CFS
S packets sent
t over VPC pee
er link: 13
CFS
S packets rece
eived over VPC
C peer link: 13
CFS
S packet error
rs: 0
r
a vPC
C peer, this trraffic is repliccated to the ports
p
that joinned a given
When multticast traffic reaches
group as well as to the peer
p
link. Thee usual dupliccate preventioon rule of vPC
C applies, andd as Figure
19 shows, the
t traffic goees from S1 too S2 over the peer link (M
M1 to M2), buut Link 4 (L44) does not
forward this traffic becauuse L4 is a vP
PC member port.
p
Figure 19 Multicast Trraffic Forwaarding with vPC
v

A rights reserved.
Cisco Confidential
20
Multicast traffic
t
is copiied over the peer link to help ensure that orphan ports get thee multicast
stream and to help with failure scenarrios, such as the
t loss of Linnk 3 (L3) in Figure
F
19. Thhis happens
regardless of
o the presencce of receiverrs on the vPC peer.
Because off this it is impportant to propperly size the peer link to prevent
p
the peer link from
m becoming
the bottleneeck in the infrrastructure.
Thus, as a best practicce for vPC designs,
d
you should be suure to provission the peerr link with
sufficient links
l
accordinng to the bandwidth needds of your multicast
m
traff
ffic. Remembber that all
multicast trraffic traversees the peer linnk.
Protocol In
ndependentt Multicast and
a vPC
At the timee of this writing, vPC workks with Protoocol Independdent Multicast Any Sourcee Multicast
(PIM-ASM
M) but not withh Bidirectionaal (Bidir-PIM
M) or PIM Souurce-Specific Multicast (PIIM-SSM).
In PIM-Spaarse Mode thee PIM Designnated Router (DR) encapsuulates the trafffic from a givven source
and unicastts it to the renndezvous poinnt. Converselly, traffic from
m a source is drawn towarrd the PIM
designated router for forrwarding on a VLAN.
vironments, both aggregaation-layer deevices operatte as PIM deesignated rouuters. This
In vPC env
behavior alllows a multiicast source to send trafficc and have thhe traffic hashhed to either vPC peer,
which will then simply forward
f
the trraffic to the reendezvous pooint.
When a receiver is located in a vPC VLAN, thhe IGMP reports are syncchronized, annd Layer 3
G are createdd on both vPC
C peers. Both vPC peers seend PIM (*, G)
G joins to
forwarding entries (*, G)
m rendezvouss point. As a result, both vPC
v peer switches draw traaffic, causing temporary
the upstream
duplicates.
After a mullticast source starts sendingg traffic, onlyy one vPC peeer becomes thhe forwarder for
f a given
source and sends (S, G) joins. The chhoice of the foorwarder is baased on the distance
d
to thee source (if
the distancces are identiical, the vPC
C primary is chosen) andd converges on
o the designnated data
forwarder for
f these VLA
ANs on a per-stream basis, to prevent duuplicates.
In summary
y, with the duual-designatedd-router approoach, both vP
PC peers havee IGMP routees, but only
one of the peers
p
has the Outoing Intefface List for (S,
( G).
As with Laayer 2 traffic, multicast traaffic receivedd from the corre is copied to the peer linnk to reach
potential orrphan ports.
vPC Failure Scenarioss

This section
n describes thhe expected behavior of a vPC
v design foor various linkk failures.
vPC Memb
ber Port Faillure
If one vPC member portt goes down
for instance, if a link from
m a NIC goess downthe member is
removed from the PortC
Channel withoout bringing down
d
the vPC
C entirely. Coonversely, thee switch on
which the remaining
r
poort is located will allow frames
f
to be sent from thhe peer link to
t the vPC
orphan portt (ports; recalll the vPC dupplicate avoidaance techniquue). The Layeer 2 forwardinng table for
the switch that
t detected the failure is also updatedd to point the MAC
M
addressses that were associated
with the vP
PC port to the peer link.
vPC Comp
plete Dual-Acctive Failure
e (Double Faailure)
If both the peer link andd the peer-keeepalive link arre disconnectted, the Ciscoo Nexus switcch does not
bring down
n the vPC, because each Cisco
C
Nexus switch
s
cannott discriminatee between a vPC
v
device
reload and a combined peer-link
p
and peer-keepalivve-link failuree.
p
with a dual-active scenario is thhe lack of synnchronizationn between the vPC peers
The main problem
over the peeer link. This behavior causes IGMP snooping to malfunctionn, which in tuurn causes
multicast trraffic to drop.
As describ
bed previouslly, a vPC topology
t
intrrinsically prootects against loops in dual-active
d
scenarios. Each
E
vPC peeer, upon losiing peer-linkk connectivityy, starts forwaarding BPDU
Us on vPC
member po
orts. With the peer-switch feature, bothh vPC peers seend BPDUs with
w the samee bridge ID
to help ensu
ure that the doownstream deevice does noot detect a spaanning-tree misconfiguratio
m
on.
When the peer
p
link andd the peer-keeepalive link are
a simultaneeously lost, both
b
vPC peeers become
operationall primary. At the time of this
t
writing, when
w
connecctivity betweeen the peers is
i restored,
the vPC seccondary (operrational primaary) stays primary, and thee vPC primarry (operationaal primary)
becomes th
he vPC primarry (operationaal secondary).
If you wantt to restore the primary rolle on the vPC primary, youu can change the priority on
o one vPC
of the primaary and secondary roles.
switch and then flap thee peer-link, which
w
causes renegotiation
r
This proced
dure is disrupptive and it is described inn the section vPC
Role annd Priority under

u
vPC
Domain Co
onfiguration..
vPC Peer-LLink Failure

To preventt problems caaused by duaal-active devices, vPC shuuts down vPC
C member poorts on the
secondary switch
s
when the
t peer link is lost but thee peer keepaliive is still present.
When the peer

p link fails, the vPC peeers verify theiir reachabilityy over the peeer-keepalive link,
l
and if
they can co
ommunicate thhey take the following
f
actiions:
The op
perational secoondary vPC peer
p
(which may
m not matcch the configuured secondarry because
vPC is nonpreemptiive) brings doown the vPC member porrts, including the vPC mem
mber ports
located
d on the fabriic extenders in
i the case of
o a Cisco Neexus 5000 Seeries design with
w fabric
extendeers in straightt-through modde.
The seccondary vPC peer brings down the vP

PC VLAN SV
VIs: that is, all SVIs for thhe VLANs
that haappen to be configured
c
onn the vPC peeer link, whetther or not thhey are used on a vPC
membeer port.
Note
To keep the SVI

S interface up
u when a peerr link fails, use the command dual-active exxclude
interface-vlaan.
At the timee of this writting, if the peeer link is loost first, the vPC
v
secondarry shuts dow
wn the vPC
member po
orts. If this faiilure is follow
wed by a vPC
C peer-keepaliive failure, thhe vPC seconddary keeps
the interfacces shut dow
wn. This behaavior may chhange in the future with the introducttion of the
autorecoverry feature, whhich will allow
w the secondaary device to bring up the vPC ports as a result of
this sequence of events.
vPC Peer-K
Keepalive Faailure
If connectiv
vity of the peeer-keepalive link is lost but
b peer-link connectivity
c
i not changeed, nothing
is
happens; bo
oth vPC peerrs continue too synchronizee MAC addreess tables, IG
GMP entries, and so on.
The peer-keepalive link is mostly ussed when the peer link is lost,
l
and the vPC peers usse the peer
keepalive to
t resolve thee failure and determine which
w
device should shut down
d
the vPC
C member
ports.
Examples
Figure 20 illustrates
i
whhat happens during
d
vPC peer-link
p
failuure for vPC ports.
p
Agg1 is
i the vPC
primary, an
nd Agg2 is thee vPC seconddary.
The sequen
nce of events is
i as follows:
The vP
PC peer link fails, but Aggg1 and Agg22 can still coommunicate through the roouted path
with the vPC peer-keepalive protocol.
o vPC Po51 and Po52

Eth2/9 and eth2/10 on Agg2 arre shut downn because theey are part of
respectively, and Aggg2 is the opeerational seconndary vPC deevice.
SVI VL
LAN50 (vPC
C-VLAN) is shut down on
o the operaational seconddary device to prevent
traffic from
f
the coree routers from
m reaching thhe vPC secondary device on
o which the vPC ports
are shu
ut down.
Figure 20 Peer-Link Failure

F
Core1
Core2
P
Peer
Keepalive
Peer-Link
k
ECMP
P
SVI VLAN 50
SVI VLAN 50
vPC
Primary
Agg1
Agg2
100
110
Po51 eth2/9
vP
PC
Seco
ondary
Po52 eth
h2/10
eth2/1
eth2/2
n5k01
eth
h2/1
eth2/2
n5
5k02
VLAN 50

A rights reserved.
Cisco Confidential
21
t left througgh the vPC

As a result of the peer-liink failure, all traffic in Figgure 20 takess the path on the
vice. This is true
t
both for the
t client-to-sserver traffic and the serveer-to-client traaffic.
primary dev
The following show com
mmand entereed on the secoondary vPC peer
p
illustratees the results of
o the vPC
peer-link faailure:
tc-nexus7k0
02-vdc2# show vpc br
vPC domain id
Peer status
s
vPC keep-al
live status
vPC role
Dual Active
e Detected
:
:
:
:
1
pee
er link is dow
own
pee
er is alive
sec
condary
vPC Peer Li
ink Status
--------------------------------------------------------------------id Port Status Activ
ve vLANs
-- ---- ------ -------------------------------------------------1
Po10 down vPC status
---------------------------------------------------------------------id Port Status Consi
istency Reason
n
Acti
ive vLANs
-- ---- ------ ----------- -------------------------- -----------51 Po51 down succe
ess
succes
ss
-
The access switch uses the

t remainingg link:
tc-nexus5k0
01# show port channel summa
ary
--------------------------------------------------------------------------------
Group PortType
Protocol Member Ports
Chann
nel
-------------------------------------------------------------------------------51
Po51(SU)
Eth
LACP
Eth2/2(D)
Eth2/1(P)
keepalive com
mmunication helps ensuree that the looss of the peer-link pathh does not
The peer-k
introduce any
a unwanted flooding or split-subnet
s
sccenarios.
Figure 21 shows the faailure scenarioo in the pressence of a faabric extenderr. The vPC operational
o
secondary shuts
s
down thhe vPC membber port to hoost 1, which is directly attaached to N5kk01 and the
vPC member port of hosst 2 connectedd to Cisco Neexus 2000 Serries Fabric Exxtender N2k01.
Figure 21 vPC Peer-Liink Failure on
o the Cisco Nexus 5000 Series
S
P
Peer
Keepalive Path
mgm
mt0 vrf
Peer Link
vPC
Ope
erational
Prrimary
N5k01
N5k02
2
N2k01
vPC
Operational
Secondary
N2k02
2

A rights reserved.
Cisco Confidential
22
vPC with Faabric Exten

nder Active
e-Active Design
D
The case off fabric extender dual-connnected to the Cisco Nexus 5000 Series in vPC modee is slightly
different fro
om that of othher vPC desiggns.
Starting fro
om Cisco NX
X-OS 4.1(3), you can connnect a fabric extender to two Cisco Nexus
N
5000
Series deviices configureed for vPC. The fabric exxtender is a satellite switch that depennds on the
Cisco Nexu
us 5000 Series for both connfiguration annd forwardingg.
In such a design,
d
both Cisco
C
Nexus 5000
5
Series Switches
S
havee equal rightss to configuree the fabric
extender sw
witch ports.
To address this design, in which eacch fabric extender is controlled by two entities (the two Cisco
Nexus 5000
0 Series Swittches), the im
mplementationn relies on thhe modeling of
o each fabric extender
port as if it were two inddependent ports configuredd for vPC. Thhe same fabriic extender poort appears
witch, and the Cisco Nexuss 5000 Series vPC peers opperate as if
on each Cissco Nexus 50000 Series Sw
these two ports
p
were foorming a PorttChannelannd in fact the Cisco Nexus 5000 Seriess Switches
are configu
ured in vPC mode
m
accordinng to all the prreviously described guidellines.
The 10 Gig
gabit Ethernett ports conneecting the Cisco Nexus 5000 to the fabric
f
extendeer (switchport mode fabric) are configured
c
ass vPC membber ports, andd the individuual ports on the fabric
extender, su
uch as port eth100/1/1,
e
apppear on bothh nexus5k01 and nexus5kk02, as shownn in Figure
22.
Figure 22 Fabric Exten
nder Active--Active Desiggn
Peer Keepalive Connectivity

C
vPC Peer Lin
nk
mgmt0
m
vrf
nexu
us5k02
nexus5k01
1
vPC Peer
vPC
vPC
Member
Port
eth100/1/1
This Port Counts as If
C Port
It Were a vPC

A rights reserved.
Cisco Confidential
23
To keep th
he nexus5k01 and nexus5kk02 configuraations synchrronized, startiing from Ciscco NX-OS
5.0(2)N1(1) you can usse the configuuration synchhronization feature
fe
to deffine the fabric extender
port configu
uration in a sw
witch profile to help ensurre consistencyy between thee two configurrations.
With this topology, PorrtChannels woork on fabricc extenders, but
b you cannoot create a vP
PC from a
server that is split betw
ween two fabbric extenderss (for this, you
y need to use
u the fabricc extender
straight-through topologgy).
p
deescribed for vPC
v
memberr ports applyy equally to the fabric
The failuree scenarios previously
extender po
orts. If the peeer link is lost,, the vPC secoondary devicee shuts down the fabric poorts that are
connected to
t the secondaary Cisco Nexxus 5000 Seriies device.
vPC Configu
uration Be
est Practice
es
vPC Domaain Configuraation
vPC Role and
a Priority
A domain needs to be defined (as indicated byy the domain ID) as well as prioritiess to define
primary and
d secondary roles
r
in the vPC
v
configuraation. The low
wer number has
h higher priiority, so it
wins. For tw
wo switches (vPC peers) to
t form a vPC
C system, thee domain IDs of these switches need
to match. As
A previouslyy described, the
t domain ID
I is used too generate thee LAGID in the LACP
negotiation
n.
agg1(config
g)# vpc domain
n <domain-id>
agg1(config
g-vpc-domain)#
# role priorit
ty 100
agg2(config
g)# vpc domain
n <domain-id same as agg
g1>
g-vpc-domain)#
# role priorit
ty 110
agg2(config
Note that th
he role is nonppreemptive, so
s a device may
m be operatiionally primarry but seconddary from a
configuratio
on perspectivve. Because spanning treee is preempttive, this behhavior may result
r
in a
mismatch between
b
the spanning-treee root and the vPC opperational priimary devicee, with no
consequencces for traffic forwarding.
Although mismatched
m
s
spanning-tree
fic forwardingg, you still
and vPC priiorities do noot affect traffi
should keep
p the priorities matched to
t have the sppanning-tree root and vPC
C primary onn the same
device and
d the spanninng-tree seconndary root and vPC secoondary on thhe same devvice where
applicable (this recomm
mendation appplies only at the
t aggregatioon layer). The main beneffit is easier
managemen
nt. When the peer-switch command is used, both devices
d
are coonfigured withh the same
spanning-trree priority, soo this recomm
mendation doees not apply.
After failov
ver, the vPC
C operational primary andd vPC operaational seconddary do not match the
original con
nfiguration. You
Y can restoore matching by followingg these configguration stepss: from the
vPC operattional primaryy, you can chhange the rolle priority to the highest value
v
(32768) and then
enter a shutt/no shut com
mmand on thee peer-link PoortChannel.
You can alsso use a script such as the following:
7k-1(config
g)# cli alias name vpcpreem
mpt conf t ; vpc
v domain <d
domain-id> ;
role priori
ity 32767 ; in
nt <peer-link>
> ; shut ; no
o sh *
Reload Reestore
If the Cisco
o NX-OS verrsion supportss vPC reload
d restore, youu should conffigure this feaature under
the vPC dom
main configuuration:
vpc domain 1
role prio
ority 100
peer-keep
palive destin
nation 10.51.35.140 source 10.51.35.133
reload re
estore
If you hav
ve a Cisco Nexus
N
5000 Series
S
Switchh running an NXOS versiion earlier thhan NXOS
5.0(2)N1(1) and the relooad restore feature
f
is not available, yoou can configuure peer-con
nfig-checkbypass as follows:
f
vpc domain 2
role prio
ority 100
peer-keep
palive destin
nation 10.51.35.18
peer-conf
fig-check-byp
pass
Peer Gateway
If the vPC
C switch is allso performinng Layer 3 switching,
s
it is useful to add the peer-gateway
configuratio
on in the vPC
C domain defiinition:
vpc domain 1
ority 100
role prio
peer-keep
palive destin
nation 10.51.35.140 source 10.51.35.133
peer-gate
eway
reload re
estore
vPC Peer Link

L
The peer-lin
nk PortChannnel connects vPC
v peers and carries all access
a
VLAN
Ns (defined byy the user).
This link also
a
carries additional trafffic that the user
u
does not need to deffine: more sppecifically,
BPDUs and
d HSRP helloos and MAC address
a
synchhronization beetween the vP
PC peers.
This link iss by far the most
m importannt component of the vPC system.
s
Althoough its failurre does not
disrupt exissting vPC flow
ws, its failuree can impair the
t establishm
ment of new flows
fl
and isollate orphan
ports. Conffiguring the peer link in a redundantt fashion hellps ensure esssentially uniinterrupted
connectivity
y between thee vPC peers. The followinng script illusttrates how to configure thee peer link,
which in th
his case is PorrtChannel 10:
agg(config)# interface port-channel10
p
0
agg(config-if)# vpc pee
er-link
agg(config-if)# switchp
port trunk all
lowed vLAN <a
all access vLA
ANs>
The configuration of thee peer link auutomatically installs Bridgge Assurancee on the peer link. This
on is compatiible with ISSU
U, so you cann keep Bridgee Assurance ennabled on this link.
configuratio
The peer lin
nk carries a copy of the muulticast trafficc regardless of
o whether theere are orphann ports that
need to receeive it. You should
s
provisiion the bandw
width for the peer
p link accoordingly.
vPC Peer Keepalive

K
The peer-keeepalive connnectivity shouuld never be carried
c
as a VLAN
V
on thee peer link; ottherwise, it
will not pro
ovide any bennefit. Insteadd, it should bee carried overr a routed inffrastructure, and
a it does
not need to be a direct pooint-to-point link.
wing configuraation illustrattes the use of a dedicatedd Gigabit Ethhernet interface for this
The follow
purpose:
vrf context
t vpc-keepaliv
ve
interface Ethernet8/16
E
descripti
ion tc-nexus7k
k02-vdc2 - vPC
C Heartbeat Link
L
vrf membe
er vpc-keepali
ive
ip addres
ss 192.168.1.1
1/24
no shutdo
own
vpc domain 1
peer-keep
palive destina
ation 192.168.1.2 source 192.168.1.1 vr
rf vpc-keepali
ive
You should
d not use the mgmt0
m
interfa
face for a direect back-to-baack connection between Ciisco Nexus
7000 Seriess systems beccause you caannot determiine which suppervisor is acctive at any given
g
time.
You can use it instead onn the Cisco Nexus
N
5000 Seeries.
The mgmt0
0 interface cann be used botth for manageement and foor routing the peer keepalivve through
the out-of-band manageement netwoork. In this case,
c
each Cisco
C
Nexus 7000 Series Switch is
connected to
t the managgement netwoork through mgmt0
m
of suppervisor slots 5 and 6 andd the Cisco
Nexus 5000
0 Series throuugh the singlee mgmt0 interrface.
By followin
ng this approach, regardless of which supervisor
s
is active, the Ciisco Nexus 7000 Series
Switch has one of the mgmt0
m
interfacces connectedd to the manaagement netw
work, which can
c then be
used for peer-keepalive purposes.
p
vPC Ports
PortChanneels are configgured by bunddling Layer 2 ports (switchh ports) on eaach Cisco Nexxus switch
through thee command vpc,
v
as shownn in the follow
wing code. Thhe system sennds an error message
m
if
the PortChaannel was nott previously configured
c
as a switch portt.
agg1(config
g)#interface ethernet2/9
e
agg1(config
g-if)# channel
l-group 51 mod
de active
agg1(config
g)#interface Port-channel
P
5
51
agg1(config
g-if)# switchp
port
agg1(config
g-if)# vpc 51
!
g)#interface ethernet2/9
e
agg2(config
agg2(config
g-if)# channel
l-group 51 mod
de active
agg2(config
g)#interface Port-channel
P
5
51
agg2(config
g-if)#switchpo
ort
agg2(config
g-if)# vpc 51
If the conssistency checkk does not show

s
success, you shouldd verify the consistency
c
p
parameters.
Typical reaasons that a vP
PC may not form
fo include the
t following:
Thee VLAN that is defined in the trunk doees not exist, or

o it is not deffined on the peer link.
On
ne member poort is configurred as the acceess and the otther as the truunk.
Mismatches exisst in the VLA

ANs that are carried on the trunk, etc.
wing example shows how to

t verify that the vPC configuration is consistent beetween two
The follow
vPC peers for
f the globall consistency parameter as well as for a specific PortC
Channel:
tc-nexus7k0
01-vdc2# show vpc consisten
ncy-parameter
rs global
tc-nexus7k0
01-vdc2# show vpc consisten
ncy-parameter
rs int port-ch
hannel 51
Legend:
Typ
pe 1 : vPC wil
ll be suspende
ed in case of
f mismatch
Name
------------STP Port Ty
ype
STP Port Gu
uard
STP MST Sim
mulate PVST
Allowed vLA
ANs
Type
---1
1
1
-
Loc
cal Value
---------------------Def
fault
Non
ne
Def
fault
10-14,21-24,50,60
Peer Value
----------------------Defau
ult
None
Defau
ult
10-14
4,21-24,50,60
After a portt is defined ass part of a PoortChannel, anny additional configurationns, such as acctivation or
disablemen
nt of Bridge Assurance
A
orr trunking moode, are perfformed in thee interface PoortChannel
configuratio
on mode. Tryying to configgure spanningg-tree propertties for the phhysical interfaace instead
of the PortC
Channel will result
r
in an errror message.
LACP
You should
d use LACP for
f dynamic bundling
b
of thhe ports in thee vPC group, because LAC
CP verifies
that the ports being bunndled are acttually part off the same phhysical or virrtual switch, preventing
p
c
s.
erroneous configurations
For examplle, if the PortC
Channel is coonfigured as active
a
on the Cisco
C
Nexus 7000 Series Switch
S
and
the downstrream switch is not configuured for PortC
Channel, the PortChannel ports will bee shown as
in the indiv
vidual (I) statee and will runn regular spannning tree.
After the access layerr switches are
a configureed for LAC
CP, the negootiation com
mpletes the
PortChanneel forms:
tc-nexus5k0
01(config)# in
nt eth2/1-2
tc-nexus5k0
01(config-if-r
range)# channe
el-group 51 mode
m
passive
The PortCh
hannel on thee Cisco Nexus 5000 Seriess access switcches becomes active, indicating that
the LACP negotiation
n
iss functioning between the upstream vPC
C system andd the Cisco Nexus
N
5000
Series:
tc-nexus5k0
01# show port-channel summa
ary
Flags: D - Down
P - Up in por
rt-channel (m
members)
I - Individual H - Hot-stand
dby (LACP only
y)
s - Suspended r - Module-re
emoved
S - Switched
R - Routed
U - Up (port-cha
annel)
-------------------------------------------------------------------------------Group PortType
Protocol Member Ports
Chann
nel
-------------------------------------------------------------------------------Eth2/1(P)
51
Po51(SU)
Eth
LACP
Eth2/2(P)
The PortCh
hannel on the Cisco Nexus 7000 Series Switch also becomes
b
activve because off the LACP
negotiation
n:
tc-nexus7k0
01-vdc2# show vpc br
[]
ink status
vPC Peer-li
--------------------------------------------------------------------id Port Status Activ
ve vLANs
----------------------------------------------- ---- ------ ----1
Po10 up
10-14
4,21-24,50,60
vPC status
---------------------------------------------------------------------id Port Status Consi
istency Reason
n
Acti
ive vLANs
------- -------------------------- ------------- ---- ------ ----51 Po51 up
succe
ess
succes
ss
10-1
14,21-24
,50,60
If the PortC
Channel portss are suspendeed, a mismatcch occurred inn the PortChaannel ports beetween the
switches th
hat are suppossed to bring up
u the PortChhannel. For example,
e
a vP
PC on the Ciisco Nexus
7000 Seriess is configureed with ports that individually connect to

t two differeent PortChannnels on the
Cisco Nexu
us 5000 Series.
Alternatively, if the acceess-layer portts are not connfigured for a channel, the Cisco Nexuss 7000 and
5000 Seriess will operatee normally wiith spanning tree. If the poorts on the Ciisco Nexus 5000 Series
are configu
ured in passive channel-ggroup mode and
a the Cisco Nexus 70000 Series porrts are not
configured for PortChannnels, the Cissco Nexus 70000 and 50000 Series run spanning treee again on
those ports..
For More In
nformation
n
Cisco Nexu
us 5000 pagge: http://www
w.cisco.com//go/nexus5000
Cisco Nexu
us 7000 pagge: http://www
w.cisco.com//go/nexus7000
[[NOTE: IN LAY
YOUT, PLEASE USE THE CU
URRENT LEGA
AL BLOCK (10005R)]]

Guide - Chap 3 - Cisco NX-OS Software Virtual PortChannel Fundamental Concepts With NXOS 5 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Guide - Chap 3 - Cisco NX-OS Software Virtual PortChannel Fundamental Concepts With NXOS 5 PDF

Uploaded by

Copyright:

Available Formats

Cisco N

Traffic Flows ...................................................................................................................................................9

vPC Configguration Chaanges Whenn the Peer Linnk Fails ...............................................................................15

vPC Configguration Con

Layer 3 Linnk Between vPC Peers .................................................................................................................28

2 - Nexus Data Cennter Design with vPC

Layer 3 Linnk to the Co

vPC Peer Link

For More Infoormation .......................................................................................................................................42

3 - Nexus Data Cennter Design with vPC

4 - Nexus Data Cennter Design with vPC

Peer Keepallive Link

vPC Member Ports

2009 Cisco Systems, Inc. All

Cisco Nexus 2000 Series Straight-Through Mode vPC

2 Gigabit Etthernet Ports

2009 Cisco Systems, Inc. All

4 Gigabit Ethernet Ports

5 - Nexus Data Cennter Design with vPC

Figure 3 illlustrates the main

2009 Cisco Systems, Inc. All

vPC Peer Link

6 - Nexus Data Cennter Design with vPC

vPC Ports, and Orphan

7 - Nexus Data Cennter Design with vPC

Peer Link: VLANs 10, 20, 30, 40, and

2009 Cisco Systems, Inc. All

vPC Topology with Fa

8 - Nexus Data Cennter Design with vPC

2009 Cisco Systems, Inc. All

A Layeer 3 Gigabit Ethernet

vPC meember ports forming

9 - Nexus Data Cennter Design with vPC

2009 Cisco Systems, Inc. All

Dual-Conttrol Plane wiith Single Laayer 2 Node Behavior

Link Aggreegation Group Identifierr

You neeed a means of

For more information,

In Figure 8, switch 1 annnounces ports 1 and 2 as part

2009 Cisco Systems, Inc. All

System ID in a vPC Sysstem

witch option has

It reduces the disruuption associaated with a dual-active

PDUs if the prrimary and seecondary rolees change.

Cisco Fabrric Services over

vPC Configguration Chaanges When

2009 Cisco Systems, Inc. All

The initial implementattion of vPC did

2009 Cisco Systems, Inc. All

Status Consistency Reason

------ ------------ ------ ------------ --------------------------- -----------8

Status Consistency Reason

------ ------------ ------ ------------ --------------------------- -----------8

To restore connectivity, you need to

A vPC switch that has

narios: Peer Link

2009 Cisco Systems, Inc. All

Case 1 haas been adddressed by the

The peer-config-check--bypass comm

2009 Cisco Systems, Inc. All

w the peerr link down, you

vPC Reload Restore

-- ---- ------ ----------- --------------------------- ------------51 Po51 up