Cisco 7000 Nexus Hardware

Cisco Nexus 7000 Hardware Architecture
BRKARC-3470
BRKARC-3470
2012 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Session Goal
To provide you with a thorough understanding of the Cisco Nexus 7000 switching architecture,
supervisor, fabric, and I/O module design, packet flows, and key forwarding engine functions
This session will not examine NX-OS software architecture or other Nexus platform architectures
Related sessions:
BRKDCT-2204
Nexus 7000/5000/2000/1000v Deployment Case Studies
BRKIPM-3062
Nexus Multicast Design Best Practices
BRKDCT-2121
VDC Design and Implementation
BRKDCT-2048
Deploying Virtual Port Channel in NX-OS
BRKARC-3472
NX-OS Routing & Layer 3 Switching
BRKDCT-2081
Cisco FabricPath Technology and Design
TECDCT-3297
Operating and Deploying NX-OS
BRKCRS-3144
Troubleshooting Cisco Nexus 7000 Series Switches
LTRCRT-5205
Configuring Nexus 7000 Virtualization Lab
LTRDCT-1142
FabricPath Deployment in the Data Center Lab
BRKARC-3470
Cisco Public
What Is Nexus 7000?

Data-center class Ethernet switch designed to deliver high-availability,
system scale, usability, investment protection
Supervisor Engine
I/O Modules
Chassis
Fabrics
BRKARC-3470
Cisco Public
Agenda
Chassis Architecture
Supervisor Engine and I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
Nexus 7000 Chassis Family

Nexus 7010
NX-OS 4.1(2) and later
Nexus 7018
25RU
21RU
Front
N7K-C7010
Rear
Front
N7K-C7018
Rear
NX-OS 5.2(1) and later
Nexus 7009
14RU
Front
BRKARC-3470
N7K-C7009
Rear
Cisco Public
Key Chassis Components

Common components:
Supervisor Engines
I/O Modules
Power Supplies
Chassis-specific components:
Fabric Modules
Fan Trays
BRKARC-3470
Cisco Public
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
Supervisor Engine 1
Performs control plane and management functions
Dual-core 1.66GHz x86 processor with 8GB DRAM
2MB NVRAM, 2GB internal bootdisk, compact flash slots, USB
Console, aux, and out-of-band management interfaces

Interfaces with I/O modules via 1G switched EOBC
Houses dedicated central arbiter ASIC
Controls access to fabric bandwidth via dedicated arbitration path to I/O modules
N7K-SUP1
ID LED
Status
LEDs
BRKARC-3470
AUX Port
Console Port
USB Ports
Management
Ethernet
Compact Flash
Slots
CMP Ethernet
Reset Button
Cisco Public
Nexus 7000 I/O Module Families

M Series and F Series
M family L2/L3/L4 with large forwarding tables and rich feature set
N7K-M148GT-11/N7K-M148GT-11L
N7K-M132XP-12/
N7K-M132XP-12L
N7K-M108X2-12L
N7K-M148GS-11/N7K-M148GS-11L
F family High performance, low latency, low power with streamlined

feature set
N7K-F132XP-15
BRKARC-3470
N7K-F248XP-25
Cisco Public
10
Supported in NX-OS release 5.0(2a) and later
8-Port 10GE M1 I/O Module

N7K-M108X2-12L
8-port 10G with X2 transceivers
80G full-duplex fabric connectivity
Two integrated forwarding engines (120Mpps)
Support for XL forwarding tables (licensed feature)
Distributed L3 multicast replication

802.1AE LinkSec
BRKARC-3470
N7K-M108X2-12L
Cisco Public
12
8-Port 10G XL M1 I/O Module Architecture

N7K-M108X2-12L
EOBC
To Central Arbiter
To Fabric Modules
Fabric 1
LC
CPU
VOQs
Forwarding
Engine
VOQs
Forwarding
Engine
Replication
Engine
Replication
Engine
Replication
Engine
Replication
Engine
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
Linksec
Linksec
Linksec
Linksec
Linksec
Linksec
Linksec
Linksec
Front Panel Ports

BRKARC-3470
Cisco Public
13
N7K-M132XP-12 Supported in all releases

N7K-M132XP-12L Supported in NX-OS release 5.1(1) and later
32-Port 10GE M1 I/O Modules

N7K-M132XP-12, N7K-M132XP-12L
32-port 10G with SFP+ transceivers
Integrated 60Mpps forwarding engine
XL forwarding engine on L version
Oversubscription option for higher density (up to 4:1)

N7K-M132XP-12/
N7K-M132XP-12L
Supports Nexus 2000 (FEX) connections

Distributed L3 multicast replication
LISP support
802.1AE LinkSec
BRKARC-3470
Cisco Public
15
Shared vs. Dedicated Mode

To Fabric
rate-mode shared
(default)
One port group
10G
11
13
15
Shared mode
Four interfaces in port group share 10G
bandwidth
Another port group
Port group group of contiguous

even or odd ports that share 10G of
bandwidth (e.g., ports 1,3,5,7)
To Fabric
rate-mode dedicated
10G
Dedicated mode
9
11
13
15
First interface in port group gets 10G

bandwidth
Other three interfaces in port group
disabled
BRKARC-3470
Cisco Public
16
32-Port 10G M1 I/O Module Architecture

N7K-M132XP-12, N7K-M132XP-12L
EOBC
To Central Arbiter
To Fabric Modules
Fabric 1
LC
CPU
VOQs
VOQs
Forwarding Engine
Replication
Engine
Replication
Engine
Replication
Engine
Replication
Engine
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
1 3 5 7
9 11 13 15
17 19 21 23
25 27 29 31
2 4 6 8
10 12 14 16 18 20 22 24 26 28 30 32
Front Panel Ports

BRKARC-3470
Cisco Public
17
Supported in NX-OS release 5.1(1) and later
32-Port 1G/10GE F1 I/O Module

N7K-F132XP-15
32-port 1G/10G with SFP/SFP+ transceivers
230G full-duplex fabric connectivity (320G local switching)
System-on-chip (SoC)* forwarding engine design
16 independent SoC ASICs
Layer 2 forwarding with L3/L4 services (ACL/QoS)

FabricPath-capable
FCoE-capable
N7K-F132XP-15
* sometimes called switch-on-chip

BRKARC-3470
Cisco Public
20
32-Port 1G/10G F1 I/O Module Architecture

N7K-F132XP-15
EOBC
To Fabric Modules
To Central Arbiter
Arbitration
Aggregator
LC
CPU
Fabric 1
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
Fabric 1
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
9 10 11 12 13 14 15 16
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
2 X 10G
SoC
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Front Panel Ports
BRKARC-3470
Cisco Public
21
Supported in NX-OS release 6.0(1) and later
48-Port 1G/10GE F2 I/O Module

N7K-F248XP-25
48-port 1G/10G with SFP/SFP+ transceivers
System-on-chip (SoC)* forwarding engine design
12 independent SoC ASICs
Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QoS)

Supports Nexus 2000 (FEX) connections
N7K-F248XP-25
FabricPath-capable
FCoE-ready
* sometimes called switch-on-chip

BRKARC-3470
Cisco Public
22
48-Port 1G/10G F2 I/O Module Architecture

N7K-F248XP-25
To Central Arbiters
To Fabric Modules
EOBC
Arbitration
Aggregator
LC
CPU
Fabric 2
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
13
17
21
25
29
33
37
41
45
3
2
7
6
11
10
12
14
15
16
18
19
20
22
23
24
26
27
28
30
31
32
34
35
36
38
39
40
42
43
44
46
47
48
Front Panel Ports
BRKARC-3470
Cisco Public
23
F2-Only VDC
Communication between F2only VDC and M1/F1 VDC must

be through external connection
F2 module
F2 modules do not interoperate with other

Nexus 7000 modules
F2 module
F2-only
VDC
F2 module
Must deploy in an F2 only VDC

Can be default VDC, or any other VDC
Use the limit-resource module-type f2 VDC
configuration command
M1 module
System with only F2 modules and empty

configuration boots with F2-only default VDC
automatically
F1 module
M1 module
M1/F1
VDC
F1 module
M1/F1 modules can exist in

same chassis as F2 modules,
but not in the same VDC
BRKARC-3470
Cisco Public
25
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
28
M1 Forwarding Engine Hardware

Hardware forwarding engine(s) integrated on
every I/O module
OTV
IGMP snooping
60Mpps per forwarding engine Layer 2

bridging with hardware MAC learning
RACL/VACL/PACL
60Mpps per forwarding engine Layer 3 IPv4

and 30Mpps Layer 3 IPv6 unicast
Policy-based routing (PBR)
Layer 3 IPv4 and IPv6 multicast support (SM,

SSM, bidir)
QoS remarking and policing policies

Unicast RPF check and IP source guard
Ingress and egress NetFlow (full and sampled)
MPLS
BRKARC-3470
Hardware Table
M1 Modules
M1-XL Modules
without License
M1-XL Modules with

License
FIB TCAM
128K
128K
900K
Classification TCAM (ACL/QoS)
64K
64K
128K
MAC Address Table
128K
128K
128K
NetFlow Table
512K
512K
512K
Cisco Public
29
M1 Forwarding Engine Architecture

FE Daughter
Card
Ingress NetFlow
collection
Ingress Pipeline
Ingress policing
FIB TCAM and

adjacency table
lookups for
Layer 3
forwarding
ECMP hashing
Multicast RPF
check
Egress Pipeline
Ingress ACL
and QoS
classification
Layer 3
Engine
Unicast RPF
check
Egress NetFlow
collection
Egress policing
Egress ACL and QoS classification

Ingress MAC
table lookups
IGMP snooping
lookups
IGMP snooping
redirection
Layer 2
Engine
Packet Headers from

I/O Module Replication Engine
BRKARC-3470
Egress MAC
lookups
IGMP snooping
lookups
Final lookup result to
I/O Module Replication Engine
Cisco Public
30
F2 Forwarding Engine Hardware

Each SoC forwarding engine services 4 frontpanel 10G ports (12 SoCs per module)
RACL/VACL/PACL
QoS remarking and policing policies
60Mpps per SoC Layer 2 bridging with

hardware MAC learning
Policy-based routing (PBR)
60Mpps per forwarding engine Layer 3 IPv4/

IPv6 unicast
FabricPath forwarding
Unicast RPF check and IP source guard

Ingress sampled NetFlow (future)
Layer 3 IPv4 and IPv6 multicast support (SM,

SSM)
FCoE (future)
IGMP snooping
Hardware Table
Per F2 SoC
Per F2 Module
MAC Address Table
16K
256K*
FIB TCAM
32K IPv4/16K IPv6 32K IPv4/16K IPv6
16K
192K*
* Assumes specific configuration to scale SoC resources
BRKARC-3470
Cisco Public
31
F2 Forwarding Engine Architecture

To/From Central
Arbiter
To Fabric
Virtual output
queues
From Fabric
Ingress Buffer
(VOQ)
Egress fabric
receive buffer
Ingress and egress

forwarding decisions
(L2/L3 lookups,
ACL/QoS, etc.)
Forwarding Engine
Pre-Forwarding
Ingress Buffer
Skid buffer
Accommodates pause
reaction time
1G and 10G
capable interface
MAC
Four front-panel
interfaces per
ASIC
BRKARC-3470
Egress Buffer
4 X 10G
SoC
FIB, ADJ, MAC,

ACL, QoS, MET
1G/10G MAC
Port A
1G/10G
Forwarding
tables
1G/10G MAC
Port B
1G/10G
Port C
1G/10G
Port D
1G/10G
Cisco Public
32
F1 Forwarding Engine Hardware

Each SoC forwarding engine services 2 frontpanel 10G ports (16 SoCs per module)
30Mpps per SoC Layer 2 bridging with

hardware MAC learning
IGMP snooping
VACL/PACL
QoS remarking policies
FabricPath forwarding
FCoE
Hardware Table
Per F1 SoC
Per F1 Module
MAC Address Table
16K
256K*
1K in/1K out
16K in/16K out*
* Assumes specific configuration to scale SoC resources
BRKARC-3470
Cisco Public
33
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
35
Crossbar Switch Fabric Modules
N7K-C7018-FAB-1/FAB-2
Two fabric generations available Fabric 1 and Fabric 2

Fabric
Per-fabric module bandwidth
Total bandwidth with 5 fabric modules
Fabric 1
46Gbps per slot
230Gbps per slot
Fabric 2
110Gbps per slot
550Gbps per slot
Each installed fabric increases available per-payload slot bandwidth

Different I/O modules leverage different amount of fabric bandwidth
All I/O modules compatible with both Fabric 1 and Fabric 2
Access to fabric bandwidth controlled using QoS-aware central arbitration with VOQ
N7K-C7009-FAB-2
BRKARC-3470
N7K-C7010-FAB-1/FAB-2
Cisco Public
36
Multistage Crossbar
Nexus 7000 implements 3-stage crossbar switch fabric
Stages 1 and 3 on I/O modules

Stage 2 on fabric modules
Fabric Modules
1
2nd stage
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
2 x 23Gbps (Fab1) or
2 x 55Gbps (Fab2)
per slot per fabric module
Up to 230Gbps (Fab1) or
550Gbps (Fab2)
per I/O module with
5 fabric modules installed
1st stage
BRKARC-3470
Fabric ASIC
20 x 23Gbps (Fab1) or
20 x 55Gbps (Fab2)
channels per fabric module
Fabric ASIC
Fabric ASIC
Ingress
Module
Egress
Module
3rd stage
Cisco Public
37
I/O Module Capacity Fabric 1

Fabric 1 Modules
230Gbps
46Gbps
184Gbps
138Gbps
92Gbps
per slot bandwidth
Local Fabric 2
(480G)
46Gbps/slot
Fabric 1
ASICs
46Gbps/slot
Fabric 1
ASICs
46Gbps/slot
Fabric 1
ASICs
46Gbps/slot
Fabric 1
ASICs
46Gbps/slot
Fabric 1
ASICs
One fabric
Any port can pass traffic to any
other port in system
Two fabrics
80G M1 module has full
bandwidth
Local Fabric 1
(230G)
Five fabrics
230G F1 module has maximum
bandwidth
480G F2 module limited to 230G
per slot
BRKARC-3470
Local Fabric 1
(80G)
Cisco Public
38
I/O Module Capacity Fabric 2
Fabric channels run at

lowest common speed
Fab2 does NOT make Fab1-based

modules faster!!
550Gbps
110Gbps
440Gbps
220Gbps
330Gbps
Fabric 2 Modules
110Gbps/slot
Fabric 2
ASICs
Local Fabric 2
(480G)
per slot bandwidth

Fabric 2
ASICs
One fabric
Any port can pass traffic to any
other port in system
Two fabrics
Local Fabric 1
(230G)
Fabric 2
ASICs
80G M1 module has full

bandwidth
Five fabrics
bandwidth
Fabric 2
ASICs
Local Fabric 1
(80G)

bandwidth
BRKARC-3470
Fabric 2
ASICs
Cisco Public
39
Fabric 1 to Fabric 2 Migration

Online, non-disruptive migration of Fabric 1 to Fabric 2 supported
Upgrade to software release supporting Fabric 2

Remove one Fabric 1 module at a time, replace with Fabric 2 module
Allow new Fabric 2 module to come completely online before removing next
Fabric 1 module
Mix of Fabric 1/Fabric 2 not recommended or supported for longer than

duration of the migration
Within 12 hours of install of first Fabric 2 module, system syslogs warning to
complete migration
http://www.cisco.com/en/US/docs/switches/datacenter/hw/nexus7000/installation/guide/n7k_replacing.html
BRKARC-3470
Cisco Public
40
Arbitration, VOQ, and Crossbar Fabric

Arbitration, VOQ, and fabric combine to provide all necessary
infrastructure for packet transport inside switch
Central arbitration Controls scheduling of traffic into fabric based on
fairness, priority, and bandwidth availability at egress ports
Virtual Output Queues (VOQs) Provide buffering and queuing for
ingress-buffered switch architecture
Crossbar fabric Provides dedicated, high-bandwidth interconnects
between ingress and egress I/O modules
BRKARC-3470
Cisco Public
41
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
45
Buffering, Queuing, and Scheduling

Buffering storing packets in memory
Needed to absorb bursts, manage congestion
Queuing buffering packets according to traffic class

Provides dedicated buffer for packets of different priority
Scheduling controlling the order of transmission of buffered packets

Ensures preferential treatment for packets of higher priority and fair treatment for packets of
equal priority
Nexus 7000 uses queuing policies and network-QoS policies to define buffering,
queuing, and scheduling behavior
Default queuing and network-QoS policies always in effect in absence of any user
configuration
BRKARC-3470
Cisco Public
46
I/O Module Buffering Models

Buffering model varies by I/O module family
M1 modules: hybrid model combining ingress VOQ-buffered architecture with
egress port-buffered architecture
F1/F2 modules: pure ingress VOQ-buffered architecture
All configuration through Modular QoS CLI (MQC)

Queuing parameters applied using class-maps/policy-maps/service-policies
BRKARC-3470
Cisco Public
47
Hybrid Ingress/Egress Buffered Model

M1 I/O Modules
Ingress port buffer Manages congestion in ingress forwarding/replication engines only
Ingress VOQ buffer Manages congestion toward egress destinations over fabric
Egress VOQ buffer Receives frames from fabric; also buffers multidestination frames
Egress port buffer Manages congestion at egress interface
Ingress
port buffer
Ingress
VOQ buffer
Egress
VOQ buffer
Ingress Module
Ingress Module
Crossbar
Fabric
Egress
port buffer
Egress Module
Ingress Module
BRKARC-3470
Cisco Public
48
Ingress Buffered Model

F1/F2 I/O Modules
Ingress skid buffer Absorbs packets in flight after external flow control asserted
Ingress VOQ buffer Manages congestion toward egress destinations over fabric
Egress VOQ buffer Receives frames from fabric; also buffers multidestination frames
Ingress
skid buffer
Ingress
VOQ buffer
Egress
VOQ buffer
Ingress Module
Ingress Module
Crossbar
Fabric
Egress Module
Ingress Module
BRKARC-3470
Cisco Public
50
Distributed Buffer Pool

Ingress-buffered architecture implements large, distributed buffer pool to absorb congestion
Absorbs congestion at every ingress port contributing to congestion, leveraging all per-port ingress
buffer
Excess traffic does not consume fabric bandwidth, only to be dropped at egress port
8:1 Ingress:Egress
2:1 Ingress:Egress
Ingress
VOQ buffer
Available buffer
for congestion
management:
Available buffer
for congestion
management:
2
3
Fabric
Fabric
2
5
Egress
Egress
7
Ingress
8
Ingress
BRKARC-3470
Cisco Public
52
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
54
Layer 2 Forwarding
Layer 2 forwarding traffic steering based on destination MAC address
Hardware MAC learning

CPU not directly involved in learning
Forwarding engine(s) on each module have copy of MAC table

New learns communicated to other forwarding engines via hardware flood to
fabric mechanism
Software process ensures continuous MAC table sync
Spanning tree (PVRST or MST), Virtual Port Channel (VPC), or

FabricPath ensures loop-free Layer 2 topology
BRKARC-3470
Cisco Public
55
Hardware Layer 2 Forwarding Process

In Classic Ethernet and FabricPath edge switches, MAC table lookup
drives Layer 2 forwarding
Source MAC and destination MAC lookups performed for each frame, based on
{VLAN,MAC} pairs
Source MAC lookup drives new learns and refreshes aging timers
Destination MAC lookup dictates outgoing switchport (CE/FabricPath local) or
destination Switch ID (FabricPath remote)
In FabricPath core switches, Switch ID (routing) table lookup drives

Layer 2 forwarding
Destination SID lookup dictates outgoing FabricPath interface and next hop
BRKARC-3470
Cisco Public
56
HDR
= Packet Headers
= Packet Data
DATA
M1 L2 Packet Flow
Supervisor Engine
Credit grant
for fabric
access
= Internal Signaling
CTRL
Fabric Module 1
Fabric Module 2
Fabric Module 3
Fabric ASIC
Fabric ASIC
Fabric ASIC
Receive from
fabric
Return buffer
credit
11
VOQ arbitration
and queuing
Fabric ASIC
Ingress port QoS
BRKARC-3470
Receive
packet
from wire
Layer 2
Engine
Replication
Engine
10G MAC
Forwarding
Engine
6
Layer 3
Engine
Layer 2
Engine
L2 SMAC/
DMAC lookups
Forwarding
Engine
Return result
2
LinkSec decryption
VOQs
Replication
Engine
Egress
port QoS
13
Module 2
Module 1
e1/1
Fabric ASIC
10G MAC
Linksec
ACL/QoS/
NetFlow
lookups
Layer 3
Engine
VOQs
Submit packet
headers for
lookup
12
Transmit
to fabric
Return
credit
to pool
Central Arbiter
10
15
Transmit
packet on
wire
Cisco Public
Linksec
LinkSec
encryption
14
e2/1
57
HDR
DATA
= Packet Headers
= Packet Data
CTRL
F1/F2 L2 Packet Flow

7
Supervisor Engine
Credit grant for

fabric access
Return
credit
to pool
10
Central Arbiter
Fabric Module 1
Fabric Module 2
Fabric Module 3
Fabric Module 4
Fabric Module 5
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Transmit
to fabric
VOQ arbitration
Fabric ASIC
Submit packet headers for lookup
Fabric ASIC
VOQ FE
Receive from fabric

Return buffer credit
Ingress L2 SMAC/ DMAC

lookups, ACL/QoS lookups
VOQ
Return result
SoC
2
BRKARC-3470
Ingress
port QoS
(VOQ)
e1/1
SoC
Module 1
Receive
packet
from wire
11
Transmit
packet
on wire
Module 2
e2/1
Cisco Public
58
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
59
IP Forwarding
Nexus 7000 decouples control plane and data plane
Forwarding tables built on control plane using routing protocols or static

configuration
OSPF, EIGRP, IS-IS, RIP, BGP for dynamic routing
Tables downloaded to forwarding engine hardware for data plane

forwarding
FIB TCAM contains IP prefixes
Adjacency table contains next-hop information
BRKARC-3470
Cisco Public
60
Hardware IP Forwarding Process

FIB TCAM lookup based on destination prefix (longest-match)
FIB hit returns adjacency, adjacency contains rewrite information (nexthop)

Pipelined forwarding engine architecture also performs ACL, QoS, and
NetFlow lookups, affecting final forwarding result
BRKARC-3470
Cisco Public
61
IPv4 FIB TCAM Lookup

Generate TCAM lookup key
(destination IP address)
Generate
Lookup Key
10.1.1.10
HIT!
Forwarding Engine
Flow
Data
Index, # next-hops
10.1.1.3
Index, # next-hops
10.1.1.4
Index, # next-hops
10.10.0.10
Index, # next-hops
10.10.0.100
Index, # next-hops
10.10.0.33
Index, # next-hops
10.1.1.xx
10.1.2.xx
Index, # next-hops
10.1.3.xx
Index, # next-hops
10.10.100.xx
Index, # next-hops
10.1.1.xx
Index, # next-hops
10.100.1.xx
Index, # next-hops
10.10.0.xx
Index, # next-hops
FIB TCAM
BRKARC-3470
Compare
lookup
key
10.1.1.2
10.100.1.xx
Ingress
unicast IPv4
packet header
Hit in FIB
Index,
returns
result#
in FIB DRAM
next-hops
Next-hop 1 (IF, MAC)
Load-Sharing
Hash

Offset
mod
Return
lookup
result

# nexthops

Adj Index
Adjacency
index
identifies ADJ
block to use
Modulo
function selects
exact next hop
entry to use
FIB DRAM
Adjacency Table
Cisco Public
Result
ECMP Load Sharing

Up to 16 hardware load-sharing paths per prefix
10.10.0.0/16
Use maximum-paths command in routing protocols to

control number of load-sharing paths
Load-sharing is per-IP flow
Configure load-sharing hash options with global

ip load-sharing command:
Source and Destination IP addresses

Source and Destination IP addresses plus L4 ports (default)
Destination IP address and L4 port
Additional randomized number added to hash

prevents polarization
10.10.0.0/16
via Rtr-A
via Rtr-B
Automatically generated or user configurable value
BRKARC-3470
Cisco Public
63
= Packet Headers
HDR
M1 L3 Packet Flow
DATA
= Packet Data
CTRL
Supervisor Engine
Credit grant for

fabric access
Fabric Module 2
Fabric Module 3
Fabric ASIC
Fabric ASIC
Fabric ASIC
Receive from
fabric
Return buffer
credit
10
8
Fabric ASIC
Layer 3
Engine
VOQs
Submit packet
headers for
lookup
Ingress port QoS
Layer 2
Engine
Replication
Engine
Forwarding
Engine
BRKARC-3470
Fabric ASIC
Layer 3
Engine
VOQs
Layer 2
Engine
Replication
Engine
Forwarding
Engine
Return result
10G MAC
2
Linksec
11
L2 ingress and
egress SMAC/
DMAC lookups
10G MAC
3
Receive
packet from
wire
L3 FIB/ADJ lookup
Ingress and egress
ACL/QoS/NetFlow
lookups
Egress port
QoS
13
Module 2
LinkSec decryption
Module 1
e1/1
12
Transmit to
fabric
Return
credit to
pool
Central Arbiter
Fabric Module 1
VOQ arbitration
and queuing
15
Transmit
packet on
wire
Linksec
LinkSec
encryption
14
e2/1
Cisco Public
64
HDR
DATA
= Packet Headers
= Packet Data
CTRL
F2 L3 Packet Flow
7
Supervisor Engine
Credit grant for

fabric access
Return
credit
to pool
10
Central Arbiter
Fabric Module 1
Fabric Module 2
Fabric Module 3
Fabric Module 4
Fabric Module 5
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Transmit
to fabric
VOQ arbitration
Fabric ASIC
Submit packet headers for lookup
VOQ FE
Return result
SoC
2
BRKARC-3470
Fabric ASIC
Receive from fabric

Egress port QoS
Return buffer credit
Ingress
port QoS
(VOQ)
Module 1
e1/1
Receive
packet
from wire
L2 ingress and egress SMAC/

DMAC lookups
L3 FIB/ADJ lookup
Ingress and egress ACL/QoS
lookups
11
Transmit
packet
on wire
VOQ
SoC
Module 2
e2/1
Cisco Public
65
Layer 3 Forwarding with F1 I/O Modules

F1 modules do not natively provide Layer 3 switching
Cannot inter-VLAN route on their own
However, one or more M1/M1-XL modules can provide proxy Layer 3 services
M1 forwarding engines can proxy route for F1 modules
Proxy L3 forwarding enabled by default in M1/F1 VDC
Packets destined to router MAC forwarded to M1 modules for Layer 3 via internal
Router Port-Channel
Selection of which port on which M1 module based on EtherChannel hash function
Traffic requiring L3 from F1 modules traverses the fabric, vectoring toward M1 ports enabled
for proxy L3
M1 module receiving such packets programmed to perform full ingress/egress L3 lookups
BRKARC-3470
Cisco Public
66
Proxy L3 Forwarding Conceptual

From F1 perspective, Router MAC reachable through giant port-channel
All packets destined to Router MAC forwarded through fabric toward one
member port in that channel
10.1.10.100
vlan 10
All F1 modules
Up to 128 links
All M1 modules
interface vlan 10
ip address 10.1.10.1/24
!
interface vlan 20
ip address 10.1.20.1/24
10.1.20.100
vlan 20
BRKARC-3470
Cisco Public
67
Proxy L3 Forwarding Actual

VLAN
10
DMAC
router_mac
Programming of all
F1 forwarding engines
Dest Port
internal_channel (e3/1-8,e4/1-8)
EtherChannel Hash Function

hash_input (from packet) select_member_port
M1
VOQs
5
FE
FE
SoC
F1
FE
e2/1
SoC
10.1.20.100
vlan 20
Fabric
FE
Fabric
VOQs
10
Fabric
M1
Fabric
VOQs
Fabric
Modules
F1
FE
Replication
Engine
9
e3/8
e3/7
e3/6
e3/5
Replication
Engine
Replication
Engine
Replication
Engine
Replication
Engine
e3/4
e3/3
e3/2
e3/1
e4/8
e4/7
e4/6
e4/5
Fabric
FE
VOQs
BRKARC-3470
Replication
Engine
Ingress MAC:
VLAN DMAC
10
router_mac
Routing:
DIP
10.1.20.100
Egress MAC:
VLAN DMAC
20
server_2_mac
Fabric
e1/1
10.1.10.100
vlan 10
Can be up to 128 ports

on M1 modules
Replication
Engine
Replication
Engine
e4/4
e4/3
e4/2
e4/1
Cisco Public
Dest Port
L3_lookup
Next Hop
server_2_mac (v20)
Dest Port
e2/1
Programming of all
M1 forwarding engines
interface vlan 10
ip address 10.1.10.1/24
!
interface vlan 20
ip address 10.1.20.1/24
68
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
70
Forwarding tables built on control plane using
multicast protocols
PIM-SM, PIM-SSM, PIM-Bidir, IGMP, MLD
Tables downloaded to:

Forwarding engine hardware for data plane forwarding (FIB/ADJ)
Replication engines for data plane packet replication (Multicast Expansion Table
MET)
BRKARC-3470
Cisco Public
71
IPv4 Multicast FIB TCAM Lookup

Ingress
multicast
packet header
Generate TCAM lookup

key (source and group
IP address)
Generate
Lookup Key
10.1.1.10, 239.1.1.1
Forwarding Engine
Compare
lookup key
10.1.1.12, 239.1.1.1
RPF, ADJ Index
MET Index
10.1.1.10, 232.1.2.3
RPF, ADJ Index
MET Index
10.4.7.10, 225.8.8.8
RPF, ADJ Index
MET Index
HIT! 10.1.1.10, 239.1.1.1
RPF, ADJ Index
MET Index
10.6.6.10, 239.44.2.1
RPF, ADJ Index
FIB TCAM
FIB DRAM
MET Index
Result
Adjacency Table
Hit in FIB
returns result
in FIB DRAM
Replication
Engine
Adj Index
Identifies multicast
adjacency entry
Return
lookup
result
OIFs
OIFs
Replication for
each OIF in
MET block
Replicate
OIFs
OIFs
MET
BRKARC-3470
MET index used to

find OIFs for
replication
Cisco Public
72
Egress Replication
IIF
Local
OIF
Module 1
Replication
MET
Engine
Distributes multicast replication load among

replication engines of all I/O modules with OIFs
Fabric ASIC
Input packets get lookup on ingress forwarding

engine
For OIFs on ingress module, ingress replication
engine performs the replication
For OIFs on other modules, ingress replication

engine replicates a single copy of packet into fabric
for those egress modules, fabric replicates as
needed
Fabric
Module
Fabric
Copy
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Replication
MET
Engine
Replication
MET
Engine
Replication
MET
Engine
Each egress forwarding engine performs lookup to

drive replication
Replication engine on egress module performs
replication for local OIFs
BRKARC-3470
Local
OIFs
Cisco Public
Local
OIFs
73
= Packet Headers
HDR
M1 L3 Multicast Packet Flow
Fabric replication
Fabric Module 1
Fabric Module 2
Fabric Module 3
Fabric ASIC
Fabric ASIC
Fabric ASIC
Transmit to
fabric
13
VOQ queuing
10
Fabric ASIC
Transmit
multicast fabric
distribution
packet
Ingress port QoS
VOQs
Replication
Engine
Layer 3
Engine
L3 multicast FIB
lookup
Ingress ACL/QoS/
NetFlow lookups
Egress ACL/QoS/
NetFlow lookups
Layer 2
Engine
L2 ingress
snooping
lookup
Forwarding
Engine
Return MET
result
10G MAC
3
Receive
packet from
wire
Linksec
e1/1
Replicate for
fabric delivery
17
Module 1
LinkSec decryption
16
VOQs
Layer 2
Engine
Replicate for
local OIF
delivery
14
Replication
Engine
Forwarding
Engine
Submit packet
headers for
egress lookups
15
10G MAC
L2 egress
snooping
lookup
Egress port
QoS
18
Module 2
Fabric ASIC
Layer 3
Engine
20
BRKARC-3470
12
Dequeue multicast
distribution copy
from fabric
11
Submit packet
headers for
lookup
= Packet Data
DATA
Transmit
packet on
wire
Linksec
LinkSec
encryption
19
e2/1
Cisco Public
74
= Packet Headers
HDR
F2 L3 Multicast Packet Flow
= Packet Data
Fabric replication
Fabric Module 1
Fabric Module 2
Fabric Module 3
Fabric Module 4
Fabric Module 5
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
DATA
Transmit multicast fabric

distribution packet
10
VOQ queuing
Fabric ASIC
L2 ingress snooping
lookup
L3 multicast FIB lookup
Ingress ACL/QoS lookups
VOQ FE
Return
MET result
Ingress
port QoS
(VOQ)
BRKARC-3470
Fabric ASIC
SoC
e1/1
Egress
port QoS
13
Module 2
VOQ FE
SoC
Egress ACL/QoS
lookups and L2 egress
snooping lookup for
each copy
12
Module 3
Submit packet
headers for lookup
Receive
packet
from wire
Fabric ASIC
Replicate for local

OIF delivery
VOQ FE
SoC
Module 1
11
Replicate for
fabric delivery
Receive from fabric
e2/1
14
Transmit
packet
on wire
e3/1
Cisco Public
75
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
76
What Is Classification?
Matching packets
Layer 2, Layer 3, and/or Layer 4 information
Used to decide whether to apply a particular policy to a packet

Enforce security, QoS, or other policies
Some examples:
Match TCP/UDP source/destination port numbers to enforce security policy
Match destination IP addresses to apply policy-based routing (PBR)
Match 5-tuple to apply marking policy
Match protocol-type to apply Control Plane Policing (CoPP)
etc.
BRKARC-3470
Cisco Public
77
Security ACL
CL TCAM Lookup ACL

Packet header:
SIP: 10.1.1.1
DIP: 10.2.2.2
Protocol: TCP
SPORT: 33992
DPORT: 80
Generate TCAM
lookup key
Generate
Lookup Key
SIP | DIP | Pr | SP | DP
10.1.1.1 | 10.2.2.2 | tcp | 33992 | 80

Compare lookup
key to CL TCAM
entries
Comparisons
(X = Mask)
Forwarding Engine
xxxxxxx| 10.1.2.100
| 10.2.2.2 | |xx
| xxx
xxxxxxx
xx| xxx
| xxx
| xxx
Permit
xxxxxxx | 10.1.68.44 | xx | xxx | xxx
Deny
xxxxxxx | 10.33.2.25 | xx | xxx | xxx
Deny
xxxxxxx
xxxxxxx| |tcp
tcp| xxx
| xxx
22
xxxxxxx | xxxxxxx
| |80
Permit
xxxxxxx | xxxxxxx | tcp | xxx | 23
Deny
xxxxxxx | xxxxxxx | udp | xxx | 514
Deny
HIT! xxxxxxx | xxxxxxx | tcp | xxx | 80
SIP
DIP
Results
Permit
xxxxxxx | xxxxxxx | udp | xxx | 161
Permit
Result
Result affects
final packet
handling
| Pr | SP | DP
CL TCAM
BRKARC-3470
ip access-list example
permit ip any host 10.1.2.100
deny ip any host 10.1.68.44
deny ip any host 10.33.2.25
permit tcp any any eq 22
deny tcp any any eq 23
deny udp any any eq 514
permit tcp any any eq 80
permit udp any any eq 161
Hit in CL TCAM
returns result in
CL SRAM
CL SRAM
Cisco Public
Return
lookup
result
79
CL TCAM Lookup QoS

Packet header:
SIP: 10.1.1.1
DIP: 10.2.2.2
Protocol: TCP
SPORT: 33992
DPORT: 80
QoS Classification ACLs

ip access-list police
permit ip any 10.3.3.0/24
permit ip any 10.4.12.0/24
ip access-list remark-dscp-32
permit udp 10.1.1.0/24 any
ip access-list remark-dscp-40
permit tcp 10.1.1.0/24 any
ip access-list remark-prec-3
permit tcp any 10.5.5.0/24 eq 23
Generate TCAM
lookup key
Generate
Lookup Key
SIP | DIP | Pr | SP | DP
10.1.1.1 | 10.2.2.2 | tcp | 33992 | 80
Forwarding Engine
Compare
lookup key
Comparisons
(X = Mask)
xxxxxxx | 10.2.2.xx
10.3.3.xx | xx | xxx | xxx
Policer ID 1
xxxxxxx | 10.4.12.xx | xx | xxx | xxx
Policer ID 1
10.1.1.xx
xxx |xxx
xxx
10.1.1.xx || xxxxxxx
xxxxxxx || udp
tcp || xxx|
Remark DSCP 32
HIT! 10.1.1.xx | xxxxxxx | tcp | xxx | xxx
Remark DSCP 40
xxxxxxx | 10.5.5.xx| tcp | xxx | 23
Remark IP Prec 3
SIP
DIP
| Pr | SP | DP
CL TCAM
BRKARC-3470
Hit in CL TCAM
returns result in
CL SRAM
CL SRAM
Cisco Public
Results
Result
Result affects
final packet
handling
Return
lookup
result
80
Atomic Policy Programming

Avoids packet loss during policy updates
Enabled by default
Atomic programming process:
Program new policy in free/available CL TCAM entries
Enable new policy by swapping the ACL label on interface
Free CL TCAM resources used by previous policy
BRKARC-3470
Cisco Public
82
Atomic Policy Programming Cont.

To support atomic programming, software reserves 50% of available TCAM
If insufficient resources available, system returns an error and no modifications
made in hardware
Failed to complete Verification: Tcam will be over used, please turn
off atomic update
Disable with no platform access-list update atomic

Disabling may be necessary for very large ACL configurations
Atomic programming attempted but not mandatory
User can disable atomic programming and perform update non-atomically

(assuming ACL fits in CL TCAM)
Default ACL result (deny by default) returned for duration of reprogramming
Use [no] hardware access-list update default-result permit to control default result
BRKARC-3470
Cisco Public
83
Classification Configuration Sessions

Two ways to configure ACL/QoS policies:
Normal configuration mode (config terminal)
Configuration applied immediately line by line
Recommended only for small ACL/QoS configurations, or non-data-plane ACL configuration
Session config mode (config session)

Configuration only applied after commit command issued
Recommended for large ACL/QoS configurations
Config session mode also provides verify facility to dry-run the configuration
against available system resources
No change to existing hardware configuration after verification (regardless of verification result)
BRKARC-3470
Cisco Public
84
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
85
NetFlow on Nexus 7000

NetFlow collects flow data for packets traversing forwarding engines
Per-interface full and sampled NetFlow provided by M1 module
hardware
M1M1
M1F1
F1M1
F1F1
F2F2
Bridged
Yes
Yes
No
No
No**
Routed
Yes
Yes
Yes*
Yes*
No**
* From release 5.2(1)

** Hardware supports ingress
sampled NetFlow
Each M1 module maintains independent NetFlow table

512K hardware entries per forwarding engine
Hardware NetFlow entry creation

CPU not involved in NetFlow entry creation/update
BRKARC-3470
Cisco Public
86
Full vs. Sampled NetFlow

NetFlow configured per-direction and per-interface
Ingress and/or egress on per-interface basis
Each interface can collect full or sampled flow data

Full NetFlow: Accounts for every packet of every flow on interface, up to
capacity of NetFlow table
Sampled NetFlow: Accounts for M in N packets on interface, up to
capacity of NetFlow table
BRKARC-3470
Cisco Public
87
Sampled NetFlow Details

Random packet-based sampling
M:N sampling: Out of N consecutive packets, select M consecutive

packets and account only for those flows in the hardware NetFlow table
Sampled flows aged and exported from NetFlow table normally
Advantages
Reduces NetFlow table utilization
Reduces CPU load on switch and collector
Disadvantages
Some flows may not be accounted
Collector extrapolates total traffic load based on configured sampling rate
BRKARC-3470
Cisco Public
88
Netflow Data Export (NDE)

Process of exporting statistics data from network devices to a
collector
Allows long-term baselining, trending, and analysis of NetFlow

data
Exported data sent via UDP
Variety of export formats exist
Exported data and format of records varies
from version to version
BRKARC-3470
Cisco Public
89
NetFlow Data Export Process

To NetFlow Collector
Generate NetFlow v5
or v9 export packets
M1 Module
Fabric
ASIC
Supervisor
Engine
LC
NetFlow
CPU
Table
Aged Flows
Forwarding
Engine
via Inband
Hardware
Flow Creation
VOQs
M1 Module
Main
CPU
via mgmt0
Switched
EOBC
Mgmt
Enet
LC
NetFlow
CPU
Table
Aged Flows
Forwarding
Engine
M1 Module
LC
CPU
To NetFlow Collector
BRKARC-3470
Hardware
Flow Creation
NetFlow
Table
Aged Flows
Forwarding
Engine
Cisco Public
Hardware
Flow Creation
90
Agenda
Fabric Architecture
I/O Module Queuing
Layer 2 Forwarding
IP Forwarding
Classification
NetFlow
Conclusion
BRKARC-3470
Cisco Public
91
Nexus 7000 Architecture Summary

I/O Modules
Future-proofed
chassis designs
with density and
airflow options
Control plane
protocols, system and
network management
Supervisor Engine
Chassis
Fabrics
Variety of front-panel
interface and transceiver
types with hardware-based
forwarding and services,
including unicast/multicast,
bridging/routing, ACL/QoS
classification, and NetFlow
statistics
BRKARC-3470
High-bandwidth fabric to
interconnect I/O modules and
provide investment protection
Cisco Public
92
Conclusion
You should now have a thorough understanding of
the Nexus 7000 switching architecture, I/O module
design, packet flows, and key forwarding engine
functions
Any questions?
BRKARC-3470
Cisco Public
93
93
Complete Your Online

Session Evaluation
Give us your feedback and you
could win fabulous prizes.
Winners announced daily.
Receive 20 Passport points for each
session evaluation you complete.
Complete your session evaluation

online now (open a browser through
our wireless network to access our Dont forget to activate your
Cisco Live Virtual account for access to
portal) or visit one of the Internet
stations throughout the Convention all session material, communities, and
on-demand and live activities throughout
Center.
the year. Activate your account at the
Cisco booth in the World of Solutions or visit
www.ciscolive.com.
BRKARC-3470
Cisco Public
95
Final Thoughts
Get hands-on experience with the Walk-in Labs located in World of
Solutions, booth 1042
Come see demos of many key solutions and products in the main Cisco
booth 2924
Visit www.ciscoLive365.com after the event for updated PDFs, ondemand session videos, networking, and more!
Follow Cisco Live! using social media:
Facebook: https://www.facebook.com/ciscoliveus
Twitter: https://twitter.com/#!/CiscoLive
LinkedIn Group: http://linkd.in/CiscoLI
BRKARC-3470
Cisco Public
BRKARC-3470
Cisco Public
97

Cisco 7000 Nexus Hardware

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisco 7000 Nexus Hardware

Uploaded by

Copyright:

Available Formats

Cisco Nexus 7000 Hardware Architecture

2012 Cisco and/or its affiliates. All rights reserved.

Nexus 7000/5000/2000/1000v Deployment Case Studies

Nexus Multicast Design Best Practices

VDC Design and Implementation

Deploying Virtual Port Channel in NX-OS

NX-OS Routing & Layer 3 Switching

Cisco FabricPath Technology and Design

Operating and Deploying NX-OS

Troubleshooting Cisco Nexus 7000 Series Switches

Configuring Nexus 7000 Virtualization Lab

FabricPath Deployment in the Data Center Lab

2012 Cisco and/or its affiliates. All rights reserved.

What Is Nexus 7000?

2012 Cisco and/or its affiliates. All rights reserved.

2012 Cisco and/or its affiliates. All rights reserved.

Nexus 7000 Chassis Family

NX-OS 4.1(2) and later

NX-OS 5.2(1) and later

2012 Cisco and/or its affiliates. All rights reserved.

Key Chassis Components

2012 Cisco and/or its affiliates. All rights reserved.

2012 Cisco and/or its affiliates. All rights reserved.

Console, aux, and out-of-band management interfaces

2012 Cisco and/or its affiliates. All rights reserved.

Nexus 7000 I/O Module Families

F family High performance, low latency, low power with streamlined

2012 Cisco and/or its affiliates. All rights reserved.

Supported in NX-OS release 5.0(2a) and later

8-Port 10GE M1 I/O Module

Distributed L3 multicast replication

2012 Cisco and/or its affiliates. All rights reserved.

8-Port 10G XL M1 I/O Module Architecture

Front Panel Ports

2012 Cisco and/or its affiliates. All rights reserved.

N7K-M132XP-12 Supported in all releases

32-Port 10GE M1 I/O Modules

Oversubscription option for higher density (up to 4:1)

Supports Nexus 2000 (FEX) connections

2012 Cisco and/or its affiliates. All rights reserved.

Shared vs. Dedicated Mode

Another port group

Port group group of contiguous

First interface in port group gets 10G

2012 Cisco and/or its affiliates. All rights reserved.

32-Port 10G M1 I/O Module Architecture

Front Panel Ports

2012 Cisco and/or its affiliates. All rights reserved.

Supported in NX-OS release 5.1(1) and later

32-Port 1G/10GE F1 I/O Module

Layer 2 forwarding with L3/L4 services (ACL/QoS)

* sometimes called switch-on-chip

2012 Cisco and/or its affiliates. All rights reserved.

32-Port 1G/10G F1 I/O Module Architecture

Front Panel Ports

2012 Cisco and/or its affiliates. All rights reserved.

Supported in NX-OS release 6.0(1) and later

48-Port 1G/10GE F2 I/O Module

Layer 2/Layer 3 forwarding with L3/L4 services (ACL/QoS)

* sometimes called switch-on-chip

2012 Cisco and/or its affiliates. All rights reserved.

48-Port 1G/10G F2 I/O Module Architecture

Front Panel Ports

2012 Cisco and/or its affiliates. All rights reserved.