BRKDCT-1044 0101

FCoE for the IP Network Engineer
Session ID-BRKDCT-1044
Housekeeping
We value your feedback- don't forget to complete your online session evaluations after each session & complete the Overall Conference Evaluation which will be available online from Thursday Visit the World of Solutions Please remember this is a 'non-smoking' venue! Please switch off your mobile phones Please make use of the recycling bins provided Please remember to wear your badge at all times
Presentation_ID
2010 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Agenda
Datacenter Directions SAN Technology Trends IO Consolidation Fibre Channel Basics Enhanced Ethernet FCoE Technology Primer Translating FC to FCoE Basic Example Designs Q&A
Presentation_ID
Cisco Public
Data Center Directions
Presentation_ID
Cisco Public
Virtualization brings new services to the DC

USERS Internet, Intranet, DC access,
Resource pooling for on demand services Integrated services to reduce footprint New Delivery models for next gen apps; web 2.0, mashups, SaaS Real Time Infrastructure for improved SLAs & instant provisioning Application, Server & Storage mobility for business continuance Higher uptime with simpler upgrade, refresh, & migration maximised asset Utilisation and Reduced carbon Cabling consolidation improve airflow, lower labour & acquisition costs, fewer failures.
Cisco Public
Reduced OPEX Reduced CAPEX Service Agility Higher Availability & Resilience Granular services & tiering Green IT Business to IT synergy Improved data protection Extends the lifetime of the DC Simplifies scaling & technology adoption strategy
5
Virtualised Network
Virtualized Compute
Virtualised Network
Virtualized Compute
Virtualised Storage
Virtualised Storage
Space, power & cooling savings

DC1 DC2
The ubiquitous network touches everything, so it is the enabler for the evolving Data Center
Presentation_ID
Data Center transformation

Today
Security Security
Tomorrow
Network End to end Infrastructure

front end
Network
Compute
Apps Apps
Compute & Platform

Process
back end
Network Storage
DR/Compliance
Compliance
Storage
Facilities
Network
Facilities
Presentation_ID
How do you get organisational collaboration & synchronisation? 6

Cisco Public
SAN Technology Trends
Presentation_ID
Cisco Public
SAN Consolidation & Virtualization

SAN switch innovation Dense Director switches with Logical SANs (VSAN) to reduce footprint and increase HA Performance monitoring & trending, QoS, encryption and replication requirements are integrated Intelligent applications as embedded features or plug in modules Accelerated SAN Extension and SAN Routing are integral to the switch Features to enhance VM performance, scalability, mobility are driving Storage connectivity development Multi Protocol connectivity FCoE, FC, iSCSI, FCIP High Bandwidth 8Gig, 10Gig,16Gig,
Presentation_ID 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public
SAN Density, Performance and Availability

Tiered Connectivity, Investment Protection, Operational Efficiency
MDS 9513 (528 Ports)
Integrated Call-Home
Non-Disruptive
Online Software Upgrades, instrumentation
Stateful Software
Failover and Re-startable Software Modules
End-to-End Data Integrity

ECC, Parity, CRC
Logical Redundancy
VSANs, VRRP, ISL Bundling, Load Balancing
Physical Redundancy
Supervisors, Power Supplies, Fabrics
Integrated SAN Services

Saving power, space, and simplifying management
SAN Extension (DR/BC)
WDM Optics, FCIP, Spoofing, Increased Buffer credits
SAN Routing
Inter VSAN, service isolation, Consolidated backup
Storage Virtualization
Virtual SAN fabrics (VSAN) Trunked Fabric ports, NPIV, Multi-Tenancy
Media Servers for Backup

Tape Acceleration, DMM VTL enhancements,
Security Services
Encryption, RBAC, VSAN ACL, Binding, AAA
10
Logical Segmentation for SAN Consolidation
Overlay isolated virtual fabrics on same physical infrastructure
With VSANs
Number of Switches Switch Utilization Simplified Management On-demand Flexibility Overall TCO Fewer Optimal Yes Yes Low
Presentation_ID
Cisco Public
11
IO Consolidation
Presentation_ID
Cisco Public
12
FCoE Connectivity Extends FC SANs
FC
SAN Extension VSAN SAN Security Zoning
FC
FICON
FC FICON
iSCSI
QoS
FCoE
FCIP
SAN Fabric
Preserves FC investments Simplifies SAN-attach of servers
Presentation_ID
Cisco Public
13
Before I/O Consolidation

Parallel LAN/SAN Infrastructure Inefficient use of Network Infrastructure 5+ connections per server higher adapter and cabling costs
Adds downstream port costs; cap-ex and op-ex Each connection adds additional points of failure in the fabric
LAN
SAN A
SAN B
Longer lead time for server provisioning Multiple fault domains complex diagnostics Management complexity
Ethernet FC
Presentation_ID
Cisco Public
14
I/O Consolidation
LAN SAN A SAN B
Nexus 5000
N4K-1
N4K-2
Reduction of server adapters Simplification of access layer and cabling Gateway free implementationfits in installed base of existing LAN and SAN L2 Multipathing Access Distribution Lower total cost of ownership Fewer cables Investment protection (LANs and SANs) Consistent operational model
Blade Chassis
Enhanced Ethernet and FCoE Ethernet FC
Presentation_ID
Cisco Public
15
FC over Ethernet (FCoE)

FCoE
Mapping of FC Frames over Ethernet Enables FC to Run on a Lossless Ethernet Network
Benefits
Fewer Cables
Both block I/O & Ethernet traffic co-exist on same cable
Fewer adapters needed Overall less power Interoperates with existing SANs
Ethernet Fibre Channel Traffic
Management SANs remains constant
No Gateway
Presentation_ID
Cisco Public
16
FCoE proof points

LAN
4
SAN-A
SAN-B
LAN
4
SAN-A
SAN-B
Nearly Half the Cables

16 Servers Adapters Switches Cables Mgmt Pts Enet 16 2 36 2 FC 16 2 36 2 Total 32 4 72 4 16 Servers Adapters Switches Cables Mgmt Pts Enet 16 2 36 2 FC 0 0 4 0 Total 16 2 40 2
Presentation_ID
Cisco Public
17
Why Virtualize with Converged Fabric?

Consolidates separate LAN, SAN, and server cluster network environments into a Unified fabric with Fibre Channel over Ethernet (FCoE) for I/O consolidation, single unified fabric. reducing power and cabling Same operational model as today requirements and simplifying data center networks, especially for SAN Wire once infrastructure the road to consolidation of Fibre Channel stateless computing Lower IT total cost of ownership
Presentation_ID
Cisco Public
18
Fibre Channel Basics
Presentation_ID
Cisco Public
19
Network Behavior & Characteristics

Ethernet is non-deterministic.
Flow control is destination-based Relies on TCP drop-retransmission / sliding window
Fibre-Channel is deterministic.
Flow control is source-based (B2B credits) Services are fabric integrated (no loop concept)
Presentation_ID
Cisco Public
20
Fibre Channel Port Types

Fibre Channel Switch
Input Port Fabric X Output Port
Fabric Switch E_Port Fabric TE_Port Switch
E_Port
F_Port
NP_Port
NPV Switch
TE_Port
End Node End Node
F_Port
N_Port
F_Port
N_Port
Presentation_ID
Cisco Public
21
Virtual SANs (VSANs)

Virtual Fabric Seperation
A Virtual SAN (VSAN) Provides a Method to Allocate Ports within a Physical Fabric and Create Virtual Fabrics
Analogous to VLANs in Ethernet Virtual fabrics created from larger costeffective redundant physical fabric Reduces wasted ports of a SAN island approach Fabric events are isolated per VSAN which gives further isolation for High Availability FC Features can be configured on a per VSAN basis.
VSANs supported on MDS and Nexus 5000 Product lines
Physical SAN Islands Are Virtualized onto Common SAN Infrastructure
22
Start From the Beginning

Start with the host and a target that need to communicate Host has 2 HBAs (one per fabric) each with a WWN Target has multiple ports to connect to fabric Connect to a FC Switch Port Type Negotiation Speed Negotiation FC Switch is part of the SAN fabric Most commonly, dual fabrics are deployed for redundancy
HBA FABRIC A
Target
FC Fabric
Initiator
23
Buffer to Buffer Credits

Fibre Channel Flow Control
B2B Credits used to ensure that FC transport is lossless # of credits negotiated between ports when link is brought up # Credits decremented with each packet placed on the wire Independent of packet size If # credits == 0, no more packet transmission # of credits incremented with each transfer ready received B2B Credits need to be taken into consideration as distance and/or bandwidth increases
16 15 16 Host
16
R_RDY Packet
24
Fabric Channel ID (FCID)

Fibre Channel Addressing Scheme
FCID assigned to every WWN corresponding to an N_Port FCID made up of switch domain, area and device domain ID is native to a single FC switch limitation of domain IDs in a single fabric Forwarding decisions made on domain ID found in first 8 bits of FCID
Domain ID
FC Fabric
Switch Topology Model

Session_ID Presentation_ID
Switch Domain
Area
Device
Fabric Shortest Path First

Fibre Channel Forwarding FSPF routes traffic based on destination domain ID For FSPF a domain ID identifies a single switch
This limits the max number of switches that can support in the Fabric to 239 when FSPF is supported
FSPF performs hop-by-hop routing FSPF uses total cost as the metric to determine most efficient path FSPF supports equal cost load balancing across links
Presentation_ID
Cisco Public
26
My port is upcan I talk now?

FLOGIs/PLOGIs
Step 1: Fabric Login (FLOGI) determines the presence or absence of a Fabric exchanges Service Parameters with the Fabric switch identifies the WWN in the service parameters of the accept frame and assigns a Fibre Channel ID (FCID) initializes the buffer-to-buffer credits Step 2: Port Login (PLOGI) required between nodes that want to communicate similar to FLOGI transports a PLOGI frame to the designation node port In p2p topology (no fabric present), initializes buffer-to-buffer credits
Target
FC Fabric E_Port
F_Port N_Port
HBA
Initiator
27
Login completealmost there

Fabric Zoning
Zones are the basic form of data path security A bidirectional ACL. zone members can only see and talk to other members of the zone devices can be members of more than one zone By default, devices not in a zone are isolated from other devices. Zones belong to a zoneset Zoneset must be active to enforce zoning Only one active zoneset per fabric or per VSAN
fcid 0x10.00.01 [pwwn 10:00:00:00:c9:76:fd:31] [tnitiator] fcid 0x11.00.01 [pwwn 50:06:01:61:3c:e0:1a:f6] [target]
pwwn 50:06:01:61:3c:e0:1a:f6
Target
FC Fabric
pwwn 10:00:00:00:c9:76:fd:31
Initiator
28
What is NPIV?
N-Port ID Virtualization (NPIV) provides a means to assign multiple FCIDs to a single N_Port Allows multiple applications to share the same Fiber Channel adapter port Different pWWN allows access control, zoning, and port security to be implemented at the application level Usage applies to platforms such as VMWare, MS Virtual Server and Citrix
Application Server FC NPIV Core Switch
Email
Email I/O N_Port_ID 1 Web I/O N_Port_ID 2 File Services I/O N_Port_ID 3
F_Port
F_Port
Web
File Services
Presentation_ID
N_Port
Cisco Public
29
What is NPV?
N-Port Virtualizer (NPV) utilizes NPIV functionality to allow a switch to act like a server performing multiple logins through a single physical link Physical servers connected to the NPV switch login to the upstream NPIV core switch
Physical uplink from NPV switch to FC NPIV core switch does actual FLOGI Subsequent logins are converted (proxy) to FDISC to login to upstream FC switch
No local switching is done on an FC switch in NPV mode FC edge switch in NPV mode Does not take up a domain ID Scalability will be dependent on FC login limitation
Nexus 5000, MDS 91xx, MDS blade switches, UCS Fabric Interconnect
F-Port
FC NPIV Core Switch
Eth1/1
Server1 N_Port_ID 1 Server2 N_Port_ID 2 Server3 N_Port_ID 3
NP-Port
F-Port
Eth1/2
F_Port
Eth1/3
N-Port
Presentation_ID
Cisco Public
30
Enhanced Ethernet
Presentation_ID
Cisco Public
31
Standards for I/O Consolidation

Developed by IEEE 802.1 Data Center Bridging Task Group (DCB) All technically stable Final standards expected by mid 2010 Standard / Feature
IEEE 802.1Qbb Priority-based Flow Control (PFC) IEEE 802.3bd Frame Format for PFC IEEE 802.1Qaz Enhanced Transmission Selection (ETS) and Data Center Bridging eXchange (DCBX) T11 FC-BB-5 FCoE Standard
Status of the Standard

Completed first sponsor ballot. Draft 2.3 available. Basically done. Completed first sponsor ballot. Draft 2.1 available. Basically done. Completed WG recirculation ballot. Draft 1.4 available. Going in sponsor ballot after the July plenary. Approved on June 3rd 2009.
Presentation_ID
Cisco Public
32
Why 10 Gigabit Ethernet to the Server?

Multi core CPU architectures driving increased network bandwidth demands Virtual Machines driving increased I/O connections and I/O bandwidth per server
Low latency 10GE affordability (even optics) Increased adoption of NAS (NFS/CIFS) and iSCSI Consolidation of networks
Unified Fabrics with ethernet
Ubiquity of large scale Ethernet networks

Extensive range of management, diagnostic and troubleshooting tools Massive base of manufacturers, suppliers and integrators competition, price, services and innovation
Presentation_ID
Cisco Public
33
Twinax Copper Cable

Low power consumption Low cable cost
SFP+ CX-1 Copper (SFF 8431)
SAN A LAN SAN B
Low transceiver latency Low error rate (1017) Thinner cable with higher bend radius
16x10 GE Cables
Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server Application Server
Supports 10GE passive direct attached up to 10 meters
Easier to manage cabling solution reduces deployment time All copper cables are contained within rack
Presentation_ID
16x10 GE Cables
34
Cisco Public
10 Gigabit Ethernet Media

10G Options
Connector (Media)
SFP+ CU*
copper
In-rack & Cross-rack

Cable
Twinax Twinax MM OM2 MM OM3 MM OM2 MM OM3
Cat6 Cat6a/7 Cat6a/7
Distance
<10m 15m 10m 100m 82m 300m
55m 100m 30m
Power (each side)

~ 0.1W 4W 1W 1W
~ 6W*** ~ 6W*** ~ 4W***
Transceiver Latency (link)

~ 0.1s ~ 0.1s ~0 ~0
2.5s 2.5s 1.5s
Standard
SFF 8431** IEEE 802.3ak none IEEE 802.3ae
X2 CX4
copper
SFP+ USR
MMF, ultra short reach
SFP+ SR
MMF,short reach
RJ45 10GBASE-T
copper
IEEE 802.3an
* Terminated cable
** Draft
*** As of 2008; expected to decrease over time
Across racks
35
Priority Flow Control

Fibre Channel over Ethernet Flow Control
Enables lossless Ethernet using PAUSE based on a COS as defined in 802.1p When link is congested, CoS assigned to FCoE will be PAUSEd so traffic will not be dropped Other traffic assigned to other CoS will continue to transmit and rely on upper layer protocols for retransmission
Transmit Queues
Fibre Channel
One Two Three
R_RDY
STOP
Ethernet Link
Receive Buffers
One Two
PAUSE
Three Four Five Six Seven Eight
Four Five Six Seven
Eight Virtual Lanes
B2B Credits
Presentation_ID
Packet
Eight
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public
36
Can Ethernet Be Lossless?

Yes, with Ethernet PAUSE Frame
Ethernet Link
STOP
Switch A
PAUSE
Queue Full
Switch B
Defined in IEEE 802.3Annex 31B
The PAUSE operation is used to inhibit transmission of data frames for a specified period of time
Ethernet PAUSE transforms Ethernet into a lossless fabric, a requirement for FCoE.
Presentation_ID
Cisco Public
37
Enhanced Transmission Standard

Bandwidth Management -- IEEE 802.1Qaz
Required when consolidating I/O Prevents a single traffic class of hogging all the bandwidth and starving other classes When a given load doesnt fully utilize its allocated bandwidth, it is available to other classes Helps accommodate for classes of a bursty nature
Offered Traffic
10 GE Link Realized Traffic Utilization

3G/s HPC Traffic 3G/s 2G/s
3G/s
3G/s
2G/s
3G/s
3G/s 3G/s 3G/s
Storage Traffic 3G/s
3G/s
3G/s
4G/s
6G/s
3G/s
LAN Traffic 4G/s
5G/s
t1
Presentation_ID
t2
t3
t1
t2
t3 38
DataCenter Bridging Protocol (DCBX)

Just like CDP with VOIP
Auto-negotiation of capability and configuration
Priority Flow Control capability and associated CoS values
Allows one link peer to push config to link peer

Link partners can choose supported features and willingness to accept
Nexus 5000
Discovers lossless Ethernet Capabilities Responsible for Logical Link Up/Down signaling of Ethernet and FC DCBX negotiation failures will result in:
Per-priority-pause not enabled on CoS values with PFC configuration vfc not coming up when DCBX is being used in FCoE environment
39
FCoE Technology Primer
Presentation_ID
Cisco Public
40
Unified Fabric
Fibre Channel over Ethernet (FCoE)
FCoE is Fibre Channel.
Easy to Understand Same Aligned with the Operational Model FC-BB-4 Model, Standardized in FC-BB-5 Same Techniques of Traffic Management Same Management and Security Models
Completely based on the FC model Same host-to-switch and switch-toswitch behavior of FC E.g., in order delivery or FSPF load balancing WWNs, FC-IDs, hard/soft zoning, nameserver, RSCN
Presentation_ID
Cisco Public
41
Protocol Organization
FCoE itself
Is the data plane protocol It is used to carry most of the FC frames and all the SCSI traffic
FIP (FCoE Initialization Protocol)

It is the control plane protocol It is used to discover the FC entities connected to an Ethernet cloud It is also used to login to and logout from the FC fabric Uses unique BIA on CNA for MAC
The two protocols have:

Two different Ethertypes Two different frame formats Both are defined in FC-BB-5 http://www.cisco.biz/en/US/prod/collateral/switches/ps9441/ps9670/white_paper_c11-560403.html Presentation_ID 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
FIP: FCoE Initialization Protocol

FCoE VLAN discovery
Automatic discovery of FCoE VLANs
Device discovery
ENodes discover VF_Port capable FCF-MACs for VN_Port to VF_Port Virtual Links VE_Port capable FCF-MACs discover other VE_Port capable FCF-MACs for VE_Port to VE_Port Virtual Links The protocol verifies the Lossless Ethernet network supports the required Max FCoE Size
Virtual Link instantiation

Builds on the existing Fibre Channel Login process Fabric Provided MAC Address (FPMA),
Virtual Links maintenance

Timer based
43
Encapsulation Technologies
Operating System / Applications SCSI Layer FCP iSCSI TCP IP FC
1, 2, 4, 8, 10 Gbps
FCP FCIP TCP IP
FCP iFCP TCP IP
FCP
SRP
FCoE IB
10, 20 Gbps
Ethernet
1, 10 . . . Gbps
FCoE is non routable, localised DC transport solution with lower protocol overhead than FCIP or iSCSI
Presentation_ID
Cisco Public
44
FCoE Frame versus iSCSI and Ethernet
Presentation_ID
Cisco Public
45
FCoE Frame
Ethernet Header FCoE Header FC Header
12 Bytes (MAC Addresses) + 4 Bytes (802.1Q Tag) 16 Bytes 24 Bytes
Total: 2180 Bytes
CRC EOF FCS

Presentation_ID
FC Payload
Up to 2112 Bytes 4 Bytes FCoE standard (FC-BB-5) requires jumbo support; 2.5KB = baby jumbo
1 Byte (EOF) + 3 Bytes (Padding) 4 Bytes

46
FCoE Addressing and Forwarding

FCoE frames have:
MAC addresses (hop-by-hop) FC addresses (end-to-end)
Presentation_ID
Cisco Public
47
Converged Network Adapters (CNA)

Replaces multiple adapters per server, consolidating both Ethernet and FC on a single interface Appears to the operation system as individual interfaces (NICs and HBAs)
10GbE
Link
10GbE Fibre Channel
FC Driver bound to FC HBA PCI address
Ethernet Driver bound to Ethernet NIC PCI address
PCIe
Ethernet
Fibre Channel Drivers
Ethernet Drivers
Operating System
Presentation_ID
Cisco Public
48
FC-BB-5 (FCOE) Terminology

Unchanged from previous FC standard
VN_Port: Virtual N_Port VF_Port: Virtual F_Port VE_Port: Virtual E_Port
N F
sw1
E
sw2
F N
FC Hosts
Added to support FCoE
FCoE_LEP (FCoE link endpoint): The data forwarding component that handles FC frame encapsulation/decapsulation, and transmission/reception of FCoE frames FCoE Controller: the entity that implement the FIP protocol
Presentation_ID
Cisco Public
49
FCoE Forwarding
FCF (Fibre Channel Forwarder) is the forwarding entity inside an FCoE switch
- Fibre Channel login happens at the FCF - contains an FCF-MAC address - consumes a Domain ID
FCoE encap/decap happens within the FCF NPV devices are not FCFs and do not have domains
FCoE Switch
FC Domain ID : 15 FC port FC port FC port FC port Eth port
Presentation_ID
FCF Ethernet Bridge
Eth port
Eth port
Eth port
Eth port
Cisco Public
Eth port
Eth port
Eth port
50
FCoE Link
ENode
FC-3/FC-4s FC-3/FC-4s FC-3/FC-4s FC Switching Element
FCF
VN_Port(1) FC-2V
FCoE_LEP
VN_Port(2) FC-2V
FCoE_LEP
VN_Port(3) FC-2V
FCoE_LEP FCoE Controller
VF_Port
FC-2V FCoE Ethernet

FCoE Controller FCoE_LEP
FC-2V
FCoE_LEP FCoE_LEP
Lossless Ethernet MAC
Ethernet_Port
Virtual Links
Lossless Ethernet MAC
Ethernet_Port
MAC VN_Port(1)
MAC VN_Port(2)
MAC VN_Port(3)
FCF-MAC
FCF-MAC
FCF-MAC
Lossless Ethernet network
Each FCF has a single MAC

51
Enode MAC Address

Fibre Channel over Ethernet Addressing Scheme
Domain ID
Enode MAC assigned for each FCID Enode MAC composed of a FC-MAP and FCID
FC Fabric
FC-MAP is the upper 24 bits of the Enodes MAC FCID is the lower 24 bits of the Enodes MAC
FCoE forwarding decisions still made based on FSPF and the FCID within the Enode MAC Fibre Channel FCID Addressing
FC-MAP (0E-FC-xx)
FC-ID 10.00.01
FC-MAC Address
Session_ID Presentation_ID
FC-MAP (0E-FC-xx)
FC-ID 7.8.9
Translating FC to FCoE
Presentation_ID
Cisco Public
53
FCoE, Same Model as FC

Same host to target communication Host has 2 CNAs (one per fabric) Target has multiple ports to connect to fabric Connect to a capable switch Port Type Negotiation (FC port type will be handled by FIP) Speed Negotiation DCBX Negotiation Access switch is a Fibre Channel Forwarder (FCF) Dual fabrics are still deployed for redundancy
CNA Unified Wire Ethernet Fabric FC Fabric
Target
DCB capable Switch acting as an FCF
ENode
54
Fibre Channel over Ethernet Port Types
FCoE Switch
Presentation_ID
Cisco Public
55
My port is upcan I talk now?

FIP and FCoE Login Process
Step 1: FIP Discovery Process Enables FCoE adapters to discover which VLAN to transmit & receive FCoE frames Enables FCoE adapters and FCoE switches to discovers other FCoE capable devices Verifies Lossless Ethernet is capable of FCoE transmission Step 2: FIP Login Process
Simular to existing Fibre Channel Login process - sends FLOGI to upstream FCF Adds the negotiation of the MAC address to use Fabric Provided MAC Address (FPMA) FCF assigns the host a Enode MAC address to be used for FCoE forwarding ENode
Target FC or FCoE Fabric
E_ports or VE_Port VF_Port VN_Port
CNA FIP Discovery

56
Login completealmost there

Fabric Zoning
FCoE fabric zoning done the same as FC fabric zoning Zoning is enforced at the FCF Zoning can be configured on the Nexus 5000 using the CLI or Fabric Manager If Nexus 5000 is in NPV mode, zoning will be configured on the upstream core switch and pushed to the Nexus 5000
FC/FCoE Fabric
pwwn 50:06:01:61:3c:e0:1a:f6
Target
FCF with Domain ID 10
fcid 0x10.00.01 [pwwn 10:00:00:00:c9:76:fd:31] [tnitiator] fcid 0x11.00.01 [pwwn 50:06:01:61:3c:e0:1a:f6] [target]
pwwn 10:00:00:00:c9:76:fd:31
Initiator
57
Basic Example Designs
Presentation_ID
Cisco Public
58
Nexus 5000 FC Module Modes

Nexus 5000 supports two modes of operation
Switch mode N-Port Virtualization (NPV) mode
Modes of operation describe how native FC traffic is handled and therefore requires an FC module Modes are independent of the Nexus 5000 acting as an FCF or FCoE Pass-Through for FCoE traffic Modes are consistent with current FC edge switches (MDS 9124/9134/9222i)
Presentation_ID
Cisco Public
59
Nexus 5000 in Switch Mode

Nexus 5000 FC module can be ISLed to another FC switch via E_ports or TE_ports Zoning enforced on the Nexus 5000 Domain manager, FSPF, zone server, fabric login server, name server run on Nexus 5000 Domain ID is required for every VSAN declared on the Nexus 5000 Interop mode considerations when connecting to non-Cisco FC switches Is viewed within Fabric Manager like any other FC edge switch on the fabric FC traffic is locally switched at the Nexus 5000
** Nexus 5000 supports direct connectivity to FC and FCoE targets (storage arrays) when in switch mode
Presentation_ID
Cisco Public
60
Nexus 5000 in NPV mode

No local switching
All traffic sent to the core SAN switches
Nexus 5000 FC module can work in NPV mode

Server-facing ports are F_ports Uplinks toward SAN NPIV enabled core are NP_ports
Nexus 5000 switches assign FCIDs to attached devices

First byte (domain ID) in FCID received from core SAN switch
Multiple VSANs per F_Port Port-Channel Zoning, DPVM, etc. are not enforced on the Nexus 5000 Domain manager, FSPF, zone server, fabric login server, name server run on NPIV core switch
Presentation_ID
Cisco Public
61
Nexus 4000
embedded FCoE switch for traditional blade systems
Solution Comparison Traditional LAN & SAN in Blade Servers
LAN SAN A SAN B
Unified I/O with Nexus 4000

LAN SAN A SAN B
DCB
Ethernet
FC
Cleaner, Simpler Design

62
Nexus 4000 Solution Comparison

Single Chassis comparison using Virtualized Servers w/ 1GbE & 4G FC Traditional I/O Switches Adapters Power (Switch Only) Cables Management Points 8 48 480 W 32 8 Unified I/O Nexus 4000 2 16 120 W 8 2 % Reduction 75% 66% 75% 75% 75%
Unified I/O lowers CapEx by reducing switches, cables & adapters Additional OpEx savings by reduced power & points of management
63
FCoE: Initial Deployment

The first step is access consolidation: it provides significant cost benefit maintaining a conservative design
SAN A SAN B 10GE Backbone
VF_Ports VN_Ports
Presentation_ID
Cisco Public
64
FCoE: Adding Native FCoE Storage

SAN A SAN B 10GE Backbone
VN_Ports VF_Ports
10GE VN_Ports 4/8 Gbps FC
Presentation_ID
Cisco Public
65
FCoE: Adding VE_ports

SAN A
SAN B
10GE Backbone
VE_Ports VF_Ports 10GE VN_Ports 4/8 Gbps FC
Presentation_ID
Cisco Public
66
Unified I/O: Operations View

Server Ops
Business Processes: Same as today o Converged Network Adapter (CNA) config equivalent to NIC and HBA. Box Mgmt: No impact Network Mgmt System: o Existing tools (Tivoli, BMC Patrol)
SAN Ops
SAN B
Business Process: Same as today o Configure FC Interface mode, speed, VSAN, Port Channels, Trunks, NPV etc. o Monitor SAN specific MIBs, Syslog, RMON. Box Mgmt: o CLI (Role based) o XML API Network Mgmt System: o Existing tool - Fabric Manager
SAN A
Nexus 5K
LAN
Network Ops
Business Process: Same as today o Configure Ethernet port, VLAN, STP, Port Channels, Trunks etc. o Monitor LAN specific MIBs, Syslog, RMON. Box Mgmt: o CLI (Role based) o XML API Network Mgmt System: o Existing GNCC tools (Smarts, Infovista, Alterpoint, Opnet) o Cisco tool - DCNM
67
Converged Activities
Business Processes: o Switch reboot o Image upgrade o User roles o Initial creation of virtual interfaces (veth, vfc) Box Mgmt: Creation of Mgmt Interface Day-2 System level support: Memory, CPU error, Call Home feature, Span ports.
Presentation_ID 2010 Cisco and/or its affiliates. All rights reserved.
1 GbE 10 GbE FC 10GbE Public FCOE Cisco
Q&A Wrap Up
Presentation_ID
Cisco Public
68
Recommended Reading
Presentation_ID
Cisco Public
69
Follow this session up with

BRKDCT-2023 Evolution of the Data Center Access Architecture BRKDCT-2079 The Evolution of Data Center Networks BRKSAN-2047 FCoE - Design, operations and management best practices
Presentation_ID
Cisco Public
70
Complete Your Online Session Evaluation

Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Cisco Preferred Access points for each session evaluation you complete. Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.
Dont forget to activate your Cisco Live and Networkers Virtual account for access to all session materials, communities, and on-demand and live activities throughout the year. Activate your account at any internet station or visit www.ciscolivevirtual.com.
Cisco Public
Presentation_ID
71
Presentation_ID
Cisco Public
72

BRKDCT-1044 0101

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BRKDCT-1044 0101

Uploaded by

Copyright:

Available Formats

FCoE for the IP Network Engineer

2010 Cisco and/or its affiliates. All rights reserved.

2010 Cisco and/or its affiliates. All rights reserved.

Data Center Directions

2010 Cisco and/or its affiliates. All rights reserved.

Virtualization brings new services to the DC

Space, power & cooling savings

2010 Cisco and/or its affiliates. All rights reserved.

Data Center transformation

Network End to end Infrastructure

Compute & Platform

2010 Cisco and/or its affiliates. All rights reserved.

How do you get organisational collaboration & synchronisation? 6

SAN Technology Trends

2010 Cisco and/or its affiliates. All rights reserved.

SAN Consolidation & Virtualization

SAN Density, Performance and Availability

MDS 9509 (336 Ports)

End-to-End Data Integrity

MDS 9506 (192 Ports)

Integrated SAN Services

Media Servers for Backup

Logical Segmentation for SAN Consolidation

Overlay isolated virtual fabrics on same physical infrastructure

2010 Cisco and/or its affiliates. All rights reserved.

2010 Cisco and/or its affiliates. All rights reserved.

FCoE Connectivity Extends FC SANs

Preserves FC investments Simplifies SAN-attach of servers

2010 Cisco and/or its affiliates. All rights reserved.

Before I/O Consolidation

2010 Cisco and/or its affiliates. All rights reserved.

Enhanced Ethernet and FCoE Ethernet FC

2010 Cisco and/or its affiliates. All rights reserved.

FC over Ethernet (FCoE)

2010 Cisco and/or its affiliates. All rights reserved.

FCoE proof points

Nearly Half the Cables

2010 Cisco and/or its affiliates. All rights reserved.

Why Virtualize with Converged Fabric?

2010 Cisco and/or its affiliates. All rights reserved.

Fibre Channel Basics

2010 Cisco and/or its affiliates. All rights reserved.

Network Behavior & Characteristics

2010 Cisco and/or its affiliates. All rights reserved.

Fibre Channel Port Types

Fabric Switch E_Port Fabric TE_Port Switch

2010 Cisco and/or its affiliates. All rights reserved.

Virtual SANs (VSANs)

VSANs supported on MDS and Nexus 5000 Product lines

Physical SAN Islands Are Virtualized onto Common SAN Infrastructure

Start From the Beginning

Buffer to Buffer Credits

Fibre Channel Switch

Fabric Channel ID (FCID)

Switch Topology Model

Fabric Shortest Path First

2010 Cisco and/or its affiliates. All rights reserved.

My port is upcan I talk now?

Login completealmost there

2010 Cisco and/or its affiliates. All rights reserved.

FC NPIV Core Switch

Server1 N_Port_ID 1 Server2 N_Port_ID 2 Server3 N_Port_ID 3