CFP270 SG 1008

CFP270: Brocade 8 Gbit/sec Introduction to Administration and Theory
Student Guide
Revision 1008
CFP270
Corporate Headquarters - San Jose, CA USA T: (408) 333-8000 info@brocade.com European Headquarters - Geneva, Switzerland T: +41 22 799 56 40 emea-info@brocade.com Asia Pacific Headquarters - Singapore T: +65-6538-4700 apac-info@brocade.com
2008 Brocade Communications Systems, Inc. All Rights Reserved. Brocade, the Brocade B-weave logo, Fabric OS, File Lifecycle Manager, MyView, Secure Fabric OS, SilkWorm, and StorageX are registered trademarks and the Brocade B-wing symbol and Tapestry are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. FICON is a registered trademark of IBM Corporation in the U.S. and other countries. All other brands, products, or service names are or may be trademarks or service marks of, and are used to identify, products or services of their respective owners. Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in this document may require an export license from the United States government. Revision: October, 2008
Table of Contents
Module 1 Course Introduction CFP270 Course Overview ......................................................................................................................... 2 Brocade Certification Program ...................................................................................................................... 3 What is a Brocade Certification Worth? ........................................................................................................ 4 CFP270 Course Objectives........................................................................................................................... 5 CFP270 Course Objectives (cont.) ............................................................................................................... 6 CFP270 Course Prerequisites ...................................................................................................................... 7 CFP270 Instructor-Led Course Agenda........................................................................................................ 8 Registering for the BCFP Exam .................................................................................................................... 9 The Fulfillment Process............................................................................................................................... 10 Training Facility and Training Policies ........................................................................................................ 11 Introductions ................................................................................................................................................ 12 Module 2 - Hardware Product Features Objectives...................................................................................................................................................... 2 The Brocade Family of Products ................................................................................................................... 3 Brocade Switches ......................................................................................................................................... 4 Condor2 / Condor ASIC Comparison ........................................................................................................... 5 Virtual Channels (VCs)1................................................................................................................................ 6 Condor2 ASIC ............................................................................................................................................... 7 Condor2 ASIC (cont.) .................................................................................................................................... 8 GoldenEye2 ASIC Features ....................................................................................................................... 9 GoldenEye2 vs. GoldenEye ASIC Comparison .......................................................................................... 10 Brocade 300 vs. Brocade 200E .................................................................................................................. 11 Brocade 5100 vs. Brocade 5000 ................................................................................................................. 12 Brocade 5300 vs. Brocade 4900 ................................................................................................................. 13 Brocade Backbone and Director Family ..................................................................................................... 14 DCX Data Center Backbone ....................................................................................................................... 15 DCX Port Side .......................................................................................................................................... 16 DCX Non-Port Side .................................................................................................................................. 17 CP8 Blade / CR8 (Core) Blade ................................................................................................................... 18 CP8 Blade Overview ................................................................................................................................ 19 CR8 (Core) Blade Overview..................................................................................................................... 20 ICLs Double the Core.................................................................................................................................. 21 ICL Details ................................................................................................................................................... 22 ICL Connections .......................................................................................................................................... 23 ICL Cabling ................................................................................................................................................. 24 Other ICL Cabling Supported Configurations ............................................................................................. 25 ICL Inter Chassis Link .............................................................................................................................. 26 DCX Dual Core Fabric ................................................................................................................................ 27 ICL Ports ..................................................................................................................................................... 28 Brocade 48000 Director .............................................................................................................................. 29 Director CP4 Blades.................................................................................................................................... 30 CP Redundancy .......................................................................................................................................... 31 DCX vs. 48000 ..................................................................................................................................... 32 Chassis: Dimensions / Weight .................................................................................................................... 33 FC8-16 Blade (DCX and 48000) ................................................................................................................. 34 FC8-32 Blade (DCX and 48000) ................................................................................................................. 35 FC8-48 Blade (DCX and 48000) ................................................................................................................. 36 FC4-16 Blade (48000 Only) ........................................................................................................................ 37 FC4-32 Blade (48000 Only) ........................................................................................................................ 38 FC4-48 Blade (48000 Only) ........................................................................................................................ 39 FC10-6 Blade (DCX and 48000) ................................................................................................................. 40
FR4-18i Blade (DCX and 48000) ................................................................................................................ 41 FC4-16IP Blade (48000 Only) ..................................................................................................................... 42 Director Blade Support Matrix ..................................................................................................................... 43 FC Port Blade Frame Flow.......................................................................................................................... 44 DCX Architecture 16-Port Blade .............................................................................................................. 45 DCX Architecture 32-Port Blade .............................................................................................................. 46 DCX Architecture 48-Port Blade .............................................................................................................. 47 Mi10K Director ............................................................................................................................................ 48 The Mi10K 4Gbps Blade (LMQ) .................................................................................................................. 49 Line Module Cards (LIMs) ........................................................................................................................... 50 Optical Paddles ........................................................................................................................................... 51 M6140 Director ............................................................................................................................................ 52 M6140 Control Processor Card (CTP) ........................................................................................................ 53 Brocade 7500 FCR/FCIP ........................................................................................................................... 54 Brocade 7500E FCR/FCIP .......................................................................................................................... 55 7500 and 7500E Feature Matrix ................................................................................................................. 56 Brocade USD-X ........................................................................................................................................... 57 SFP and XFP Transceivers......................................................................................................................... 58 SFP Types................................................................................................................................................... 59 Optical Transceiver Capability .................................................................................................................... 60 Summary ..................................................................................................................................................... 61 Module 3 Fibre Channel Theory Objectives...................................................................................................................................................... 2 Fibre Channel Networking Model .................................................................................................................. 3 Fibre Channel Classes of Service ................................................................................................................. 4 Fibre Channel Frame Format ........................................................................................................................ 5 Node WWN Name Format NWWN ............................................................................................................ 6 Port WWN Name Format PWWN .............................................................................................................. 7 NWWN and PWWN Example ....................................................................................................................... 8 Port Types ..................................................................................................................................................... 9 Fabric Initialization Process ........................................................................................................................ 10 Node Types and Port Types ....................................................................................................................... 11 Fabric Generic Services .............................................................................................................................. 12 Well-Known Addresses ............................................................................................................................... 13 Fibre Channel Network Addressing ............................................................................................................ 14 Fabric Login at FFFFFE .............................................................................................................................. 15 Name Server at FFFFFC ............................................................................................................................ 16 FFFFFC Port and Node Attributes .............................................................................................................. 17 Brocade B-Series Name Server Commands .............................................................................................. 18 nsshow [-rt] .................................................................................................................................................. 19 nscamshow [-t] ............................................................................................................................................ 20 nsallshow..................................................................................................................................................... 21 Fabric Controller at FFFFFD ....................................................................................................................... 22 Brocade RSCN Delivery.............................................................................................................................. 23 Device Communication Example ................................................................................................................ 24 Fabric Initialization Process ....................................................................................................................... 26 Buffer-to-Buffer (BB) Credits ....................................................................................................................... 29 Port Indexes - FCx-48 Blades ..................................................................................................................... 30 Port Indexes - Mixed Blades ...................................................................................................................... 31 Shared Area Numbers - FCx-48 Blades ..................................................................................................... 33 Shared Area Numbers - switchshow........................................................................................................... 35 N_Port ID Virtualization (NPIV) ................................................................................................................... 36 NPIV Commands ........................................................................................................................................ 38 Summary ..................................................................................................................................................... 39
ii
Module 4 Installation and Configuration Objectives...................................................................................................................................................... 2 Environmental Concerns ............................................................................................................................... 3 Brocade Management Interfaces and Tools ................................................................................................. 4 Command Line Interface Shortcuts .............................................................................................................. 5 Initial Configuration ....................................................................................................................................... 6 Log in Through the Serial Port ...................................................................................................................... 7 Set the IP Address ........................................................................................................................................ 8 Log in Through the Ethernet Interface .......................................................................................................... 9 Set Command Line Session Timeout ......................................................................................................... 10 Set the Fabric-Wide Clock .......................................................................................................................... 11 Set Switch Time Zone ................................................................................................................................. 12 Set Login Banner ........................................................................................................................................ 13 Activate Licensed Features ......................................................................................................................... 14 Set the Switch Name................................................................................................................................... 15 Set syslog Server ........................................................................................................................................ 16 Initial Security Configuration ....................................................................................................................... 17 Disable Telnet ............................................................................................................................................. 18 Set Password Rules .................................................................................................................................... 19 Set Password Rules (cont.)......................................................................................................................... 20 Set Password Rules (cont.) Change Default Passwords............................................................................ 21 Set Password Rules (cont.)......................................................................................................................... 22 Set Role Based Access Control (RBAC) .................................................................................................... 23 Set LDAP to RBAC Roles ........................................................................................................................... 29 RADIUS Authentication ............................................................................................................................... 32 Initial Interoperability Configuration ............................................................................................................. 38 Fabric Interoperability Overview .............................................................................................................. 39 Verifying Switch Status ............................................................................................................................... 45 Switch Status Commands ........................................................................................................................... 46 switchstatuspolicyset................................................................................................................................... 48 Port Status................................................................................................................................................... 49 Port Speeds ................................................................................................................................................ 50 Port Settings & Port Setting Commands ..................................................................................................... 51 Verifying Switch Operation .......................................................................................................................... 53 Port Status LEDs ......................................................................................................................................... 54 Fabric Parameters....................................................................................................................................... 55 Configuration Parameters ........................................................................................................................... 56 Consistent Fabric Parameters .................................................................................................................... 57 Booting a Switch ......................................................................................................................................... 58 Summary ..................................................................................................................................................... 59 Appendix - Installation and Configuration ................................................................................................... 61 Initial Interoperability Configuration ............................................................................................................. 62
iii
Module 5 Zoning Objectives...................................................................................................................................................... 2 Zoning Overview ........................................................................................................................................ 3 Process to Implement Zoning ....................................................................................................................... 4 Hierarchy of Objects ...................................................................................................................................... 5 Zone Management ........................................................................................................................................ 6 Zone Aliases ................................................................................................................................................. 7 Zoning Example ............................................................................................................................................ 8 Enabling Zoning .......................................................................................................................................... 12 Disabling Zoning ......................................................................................................................................... 14 Clearing Zoning ........................................................................................................................................... 15 Saving Zoning ............................................................................................................................................. 16 Maximum Zoning Database Size ................................................................................................................ 17 Zone Object Commands ............................................................................................................................. 19 Zoning Display Commands nsaliasshow .................................................................................................... 20 Zoning Display Commands (cont.) nodefind ............................................................................................... 21 Zoning Display Commands (cont.) nszonemember .................................................................................... 22 Zoning Display Commands (cont.) nszonemember -u ............................................................................... 23 Additional Zone Management Commands.................................................................................................. 24 Default Zoning ............................................................................................................................................. 25 Default Zoning defzone .............................................................................................................................. 26 Web Tools Zoning Administration ............................................................................................................ 30 Zoning Enforcement .................................................................................................................................... 31 Zoning Enforcement Command portzoneshow........................................................................................... 35 Implementation Considerations .................................................................................................................. 36 Zoning Best Practices ................................................................................................................................. 37 Adding a New Switch to a Zoned Fabric ..................................................................................................... 39 Ensure New Switch Has No Zoning ............................................................................................................ 40 Connect New Switch to Existing Fabric ...................................................................................................... 41 Merging Two Zoned Fabrics: Segmentation ............................................................................................... 42 Other Merge Fabric Considerations ........................................................................................................... 43 Fabric Manager Fabric Merge Check ......................................................................................................... 44 Fabric Selection .......................................................................................................................................... 45 Merging Check Results Successful ......................................................................................................... 46 Other Zoning Tools .................................................................................................................................... 47 SAN Health Sample Zoning Spreadsheet................................................................................................ 48 Summary .................................................................................................................................................... 49
Module 6 FSPF Routing Objectives...................................................................................................................................................... 2 Routing Overview ....................................................................................................................................... 3 Fabric Terminology ....................................................................................................................................... 4 Principal Switch Path .................................................................................................................................... 6 Principal Switch Commands ......................................................................................................................... 7 Routing Terminology ..................................................................................................................................... 8 Over-Subscribing ISLs .................................................................................................................................. 9 Virtual Channels .......................................................................................................................................... 10 Virtual Channels 1/2/4 Gbit/s ASICs ........................................................................................................ 11 Virtual Channels 8 Gbit/sec ASICs .......................................................................................................... 12 Fabric Shortest Path First (FSPF) ............................................................................................................... 13 FSPF Link Cost ........................................................................................................................................... 14 FSPF Dynamic Load Sharing (DLS) ........................................................................................................... 15 FSPF In-Order Delivery............................................................................................................................... 16 Routing Policies ......................................................................................................................................... 17 Exchange-based Routing ............................................................................................................................ 18
iv
Exchange-based Routing and DLS............................................................................................................. 19 Exchange-based Routing and DLS (cont.) ................................................................................................. 20 Port-based Routing ..................................................................................................................................... 21 Port-based Routing and DLS ...................................................................................................................... 22 Routing Policy Selection ............................................................................................................................. 23 Display Routing Information Overview ..................................................................................................... 24 Display Routing Information fcping .......................................................................................................... 25 fcping Example ............................................................................................................................................ 26 Display Routing Information pathinfo ....................................................................................................... 27 Display Routing Information pathinfo (cont.) ............................................................................................ 28 Display Routing Information topologyshow Exchange-based Routing .................................................... 29 Display Routing Information topologyshow Port-based Routing.............................................................. 30 Interop Mode Considerations Routing ..................................................................................................... 31 What is Brocade B-Series ISL Trunking? ................................................................................................... 32 Trunking Requirements ............................................................................................................................. 33 Condor 4 Gbit/sec Trunking Overview ..................................................................................................... 34 GoldenEye 4 Gbit/sec Trunking Overview ............................................................................................... 35 8 Gbit/sec Trunking Overview .................................................................................................................. 36 Port-based Routing without Trunking: Unpredictable Performance........................................................... 37 Trunking Frame Allocation .......................................................................................................................... 38 Port-based Routing over Trunks ................................................................................................................ 39 Exchange-based Routing over Trunks........................................................................................................ 40 One Port Group with Multiple ISL Trunks ................................................................................................... 41 ASIC Trunk Pseudo-Master ....................................................................................................................... 42 The Deskew Counter .................................................................................................................................. 43 Trunking Related Commands Overview .................................................................................................. 44 trunkshow .................................................................................................................................................... 45 switchshow .................................................................................................................................................. 46 islshow......................................................................................................................................................... 47 portcfgtrunkport/portcfgshow....................................................................................................................... 48 trunkdebug .................................................................................................................................................. 49 Brocade M-series Open Trunking ............................................................................................................... 50 Summary ..................................................................................................................................................... 52 Module 7 Fibre Channel Long Distance Objectives...................................................................................................................................................... 2 Long Distance Fabrics Overview ............................................................................................................... 3 Long Distance Cabling ................................................................................................................................. 4 Long Distance Connectivity Options ............................................................................................................. 6 Long Distance Connectivity Options Native FC over Dark Fiber .............................................................. 7 Long Distance Connectivity Options WDM ................................................................................................ 8 Long Distance Connectivity Options Time Division Multiplexing (TDM) .................................................... 9 Long Distance Connectivity Options SONET/SDH .................................................................................. 10 Long Distance Connectivity Options Extended Distance Solutions ......................................................... 11 FOS Extended Fabrics Feature .................................................................................................................. 12 Extended Fabric Configuration ................................................................................................................... 13 portshow ...................................................................................................................................................... 15 Distance Availability and Performance ....................................................................................................... 16 Extended Distance Availability Trunking ................................................................................................. 17 Extended Distance Availability Protection Switching .............................................................................. 18 Extended Distance Performance Buffer-to-Buffer Credit Allocation ........................................................ 19 E_Port Credit Recovery .............................................................................................................................. 20 portbuffershow ............................................................................................................................................ 26 Summary ..................................................................................................................................................... 27 Appendix - Fibre Channel Long Distance ................................................................................................... 29
Module 8 Administration and Maintenance Objectives...................................................................................................................................................... 2 Centralized Management EFCM & Fabric Manager ..................................................................................... 3 Which Application to Use? ............................................................................................................................ 4 Fabric Manager Overview .......................................................................................................................... 5 EFCM Offerings ............................................................................................................................................ 6 Managing Individual Switches Web Tools and EFCM Basic ........................................................................ 7 EFCM Basic Edition ...................................................................................................................................... 8 Web Tools .................................................................................................................................................... 9 ESCM (Enterprise Server Connectivity Manager) ...................................................................................... 10 SNMP Overview ....................................................................................................................................... 12 SNMP Components .................................................................................................................................... 13 SAN Monitoring with SNMP ........................................................................................................................ 14 SNMP Version Authentication ..................................................................................................................... 15 Brocade SNMP Fabric OS Commands ....................................................................................................... 16 Brocade Single-CP Switch Firmware Download Internal Process.............................................................. 17 CP Differences for Directors and Backbones ............................................................................................. 25 Non-Disruptive CP Card Failover ................................................................................................................ 26 High Availability Commands .................................................................................................................... 27 Other HA Commands ................................................................................................................................. 28 Firmware Maintenance CP Cards ............................................................................................................ 29 Firmware Download Process ...................................................................................................................... 30 48000 Firmware Upgrade .......................................................................................................................... 31 DCX Firmware Upgrade .............................................................................................................................. 45 Firmware Maintenance Single-CP Mode ................................................................................................. 55 Firmware Maintenance Other Commands ............................................................................................... 56 Displaying Control Processor Status firmwareshow .................................................................................. 57 Downloading Firmware for M-Series ........................................................................................................... 58 Archiving Switch Configuration configupload .............................................................................................. 59 Archiving Switch Configuration configupload Example ............................................................................... 60 Restoring Switch Configuration configdownload ........................................................................................ 61 File Copy Protocols ..................................................................................................................................... 62 Summary ..................................................................................................................................................... 63 CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory ................................................ 64 Appendix - Administration and Maintenance More information on firmware download ............................ 65 Appendix A Error Handling....................................................................................................................... 66 Appendix B DCX Firmware Upgrade ....................................................................................................... 69 firmwareshow -v .......................................................................................................................................... 73 CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory ................................................ 74
vi
CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory
Course Introduction

Instructor-Led Module 1 Course Introduction
Revision 1008
Revision 1008
1-1
Course Introduction
CFP270 Course Overview

CFP270, along with CFP271 are designed to prepare you for the Brocade Certified Fabric Professional (BCFP) certification exam #143-060 CFP270 is a 3-day instructor-led class that combines classroom training and lab exercises on many of Brocades basic concepts, terminologies and practices CFP271 is an extension of CFP270 and provides more detailed information about Brocade technologies and product features CFP271 is a blended learning class that combines instructor-led training and web-based training (WBT) eStudy materials and consists of:
3 days of instructor-led training for more hands-on time to reinforce all blended learning (web-based and instructor-led) 3 hours of web-based training eStudy materials provide additional material that you can access after this class is complete
CFP271 web-based training does not have to be completed before the instructor-led training
2008 Brocade Communications Systems, Inc. All Rights Reserved.
Revision 1008
1-2
Course Introduction
Brocade Certification Program

Brocade Certified Fabric Professional (BCFP) This level of certification indicates you have mastered all of the basics of the Brocade switches and are knowledgeable in Fibre Channel theory. Brocade Certified Fabric Designer (BCFD) This level of certification indicates that you have mastered the creation of a successful Data Center Fabric (DCF) design, including knowledge of the key DCF design concepts; assessing DCF requirements; identifying trade-offs involving manageability, availability, scalability, and performance; and the creation of a DCF deployment plan, including integration into existing DCF environments. Brocade Certified SAN Manager (BCSM) This level of certification indicates that you have a detailed understanding of administering Brocade switches and managing aspects of a SAN. Brocade Certified Architect for FICON (BCAF) This level of certification indicates that you have a detailed understanding of FICON concepts, can demonstrate knowledge of Brocade FICON Director and switching SAN components, and can design, install, configure, maintain, manage, and troubleshoot Brocade hardware and software products.
Revision 1008
1-3
Course Introduction
What is a Brocade Certification Worth?

In 2007, Certification Magazine ran their annual salary survey of the top IT certification programs. There were 35,066 participants from 195 countries. Here are some of the results (salaries in US $): Survey results may be found at:
http://www.nxtbook.com/nxtbooks/mediatec/cm1207/ See page 18 and 19
1. 2. 3. 4. 10. Brocade Certified Fabric Professional (BCFP) $96,630 Brocade Certified SAN Manager (BCSM) ISACA Certified Information Security Manager Cisco CCIE Brocade Certified SAN Designer (BCSD) $117,110 $115,720 $111,090 $109,510
Revision 1008
1-4
Course Introduction
CFP270 Course Objectives

After attending this course, attendees should be able to:
Describe the 8 Gbit/sec Brocade product line along with physical layer SAN components Review Fibre Channel concepts Install, configure, and verify functionality of switches in a Brocade Fabric Perform maintenance tasks that include downloading and validating firmware download processes and saving/backing up configuration files Configure, verify, administer, and describe Brocade Zoning Configure a zone merge for two separate SANs using Fabric Manager Describe and evaluate FSPF routing and routing policies Describe multi-switch fabric Trunking and distance solutions
Revision 1008
1-5
Course Introduction
CFP270 Course Objectives (cont.)

Describe management tools and interfaces used to manage Brocade Fabrics Describe and utilize Brocade error messages, commands, and diagnostics Describe and implement routing and distance extension solutions Configure switches to use either a RADIUS or LDAP server for user authentication Configure a B-Series switch for native interoperability with M-Series SANs Describe and use SAN Health to document a SAN Configure ASIC Trunking on B- Series and Open Trunking on M-Series switches
Revision 1008
1-6
Course Introduction
CFP270 Course Prerequisites

Before taking this course, attendees should have:
Completed the FC101 Fibre Channel Fundamentals course 6 months work experience associated with SCSI storage and LANs
Revision 1008
1-7
Course Introduction
CFP270 Instructor-Led Course Agenda

There are 8 modules in this training course:
Module 1 - Course Introduction Module 2 - Hardware Product Features Module 3 - Fibre Channel Theory Module 4 - Installation and Configuration Module 5 - Zoning Module 6 - FSPF Routing Module 7 - Fibre Channel Long Distance Module 8 - Administration and Maintenance
Revision 1008
1-8
Course Introduction
Registering for the BCFP Exam
Pearson VUE is our chosen test delivery vendor. They operate 5000+ testing centers worldwide. To register for an exam or locate the nearest testing center:
Visit http://www.pearsonvue.com/brocade Call 866-361-5817 toll-free in North America Visit http://www.pearsonvue.com for other contact numbers worldwide (some locations may not have toll-free numbers)
Revision 1008
1-9
Course Introduction
The Fulfillment Process

A proper email and mailing address are vital for the fulfillment process After passing the BCFP exam:
1. Within 2 weeks, you will receive a congratulatory email from our fulfillment company with a subject line of Brocade Certification 2. You will be directed to log on to their website to select your gift 3. Your gift, certificate, and information about logo access will be sent to you all in one package
10
Revision 1008
1 - 10
Course Introduction
Training Facility and Training Policies

Our normal routine includes:
Facility information and lab policies Class start and stop times Regular breaks Please:
Be open to new ideas and information Have fun Set your cellular telephone and digital pager to silent
or vibrate
Feel free to share non-compromising SAN experiences Follow lab policies (food, drink) defined at your facility Do NOT download or install applications on your lab workstation
unless your instructor directs you to do so
11
Revision 1008
1 - 11
Course Introduction
Introductions
Please take a moment and share with us:
Name Employer Where you are located? Your background with switches, SANs, Fibre Channel, storage, systems, and networking What you would like to gain from taking this class? State an interesting fact about yourself
12
Revision 1008
1 - 12
Course Introduction

End of Instructor-Led Module 1 Course Introduction
Revision 1008
Revision 1008
1 - 13
This page is intentionally blank.
Revision 1008
1 - 14
Hardware Product Features

Instructor-Led Module 2 Hardware Product Features
Revision 1008
Revision 1008
2-1
Objectives
After completing this module, attendees will be able to:
Identify the Brocade family of Fibre Channel switches, directors and backbones Describe the switch, director, and backbone hardware features and functionality List the different types of fiber optic cables and SFPs
Revision 1008
2-2
The Brocade Family of Products

Brocades product offerings include the following groups of devices that will be discussed in this module:
Switches
200E, 300, 4900, 5000, 5100 and 5300

Enterprise Directors and the Brocade Backbone
48000, Mi10K, M6140 and DCX

Fabric Extension and Routing
7500, 7500E, FR4-18i and USD-X Integrated Routing on 5100, 5300 and DCX
Revision 1008
2-3
Brocade Switches
Revision 1008
2-4
Condor2 / Condor ASIC Comparison

Condor2 ASIC 40 Ports (5 x 8 port groups)1 8 Gbit/sec port speed 1420 available BB credits per 16 ports2 QoS: 16 Virtual Channels3 Condor ASIC 32 Ports (4 x 8 port groups) 4 Gbit/sec port speed 1024 available BB credits per 32 ports QoS: 8 Virtual Channels
Footnote 1: On the 16 and 32-port blades, two of these port groups (2 x 8 = 16 total per ASIC) are used for external ports and the other three port groups are used for internal ports. On the 48 port line card three, of these port groups (3 x 8 = 24 total per ASIC) are used for external ports and the other two port groups are used for internal ports. Footnote 2: Each Condor2 ASIC has 1420 user BB credits: Each front-end port is allocated 8 credits (8 credits x 16 ports = 128 credits) which leaves 1292 credits available per Condor2 ASIC. Note: The 48-port blade has 24 front-end ports per ASIC (8 credits x 24 ports = 192 credits) which leaves 1228 credits available per Condor2 ASIC. Use the portbuffershow command to see available credits. Footnote 3: QoS: 16 Virtual Channels: Details on the next slide.
Revision 1008
2-5
Virtual Channels (VCs)1

VC 0 Class F traffic VC 1 Class 2 ACK / Link Control VC 2 Medium priority / Unassigned traffic VC 3 Medium priority / Unassigned traffic VC 4 Medium priority / Unassigned traffic VC 5 Medium priority / Unassigned traffic VC 6 Class 3 Multicast VC 7 Broadcast / multicast VC 8 Low priority traffic
Switch
VC 9 Low priority traffic VC 10 High priority traffic VC 11 High priority traffic VC 12 High priority traffic VC 13 High priority traffic VC 14 High priority traffic VC 15 Unused
Switch
Footnote 1: Low, Medium and High priority traffic refer to the QoS (Quality of Service) feature which will be covered in a later module.
Revision 1008
2-6
Condor2 ASIC
Support for: E_Port, F_Port, FL_Port1 and M_Port (Mirror Port)2 40 ports divided up into 5 x 8 port groups. 8-port Trunking up to 100 Km3 Condor2 ASIC can run at 1/2/4/8 Gbit/sec speeds
8 Gbit/sec SFP only negotiates to 8/4/2 Gbit/sec Use a 4 Gbit/sec SFP for devices that run at 1 Gbit/sec
8 buffers per port reserved for each port; 1420 available BB credits per ASIC4
Footnote 1: FL_Ports are not supported on the 48-port blade. Footnote 2: In Fabric OS v6.0.0, Mirror ports are not supported on the Condor2 ASIC. Footnote 3: In Fabric OS v6.0.0 8, Gbit/sec speed is only supported up to 10Km. Footnote 4: Each Condor2 ASIC has 1420 user credits: Each front-end port is allocated 8 credits (8 credits x 16 ports = 128 credits) which leaves 1292 credits available per Condor2 ASIC. Note: The 48-port blade has 24 front-end ports per ASIC (8 credits x 24 ports = 192 credits) which leaves 1228 credits available per Condor2 ASIC. Use the portbuffershow command to see available credits.
Revision 1008
2-7
Condor2 ASIC (cont.)

Complete backward compatibility Virtual N_Port support (NPIV) Dynamic Path Selection (DPS) (Exchanged-based routing) 16 Virtual Channels (VCs) per port Credit sharing across VCs FICON capable support1
Footnote 1: FICON support is on a platform / blade basis.
Revision 1008
2-8
GoldenEye2 ASIC Features

The Brocade 300 and 5300 port hardware is based on the GoldenEye2 ASIC:
One ASIC provides 32 FC ports at 1, 2, 4, 8 Gbit/sec Supports F/FL/E/M_Ports 32bit/33MHz PCI interface 8-port trunking
Revision 1008
2-9
GoldenEye2 vs. GoldenEye ASIC Comparison

GoldenEye2 ASIC (Brocade 300) 32 Ports1 (8-port trunk groups) 1, 2, 4, and 8 Gbit/sec port speed 676 available BB credits per 24 ports3 QoS: 16 Virtual Channels
F/FL/E/M_Ports N_Port ID Virtualization
GoldenEye ASIC (Brocade 200E) 24 Ports2 (4-port trunk groups) 1, 2, and 4 Gbit/sec port speed 272 available BB credits per 16 ports QoS: 8 Virtual Channels
F/FL/E_Ports N_Port ID Virtualization
10
Footnote 1: The GoldenEye2 ASIC is architected for 32 ports but only 24 ports are currently used (Brocade 300). Footnote 2: The GoldenEye ASIC is architected for 24 ports but only 16 ports are currently used (Brocade 200E). Footnote 3: Each GoldenEye2 ASIC has 676 user BB credits: Each front-end port is allocated 8 credits (8 credits x 24 ports = 192 credits) which leaves 484 credits available per GoldenEye2 ASIC. Use the portbuffershow command to see available credits.
Revision 1008
2 - 10
Brocade 300 vs. Brocade 200E
Brocade 300 24 FC Ports 1, 2, 4 and 8 Gbit/sec FC port speed supported Ports on Demand (8-port increment) Trunk groups (8-port) USB port 1U form factor One GoldenEye2 ASIC 16 FC Ports
Brocade 200E
1, 2 and 4 Gbit/sec FC port speed supported Ports on Demand (4-port increment) Trunk groups (4-port) USB port - not available 1U form factor One GoldenEye ASIC
11
Both switches are capable of going into Access Gateway mode (covered in CFP271).
Revision 1008
2 - 11
Brocade 5100 vs. Brocade 5000
Brocade 5100 40 FC Ports (8-port trunk groups) 1, 2, 4 and 8 Gbit/sec FC port speed Integrated Routing (EX_Ports) Available per port Two 125 W Power Supply/Fan FRUs2 USB port 1U form factor One Condor2 ASIC
Brocade 5000 32 FC Ports (8-port trunk groups) 1, 2 and 4 Gbit/sec FC port speed Not available Two 300 W Power Supply/Fan FRUs USB port - not available 1U form factor One Condor ASIC
12
Footnote 1: Support for FCR on the Brocade 5100 requires an Integrated Routing license and full Ports on Demand licenses. Footnote 2: The power supply/fan FRUs in the Brocade 5100 are not compatible with those in the Brocade 5000.
Revision 1008
2 - 12
Brocade 5300 vs. Brocade 4900
Brocade 5300 80 FC Ports 1, 2, 4 and 8 Gbit/sec FC port speed supported Support for FCR Available per port Two 300W Power Supplies USB port Three Fan FRUs2 2U form factor Nine GoldenEye2 ASICs 64 FC Ports
Brocade 4900
1, 2 and 4 Gbit/sec FC port speed supported Support for FCR Not Available Two 300W Power Supplies USB port - not available Three Fan FRUs 2U form factor Six Condor ASICs
13
Footnote 1: Support for FCR on the Brocade 5300 requires an Integrated Routing license. Footnote 2: The fans in the Brocade 5300 are larger than the fans in the Brocade 4900.
Revision 1008
2 - 13
Brocade Backbone and Director Family
14
Revision 1008
2 - 14
DCX Data Center Backbone

DCX
12 Slot Chassis - 2 CP, 2 Core, 8 Port Blades Core frame routing has been removed from CP blade, functionally has been moved to the Core (CR8) blade 16/32/48-port blades using the Condor2 ASIC 4 - 2000W Power Supplies1 3 Blowers 2 WWN Cards Chassis size identical to current 48000 Fits in standard 19" rack
SFPs:
4 and 8 Gbit/sec SFPs used on blades that contain the Condor2 ASIC must be Brocade-branded2
15
Footnote 1: Each power supply provides 2000 Watts if connected at 220 VAC or 1000 Watts if connected at 110 VAC. Footnote 2: Fabric OS 6.0+ on blades that use the Condor2 ASIC (more information on this later in this module) will check to make sure the SFP is Brocade-branded. If not Brocade-branded, the port will not come up. 4 Gbit/sec blades that use the Condor ASIC do not have this requirement.
Revision 1008
2 - 15
DCX Port Side

12 slots1
8 port blades (1-4, 9-12) 2 Core blades (5, 8) 2 CP blades (6, 7)
512 Gbit/sec of bandwidth per slot (256 Gbit/sec each direction) Slots are keyed, so blades cannot be installed into the wrong slot
1 2 3 4 5 6 7 8 9 10 11 12
Physical Slot Numbers
16
Footnote 1: Slots 1-4 show 48-port blades Slot 5 is Core blade 0 Slot 6 is CP blade 0 Slot 7 is CP blade 1 Slot 8 is Core blade 1 Slots 9 and 10 show 32-port blades Slots 11 and 12 show 16-port blades
Revision 1008
2 - 16
DCX Non-Port Side

Dual WWN boards behind Logo Plate 2000 W Power Supplies Air intake panel below PS Bay Blower FRU Blowers and power supplies plug directly into Backplane
17
If power supplies are using 220Vac input, only 2 supplies are required for N+1 protection. If power supplies are using 110Vac input, 3 power supplies would be required for N+1 protection. 2 functional blowers are required to cool the DCX.
Revision 1008
2 - 17
CP8 Blade / CR8 (Core) Blade

The switching functionally was moved off the CP blades and put on its own (Core) blade The extra space allows for more functionality to be added on the CR8 (Core) and the CP8 blades Inter Chassis Links (ICLs) are included on the CR8 blade USB port and a second IP port have been added to the CP8 blade The loss of a CP blade does not result in a loss in bandwidth
18
Revision 1008
2 - 18
CP8 Blade Overview

2 per DCX 1 USB 2.0 Port 1 RJ45 Console port 2 IP network ports1 3 IP addresses required: CP0, CP1 and SW0 Dual processors: Main and Co (future use)
USB drive RJ45 port MGMT IP port Service IP port
19
Footnote 1: The Service IP port is for future use. USB port (must be a Brocade-Branded USB drive) can be used for: firmwaredownload configupload / configdownload supportsave LEDs:
LED Power Attention Active Color Green Amber Blue Location Front Panel Front Panel Front Panel Description On = CP Power is operational Off = CP Power if failed On = If on for > 5 seconds, board is defective/faulted On = This CP is the Active CP Off = This CP is either booting, negotiating to be Active or the Standby CP On = Ethernet Port MAC Link has been established at 100/1000Mbps Off = no Link or 10Mbps On blinking = TX or RX frames activity present Off = No TX or RX activity On = USB port is enabled Off = USB port is disabled
Ethernet Link Ethernet Activity USB Port
Green
Front Panel RJ45 Top Front Panel RJ45 Bottom External
Green Green
Revision 1008
2 - 19
CR8 (Core) Blade Overview

Provides core routing of frames either from blade to blade or from DCX to DCX through ICL cable 4 Condor2 ASICs per blade 2 Inter Chassis Links (ICL) (Trunked ISLs between two DCX Chassis) Each cable provides 128 Gbit/sec bandwidth ICL cables: Copper-based proprietary connector
No SFPs
ICL1
ICL0
20
There are two LEDs on the top of the CR8 blade with the following behavior:
LED Power Attention
Color Green Amber
Description On = CP Power is operational Off = CP Power has failed On = If on for > 5 seconds, board is defective/faulted Off = Normal Operation
The two ICL connectors have two LEDs each with the following behavior:
Revision 1008
2 - 20
ICLs Double the Core

One DCX switch as the core is not big enough to meet the needs of the fabric Inter Chassis Links (ICLs) are ISL connections between two DCX switches on the CR8 (Core) blades Creates a dual DCX core fabric: Doubles the port count at the core of the fabric without the use of any user ports Two domains For FICON purposes, the ICLs are not considered a hop
21
ICL kit comes with the following: Four ICL cables Two ICL licenses (one per DCX)
Revision 1008
2 - 21
ICL Details
ICLs are 8 Gbit/sec ISL connections between two DCX switches on the CR8 (Core) blades Speed locked at 8 Gbit/sec Copper-based proprietary connector No SFPs Each cable provides 16 x 8 Gbit/sec bandwidth Licensed feature
22
Revision 1008
2 - 22
ICL Connections
CR8 (Core) blades must be in slots 5&8 ICL used to connect ONLY two DCX chassis together No user ports are required for ICL connections DPS (Dynamic Path Section) is default on the chassis Each ICL cable provides 2 x 64 Gbit/sec Trunk1
23
Footnote 1: Does not require a Trunking license on the switch to trunk the ICL ports.
Revision 1008
2 - 23
ICL Cabling
ICL0 (bottom port) must be connected to ICL1 (top port) on the other DCX ICL1 (top port) must be connected to ICL0 (bottom port) on the other DCX1 Cables can be cross-connected to the other slot. Example: Cables connected to slot 5 on one DCX can be connected to slot 8 on the other DCX ICL Cables are 2 meters in length
24
Footnote 1: Cables and connectors are color coded to avoid misconnection.
Revision 1008
2 - 24
Other ICL Cabling Supported Configurations

All cables crossed Inter cables crossed Top/bottom cables crossed
25
Footnote 1: All ICL cables are 2 meters in length. These work just like other ISL connections. If pulling a cable would result in the failover to other paths (ICL connections or external ISL connections between the two switches). The only differences between ICLs and ISL are that ICLs do not use user ports, the port speed is hard set, and Trunking is automatically enabled.
Revision 1008
2 - 25
ICL Inter Chassis Link

512 Gbit/sec total bandwidth (1 Tbit/sec bidirectional)
2 Core blades x 32 ports per blade x 8 Gbit/sec per port = 512 Gbit/sec total bandwidth between two DCX chassis Two connector ports per blade; Each connector port aggregates 16 ICL ports (128 Gbit/sec bandwidth per cable)
ICL connections can only be used to connect two DCX chassis together ICL license is required on each DCX
26
Revision 1008
2 - 26
DCX Dual Core Fabric

Maximum user port count doubles
768 user ports
512 Gbit/sec total bandwidth between two DCX chassis without the use of any user ports ICLs are oversubscribed:1
4:1 at 256 ports @ 8 Gbits/sec 6:1 at 384 ports @ 8 Gbits/sec
Two domains
27
Footnote 1: For any-to-any topologies where any port connected to one chassis may be connected to any port on the other chassis, ICLs are oversubscribed 4:1 at 8 Gbit/sec between the chassis (Core Blades in Chassis 1 to Core Blades in Chassis 2). If more inter-chassis bandwidth is required, nothing precludes the use of front ports as ISLs in addition to ICLs. When each chassis is fully configured to 384 ports and any port may connect to any other port, then the two chassis are 6:1 oversubscribed (1.5x oversubscribed at each chassis multiplied by 4x oversubscribed across ICLs) at 8 Gbit/sec. ISL oversubscription is represented as a ratio of input ports to output ports.
Revision 1008
2 - 27
ICL Ports
Appear as regular FC ports with certain restrictions Most properties are static. No portcfg allowed on these ports:
Speed is locked at 8 Gbit/sec Trunking enabled Credit sharing is on Support for QoS No long distance support
ICL ports must be reinitialized after the ICL license is applied
28
Here is what happens when you try to set the port speed on an ICL port: DCX10:admin> portcfgspeed 5/1 4 Operation failed - Configuration change is not allowed on ICL ports.
Revision 1008
2 - 28
Brocade 48000 Director

4, 8 and 10 Gbit/sec performance from port cards and CP cards based on the Condor, Condor2 and Egret ASICs 16, 32, 48 ports per port card; up to 384 user ports total
29
The Brocade 48000 director extends the high-availability and performance features introduced in prior Brocade director products. The CP and port cards are based on the Condor ASIC, which provides double the port speed (up to 4 Gbit/sec) and double the port count (up to 48 ports per port card, up to 384 ports per chassis). The Brocade 48000 continues the modular, redundant architecture with multiple field-replaceable CP cards, port cards, power supplies, blower assemblies, and WWN cards. All ports within the chassis are managed as a single domain. The routing between the Brocade 48000 port cards is based on the same non-blocking architecture used in prior Brocade director products. A fully-loaded Brocade 48000 consumes less power than prior fully-loaded Brocade director products. The Brocade 48000 protects your existing investment by using the same power supply, blower assembly, and WWN card as prior Brocade director products.
Revision 1008
2 - 29
Director CP4 Blades

The Brocade 48000 Control Processor (CP) cards perform all motherboard functions:
Runs a real-time Linux kernel Fabric OS runs as an application Connects port card-to-port card data paths Two CP4 cards per chassis (slots 5 and 6 only) Only one CP card required for operation Includes a modem port, serial port, Ethernet port, and an Active CP LED
30
Brocade director CP cards are installed in slot 5 (CP0) and slot 6 (CP1). The Brocade 48000 CP card is also called the CP4. Each CP card includes a single 1 GB Compact Flash card, that stores the Fabric OS firmware and is managed as two equal-sized partitions. There is also 16 MBytes of kernel flash memory (also managed as two equal-sized banks) for user configurations. Each CP card is powered by a CPU (800 MHz IBM Power PC 440GX). The kernel is based on MontaVista, a Linux-based, industry-standard real-time OS that allows adding new features. Fabric OS runs as a layered application on the Linux kernel. Each CP card provides a dial-in modem port (identified as RS-232) for remote management (requires an external modem); a serial port (10101) for local management; and a 10/100 Mbit/sec Ethernet port (with an unique IP address) for network management. The Brocade 48000 CP card consumes 100 watts of power.
Revision 1008
2 - 30
CP Redundancy
The Brocade 48000 CP cards are logically and electrically divided into two completely independent parts:
CP Section: Traditional CP card functions - runs Linux and Fabric OS, controls port cards, monitors all chassis hardware, and Fabric Services Core Section: Condor ASICs that connect data paths between the port cards
A CP Section failure has no effect on the data flowing through the Core Section and a Core Section failure has no effect on CP Section control
31
The CP and Core sections draw power from separate power etches, and each section can fail independent of the other section just as with each half of the Brocade director WWN card. The Brocade 48000 CP card uses Condor ASICs in the Core Section, matching the Condor ASIC deployed on the FC4-16 and FC4-32 port cards.
Revision 1008
2 - 31
DCX
vs.
48000
DCX 12 blade slots 8 port blades which run at 1, 2, 4 and 8 Gbit/sec 10 Gbit/sec port blades 512 Gbit/sec bi-directional bandwidth per 384 user ports (448 total ports)2 slot1 10 blade slots
48000
4 and 8 Gbit/sec port blades which run at 1, 2, 4 and 8 Gbit/sec 10 Gbit/sec port blades 128 Gbit/sec bi-directional bandwidth per slot 384 user ports Core routing / CP functions are on the same blades Single WWN card
Core routing / CP functions are on different blades Dual WWN cards
32
Footnote 1: In the DCX, each slot can handle 32 x 8 Gbits/sec in both directions and still be fully subscribed; where as the 48000 can only handle up to 16 x 4 Gbit/sec in both directions and still be fully subscribed. Therefore, the DCX has 4 times the bandwidth per slot than the 48000. Footnote 2: 384 user ports plus 64 ICL (ISL) ports for a total of 448 ports.
Revision 1008
2 - 32
Chassis: Dimensions / Weight
DCX
Depth (no door) 24.09 in 27.46 in
48000
Depth (with door) 28.82 in 28.82 in Weight (fully loaded) 239 lbs1 219 lbs
Director DCX 48000
Height 24.11 in 24.11 in
Width 17.22 in 17.22 in
33
Footnote 1: Fully loaded: 8 x 48-port blades, 2 Core blades, 2 CP blades, 3 blowers, 4 power supplies, 2 WWN cards and door.
Revision 1008
2 - 33
FC8-16 Blade (DCX and 48000)

16 1/2/4/8 Gbit/sec FC ports
F/FL/E/EX support1 1:1 subscription at all speeds
One Condor2 ASIC per blade LEDs2

2 per blade 1 per port
Port blades are flush with front of chassis
34
Footnote 1: EX_Port support is with the DCX only Footnote 2: Blade LEDs: Per-Blade LEDs
Per-Port LED
Revision 1008
2 - 34

F/FL/E/EX_Port1 support 1:1 subscription at all speeds
Two Condor2 ASICs per blade LEDs2

35
Footnote 1: EX_Port support is in the DCX only Footnote 2: Blade LEDs: Per-Blade LEDs
Per-Port LED
Revision 1008
2 - 35

F/E/EX_Port1 support Worst case 24:16 oversubscription at 8 Gbit/sec
Two Condor2 ASICs per blade LEDs2

36
Footnote 1: EX_Ports only supported in the DCX Footnote 2: Blade LEDs: Per-Blade LEDs
Per-Port LED
Revision 1008
2 - 36
FC4-16 Blade (48000 Only)

The FC4-16 port card provides sixteen 1, 2, or 4 Gbit/sec (auto-sensing) ports per card E/F/FL_Port modes SWL, LWL and ELWL SFPs Based on one Condor ASIC Optical slider and thumbscrew Trunk groups: ports 0-7, 8-15
37
The FC4-16 port card provides sixteen 1, 2, or 4 Gbit/sec Fibre Channel ports that may be connected to servers, storage, or other switches as needed. Port hardware is based on the same ASIC technology used in the Brocade 4100 switch. Port interfaces are compatible with SWL, LWL, and ELWL transceivers. Ports may be F_Port, FL_Port, or E_Port and can be pre-configured, or can selfconfigure. Ports are numbered on the card from 0 to 15, start at the bottom of the card (in the image above, the lower left-hand corner of the blade) and move upwards. On an FC4-16 port card, ports within the port card communicate directly. Communications to other port cards are made through the backplane. The Brocade 48000 optical slider, along with the thumb screw, are enhanced so that is it more difficult to insert and remove the card, as well as power on the card, without the thumbscrew fully in place. The FC4-16 port card consumes 40 watts of power.
Revision 1008
2 - 37

The FC4-32 port card provides thirty-two 1, 2, or 4 Gbit/sec (auto-sensing) ports per card E/F/FL_Port modes SWL, LWL and ELWL SFPs Based on two Condor ASICs Improved optical slider and thumbscrew Trunk groups: ports 0-7, 8-15, 16-23, 24-31
38
The FC4-32 port card provides thirty-two 1, 2, or 4 Gbit/sec Fibre Channel ports that may be connected to servers, storage, or other switches as needed. Port hardware is based on the same ASIC technology used in the Brocade 4100 switch. Port interfaces are compatible with SWL, LWL, and ELWL transceivers. Ports may be F_Port, FL_Port, or E_Port and can be pre-configured, or can selfconfigure. Ports are numbered as follows: Left-hand column: ports are numbered on the card from 0 to 15, start from the bottom of the card (in the image above, the lower-left corner) and move upwards. Right-hand column: ports are numbered on the card from 16 to 31, start from the bottom of the card and move upwards. The FC4-32 port card consumes 50 watts of power.
Revision 1008
2 - 38

The FC4-48 port card provides forty-eight 1, 2, or 4 Gbit/sec (auto-sensing) ports per card Contains two Condor ASICs Trunk groups: ports 0-7, 8-15, 16-23, 24-31, 32-39, 40-48 E/F_Port modes (no FL_Ports)
39
The FC4-48 port blade is similar in architecture to the FC4-32 port blade. It contains two Condor ASICs which interface to a total of 48 ports that are capable of 1/2/4 Gbit/sec speeds. It has many of the same capabilities as the FC4-32 port blade. The FC4-48 port blade has a new type of ejector and ejector handles. When the board is inserted into an empty blade slot of a Brocade 48000, the handles are pushed outward to lock it into the slot. They are pushed inward to eject it from the chassis.
Revision 1008
2 - 39

The FC10-6 director blade provides six 10 Gbit/sec FC ports Utilizes six Egret ASICs 10 Gbit/sec FC ports use a different encoding scheme 64b/66b Trunking not supported FC10-6 ports support only 10 Gbit/sec speeds, and may be attached to other FC10-6 ports as E_Ports only Uses the 10 Gbit/sec transceiver - the XFP Can install up to eight FC10-6 blades per Brocade 48000 Director or DCX Must have 4 power supplies installed1
40
FC10-6 ports may not be connected to 10 Gbit/sec ports in the Brocade Mi10K or other vendors products. The 8b/10b encoding scheme converts 8 bits of user data into 10 bits of data transmitted over the Fibre Channel link. The 64b/66b encoding scheme converts 64 bits of user data into 66 bits of data transmitted over the Fibre Channel link. The difference in encoding schemes means that one 10 Gbit/sec link can carry almost as much data as three 4 Gbit/sec links: 10 Gbit/sec link: Bandwidth = 10 Gbit/sec * (64/66) = 9.70 Gbit/sec Three 4 Gbit/sec links: Bandwidth = 3 * 4 Gbit/sec * (8/10) = 9.60 Gbit/sec The XFP (10 Gbit/sec Small Form Factor Pluggable) is a hot-swappable optical transceiver used by several 10 Gbit/sec protocols, including Fibre Channel. The Brocade FC10-6 can be installed in a Brocade 48000 director with the following blades: FC4-16, FC4-32, FC4-48, FR4-18i, FC4-16IP, and FA4-18. Footnote 1: The power supply requirement is for high-availability purposes, and is not enforced by Fabric OS. Your switch provider may require four power supplies; check switch provider supportability requirements.
Revision 1008
2 - 40
FR4-18i Blade (DCX and 48000)

The FR4-18i blade provides switch, routing, and FCIP functions 16 FC ports for devices, switching, and FC routing (1/2/4 Gbit/sec; F, FL, E, and EX_Ports) Two Gigabit Ethernet ports for FCIP connectivity and routing (VE and VEX_Ports) A maximum of 4 FR4-18i blades per Brocade 48000 Requires 4 power supplies for HA
41
The FR4-18i blade provides the following features: 16 Fibre Channel ports (based on the Condor ASIC) that support the Fibre Channel Routing Services 2 GbE ports supporting the FCIP and Fibre Channel Routing Services with link speeds up to 1 Gbit/sec: Each GbE port can support up to 8 FCIP tunnels. Each FCIP tunnel is represented and managed as a virtual Fibre Channel E_Port (VE_Port). Each FCIP tunnel can support data compression and traffic shaping, with a committed bandwidth. Fibre Channel Routing Services can be used over each FCIP tunnel (VEX_Ports). You can learn more about the Brocade FR4-18i Blade by viewing the AFS175 webbased training class from Brocade.
Revision 1008
2 - 41
FC4-16IP Blade (48000 Only)

The FC4-16IP blade provides a gateway between iSCSI hosts & Fibre Channel fabrics Supports iSCSI initiators to Fibre Channel Targets 8 1000Base-T (GbE) Ethernet ports, RJ45 Copper 8 1/2/4 Gbit/sec FC ports only (no FCR), SFP Media 64 iSCSI initiators per GbE port, 512 initiators per blade Up to 4 intelligent AP blades per Brocade 48000
Maximum of four FC4-16IP blades per chassis
RJ45 GE
Microsoft iSCSI 2.0 driver Native OS iSCSI drivers: Linux, Solaris 9, HP-UX 11i, IBM AIX 5.x Requires 4 power supplies for HA
1/2/4G FC
GE7 GE6 GE5 GE4 GE3 GE2 GE1 GE0 7 6 5 4 3 2 1 0
42
The FC4-16IP blade provides a director-based iSCSI gateway from the Fibre Channel SAN to iSCSI initiators. Both of the intelligent blades (FR4-18i and FC4-16IP) require 4 power supplies for High Availability.
Revision 1008
2 - 42
Director Blade Support Matrix

Blade 48000
DCX
FC4-16 FC4-32 FC4-48 FC10-6 10G FC FR4-18i FCIP FC4-16IP iSCSI FA4-18 Applications CP4 FC8-16 FC8-32 FC8-48 CP8 CR8 (Core)
43
Revision 1008
2 - 43
FC Port Blade Frame Flow

If the SID and DID are on the same Condor2 ASIC, local switching is used - just as with the Condor ASIC
The frame does not have to go through the CR8 (Core) blade
Frames going from one Condor2 to another Condor2 on the same port blade must go through the CR8 (Core) blade
No direct connections between the Condor2 ASICs on the same port blade - same as on the Brocade 48000
Condor2 ASIC latency is 700ns per ASIC (2.1s through the switch) 1
44
Footnote 1: Through the switch means the frame comes in on one blade goes through core blade, and out another blade on the switch. The Condor ASIC latency is about 800ns per ASIC and 2.4s through the switch.
Revision 1008
2 - 44
DCX Architecture 16-Port Blade

CR8 Blade Condor2 ICL1 Condor2 Condor2 Condor2 ICL0 Condor2
Each line is a 8 x 8 Gbit/sec Trunk
FC8-16 Blade
FC front end ports 0-15
CR8 Blade Condor2
FC8-16 Blade
ICL1 Condor2 Condor2 ICL0 Condor2 Condor2
FC front end ports 0-15
45
Each Condor2 ASIC has 40 ports; on the 16-port blade, only 32 are used: 16 external and 16 Internal. If the initiator and target are on the same ASIC, the frame would not go through the core and local switching would be used. If the frame coming out of the core blade is going out the ICL cable, each Condor2 ASIC on the core has 8 x 8 Gbit/sec Trunk going to the other DCX. If the frame coming out of the core blade is going to a 16-port blade, then each Condor2 ASIC on each core blade would have 2 x 8 Gbit/sec Trunk going to the Condor2 ASIC on the 16-port blade. If the frame coming out of the core blade is going to a 32 or 48-port blade, then each Condor2 ASIC on each core blade would have 2 x 8 Gbit/sec Trunk going to both of the Condor2 ASICs on the 32 or 48-port blade.
Revision 1008
2 - 45

CR8 Blade Condor2 ICL1 Condor2 Condor2 ICL0 Condor2
FC8-32 Blade
Condor2
FC front end ports 8-15, 24-31
Condor2
CR8 Blade Condor2
FC8-32 Blade FC front end ports 8-15, 24-31
ICL1 Condor2 Condor2 ICL0 Condor2
Condor2
Condor2
46
Each Condor2 ASIC has 40 ports; on the 32-port blade, only 32 are used: 16 external and 16 Internal. If the initiator and target are on the same ASIC, the frame would not go through the core and local switching would be used. If the frame coming out of the core blade is going out the ICL cable, each Condor2 ASIC on the core has 8 x 8 Gbit/sec Trunk going to the other DCX. If the frame coming out of the core blade is going to a 16-port blade, then each Condor2 ASIC on each core blade would have 2 x 8 Gbit/sec Trunk going to the Condor2 ASIC on the 16-port blade. If the frame coming out of the core blade is going to a 32 or 48-port blade, then each Condor2 ASIC on each core blade would have 2 x 8 Gbit/sec Trunk going to both of the Condor2 ASICs on the 32 or 48-port blade.
Revision 1008
2 - 46

CR8 Blade Condor2 ICL1 Condor2 Condor2 ICL0 Condor2
FC8-48 Blade
Condor2
Condor2
CR8 Blade Condor2
FC8-48 Blade FC front end ports 8-23, 40-47
ICL1 Condor2 Condor2 ICL0 Condor2
Condor2
Condor2
47
Each Condor2 ASIC has 40 ports; on the 48-port blade all 40 ports are used: 24 external and 16 Internal. This is why the 48-port blade is 24 to 16 oversubscribed. If the initiator and target are on the same ASIC, the frame would not go through the core and local switching would be used. If the frame coming out of the core blade is going out the ICL cable, each Condor2 ASIC on the core has 8 x 8 Gbit/sec Trunk going to the other DCX. If the frame coming out of the core blade is going to a 16-port blade, then each Condor2 ASIC on each core blade would have 2 x 8 Gbit/sec Trunk going to the Condor2 ASIC on the 16-port blade. If the frame coming out of the core blade is going to a 32 or 48-port blade, then each Condor2 ASIC on each core blade would have 2 x 8 Gbit/sec Trunk going to both of the Condor2 ASICs on the 32 or 48-port blade.
Revision 1008
2 - 47
Mi10K Director
Provides 1, 2, 4, and 10 Gbit/sec 256 user ports Create up to 4 dynamic partitions in a single chassis Dynamic creation of up to 8 individual Virtual SANs, a maximum of 4 per partition
48
Revision 1008
2 - 48
The Mi10K 4Gbps Blade (LMQ)

Operates at 4 Gbit/sec, auto-negotiates at 1, 2, and 4 Gbit/sec 32 ports per line module Coexists with all existing Mi10K LIMs
49
LMQ stands for Line Module Quad Speed.
Revision 1008
2 - 49
Line Module Cards (LIMs)

The LIM card faceplate contains:
Interfaces for attaching up to four optical paddles A bi-color (green/amber) LED 1, 2 and 10 Gbit/sec
50
Revision 1008
2 - 50
Optical Paddles
51
Revision 1008
2 - 51
M6140 Director
Provides 1, 2, 4, and 10 Gbit/sec Chassis provides slots for up to 35 cards XPM card has one port QPM Card has four ports 140 maximum ports
52
Revision 1008
2 - 52
M6140 Control Processor Card (CTP)

Each CTP card provides:
System services processor (SSP) subsystem
Runs director applications and the underlying

operating system
Communicates with director ports Controls the RS-232 maintenance port and
10/100Mbps Ethernet port
Embedded Port (EP) subsystem
Provides Class F and well known frames
53
The director is delivered with two CTP cards. The active CTP card initializes and configures the director after power on and contains the microprocessor and associated logic that coordinate director operation. The CTP card provides an initial machine load (IML) button and a RESET button (recessed) on the faceplate. When the IML button is pressed, held for three seconds, and released, the director performs an IML that reloads the firmware from FLASH memory. This operation is not disruptive to Fibre Channel traffic. When the RESET button is pressed and held for three seconds, the director performs a reset. A reset is disruptive and resets the: Microprocessor and functional logic for the CTP card and reloads the firmware from FLASH memory. Ethernet LAN interface, causing the connection to the management server to drop momentarily until the connection automatically recovers. Ports, causing all Fibre Channel connections to drop momentarily until the connections automatically recover. This causes attached devices to log out and log back in, therefore data frames lost during director reset must be retransmitted. A reset should only be performed if a CTP card failure is indicated. As a precaution, the RESET button is flush mounted to protect against inadvertent activation. Each CTP card also provides a 10/100Mbps RJ-45 twisted pair connector on the faceplate that attaches to an Ethernet local area network (LAN) to communicate with the management server or a Simple Network Management Protocol (SNMP) management station. In addition, a CTP card provides nonvolatile memory for storing firmware, director configuration information, persistent operating parameters, and memory dump files. Director firmware is upgraded concurrently (without disrupting operation). The backup CTP card takes over operation if the active card fails. Failover from a faulty card to the backup card is transparent to attached devices. Each card faceplate contains a green LED that illuminates if the card is operational and active, and an amber LED that illuminates if the card fails. Both LEDs are extinguished on an operational backup card. The amber LED blinks if FRU beaconing is enabled.
Revision 1008 2 - 53
Brocade 7500 FCR/FCIP

The Brocade 7500 is a 1U, stand-alone switch version of the Brocade FR4-18i blade:
Sixteen 4 Gbit/sec FC ports - Condor-based; F/FL/E/EX_Ports Two 1 Gbit/sec GbE ports - FCIP connections with multiple tunnels, data compression, traffic shaping, and routing Port speeds of 1, 2, or 4 Gbit/sec ISL Trunking, Extended Fabrics
Two 8-port groups (ports 0-7 and ports 8-15) each with 472 buffer credits
available for extended distances
54
The Brocade 7500 is designed for stacked deployments A stand-alone solution for enterprise and mid-range 16 port Fibre Channel Switch/Router 2 port Fibre Channel over IP Distance Extension 16 Front Panel 1, 2, or 4 Gbit/sec FC ports Fibre Channel Routing Services for all ports including load balancing and long distance support 16 4 Gbit/sec Internal Fibre Channel Router 2 Front Panel 1 Gbit/sec APs Ethernet Ports Compression Hardware Assist IPSEC Hardware Assist 8 FCIP tunnels per Port Each FCIP Tunnel is represented and managed as a Virtual FC E_Port Support for Fibre Channel Routing Services over FCIP link Fabric OS v5.1.0+ required to operate blade Fixed configuration Non-disruptive firmware download (except GbE IP-ports where there can be up to a 30sec outage) 1U by 24 deep Redundant fans, redundant power supplies
Revision 1008
2 - 54
Brocade 7500E FCR/FCIP

Same chassis as 7500 Economical solution for connecting remote sites using FCIP 4 ports:
2 Fibre Channel (E, F, FL, EX) ports 2 Gigabit Ethernet (VE, VEX) ports
Software license upgrade available to activate 14 additional Fibre Channel ports
(2) 4G FC ports
Expansion FC ports
(2) GigE ports
55
You can order and install a software upgrade license. After the Brocade 7500E firmware detects the presence of the upgrade license, the Brocade 7500E will enable functions equivalent to the Brocade 7500 as well as the high-performance extension license, activation of all 16 Fibre Channel ports, and full line-rate performance. The software upgrade license is a bundled package that includes: Hardware-based encryption Read and Write Tape pipelining Fibre Channel-based extension with Fast Write FICON (disk and tape) over metro distances Local switching Fibre Channel Routing between fabrics Call Home feature Full line rate speeds across IP WAN ports Activation of 14 additional Fibre Channel ports Seven additional tunnels per Gigabit Ethernet port
Revision 1008
2 - 55
7500 and 7500E Feature Matrix

Capability Redundant Power Supplies and Fans Hardware-based Encryption FC Tape Pipelining (over FCIP) FICON (Disk and Tape) FC-based Extension with FastWrite Qualified for local FC switching FC Routing between fabrics Call Home Fabric Isolation Per GE Port Rate Limiting (Throughput Throttling) # of FC Ports Connections or Tunnels (Remote Sites) Hardware-based Compression Open Systems Extension w/ FastWrite over FCIP Storage-Optimized TCP
* 7500 with High-performance Extension over FCIP license ** Available with the 7500 Brocade Accelerator for FICON
7500 * * -** * Up to 1G 16 8 per port * *
7500E -** Up to 50Mbps per GE port 2 1 per port
7500E Upgrade license -** Up to 1G Additional 14 8 per port -
56
Revision 1008
2 - 56
Brocade USD-X
Supports Fibre Channel, FICON, ESCON, Bus-and-Tag, or mixed environment systems and a variety of network interfaces, including:
OC-3 ATM 10/100 and Gigabit Ethernet
Delivers industry-leading throughput over thousands of miles
57
Revision 1008
2 - 57
SFP and XFP Transceivers
XFP SFPs
58
Revision 1008
2 - 58
SFP Types
Short Wavelength (SWL) Long Wavelength (LWL) Extended Long Wavelength (ELWL) Wave Division Multiplexing (WDM) 8G SFPs support 2, 4 and 8 Gbit/sec speeds 4G SFPs support 1, 2 and 4 Gbit/sec speeds
59
SWL Fiber Optic SFP Module - Based on short-wavelength 850 nm lasers supporting 2 and 4 Gbit/sec link speeds. This SFP module supports 50/62.5, multimode fiber optic cables, 50 cable is recommended. Note: Short-wavelength SPF modules may be labeled MM (for multimode), SW, or SWL (for shortwavelength laser). LWL Fiber Optic SFP Module Based on long-wavelength 1310nm lasers supporting 2 and 4 Gbit/sec link speeds. This SFP module supports 9 single-mode fiber optic cables. ELWL Fiber Optic SFP Module Based on long-wavelength 1550nm lasers supporting 2 Gbit/sec link speeds. This SFP module supports 9 single-mode fiber optic cables and distances over 50 km. WDM Fiber Optic SFP Module Based on various wavelength lasers ranging from 1470nm to 1610nm supporting 2 Gbit/sec link speeds. Requires external xWDM mux/demux module to transmit multiple wavelengths simultaneously over a single fiber optic cable. WDM devices support longdistance connectivity to 100 km and beyond.
TST_ST01_B20_1:admin> sfpshow Area Area 0: id (sw) Vendor: FINISAR CORP. 1: id (sw) Vendor: FINISAR CORP. Serial No: U85066V Serial No: H11XSF9
. . . [truncated] TST_ST01_B20_1:admin> sfpshow 0 Identifier: Connector: Encoding: Baud Rate: Length 9u: Length 9u: Length 50u: Length Cu: Vendor OUI: Vendor PN: Vendor Rev: Wavelength: Options: BR Max: BR Min: Serial No: Date Code: Current: Voltage: RX Power: TX Power: 3 7 1 42 0 0 15 0 SFP LC 8B10B (units 100 megabaud) (units km) (units 100 meters) (units 10 meters) (units 10 meters) (units 1 meter)
Transceiver: 150c402001000000 100,200,400_MB/s M5,M6 sw Inter_dist
Length 62.5u:7
Vendor Name: FINISAR CORP. 00:90:65 FTRJ8524P2BNV A 850 0 0 U85066V 050729 6.678 mAmps 3319.4 mVolts 408.0 uWatts 311.9 uWatts (units nm) 0032 Loss_of_Sig,Tx_Disable
Temperature: 28 Centigrade
TST_ST01_B20_1:admin>
Revision 1008
2 - 59
Optical Transceiver Capability

All Condor2 blades and switches require Brocade-branded optics 8 Gbit/sec Brocade-branded SFP distances: Shortwave/Multimode:
62.5m 50m 50m 21 m (OM1) 50 m (OM2) 150 m (OM3)
60
If a non-branded SFP is used on blades running the Condor2 ASIC, the port will not come online. In Fabric OS v6.0, 8 Gbit/sec LW is not supported at 8 Gbit/sec. For long distance, must use 4 Gbit/sec.
Revision 1008
2 - 60
Summary
Brocade 4 Gbit/sec products use the GoldenEye and Condor ASICs Brocade 8 Gbit/sec products are built around the GoldenEye2 and Condor2 ASICs Brocade enterprise class products include the M6140, MI10K, and B48000 directors and the DCX Backbone The B48000 CP Cards have both CP and Core routing functionality, whereas the DCX separates the CP and Core functionality on Core and CP Blades Two DCX Backbones can be connected together through ICL cables
61
Revision 1008
2 - 61

End of Instructor-Led Module 2 Hardware Product Features
Revision 1008
Revision 1008
2 - 62
The following table lists the switch type assigned to each switch and is displayed in the switchshow command.
Brocade Switch
1000 2800 2400 20x0 22x0 3800 12000 3900 3200 24000 3850 3250 4100 200E 48000 4900 7500 7600 5000 DCX 5300 5100 300
Switch Type
1 2 3 4 5 9 10 12 16 21 26 27 32 34 42 44 46 55 58 62 64 66 71
ASIC
Stitch/Flanel LOOM LOOM LOOM LOOM BLOOM BLOOM BLOOM BLOOM BLOOMII BLOOMII BLOOMII Condor GoldenEye Condor Condor Condor Condor Condor Condor2 GoldenEye2 Condor2 GoldenEye2
Revision 1008
2 - 63
Revision 1008
2 - 64
Fibre Channel Theory

Instructor-Led Module 3 Fibre Channel Theory
Revision 1008
Revision 1008
3-1
Objectives
After completing this module and associated lab, attendees will be able to:
Discuss Fibre Channel networking model Identify different Classes of Service Describe Fibre Channel frame format Understand format of a WWN List the steps of port initialization Identify port types and node types Identify Fabric Services well-known addresses Understand Fabric addressing modes Discuss NPIV Support
Revision 1008
3-2
Fibre Channel Networking Model
The FC-0 and FC-1 layers specify physical and data link functions needed to physically send data from one port to another. FC-0 specifications include information about feeds and speeds. FC-1 layer contains specifications for 1, 2, 4 and 8 Gbit/sec 8b/10b encoding, ordered set and link control communication functions. 10 Gbit/sec communication uses 64b/66b encoding. FC-2 specifies content and structure of information along with how to control and manage information delivery. This layer contains basic rules needed for sending data across the network. This includes: (1) how to divide the data into frames, (2) how much data should be sent at one time before sending more (flow control), and (3) where the frame should go. It also includes Classes of Services, which define different implementations that can be selected depending on the application. FC-3 defines advanced features such as striping (to transmit one data unit across multiple links) and multicast (to transmit a single transmission to multiple destinations) and hunt group (mapping multiple ports to a single node). While the FC-2 level concerns itself with the definition of functions with a single port, the FC-3 level deals with functions that span multiple ports. FC-4 provides mapping of Fibre Channel capabilities to pre-existing protocols, such as IP, SCSI, or ATM, etc.
Revision 1008
3-3
Fibre Channel Classes of Service

Class 1
Description
Dedicated connection, allocates full bandwidth between ports Connectionless, switch-to-switch communication between ports, transfers frames Connectionless, switch-to-switch communication between ports, transfers frames Dedicated connection, allocates requested amount of bandwidth between ports, uses virtual circuits Dedicated connection for multicast service Switch to switch communication
ACK
Yes
Brocade Supported No
Yes
Yes
No
Yes
4 6 F
Yes Yes Yes
No No Yes
4
Different types of information have different delivery requirements. In order to accommodate the different delivery requirements, Fibre Channel associates a set of delivery characteristics into what is called a Class of Service. The characteristics relate to the type of connection between the ports, confirmation of delivery, flow control mechanisms and how errors are handled. Class-1 is a connection-oriented circuit that dedicates 100% of the bandwidth between the sending and receiving ports. It also provides for a confirmation of delivery (ACK). Class-2 is a connectionless class with an acknowledgement (confirmation of delivery). No bandwidth is allocated or guaranteed. IP uses this class. Uses both Buffer-to-Buffer (BB) credits and End-to-End (EE) credits for flow control. Class-3 is a connectionless class without an acknowledgement (confirmation of delivery). No bandwidth is allocated or guaranteed. FCP uses this class. Uses Buffer-to-Buffer (BB) credits for flow control, does not use End-to-End (EE) credits. Class-4 is a connection-oriented class that uses virtual circuits and confirmation of delivery. Unlike Class-1 that reserves the entire bandwidth, Class-4 can allocate a requested amount of bandwidth. Class-6 is a variation of Class-1 that provides a one-to-many multicast service with a confirmation of delivery. Class-F is a connectionless class with acknowledgements (confirmation of delivery). between two switches. Note: Brocade supports Class-2, Class-3, and Class-F only.
Revision 1008
3-4
Fibre Channel Frame Format

Frame
S O F 4
Header
Payload
C E R O C F 4 4
24
Up to 2112
2148 Bytes
Word Bits 31-24 R_CTL CS_CTL TYPE SEQ_ID OX_ID Parameters Payload DF_CTL Bits 23-16 Bits 15-8 D_ID S_ID F_CTL SEQ_CNT RX_ID Bits 7-0
H E A D E R Payload
0 1 2 3 4 5 6-n
A frame has a header and may have a payload. The header contains control and addressing information associated with the frame. The payload contains the information being transported by the frame on behalf of the higher level service or FC-4 upper level protocol. The Fibre Channel standards allow bytes from the payload to be used for optional headers. There are many different payload formats, based on the protocol. The TYPE field (Word 2, bits 31- 24) specifies which format to use. The Routing Control INFO bits (bits 27-24) determine how to interpret the payload. Field Definitions: Routing Control (R_CTL): first 8 bits of the header. They define the type of frame and its content or function. The first 4 bits (Bits 31-28) identify the frame type. The second four bits (Bits 27-24) define the contents of the frame or identify the function of the frame (22 = ELS, 02 = CT command). Destination_ID (D_ID): 24-bit Port Identifier (PID) of the recipient. It could also be a wellknown address such as the Name Server FFFFFC. Class Specific Control Field (CS_CTL): control necessary for the different classes of service. This field is always zero for classes 2 and 3 per the standards. Classes 1 and 4 use it. Source_ID (S_ID): 24-bit Port identifier (PID) of the source. It could be a well-known address. Type: identifies the protocol of the frame content for Data Frames (i.e FC_CT, FCP, IPFC). Frame Control (F_CTL): contains miscellaneous control information regarding the frame such as who owns initiative, first frame of the exchange, last frame of the exchange, etc. Sequence ID (SEQ_ID): used to identify and track all of the frames within a sequence between a source and destination port pair. Data Field Control (DF_CTL): indicates if any optional headers are present at the beginning of the data field of the frame. Optional headers are used for information that may be required by some applications or protocol mappings. Sequence Count (SEQ_CNT): used to indicate the sequential order of frame transmission within a sequence or multiple consecutive sequences within the same exchange. This is a counter that increments as sequence of frames is transmitted. Originator_ID (OX_ID): Exchange ID assigned by the originator port. Responder_ID (RX_ID): Exchange ID assigned by the responder to the exchange. Data Field/Payload: The maximum size is 2112 bytes. Revision 1008 3-5
Node WWN Name Format NWWN

Single hexadecimal digit selected by the vendor (Brocade uses a 1)
Node
10:00:00:05:1e:00:60:02
FC Standard Reserved OUI1 Assigned by IEEE Assigned by the vendor
This format applies to: Switches Storage HBAs
Footnote1: OUI is the Organization Unique Identifier. Based on the IEEE Standard format, a Brocade Node WWN is: 10:00:00:05:1e:xx:xx:xx. Where: The first 2 bytes are always 10:00 The next 3 bytes are vendor-specific. Brocade has the following OUI codes registered: 00:00:88 00:01:0F 00:05:1E 00:05:33 00:14:c9 00:60:69 00:60:DF 08:00:88 The last 3 bytes are derived from the Brocade main board serial number. The 3-byte company ID found in the 64-bit IEEE Standard format WWN can be searched at: http://standards.ieee.org/regauth/oui/oui.txt
Revision 1008
3-6
Port WWN Name Format PWWN

Single hexadecimal digit selected by the vendor (Brocade uses a 2)
Port 9
20:09:00:05:1e:00:60:02
Set by the vendor to uniquely identify a port on a device or switch OUI Assigned by IEEE Assigned by the vendor
This format applies to: Switches Storage HBAs

Port World Wide Name: 2p:pp:00:05:1e:xx:xx:xx The three nibbles (p:pp) are used by Brocade to show the switch port number. The 0:09 in the example above indicates this is port 9 on the switch.
Revision 1008
3-7
NWWN and PWWN Example

NWWN 20:00:00:05:1e:51:39:7d
PWWN 10:00:00:05:1e:51:39:7d
PWWN (Port 1) 21:00:00:77:77:00:70:c0
PWWN (Port 2) 22:00:00:77:77:00:70:c0
Single Drive NWWN 20:00:00:77:77:00:70:c0

The Node WWN (NWWN) is like an umbrella that the port WWNs sit under like spokes. Different vendors do different things with the WWNs. The Port WWN (PWWN) sits under the Node WWN. The first two bytes are assigned by a vendor. Example: Seagate uses Node and Port WWNs in a different manner than Brocade. Seagate uses a format 2 (2x:) for both the Node and Port WWNs. In the PWWN, the port is identified in the second nibble of the first byte. The zoning module will show how the NWWN and PWWN are used.
Revision 1008
3-8
Port Types
Device Ports (Nx_Ports)
N_Port NL_Port U_Port FL_Port G_Port F_Port E_Port - Node Port, a Fabric device directly attached - Node Loop Port, a device attached to a loop - Universal Port, a port waiting to become another port type - Fabric Loop Port, a port to which a loop attaches - Generic Port, a port waiting to be an F_Port or E_Port - Fabric Port, a port to which an N_Port attaches - Expansion Port, a port used for inter-switch links (ISLs)
- Virtual E_Port (used in FCIP fabrics) - A type of E_Port used to connect to a FC Router fabric - VEX_Ports are no different from EX_Ports, except underlying transport is IP rather than FC
Switch Ports
VE_Port EX_Port VEX_Port
For a swtich port that goes through port initialization, it will arrive at an ending status of F_Port, FL_Port or E_Port.
Revision 1008
3-9
Fabric Initialization Process

U_Port
(State 1)
no
y/n
yes
Is something plugged into the port? (Transition 1)
FL_Port
(State 2)
yes
y/n
no
Do you want to talk loop? (Transition 2) Waiting for attached device to talk to me (State 3)
G_Port
F_Port
(State 5)
Fabric device
Is the attached device a fabric device or another switch? Switch (Transition 3)
E_Port
(State 4)
10
A Universal Port (U_Port) is the initial state of a port. (State 1) Is something connected (sending a light/electrical signal) to the port? If yes, continue. (Transition 1) U_Port starts mode detection process by transmitting at least 12 LIP(F7) Primitive Sequences. (Transition 2) If at least 3 consecutive LIP Primitive Sequences are received, then the port enters OPEN_INIT state and attempts FC-AL loop initialization. (State 2) If LIP Primitive Sequences are not received, the U_Port attempts OLD_PORT initialization by taking the link down then transmitting NOS primitives. If Link Initialization Protocol fails after 1 retry or LIP received after 1 second, go to FC-AL initialization. (Transition 2) When operating in the FL_Port mode, a U_Port will try the loop initialization procedure three times. If these fail, the port will be marked as faulty. To ensure N_Port, reinitialize the port and the switch port will cut the laser forcing a loss of signal state for at least 20 s. Then the switch port will bring back the laser and issue NOSs. (Transition 2) U_Port will attempt the OLD_PORT initialization (Link Initialization Protocol for point to point) by taking the link down and then transmitting NOS Primitive Sequence if LIP timeout or any of the loop initialization phases timeout or only one non-zero AL_PA is claimed in Loop Initialization Sequences (LOOP_EMPTY=false) or no non-zero AL_PAs are claimed (LOOP_EMPTY=true). If the ACTIVE state is reached, the port will operate in the G_Port mode. (State 3) The normal E_Port or F_Port mode detection procedure follows. (Transition 3) If ELP succeeds, the U_Port operates in the E_Port mode. (State 4) If a valid FLOGI is received, the U_Port becomes an F_Port. (State 5) If self loopback detected after ELP exchanges and LOOP_EMPTY = false, port exits G_Port and reinitializes as FL_Port. (State 2) Note: The firmware will automatically attempt to reinitialize a faulty port every two seconds.
Revision 1008
3 - 10
Node Types and Port Types
11
Switch to Switch (ISL) connections use E_Ports. Nodes that attach to the fabric can either be an N (Node) or NL (Node Loop). Public NL nodes can communicate with any member of the same loop and have the ability to send a frame to the fabric. Fabric Nodes (N) can communicate with any other Fabric Node and can communicate with private and public NL nodes on a loop. Switched Fabrics: An extensive storage network in which a large number of servers and storage systems are connected using Fibre Channel switches.
Revision 1008
3 - 11
Fabric Generic Services

Before we can talk about the login process, we need to understand Generic Services
Services used to manage a Fibre Channel network such as Fabric port logins, Name Server registration, etc. Usually only found in the switched fabric topology Each generic service is assigned a specific address referred to as its Well-Known Address (FFFFFx)
FFFFFx
FFFFFx
FFFFFx
12
In addition to specifying node port and topology behavior, Fibre Channel defines several generic services that are used to manage a Fibre Channel network. The specific address assigned to each generic service is of the format 0xFFFFFx. The next slide identifies the services provided and the Well-Known Address for each.
Revision 1008
3 - 12
Well-Known Addresses
Address FFFFFC FFFFFD FFFFFE
Description Directory Server (Name Server) Fabric Controller F_Port (Fabric Login) Server
13
Every switch has reserved 24-bit addresses known as Well Known Addresses. The services residing at these addresses provide a service to either nodes or management applications in the fabric. FFFFF6 Clock Synchronization Server: Clock Synchronization over Fibre Channel is attained through a Clock Synchronization Server that contains a reference clock. The Server synchronizes clients clocks to the reference clock on a periodic basis, using either Primitive Signals or ELS frames. FFFFF7 Security Server: The security-key distribution service offers a mechanism for the secure distribution of secret encryption keys. FFFFF8 Alias Server: The Alias Server manages the registration and deregistration of Alias IDs for both Hunt Groups and Multicast Groups. The Alias Server is not involved in the routing of frames for any Group. FFFFFA Management Server: The Management server provides a single point for managing the fabric. FFFFFB Time Server: The time server sends to the member switches in the fabric the time on either the principal switch or the Primary FCS switch. FFFFFC Directory(Name Server): The directory server/name server is where fabric/public nodes register themselves and query to discover other devices in the fabric. FFFFFD Fabric Controller: The fabric controller provides state change notifications to registered nodes when a change in the fabric topology occurs. FFFFFE F_Port(Fabric Server Login): Before a fabric node can communicate with services on the switch or other nodes in the fabric, an address is assigned by the fabric login server. Fabric addresses assigned to nodes are 3 bytes long and are a combination of the domain ID plus the port area number of the port the node is attached to. FFFFFF Broadcast Server: When a frame is transmitted to this address, the frame is broadcast to all operational N and NL ports.
Revision 1008
3 - 13
Fibre Channel Network Addressing

Each switch is responsible for assigning unique addresses Addresses are 24 bits:
Domain ID Area ID Node Address (8 bits) (8 bits) (8 bits) 0x01 - 0xEF 0x00 - 0xFF 00 or AL_PA
Box 1
Address types:
Fabric Public loop / NPIV Shared Area
Domain ID (8 Bits)
DD AA XX1 DD AA PP DD AA 80
Area ID (8 Bits) 24-Bit Address Node Address (8 Bits)
14
When a node attaches to the fabric, it must receive a unique 24-bit address. The network address is a three-byte address based upon the Domain ID, the Area ID and, if a loop device, its AL_PA. This address is the source address and is used for routing data thru the fabric from one device to another. Footnote1: XX will be 00 for Fabric OS switches and 13 for M-EOS switches. Fabric-attached devices use an address format of DD AA 00. This is the address of any Fabric-attached device that has logged into the fabric as point-to-point. Public Loop attached devices use an address format of DD AA PP. The DD AA bytes of the address come from the fabric login process and the PP byte is assigned during FC_AL initialization. NPIV attached devices use an address format of DD AA PP. The DD AA bytes of the address come from the fabric login process and the PP byte is assigned during Login process. More information on NPIV at the end of this module.
Revision 1008
3 - 14
Fabric Login at FFFFFE

A node (device1) performs a fabric login (FLOGI) request to join a fabric and receive its 24-bit address
FLOGI
FLOGI
15
Footnote 1: A device can be an initiator or a target. A device does a Fabric Login (FLOGI) to determine if a fabric is present and, if so, exchange service parameters with the fabric. A successful Fabric Login sends back the 24-bit address for the device in the fabric. The device must do a FLOGI before communicating with other devices in the fabric. Note: Since the device does not know its 24-bit address until after it does the Fabric Login, the SID (Source ID) in the frame header making the FLOGI request will be zeros (0x000000). Buffer-to-Buffer credits for the device and switch ports are exchanged in the FLOGI -> Accept commands. More on this later in this module.
Revision 1008
3 - 15
Name Server at FFFFFC
PLOGI ACC PLOGI
16
Each Brocade switch contains a Name Server that maintains local information. The Name Server provides the local devices with access to the Name Service. The Name Server runs a low-level task that communicates with each switch. The local information from each switch is shared with one another, and kept in the Name Server cache. When a new device gets added to a switch, that information is immediately propagated out to all switches in the Fabric. This behind-the-scenes communication is transparent to the local device. Name Server characteristics: No single point of failure Transparent distribution Server-to-server protocol (based on FC-CT) The Name Server, specified in the Fibre Channel Standard, is responsible for directory information about Fabric-connected devices. The Name Server maintains name and address information about Fabric-connected devices in a complex, diverse environment supported by interconnected Brocade switches. Brocade adds value to the Name Service in two ways: 1.By distributing the Name Server throughout the Fabric. 2.By providing automatic registration of essential device information on behalf of the devices, as well as deregistration. The Name Server functions like a telephone directory. The Name Server has no responsibility for the process of routing data among devices. During initialization, the Fabric Login allows the Fabric to discover devices.
FFFFFC Port and Node Attributes

The Name Server database contains two record types:
Port Attributes
Port Identifier (Native port address ID) Port Name (Port World Wide Name) Class of Service (2, 3) FC-4 Types (SCSI, IP) Port Type (N, NL) Device Type (Initiator or Target) Symbolic Port Name (free-form information)
Node Attributes
Node Name (Node World Wide Name) Fibre Channel IP Address Symbolic Node Name (free-form information)
17
The Symbolic Port Name and Symbolic Node Name are free-form variables and are not restricted by the Name Service. If no value is registered by the Nx_Port, then the Symbolic Port Name and Symbolic Node name default to a null value. Port Identifier - the 24-bit address assigned by the switch Fabric FC-4 Types - Most common are SCSI (8) and IP (5) Port Type - N or NL Symbolic Port/Node Name - a 256 character field that is used by the vendor of the device.
Revision 1008
3 - 17
Brocade B-Series Name Server Commands

nsshow
Displays detailed information about devices connected to this switch (Local Name Server)
nscamshow
Displays detailed information about devices connected to other switches in the fabric (Remote Name Servers)
nsallshow
Displays the 24-bit address of all devices in the fabric
18
There is no single command to display the detailed Name Server information for all devices in a multiple switch fabric. Use the nsshow command to display information about the devices connected to the switch where this command is issued. Use nscamshow to display information about devices connected to the remaining switches in the fabric. Use nsallshow to get a list of all the online node ports in the fabric. Note: Each switch uses a cache to store the information about the devices connected to the remaining switches in the fabric. It does not have to retrieve this information from the other switches when the command is issued. The nscam stands for Name Server Cache Manager.
Revision 1008
3 - 18
nsshow [-rt]
sw2:admin> nsshow -rt { Type Pid N COS PortName NodeName SCR 020500; FC4s: FCP PortSymb: [46] "BRE042 A.2 L3-25014-01B FW:01.03.19 Port 0 Fabric Port Name: 20:05:00:05:1e:08:52:cf Permanent Port Name: 10:00:00:05:1e:42:88:2e Device type: Physical Initiator Port Index: 5 Share Area: No Device Shared in Other AD: No Redirect: No <truncated output> " 3;10:00:00:05:1e:42:88:2e;20:00:00:05:1e:42:88:2e; 3
-r shows SCR value -t shows Device Type
19
Displays:
Type: U for unknown, N for N_Port, NL for NL_Port PID: 24-bit Fibre Channel address COS: List of classes of service supported by device PortName: Device port World Wide Name (WWN) NodeName: Device node WWN Permanent Port Name: Physical N_Port or NL_Port WWN
There might be additional lines if the device has registered any of the following information (the switch automatically registers SCSI inquiry data for FCP target devices):
FC4s supported IP address Port and node symbolic names Fabric Port Name. The WWN of the port on the switch to which the device is physically connected. Hard address and/or port IP address
-r This value indicates what type of RSCN a device registers to receive.

SCR=0 Reserved SCR=1 Fabric detected registration. Register to receive all RSCN requests issued by the fabric controller for events detected by the fabric. SCR=2 Nx_Port detected registration. Register to receive all RSCN requests issued for events detected by the affected Nx_Port. SCR=3 Register to receive all RSCN request issued. The RSCN request returns all effected N_Port_ID pages.
-t Displays the device type. The first part indicates the origination of the device.
Physical - Device connected to the Nx_Port, using FLOGI to login to the switch. Virtual - Device contrived by the switch. NPV - Device connected to the Nx_Port, using FDISC to login to the switch. iSCSI - Device connected to the iSCSI port.
Redirect: Frame redirection would be done by an application on the FA blade running some SA/SAS. (eg. DMM or Invista)
Revision 1008
3 - 19
nscamshow [-t]
sw2:admin> nscamshow -t nscam show for remote switches: Switch entry for 1 state rev owner known v610 0xfffc02 Device list: count 5 Type Pid COS PortName NodeName N 010100; 3;10:00:00:05:1e:42:88:2f;20:00:00:05:1e:42:88:2f; FC4s: FCP PortSymb: [46] "BRE042 A.2 L3-25014-01B FW:01.03.19 Port 1 " Fabric Port Name: 20:01:00:05:1e:07:c3:3f Permanent Port Name: 10:00:00:05:1e:42:88:2f Device type: Physical Initiator Port Index: 1 Share Area: No Device Shared in Other AD: No Redirect: No -t displays the
Device Type
20
Use this command to display the local Name Server Cache information about the devices discovered in the fabric by the Name Server Cache manager. The message No Entry found! is displayed if the Name Server Cache Manager hasn't discovered new switches and any new devices in the fabric. For each remote switch found, the output of this command shows the same information as the nsshow command. Note: The r option is not available with the nscamshow command.
Revision 1008
3 - 20
nsallshow
Very useful to get a list of all online node ports in the fabric
sw2:admin> nsallshow { 010100 0102e2 0102e4 0102e8 0102ef 020500 0206e2 0206e4 0206e8 0206ef 10 Nx_Ports in the Fabric }
21
Displays global Name Server information Use this command to display the 24-bit Fibre Channel addresses for all devices in the fabric. If the operand type is supplied, only devices of specified FC-PH type are displayed. If type is omitted, all devices are displayed. Below is the same command, only with the type 8 (SCSI): sw2:admin> nsallshow 8 { 010100 0102e2 0102e4 0102e8 0102ef 020500 0206e2 0206e4 0206e8 0206ef 10 FCP Ports }
Revision 1008
3 - 21
Fabric Controller at FFFFFD

Responsible for operation of the Fabric
Handles switch-to-switch (Class F) traffic
SCR
Receives node requests for a State Change Registration (SCR) Distributes Registered State Change Notifications (RSCNs) to registered nodes
RSCN
22
A State Change Registration (SCR) is used by initiators to request notification from the fabric when a state change occurs. When a state change occurs that the device should know about, the fabric sends back a Registered State Change Notification (RSCN). Only devices that do an SCR can receive an RSCN. The Fabric Controller, with its well-known address at FFFFFD, handles this process.
RSL1_ST05_B200:admin> nsshow -r { Type Pid N COS PortName NodeName SCR 020000; FC4s: FCP NodeSymb: [52] "Emulex LP1150-F4 FV2.10A7 DV5-5.20A9 RSL1-ST05-W2K-1" Fabric Port Name: 20:00:00:05:1e:02:1c:24 Permanent Port Name: 10:00:00:00:c9:51:39:7d The Local Name Server has 1 entries } 2,3;10:00:00:00:c9:51:39:7d;20:00:00:00:c9:51:39:7d; 3
Notice this device has registered for SCR 3 = Full Registration. 1 = Fabric RSCNs only 2 = Nx_Port RSCNs only 3 = Full (1 and 2) RSCNs
Revision 1008
3 - 22
Brocade RSCN Delivery

Fabric reconfigurations with no domain change will not cause an RSCN to be sent Device power on or shutdown will cause an aggregated RSCN to be sent to affected zoned devices only Zoning changes cause a Fabric RSCN to be sent to affected zoned devices only Once devices receive an RSCN, they should query the Directory (Name) Server to determine what devices they have access to
23
Revision 1008
3 - 23
Device Communication Example
24
The above is an example of the frame communication between a host device and the switch (fabric). Note: The HBA, among other things, is responsibility for framing packets, physical addressing and link level error checking. FLOGI: Fabric Login command: Used to establish a 24-bit address for the device logging in. Also establishes Buffer-to-Buffer credits, class of service supported. PLOGI: Port Login command: Device must login into the Directory (Name) Server to Register its information as well as query for devices this device is zoned with. SCR: State Change Registration: Device needs to register for State Change Notification so if there is a change is the fabric, such as a zoning change or a change in the state of a device that this device has access to, the device will receive an RSCN. Registration: A device will exchange registration information with the Directory (Name) Server. Query: Devices can query the Directory (Name) Server for information about the device it has access to.
Revision 1008
3 - 24
Device Communication Example (cont.)
25
PLOGI: Port Login command: Initiator must login into the target. PRLI: Process Login command: This established the operating (SCSI is the most common) environment between the two N_Ports. Inquiry: This example uses an INQ command, it could be something different such as a report LUNs command for example. Which command is used is determined by the driver on the initiator. The commands purpose is to get a list of LUNs the initiator has access to.
Revision 1008
3 - 25
Fabric Initialization Process

ELP (Exchange Link Parameters)
Contains sender information R_A_TOV / E_D_TOV WWPN / Switch Name Flow control used more Contains responder information R_A_TOV / E_D_TOV WWPN / Switch Name Flow control used more...
Switch A Domain ID: 1 WWNN: 22
ACC (accept frame)
P Switch B
Domain ID: 2 WWNN: 11
Switch C Domain ID: 3 WWNN: 33
Problems show up as:

Segmented / Isolated E_Port
Switch D Domain ID: 4 WWNN: 44
Reason / explanations
Incompatible Link Parameters Incompatible flow control Unauthorized switch name ELP timeout
ACC ELP
Switch E Domain ID: 5 WWNN: 55
26
In this example, switch E is being added to the fabric. For more information on this process, see the FC-SW4 Spec at www.t11.org
Revision 1008
3 - 26
Fabric Initialization Process (cont.)

ESC (Exchange Switch Capabilities)
Vendor specific info Virtual Fabric support
ACC (accept frame)

P Switch B
ACC ESC
27
Revision 1008
3 - 27
Fabric Initialization Process (cont.)

EFP (Exchange Fabric Parameters)
Principal switch selection
Principal switch priority Switch name Domain ID list

P Switch B
ACC (accept frame)

Principal switch selection
Principal switch priority Switch name Domain ID list

ACC EFP
28
Revision 1008
3 - 28
Buffer-to-Buffer (BB) Credits

BB Credits are used as a flow control mechanism to ensure that the transmitter does not overwhelm the receiver BB Credits are exchanged during login:
FLOGI Accept for Nx_Ports ELP Accept for E_Ports
Both sides (example device and switch) do not have to have the same amount of credits Distance, Link Speed and Frame size effect required credits to fill the link
29
Revision 1008
3 - 29
Port Indexes - FCx-48 Blades

135 134 133 132 131 130 129 128 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 271 270 269 268 267 266 265 264 263 262 261 260 259 258 257 256 143 142 141 140 139 138 137 136 151 150 149 148 147 146 145 144 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 287 286 285 284 283 282 281 280 279 278 277 276 275 274 273 272 159 158 157 156 155 154 153 152 167 166 165 164 163 162 161 160 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 303 302 301 300 299 298 297 296 295 294 293 292 291 290 289 288 175 174 173 172 171 170 169 168 183 182 181 180 179 178 177 176 63 62 61 60 59 58 57 56 55 53 53 52 51 50 49 48 319 318 317 316 315 314 313 312 311 310 309 308 307 306 305 304 191 190 189 188 187 186 185 184 199 198 197 196 195 194 193 192 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 335 334 333 332 331 330 329 328 327 326 325 324 323 322 321 320 207 206 205 204 203 202 201 200 215 214 213 212 211 210 209 208 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 351 350 349 348 347 346 345 344 343 342 341 340 339 338 337 336 223 222 221 220 219 218 217 216 231 230 229 228 227 226 225 224 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 367 366 365 364 363 362 361 360 359 358 357 356 355 354 353 352 239 238 237 236 235 234 233 232 247 246 245 244 243 242 241 240 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 383 382 381 380 379 378 377 376 375 374 373 372 371 370 369 368 255 254 253 252 251 250 249 248
1 2 3 4 9 10 11 12 ------------------------------------------------------------- Card Slots ----------------------------------------------------------------- 2008 Brocade Communications Systems, Inc. All Rights Reserved.
30
The port indexes are the same for both FC4-48 and FC8-48 blades. For each blade, 24 ports per ASIC: Ports 0-7 and 24-39 (ports below red line) on one ASIC, ports 8-23 and 40-47 (ports above red line) are on the other ASIC. The grey boxes represent port Indexes 0-127. The blue boxes represent port Indexes 128-255. The yellow boxes represent port Indexes 256-383.
Revision 1008
3 - 30
Port Indexes - Mixed Blades
FC8-48
Ports 256-271 128-143 0-15
FC8-48
Ports 272-287 144-159 16-31
FC8-48
Ports 288-303 160-175 32-47
FC8-48
Ports 304-319 176-191 48-63
FC8-48
Ports 320-335 192-207 64-79
FC8-48
Ports 336-351 208-223 80-95
FC8-48
Ports 352-367 224-239 96-111
FC8-48
Ports 368-383 240-255 112-127
FC8-32
Ports 128-143 0-15
FC8-32
Ports 144-159 16-31
FC8-32
Ports 160-175 32-47
FC8-32
Ports 176-191 48-63
FC8-32
Ports 192-207 64-79
FC8-32
Ports 208-223 80-95
FC8-32
Ports 224-239 96-111
FC8-32
Ports 240-255 112-127
FC8-16
Ports 0-15
FC8-16
Ports 16-31
FC8-16
Ports 32-47
FC8-16
Ports 48-63
FC8-16
Ports 64-79
FC8-16
Ports 80-95
FC8-16
Ports 96-111
FC8-16
Ports 112-127
1 2 3 4 9 10 11 12 -------------------------------------------------------------Card Slots ---------------------------------------------------------------- 2008 Brocade Communications Systems, Inc. All Rights Reserved.
31
The diagram above shows the port numbers that would exist for the domain in the chassis depending on the port card type inserted in the slot.
Revision 1008
3 - 31
Port Indexes - Mixed Blades (cont.)
FC8-48
Ports 304-319 176-191 48-63
FC8-48
Ports 352-367 224-239 96-111
FC8-48
Ports 368-383 240-255 112-127
FC8-32
Ports 160-175 32-47
FC8-32
Ports 192-207 64-79
FC8-32
Ports 208-223 80-95
FC8-16
Ports 0-15
FC8-16
Ports 16-31
32
The diagram above shows the port numbers that would exist for the domain in the chassis depending on the port card type inserted in the slot.
Revision 1008
3 - 32
Shared Area Numbers - FCx-48 Blades

A shared area is an Area ID that exists more than once on a single chassis or domain_ID. A shared area is differentiated by its Node Address. The FC4-48 and FC8-48 blades have some ports that use shared areas that allow for greater than 256 ports in a single domain Ports 16-47 of these blades use shared areas Shared Area port identifiers (PIDs) use a Node Address of either 0x00 or 0x80 Example of two shared areas on a FC8-48 blade in slot 1:
018000 Port 16
018080 Port 40
33
The ability to address 384 ports in a single switch required a change to the 24-bit addressing scheme (PID). The second byte of a PID is referred to as the Area ID. With 8 bits, the Area ID can address ports 0-255. Brocade now uses the third byte of the PID to address ports 256-383. The third byte of a PID is referred to as the Node Address. The Node Address was used to identify the address (ALPA) for a loop device. Since an FCx-48 port does not support loop devices, the Node Address can be used to identify ports in the 256-383 range. This also requires the Area ID to be shared (used twice).
Revision 1008
3 - 33
Shared Area Numbers - FCx-48 Blades (cont.)

135 134 133 132 131 130 129 128 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 271 270 269 268 267 266 265 264 263 262 261 260 259 258 257 256 143 142 141 140 139 138 137 136 151 150 149 148 147 146 145 144 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 287 286 285 284 283 282 281 280 279 278 277 276 275 274 273 272 159 158 157 156 155 154 153 152 167 166 165 164 163 162 161 160 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 303 302 301 300 299 298 297 296 295 294 293 292 291 290 289 288 175 174 173 172 171 170 169 168 183 182 181 180 179 178 177 176 63 62 61 60 59 58 57 56 55 53 53 52 51 50 49 48 319 318 317 316 315 314 313 312 311 310 309 308 307 306 305 304 191 190 189 188 187 186 185 184 199 198 197 196 195 194 193 192 79 78 77 76 75 335 334 333 332 331 330 329 328 327 326 325 324 323 322 321 320 207 206 205 204 203 202 201 200 215 214 213 212 211 210 209 208 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 351 350 349 348 347 346 345 344 343 342 341 340 339 338 337 336 223 222 221 220 219 218 217 216 231 230 229 228 227 226 225 224 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 367 366 365 364 363 362 361 360 359 358 357 356 355 354 353 352 239 238 237 236 235 234 233 232 247 246 245 244 243 242 241 240 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 383 382 381 380 379 378 377 376 375 374 373 372 371 370 369 368 255 254 253 252 251 250 249 248
XXa880
24-bit PID
74 73 72
XXa800
71 70 69 68 67 66 65 64
34
The grey boxes represent Port Indexes 0-127. The Area ID for these Indexes are not shared. The blue boxes represent the Port Indexes of one ASIC on each FCx-48 port card that are shared (ports 16-23 share the same Area ID with ports 40-47, respectively). The yellow boxes represent the Port Indexes on the other ASIC on each FCx-48 port card that are shared (ports 24-31 share the same Area ID with ports 32-39, respectively). With shared Area IDs, the lower port number on the card has a Node Address of 0x00, while the higher port number on the card has a Node Address of 0x80. The Fabric OS v5.2 Admin Guide section on Identifying Ports by Index also has a representation of the information in this table.
Revision 1008
3 - 34
Shared Area Numbers - switchshow

<truncated output> Index Slot Port Address Media Speed State ========================================================= 168 3 24 01a800 -N8 No_Module 169 3 25 01a900 -N8 No_Module Slot 3, Ports 24-31 170 3 26 01aa00 -N8 No_Module share Area ID with 171 3 27 01ab00 -N8 No_Module Slot 3, Ports 32-39, 172 3 28 01ac00 -N8 No_Module respectively 173 3 29 01ad00 -N8 No_Module 174 3 30 01ae00 -N8 No_Module 175 3 31 01af00 -N8 No_Module <truncated output> 288 289 290 291 292 293 294 295 3 3 3 3 3 3 3 3 32 33 34 35 36 37 38 39 01a880 01a980 01aa80 01ab80 01ac80 01ad80 01ae80 01af80 --------N8 N8 N8 N8 N8 N8 N8 N8 No_Module No_Module No_Module No_Module No_Module No_Module No_Module No_Module
35
Example: Ports 24-31 on the blade in Slot 3 share the same Area Id with ports 32-39 on the same blade, respectively. Ports 24-31 will have a Node Address of 0x00 and ports 32-39 will have a Node Address of 0x80. This makes their 24-bit addresses unique.
Revision 1008
3 - 35
N_Port ID Virtualization (NPIV)

Virtual servers require secure access to storage the same way as physical servers Without NPIV, a single physical server connection is unable to provide independent storage access to individual virtual servers Instead, all storage ports and Logical Unit Numbers (LUNs) are exposed to all virtual machines, reducing security and manageability NPIV is an ANSI standard designed to solve this problem
36
NPIV devices connected to the same switch port must have a unique 24-bit address as well as a unique device PWWN.
Revision 1008
3 - 36
NPIV (cont.)
With NPIV, a physical server Host Bus Adapter (HBA) can provide up to 255 unique World Wide Port Names (WWPNs) for use by virtual servers Fabric switches with NPIV support can then assign unique fabric IDs to each virtual server as they log in to the fabric With NPIV support, standard fabric zoning and storage LUN masking can be used with virtual machines to isolate storage ports and LUNs to the appropriate virtual server just as they are with physical servers
37
sw2:admin> nsshow { Type Pid COS PortName NodeName N 620800; 3;20:08:00:05:1e:07:c3:9f;10:00:00:05:1e:07:c3:9f; Fabric Port Name: 20:08:00:05:1e:0c:ab:65 Permanent Port Name: 20:08:00:05:1e:07:c3:9f Port Index: 8 Share Area: No Device Shared in Other AD: No Redirect: No N 620801; 3;10:00:00:05:1e:42:86:88;20:00:00:05:1e:42:86:88; FC4s: FCP PortSymb: [46] "BRE042 A.2 L3-25014-01B FW:01.03.19 Port 0 " Fabric Port Name: 20:08:00:05:1e:0c:ab:65 Permanent Port Name: 20:08:00:05:1e:07:c3:9f Port Index: 8 Share Area: No Device Shared in Other AD: No Redirect: No N 620802; 3;10:00:00:05:1e:42:86:88;20:00:00:05:1e:42:86:88; FC4s: FCP PortSymb: [46] "BRE042 A.2 L3-25014-01B FW:01.03.19 Port 0 " Fabric Port Name: 20:08:00:05:1e:0c:ab:65 Permanent Port Name: 20:08:00:05:1e:07:c3:9f Port Index: 8 Share Area: No Device Shared in Other AD: No Redirect: No The Local Name Server has 3 entries }
TTL(sec) na
na
na
Revision 1008
3 - 37
NPIV Commands
portcfgnpivport
Enables or disables NPIV functionality on a port Enabled by default on Condor-based ports Disabled by default on Bloom-based ports
configure
Configures the maximum number of virtual ports per FC port
Default is 126
Configures the maximum number of virtual ports per switch
Default is (126 * number of switch ports)
sw2:admin> switchdisable sw2:admin> configure Configure... Fabric parameters (yes, y, no, n): [no] Virtual Channel parameters (yes, y, no, n): [no] F-Port login parameters (yes, y, no, n): [no] y Maximum logins per switch: (1..4032) [480] Maximum logins per port: (1..255) [126]
38
RSL1_ST05_B41:admin> portcfgnpivport 0 0 RSL1_ST05_B41:admin> portcfgshow Ports of Slot 0 Speed Trunk Port Long Distance VC Link Init Locked L_Port Locked G_Port Disabled E_Port ISL R_RDY Mode RSCN Suppressed NPIV capability 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 AN AN AN AN ON ON ON ON .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ON ON ON ON -----------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+-AN AN AN AN ON ON ON ON .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ON ON ON AN AN AN AN ON ON ON ON .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ON ON ON ON AN AN AN AN ON ON ON ON .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ON ON ON ON
Persistent Disable.. .. .. ..
Revision 1008
3 - 38
Summary
The Fibre Channel Network model is made up of levels FC0 FC4 World Wide Names are associated with nodes and ports Well-Known Addresses provide fabric generic services Switch Port Types are U_Port, FL_Port, G_Port, F_Port, E_Port, VE_Port, EX_Port and VEX_Port Multiple devices can share the same switch port with the use of NPIV
39
Revision 1008
3 - 39

End of Instructor-Led Module 3 Fibre Channel Theory
Revision 1008
Revision 1008
3 - 40
Appendix A Fibre Channel Reference List

Title: Principles of SAN Design Authors: Josh Judd & Dan Krueger ISBN: 0-7414-2824-5 Title: Multiprotocol Routing for SANs Author: Josh Judd ISBN: 0-7414-2306-5 Title: Fibre Channel: A Comprehensive Introduction Author: Robert W. Kembel ISBN: 0-931836-84-0 Title: Fibre Channel Switched Fabric Author: Robert W. Kembel ISBN: 0-931836-71-9 Title: IP SANS: An Introduction to iSCSI, iFCP, and FCIP Protocols for Storage Area Networks Author: Tom Clark ISBN: 0-201-75277-8 Title: Fibre Channel, Gigabit Communications and I/O for Computer Networks Author: Alan F. Benner ISBN: 0-07-005669-2 Title: Designing Storage Area Networks: A Practical Reference for Implementing Fibre Channel SANs (The Addison-Wesley Networking Basics Series) Author: Tom Clark ISBN: 0-201-61584-3 Title: A Practical Guide to SNMPv3 and Network Management Author: David Zeltserman ISBN: 0-13-021453-1 Title: Storage Area Networks Designing and Implementing a Mass Storage System Authors: Ralph H. Thornburgh and Barry J. Schoenborn ISBN: 0-13-027959-5 Title: Fibre Channel for SANs Author: Alan F. Benner ISBN: 0-07-137413-2 http://www.fibrechannel.com Fibre Channel Industry Association a trade group promoting the use of fibre channel. http://www.T11.org Home page for the National Committee for Information Technology Standards (NCITS) T11 committee. This group sets the fibre channel standards. http://www.T10.org Home page for the NCITS T10 committee. This group set the SCSI standards. http://www.scsita.org SCSI Trade Association - information about SCSI. http://www.searchstorage.com A storage-specific search engine. http://www.SNIA.org Storage Network Industry Association a trade group that promotes the use of storage area networks. http://www.infostor.com InfoStor a free monthly industry journal on storage. Apply at the website to qualify for a subscription. http://compnetworking.about.com/cs/fibrechannel/index.htm
Revision 1008
3 - 41
Revision 1008
3 - 42
Installation and Configuration

Instructor-Led Module 4 Installation and Configuration
Revision 1008
Revision 1008
4-1
Objectives
After completing this module and associated lab, attendees will be able to do the following on B-Series backbones, directors and switches:
Perform out-of-box initial configuration Perform initial security configuration Configure switch interoperability Verify switch status Identify the importance of fabric parameters Perform other common administrative tasks
Revision 1008
4-2
Environmental Concerns
Power
Cable dual power switches to dual power circuits
Air
FOS switches: Air flow is from non-cable side (back) to cable side (front) M-EOS switches: Air flow is from cable side (front) to non-cable side (back)
Cable(s)
Allow for manageable cable slack to minimize stress Do not mix single (longwave) with multimode (shortwave) in patch panel Secure with Velcro straps Be wary of distances - total can add up quickly with patch panels Create a Cable Table psshow fanshow Tempshow sensorshow Displays power status Displays fan status Displays temp readings Displays all sensor readings
Monitor switch environment
Air flow for Brocade switches is from the non-cable side to the cable side because the heat-generating components of the switch (ASICs and SFPs) are on the port side. Improper cable planning can cause problems and may impact performance in the SAN. Although patch panels are helpful in a cable management scenario, attempt to minimize the number of connections as every fiber optic interconnection generates a few dB of signal loss. Keep a manageable slack to minimize cable stress. Use different color Velcro straps for trunk groups. For more information on power supplies, fans, and temperature readings see the hardware reference manual for the appropriate switch model.
Revision 1008
4-3
Brocade Management Interfaces and Tools

Command Line Interface
Serial Communication (HyperTerm or tip) Telnet (port 23) SSHv2 (port 22)
SMI-S1 Fabric Manager EFCM Web Tools

HTTP HTTPS requires a Digital Certificate to be installed on the switch2
SNMPv1 and SNMPv3
Footnote 1: The SMI Agent software provides a Common Information Model (CIM) agent for switch and director products. The SMI Agent performs the functions of a general purpose server as defined in the Storage Management Initiative Specification (SMI-S version 1.2.0) and enables a standard set of management functions to be performed by third-party CIM clients. Footnote 2: Secure Socket Layer (SSL) is part of base Fabric OS. SSL works by using a key to encrypt data transferred over an SSL connection. By convention, URLs that require an SSL connection start with https: instead of http: All Brocade supported Internet browsers support SSL. Configuration of the SSL protocol involves obtaining, installing, and configuring PKI certificates: Public Key Infrastructure (PKI) is a system of public key encryption using digital certificates from a Certificate Authority (CA) and other registration authority to verify and authenticate the validity of each party involved in an electronic transaction. The CA works as part of a Public Key Infrastructure (PKI) and therefore checks with a registration authority (RA) to verify digital certificate requestor information. Once RA verifies information CA can issue a certificate. The information that the RA verifies depends on the CA, but includes items such as owners public key; certificate expiration date; owners name and other public key owner information. Secure access implementations that involve a Public Key Infrastructure (PKI) include three parts: Part 1 Certificate Signing Request (CSR) management (SSL access CSRs have to be generated at the CLI): Generate/delete/Display/Export CSR Part 2 Certificate Authority (CA) management: Send Certificate Signing Request (CSR) to CA and Get certificate(s) back from CA (SSL uses 3rd party CAs such as Verisign, Entrust, InstantSSL, and GeoTrust.) Part 3 Import and configure certificates SSL requires 3 certificates to be imported from the CLI: A Certificate Authority (CA) certificate that authenticates the switch certificate generated by a 3rd party. CA is a chainroot certificate which means that the switch manufacturer (Brocade) does not own the ROOT CA (it was obtained from a 3rd party). This CA certificate needs to be imported and installed on each switch that will be accessed via HTTPS (Example: ComodoSecurityServicesCA.crt). A switch certificate that authenticates the switch needs to be imported and installed on each switch that will be accessed via HTTPS (Example: 192_168_188_195.crt). A server Root certificate needs to be installed on each server that will be used to access a switch using HTTPS (Example: GTECyberTrustRoot.crt). Note: The Root certificate, CA authentication certificate, and the switch certificate must come from the same CA. One pair of CA and switch certificates are needed per switch. Each switch certificate will be unique. If the same CA is used multiple switches could import the same CA certificate, but each switch will need to import it.
Revision 1008
4-4
Command Line Interface Shortcuts

Recall last command & put CLI into edit mode
ESC + K (pre v5.1.0), or Up Arrow key (v5.1.0+)
Cursor Position - while edit mode active

H, J, K, L or UP, DOWN, LEFT, RIGHT (v5.1.0+)
End of Line - while edit mode active

Shift + A or END key (v5.1.0+)
Command completion
Use the tab key to complete command1 (v6.0+)
History of commands
h
Multiple commands issued on one line

command1;command2
Help for commands

help <command>
When administrating Brocade switches using the command line interface, certain key strokes can be helpful administrating the switch. Footnote 1: The tab key will complete a command only when there is one available command option. For example, if you enter B51:admin> cfgs and then press the tab key nothing will happen as there is more than one command that starts with cfgs. However, B51:admin> cfgsa will complete the command cfgsave. If more than one command is available, pressing the tab key twice will bring up a list of remaining available commands. B51:admin> cfgs cfgsave cfgsaveactivetodefined B30:admin> cfgs cfgshow cfgsize
Revision 1008
4-5
Initial Configuration
Revision 1008
4-6
Log in Through the Serial Port
A PC with:
Terminal Emulator An available COM port
A UNIX system with:

tip An available serial port
Cable:
The required serial cable is provided with the switch
When a new switch has arrived for installation into a fabric, it is suggested to use a serial cable to configure the switch with an IP address. After the IP address is configured, the serial connection to the switch may be dropped and an SSH, telnet, or Web Tools session may be used for further switch configuration because of its convenience and speed. To configure the connection in a B-Series environment: Bits per second: 9600 Data bits: 8 Parity: None Stop bits: 1 Flow control: None To configure the connection in an M-Series environment:
Installation steps 1. Insert the serial cable provided to an RS-232 serial port on the workstation FOS switches use a straight-through cable M-EOS switches use a null modem cable 2. Verify the switch has power and is past the POST stage 3. Invoke the ipaddrset command to set the IP address, subnet mask and default gateway
Revision 1008
4-7
Set the IP Address

Default IP Address for FOS switches 10.77.77.77
M-EOS switches 10.1.1.10
Default Netmask FOS switches: 255.255.255.0

M-EOS switches 255.0.0.0
Obtain addressing information for your network

IP Address & netmask Default gateway
Backbones/Directors require more than one IP address on the same subnet

One IP Address required for switch management One IP Address required per Control Processor Default IP Addresses for Directors: 10.77.77.77 (switch management), .75 (cp0), .74 (cp1) (M-EOS switches use the same IP for all communications)
B51:admin> ifmodeshow eth0 Link mode: negotiated 100baseTx-FD, link ok B51:admin> ifmodeset eth0 Exercise care when using this command. Forcing mode not supported by the network equipment to result in an inability to communicate with the interface. It is recommended that you only use serial console port. the link to an operating which it is attached may system through its Ethernet this command from the
Are you sure you really want to do this? (yes, y, no, n): [no] yes Proceed with caution. Auto-negotiate (yes, y, no, n): [no] Force 100 Mbps / Full Duplex (yes, y, no, n): [no] Force 100 Mbps / Half Duplex (yes, y, no, n): [no] Force 10 Mbps / Full Duplex (yes, y, no, n): [no] Force 10 Mbps / Half Duplex (yes, y, no, n): [no] You must select at least one link operating mode. B51:admin> ipaddrset Ethernet IP Address [10.255.248.35]: Ethernet Subnetmask [255.255.255.192]: Fibre Channel IP Address [0.0.0.0]: Fibre Channel Subnetmask [0.0.0.0]: Gateway IP Address [10.255.248.62]: Issuing gratuitous ARP...Done. IP address is being changed...Done. Committing configuration...Done.
Revision 1008
4-8
Log in Through the Ethernet Interface

Multiple concurrent telnet sessions are allowed on Linux-based switches
Use killtelnet to terminate a telnet connection
Login using a standard telnet or SSHv2 client Telnet may be disabled to force administrators to connect through an encrypted SSHv2 session
B51 login: admin Password: Please change your passwords now. Use Control-C to exit or press 'Enter' key to proceed. Password was not changed. Will prompt again at next login until password is changed.
B51:admin> killtelnet [* abbreviated *] __________________________________________________________________________ ____ Session No LOGIN@ USER TTY IDLE FROM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ 0 1:47pm 1 1:52pm admin0 admin0 pts/0 pts/1 1.00s 9.00s 10.255.248.22 10.255.248.22
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ Enter Session Number to terminate (q to quit) 1 Please Ensure (Y/[N]): Y killing session.... Done!
Revision 1008
4-9
Set Command Line Session Timeout

Automatically terminate a telnet or SSH session after a period of inactivity Timeout value is specified in minutes
Setting a timeout value of 0 disables automatic session timeout Valid settings include 0, or a value between 1 and 99,999 minutes To display the current setting, type timeout with no arguments
B51:admin> timeout Current IDLE Timeout is 0 minutes B51:admin> timeout 15 IDLE Timeout Changed to 15 minutes The modified IDLE Timeout will be in effect after NEXT login B51:admin> login B51 login: admin Password: B51:admin> timeout Current IDLE Timeout is 15 minutes
10
Default timeout on Linux-based switches is 10 minutes.
Revision 1008
4 - 10
Set the Fabric-Wide Clock

The Principal Switch maintains time for an entire fabric Subordinate switches synchronize time from the Principal Use the tsclockserver command to instruct the Principal Switch to synchronize time with an NTP server Specify an IP address of an NTP server Specify LOCL to stop NTP synchronization (LOCL must be uppercase) Use the date command to manually set the switch date and time date with no arguments displays the current date and time date mmddhhmmyy" sets the date and time, where:
mm is the month, valid values are 01-12 dd is the date, valid values are 01-31 hh is the hour, valid values are 00-23 mm is minutes, valid values are 00-59 yy is the year, valid values are 00-99
The date command becomes read-only if an NTP server has been specified
11
B51:admin> date Tue May 16 15:00:57 UTC 2006 B51:admin> tsclockserver LOCL B51:admin> tsclockserver 128.118.25.3 Updating Clock Server configuration...done. B51:admin> tsclockserver 128.118.25.3 B51:admin> date "0516073406" External Time Synchronization in place. Cannot execute this command. B51:admin> tsclockserver LOCL Updating Clock Server configuration...done. B51:admin> tsclockserver LOCL B51:admin> date "0516073406" Tue May 16 07:34:00 UTC 2006
Revision 1008
4 - 11
Set Switch Time Zone

Set on each switch in the fabric
Individual switches maintain time zone information independently
Use the tstimezone command to set the switch time zone in relationship to Greenwich Mean Time (GMT) Example: Eastern Time (United States) is GMT-5
tstimezone -5
Use tstimezone --interactive to provide a time zone menu
12
B51:admin> tstimezone -5 Updating Time Zone configuration...done. System Time Zone change will take effect at next reboot.
Revision 1008
4 - 12
Set Login Banner

A login banner will appear prior to CLI or Web Tools login Viewed from command line or Web Tools Set using the bannerset command Remove using bannerset ""
B51:admin> bannerset Please input content of security banner (press "." and RETURN at the beginning of a newline to finish input): Unauthorized access is prohibited. Do not log in if you do not have the authorization to do so. .
13
B51:admin> bannerset Please input content of security banner (press "." and RETURN at the beginning of a newline to finis h input): Unauthorized access is prohibited. Do not log in if you do not have the authorization to do so. . B51:admin> login Unauthorized access is prohibited. Do not log in if you do not have the authorization to do so. B51 login: admin Password: B51:admin> bannershow Unauthorized access is prohibited. Do not log in if you do not have the authorization to do so. B51:admin> bannerset "" B51:admin> bannershow
Revision 1008
4 - 13
Activate Licensed Features

Used to enable Fabric OS features Based on the switch WWN
licenseidshow Note: M-EOS switches use the serial number to generate licenses
License string is up to 32 mixed-case, case-sensitive characters A single license key may activate one feature or a bundle of features License commands
licenseshow licenseadd licenseremove 2008
Brocade Communications Systems, Incorporated. CFP270 ILT 0508
14
B51:admin> licenseidshow 10:00:00:05:1e:02:ab:21 b51:admin> licenseshow bQSyeQbQcccRTzfu: Fabric license RzdczyebRdS0id0i: Second Ports on Demand license - additional 8 port upgrade license WKAHHGEEMNrBEKC4NDWR4garNA7RNtDfBJFWE: 8 Gig FC license SgYXFZAtaEK7GDLQJrRgfTKCa3ENMK4NB7RBN: Extended Fabric license Fabric Watch license Performance Monitor license Trunking license FICON_CUP license First Ports on Demand license - additional 8 port upgrade license Unknown license results when a license Integrated Routing license from a previous OS is no longer valid Adaptive Networking license Example: Web Tools is not longer a licensed feature Unknown30 license Unknown31 license
One feature per license key
Multiple features per license key
Set the Switch Name

Switch names should be unique for easier administration Naming suggestions
Site or building where switch is located Floor or room where switch is located Indicate topology (core switch vs. edge switch) Rack ID Switch Type Fabric ID Domain ID
Example: B51
Remote SAN Lab #1 Station #2 Brocade B51
Switch name is assigned using the switchname command

15
Having a well thought-out switch naming convention enables easy identification of physical switches if a problem arises. Use a switch naming convention that scales across the organization, keeping in mind that the SAN might start small but can be extended enterprise-wide over time. Switch names can be duplicated in the fabric. To see a list of the existing switch names and their IP settings, use the command fabricshow. Switch Name rules: Up to 15 characters including letters, digits, hyphens, and underscore characters Must begin with a letter No spaces
SW:admin> switchname "B51" Committing configuration... Done. B51:admin> switchname B51 B51:admin> fabricshow Switch ID Worldwide Name Enet IP Addr 10.255.248.32 10.255.248.35 FC IP Addr 0.0.0.0 0.0.0.0 Name "B30" >"B51" ------------------------------------------------------------------------1: fffc01 10:00:00:05:1e:02:12:a5 2: fffc02 10:00:00:05:1e:02:ab:21 The Fabric has 2 switches
Revision 1008
4 - 15
Set syslog Server

The system logging daemon (syslogd) on hosts can receive system events and error messages from Brocade switches If all switches and control processors escalate messages to syslogd, the administrator may view a fabric-wide log of events Configuration is simple
syslogdipadd syslogdipremove syslogdipshow
syslog records are tagged as belonging to a facility

Supports UNIX local1 - local7 facilities The default facility level is 7 Change the facility using the syslogdfacility command
Additional host configuration may be necessary, see server documentation

16
B51:admin> syslogdipshow No addresses configured B51:admin> B51:admin> syslogdipadd 10.255.248.2 B51:admin> syslogdipadd 10.255.248.3 B51:admin> syslogdipshow syslog.IP.address.1 syslog.IP.address.2 10.255.248.2 10.255.248.3
B51:admin> syslogdfacility Syslog facility: LOG_LOCAL7 B51:admin> syslogdfacility -l 6 Syslog facility changed to LOG_LOCAL6 B51:admin> syslogdipremove 10.255.248.3 B51:admin> syslogdipshow syslog.IP.address.1 B51:admin> syslogdipshow No addresses configured
10.255.248.2
B51:admin> syslogdipremove 10.255.248.2
Initial Security Configuration
Revision 1008
4 - 17
Disable Telnet
Use the IPFilter policy to disable telnet1 If telnet is disabled from within a telnet session, all telnet sessions will be disabled To avoid losing your session, disable telnet through an alternate interface
Serial port session SSHv2 session Web Tools
18
Footnote 1: To create an IP Filter policy: ipfilter --create <policyname> -type <ipv4 | ipv6> <policyname> is a unique string composed of maximum 20 alpha, numeric or underscore characters The name default is reserved (default_ipv4 and default_ipv6) The policy name is case insensitive but is always stored as lower case ipfilter --addrule <policyname> -rule <rule_number> -sip <source_IP> -dp <dest_port> -proto <protocol> -act <permit | deny> Note: Previous to Fabric OS v5.3 use the configure command to disable telnet.
Revision 1008
4 - 18
Set Password Rules

Password rules are enforced only when defining new passwords Passwords that have already been defined will not be checked for policy compliance Set password rules with passwdcfg --set command Set password strength policy by specifying the minimum number of:
Lowercase letters Uppercase letters Digits (0-9) Punctuation Minimum characters1 length2 -lowercase -uppercase -digits -punctuation -minlength
Limit password re-use by setting the password history policy

Passwords kept in history3 -history
19
Footnote 1: All printable punctuation characters except colon ":" are allowed. Footnote 2: The minimum password length may be set from 8 to 40 characters in length. The password length is the total number of lowercase, uppercase, digits, and punctuation characters. The total number of these characters may not exceed 40. Keep this in mind as you specify the minimum number of each type of character required. Footnote 3: The password history policy is not enforced when an administrator sets a password for another user, but the password set by the administrator is recorded in the user's password history. swd77:admin> passwdcfg --set -lowercase 3 -uppercase 1 -digits 2 punctuation 2 -minlength 10 -history 3 swd77:admin> passwd Changing password for admin Enter old password: Enter new password: Password must be between 10 and 40 characters long. Enter new password: Insufficient number of upper case letters Enter new password: Insufficient number of lower case letters [* abbreviated *] Enter new password: Insufficient number of digits in password Enter new password: Re-type new password: passwd: all authentication tokens updated successfully Saving password to stable storage. Password saved to stable storage successfully.
Revision 1008
4 - 19
Set Password Rules (cont.)

Avoid stale passwords by setting a password expiration policy1
Minimum age Maximum age Expiration warning (days)2 -minpasswordage -maxpasswordage -warning -lockoutthreshold -lockoutduration
Set the account lockout policy3

Password failures allowed Set lockout duration (minutes)
B51:admin> passwdcfg --set -minpasswordage 20 -maxpasswordage 30 -warning 5 B51:admin> passwdcfg --set -lockoutthreshold 5 -lockoutduration 15
20
Footnote 1: The password expiration policy is not enforced for root and factory accounts. Footnote 2: The user will begin seeing warning messages when they login a number of days prior to password expiration. They will be compelled to change their password when it has expired. Footnote 3: The account lockout policy is not enforced for root, factory, and admin role accounts.
swd77:admin> passwdcfg --set -minpasswordage 20 -maxpasswordage 30 warning 5 swd77:admin> userconfig --show -a [* abbreviated *] Account name: root Role: root Description: root Enabled: Yes Password Last Change Date: Unknown Password Expiration Date: Not Applicable Locked: No Account name: admin Role: admin Description: Administrator Enabled: Yes Password Last Change Date: Wed May 24 2006 Password Expiration Date: Fri Jun 23 2006 Locked: No
Revision 1008
4 - 20

Change Default Passwords
Open a serial port, telnet, or SSH connection to the switch
Default administrative accounts: admin, user Default password for both: password Other default accounts: root, factory
Use of root and factory accounts is not supported Log in as root or factory only if directed by your support provider
When prompted, change all default passwords

Passwords must be between 8 and 40 characters by default
B51 login: admin Password: Please change your passwords now. Use Control-C to exit or press 'Enter' key to proceed. Password was not changed. Will prompt again at next login until password is changed.
Use the passwdcfg --showall command to display password rules
21
B51 login: admin Password: Please change your passwords now. Use Control-C to exit or press 'Enter' key to proceed. Warning: proper Access to of the Root and Factory accounts may be required ensure the Root and for
support
the switch. Please
Factory
passwords are
documented in a secure location. Recovery of a lost Root
or Factory password will result in fabric downtime. for user - root Changing password for root Enter new password: Re-type new password: passwd: all authentication tokens updated successfully Please change your passwords now. for user - factory for user - admin for user - user passwd: all authentication tokens updated successfully Saving passwords to stable storage. Passwords saved to stable storage successfully
Revision 1008
4 - 21

Use passwdcfg --setdefault command to restore the factory default password policy
B51:admin> passwdcfg --setdefault B51:admin> passwdcfg --showall passwdcfg.minlength: 8 passwdcfg.lowercase: 0 passwdcfg.uppercase: 0 passwdcfg.digits: 0 passwdcfg.punctuation: 0 passwdcfg.history: 1 passwdcfg.minpasswordage: 0 passwdcfg.maxpasswordage: 0 passwdcfg.warning: 0 passwdcfg.lockoutthreshold: 0 passwdcfg.lockoutduration: 30 passwdcfg.status: 0
22
Revision 1008
4 - 22
Set Role Based Access Control (RBAC)

RBAC is a role based approach to restricting system access to authorized users
RBAC introduced the distinction between a user account and the role assigned to the account Each account has an administratively defined role This distinction allows the switch to track login activity
There are three main parts to RBAC based accounts:

Account name - Login name and password Role - Grants or denies access to switch commands Description - Optional parameter to add additional detail about the account
23
Revision 1008
4 - 23
Set Role Based Access Control (cont.)

Fabric OS implements two classes of accounts: Default Accounts (root, factory, admin, user)
Each default account has a hard-coded set of permissions The permissions define roles with privileges corresponding to the account name The privileges and account names cannot be changed
User-Defined Accounts
256 user-defined accounts available per switch 32 simultaneous session logons per switch (includes the default accounts)
24
Revision 1008
4 - 24

Role Name
Admin BasicSwitchAdmin FabricAdmin Operator SecurityAdmin
Duties
All administration Restricted switch administration Fabric and switch administration General switch administration Restricts security functions Local switch administration Monitoring only Zone administration
Description
All administrative commands Mostly monitoring with limited switch (local) commands All switch and fabric commands; excludes user management and AD commands Routine switch maintenance commands All switch security and user management functions Most switch (local) commands; excludes security, user management and zoning commands Non-administrative use Zone management commands only
SwitchAdmin User ZoneAdmin
25
For each role there is a set of pre-defined permissions on the commands that can be performed. FOS v5.3 added the securityadmin role. FOS v5.2 added the zoneadmin, basicswitchadmin, operator, and fabricadmin roles. FOS v5.0 added the switchadmin role.
Revision 1008
4 - 25

Role permission depends on category of commands
Permission Type
Observe Modify Observe & Modify None
Abbr.
O M OM N
Description
Can run commands using options that display information only Can run commands using options that create, change & delete Can run commands using both observe & modify options Not allowed to run commands in given category
26
Each role is assigned a permission type for a category of commands. The permission applies to all commands within the category.
Revision 1008
4 - 26

Role permissions matrix (example) See Fabric OS Command Reference for complete matrix
Command Category User Operator Switch Admin Role Permission Zone Admin Fabric Admin Basic Switch Admin N O O O Admin Security Admin
Fabric Distribution Name Server Switch Configuration Zoning
N O O O
N O OM O
N OM OM O
N O N OM
OM OM OM OM
OM OM OM OM
OM N OM O
27
The categories above were chosen as an example to illustrate the permissions assigned to each role. For a complete list of commands and role permissions, see the Fabric OS Command Reference.
Revision 1008
4 - 27

Use the userconfig command to administer accounts
userconfig --show userconfig --change userconfig --add userconfig --delete
User-defined accounts assist in tracking who did what, when

Enable enhanced change tracking with trackchangesset 1
28
B51:admin> userconfig --show -a Account name: root Role: root Description: root Enabled: Yes Account name: factory Role: factory Description: Diagnostics Enabled: Yes Account name: admin Role: admin Description: Administrator Enabled: Yes Account name: user Role: user Description: User Enabled: Yes
B51:admin> userconfig --add jdoe -r admin d "Jane Doe" Setting initial password for jdoe Enter new password: Re-type new password: Account jdoe has been successfully added. B51:admin> login B51 login: jdoe Password: B51:jdoe> userconfig --show jdoe Account name: jdoe Role: admin Description: Jane Doe Enabled: Yes B51:jdoe> userconfig --change admin -e no Broadcast message from root (pts/0) Wed May 17 09:14:48 2006... Security Policy, Password or Account Attribute Change: admin will be logged out Attribute for account admin has been successfully changed. B51:jdoe> userconfig --show admin Account name: admin Role: admin Description: Administrator Enabled: No B51:jdoe>
DANGER
Your company policy may require you to disable default accounts or group-access accounts such as the default user and admin accounts. Before you disable the default account admin, be certain you have created at least one user-defined account assigned to the admin role. Without an account with admin privileges, you will not be able to manage your switch.
Revision 1008
4 - 28
Set LDAP to RBAC Roles

Fabric OS v6.1+ allows mapping of Lightweight Directory Access Protocol (LDAP) Active Directory server roles to Fabric OS RBAC roles
Creates an alias for a customer-defined group Allows a user belonging to that group to login to the switch with the permissions associated with the mapped switch role Supports one-to-one role mapping only
Example, user can map the "SANAdministrator" role on the Active

Directory server to the "admin" role on the switch
Example, user can map the "SANMaintainence" role to the switch

"operator" role
Command fails if you attempt to map an already mapped Active Directory server role
29
To implement a highly available solution that provides redundancy and minimizes the effect of network outages: Implement multiple LDAP servers on the network, configure all switches to authenticate with all LDAP servers, and configure all switches to use a local database as secondary authentication. Use the aaaconfig command to add a LDAP server.
Revision 1008
4 - 29
Set LDAP to RBAC Roles (cont.)

Configured only from CLI
To map an LDAP role to a switch role ldapcfg --maprole <ldaprole name> <switch role> Example: B51:admin> ldapcfg -maprole SANAdministrator admin To un-map a role ldapcfg --unmaprole <ldaprole name>
30
This command has the following operands: --maprole - Maps an LDAP role to a specified switch role. The following operands are required: --ldaprole - Specifies the LDAP role to be mapped to a switch role. The role must be a valid AD server role. --switchrole - Specifies the switch role to which the LDAP role is mapped. Valid switch roles include the following:
- admin - user - switchadmin - zoneadmin - fabricadmin - basicswitchadmin - operator - securityadmin
--unmaprole - Removes the mapping between an LDAP role and a switch role. Use the --show option for a listing of existing mappings. The following operand is required: --ldaprole - Specifies the LDAP AD sever role to be removed from the mapping. --show - Displays a table of existing mappings between LDAP roles and their corresponding switch role. --help - Displays command usage.
Revision 1008
4 - 30
Set LDAP to RBAC Roles (cont.)

To view current map roles
B51:admin> ldapcfg --show LDAP Role | Switch Role admin user fabricadmin zoneadmin operator securityadmin user securityadmin zoneadmin
-----------------------------------------------SANAdministrator | ldapuser SANfabadmin SANzoneadmin SANoperator LDAPSANsecadm SANuser SAN01secadmin LD_02zoneadmin | | | | | | | |
------------------------------------------------
31
LDAP roles are just names assigned to a switch role. They only apply to the available RBAC roles and not the root account.
Revision 1008
4 - 31
RADIUS Authentication
To centrally control user logins, Fabric OS supports the openstandard RADIUS protocol
Provides remote user access authentication, authorization, and accounting Client/Server model: A Brocade switch acts as a RADIUS client to a RADIUS server Network Security: All RADIUS client/server traffic is authenticated via a shared secret Focused on user logins, not FC device logins or switch attachment
When RADIUS is enabled on a switch:

All logins are authenticated through a RADIUS server (bypasses local database) All switch passwords are managed through the RADIUS server - the switch/Director local password database is bypassed Monitor user logins on a RADIUS-enabled system through the RADIUS server
32
The Remote Authentication Dial-In User Service (or RADIUS) is a protocol for carrying Authentication, Authorization, and Accounting (AAA) information about remote user access between a Network Access Server (which desires to authenticate its links) and a shared Authentication Server. RADIUS is an open standard (IETF RFC 2865 and RFC 2866). Client/server: The RADIUS client must pass user information to designated RADIUS servers, and act on the returned response. The RADIUS server receives user connection requests, authenticates the users, and then returns all configuration information needed for the RADIUS client to deliver service. In this case, a Brocade switch is configured as a Network Access Server that acts as a RADIUS client. Network Security: To ensure that user names and passwords remain private, all client/server communication is encrypted, and authenticated with a shared secret key. RADIUS is focused on authenticating, authorizing, and accounting remote user access in particular, logins and logouts. RADIUS does not perform these roles for devices or switches entering a fabric these roles continue to be handled by existing Fibre Channel protocols. In a fabric with switches running a mix of Fabric OS versions, the way a switch authenticates users depends on whether a RADIUS server is set up for that switch. For a switch with RADIUS support and configuration enabled, authentication bypasses the local password database. On a RADIUS-enabled switch, logins through the console port are not authenticated with the RADIUS server, but through the local switch database. For a switch with RADIUS support or configuration disabled, authentication uses switch local account names and passwords.
Revision 1008
4 - 32
RADIUS Authentication (cont.)

Fabric OS switches have two login authentication databases:
1.The default primary database is Switch Database1
It consists of the default switch login accounts: root, factory, admin, and user It can also contain user-defined RBAC accounts/roles There is no secondary login authentication database available when the
primary database is Switch Database; the only option is None
2.When RADIUS is configured as the primary login authentication database there are two options available for secondary login authentication: None (default) and Switch Database
33
Footnote 1: Web Tools Switch Database is referred to as switchdb at the CLI. If a denial (incorrect user name / password) is received from the RADIUS server that is authenticating the login, then a secondary RADIUS server or authentication database login is not attempted. If a configuration parameter is incorrect in either the RADIUS server or the switch AND if the secondary database is Switch Database, then telnet or Web Tools Admin access could be gained, after timeout(s), using a local account that authenticates via the Switch Database.
Revision 1008
4 - 33

Management access to a switch with primary database configured to RADIUS will attempt authentication via configured RADIUS server(s) Possible RADIUS server responses: grant, deny, or time out If response from all RADIUS servers is timeout, serial access can authenticate using switch database; ALL servers can similarly access IF secondary RADIUS server is set to Switch Database
34
In the example above, the primary database is RADIUS, and the RADIUS servers have been properly configured with user names and passwords. When a management station attempts access, the following scenario occurs: 1.The management server attempts to access the switch (RADIUS client) using a user name/password combination configured on the RADIUS servers. 2.The authentication request is sent to the first RADIUS server in the RADIUS configuration (RADIUS Server 1). 3.If the response from RADIUS Server 1 is accept, management access is achieved; if the response is deny, the management server does not get access. 4.If there is a timeout from RADIUS Server 1, then the authentication request is sent to the second RADIUS server in the RADIUS configuration (RADIUS Server 2). 5.If the response from RADIUS Server 2 is accept, management access is achieved; if the response is deny, the management server does not get access. 6.If there is a timeout from RADIUS Server 2 AND the switch database is configured as a secondary database, then the user name/password is authenticated on the local switch. Note: Up to five RADIUS servers can be configured.
Revision 1008
4 - 34

Use the aaaconfig RADIUS configuration command to configure and modify RADIUS settings
Add a RADIUS server:
aaaconfig --add <server> [p <port>] [-s <sharedsecret>] [-t <timeout>] [-a pap|chap|peap-mschapv2]
Modify an already-configured RADIUS server:

aaaconfig --change <server> [p <port>] [-s <sharedsecret>] [-t <timeout>] [-a pap|chap|peap-mschapv2]
35
Authentication protocols that can be configured include: Authentication Protocol (PAP) Challenge-Handshake Authentication Protocol (CHAP) Protected Extensible Authentication Protocol (PEAP)
Revision 1008
4 - 35

Web Tools can also be used to configure RADIUS on a switch
First select First select Switch Admin Switch Admin Next select Next select Show Advance Mode Show Advance Mode
36
1. Select Switch Admin 2. Select Show Advanced Mode
Revision 1008
4 - 36

Next select the AAA Next select the AAA Service tab Service tab select Add select Add Last, add RADIUS Last, add RADIUS server information server information
37
3. Select the AAA Service tab 4. Select Add 5. Add RADIUS server information (AAA = Authentication, Authorization, & Accounting)
Revision 1008
4 - 37
Initial Interoperability Configuration
Revision 1008
4 - 38
Fabric Interoperability Overview

Fabric Interoperability is switch-to-switch connectivity between B-Series and M-Series platforms
Established using direct E_Port connections1 Provides SAN functionality to a common set of hosts and storage devices2
Other connectivity solutions include:

Access Gateway (AG)3 Fibre Channel Router (FCR)4
39
Footnote 1: This is a generic definition of fabric interoperability. Fabric OS v6.0 fabric interoperability is only supported between classic Brocade (B-Series) and classic McDATA (M-Series) switches. Direct E_Port connections are established using ISLs with no FC Routing or Access gateway functionality. Fabric OS Fabric OS: the firmware on B-Series switches and Directors (e.g. 5000, 48000, etc.). M-EOS McDATA Enterprise OS, aka M-EOS: the firmware on B-Series switches and Directors (e.g. M4700, M6140, etc.). M-EOS includes both M-EOSc for Sphereon and M6000s and M-EOSn for the Mi10K. Footnote 2: Enables the exchange of fabric parameters, allowing switches to merge into one fabric with one principal switch with each switch having a unique domain ID. Brocades goal is to provide E_Port interoperability between B-Series (classic Brocade) and M-Series (classic McDATA) products at the physical, services, and management layers. This approach will give customers the flexibility to expand their existing SANs and to build new networks in the most seamless, efficient, and supportable manner possible. Footnote 3: AG connectivity capability uses Node Port Identifier Virtualization (NPIV) functionality in Brocade embedded switches and the Brocade 200E in AG mode. NPIV enables multiple hosts to connect through one port to any fabric switch that supports NPIV. Fabric OS v6.0 enhancements to AG are discussed in another instructor-led module associated with this course. Footnote 4: FCR provides physical connectivity between hosts/storage in different fabrics while keeping the fabrics isolated.
Revision 1008
4 - 39
Fabric Interoperability Overview (cont.)

Fabric OS v6.0+ support three fabric interop settings:
Interopmode 0 (Brocade Native Mode) Interopmode 1 (no longer available) Interopmode 2 (McDATA Fabric Mode)1 Interopmode 3 (McDATA Open Fabric Mode)
40
Footnote 1: Fabric OS v6.0 introduces a new interopmode 2 built from the ground up. Enhances Fabric OS v5.2.1_NI interopmode 2 functionality Interop mode restrictions: The 8 Gbit/sec Brocade DCX supports interop modes 2 or 3 starting in Fabric OS v6.0.0a+, check release notes for current support. The Brocade 4100 can load Fabric OS v6.0 but it will not run interop modes 2 or 3
Fabric OS Interop Mode FOS Native Mode FOS Interop Mode 1, obsolete
interopmode # 0 1
CLI Documentation Description interopMode off Not listed in the Fabric OS v6.0+ CLI guide McDATA Fabric Mode McDATA Open Fabric Mode
FOS McDATA Native Mode FOS McDATA Open Fabric Mode
2 3
Revision 1008
4 - 40

Brocade Native Mode (Interopmode 0): Default setting, all units ship in this mode Most robust option for Brocade Fabric OS based products Should ALWAYS be used for pure Fabric OS fabrics Non-disruptive firmware upgrades
41
Revision 1008
4 - 41

Fabric OS McDATA Fabric Mode (Interopmode 2): Ideal option for seamlessly adding Fabric OS products to M-EOS fabrics in McDATA Fabric Mode
Only supports connections to M-Series products no other vendors
Supports zoning from both B-series and M-Series switches1

When Interopmode 2 is first enabled, all existing zone configurations, defined and effective, are erased Only WWN zoning is supported
Non-disruptive firmware upgrade Requires a domain ID 96-127 to be used Not supported and should NEVER be used in fabrics with all B-Series Switches/Directors Requires M-EOS 9.06.02 +
42
Footnote 1: When zoning from a B-Series switch for the first time, use the cfgsaveactivetodefined command to save the active config to the defined config in order to make zoning changes. Zoning changes on B-Series switches are performed and saved to the defined config. M-Series switches do not use a defined config.
Revision 1008
4 - 42

Fabric OS McDATA Open Fabric Mode (Interopmode 3):
Ideal option for adding Fabric OS products to M-EOS fabrics in Open Fabric Mode
Only supports connections to M-Series products no other vendors When Interopmode 3 is first enabled, all existing zone configurations,
defined and effective, are erased
Not supported and should NEVER be used in fabrics with all B-Series Switches/Directors M-EOS 9.06.02 minimum required Requires a domain ID 96-127 be used Zoning can only be done via M-EOS switches1
Only WWN zoning is supported

Hot-Code Activation (HCA) is not supported2 Fabric OS features that are not supported include but are not limited to: Traffic Isolation zones, Virtual Channels between switches, Administrative Domains, and Frame Redirection3
43
Footnote 1: Zoning can NOT be enabled or configured from a Fabric OS switch in McDATA Open Fabric Mode. Initiating zone activations and copying the M-EOS fabric Effective zone configuration into the Define zone configuration database are not permitted. The following commands are not permitted in mode 3: cfgenable/cfgdisable, cfgsaveactivetodefined, and cfgmcdtmode These commands generate an error message when invoked in mode 3:
Error: This command cannot be executed in McDATA Open Fabric mode.
Footnote 2: HCA may work but it is not supported. Please check release notes to see if supported. Footnote 3: Check the latest release notes for an up-to-date list of Fabric OS v6.x features qualified in Open Fabric Mode (interopmode 3). For Fabric OS v6.0 most proprietary Brocade features will be disabled. This includes but is not limited to: Frame Level Trunking, Virtual Channels between switches, Administrative Domains, and Frame Redirection. Fabric OS v6.1 added support for Frame level Trunking between B-Series switches while in Open Fabric Mode (interopmode 3).
Revision 1008
4 - 43

Use the interopmode command to configure interoperability
B51:admin> interopmode Interopmode: off usage: InteropMode [0|2|3 [-z McDataDefaultZone] [-s McDataSafeZone]] 0: to turn interopMode off 2: to turn McDATA Fabric mode on Valid McDataDefaultZone: 0 (disabled), 1 (enabled) Valid McDataSafeZone: 0 (disabled), 1 (enabled) 3: to turn McDATA Open Fabric mode on
44
Revision 1008
4 - 44
Verifying Switch Status
Revision 1008
4 - 45
Switch Status Commands

Display overall status of switch with switchstatusshow Display current policy settings with switchstatuspolicyshow
Marginal Status
Yellow color when displayed in Web Tools or Fabric Manager Entry in error log, viewed with errshow, flagged as marginal
Down Status
Red color when displayed in Web Tools or Fabric Manager Entry in error log, viewed with errshow, flagged as faulty Display temperature, fan, and power supply status with sensorshow
46
B51:admin> switchstatusshow Switch Health Report 05/06/2008 09:54:21 PM Switch Name: IP address: SwitchState: Duration: B51 192.168.176.59 HEALTHY 06:23 HEALTHY HEALTHY HEALTHY HEALTHY HEALTHY HEALTHY HEALTHY Report time:
Power supplies monitor Temperatures monitor Fans monitor Flash monitor Marginal ports monitor Faulty ports monitor Missing SFPs monitor
All ports are healthy
Revision 1008
4 - 46
Switch Status Commands (cont.)

B51:admin> sensor 1: sensor 2: sensor 3: sensor 4: sensor 5: sensor 6: sensor 7: sensor 8: sensorshow (Temperature ) (Temperature ) (Temperature ) (Temperature ) (Fan ) (Fan ) (Power Supply) (Power Supply) is is is is is is is is Ok, value is 32 C Ok, value is 17 C Ok, value is 24 C Ok, value is 19 C Ok,speed is 14361 RPM Ok,speed is 14361 RPM Ok Ok
B51:admin> switchstatuspolicyshow
The current overall switch status policy parameters: Down Marginal ---------------------------------PowerSupplies 1 1 Temperatures 2 1 Fans 2 1 Flash 0 1 MarginalPorts 10 4 FaultyPorts 10 4 MissingSFPs 0 0
47
switchstbatusshow will display the overall status of the switch that include internal switch status, faulty ports, missing SFPs, power supplies, temperatures, fans, portstatus, and ISLStatus. The status may be one of the following: marginal/warning or down/failed. switchstatuspolicyshow: This command prints the current policy parameters for calculating the overall status of the switch. The tolerances for calculating the status of the switch can be configured with switchstatuspolicyset.
B300E:admin> switchstatusshow Switch Health Report Report time: 05/21/2006 09:37:31 AM Switch Name: IP address: SwitchState: Duration: B300E 10.255.248.32 HEALTHY 70:56 HEALTHY HEALTHY HEALTHY HEALTHY HEALTHY HEALTHY HEALTHY
Power supplies monitor Temperatures monitor Fans monitor Flash monitor Marginal ports monitor Faulty ports monitor Missing SFPs monitor All ports are healthy Revision 1008
4 - 47
switchstatuspolicyset
Seven parameters that determine switch status
Power Supplies Temperatures Fans Flash Marginal Ports Faulty Ports Missing SFPs
Marginal triggers
Yellow status in Web Tools
Down triggers
Red status in Web Tools
These states are policy-based and do not necessarily reflect the operational state of the switch
48
B51:admin> switchstatuspolicyset To change the overall switch status policy parameters The current overall switch status policy parameters: Down Marginal ---------------------------------PowerSupplies 1 1 Temperatures 2 1 Fans 2 1 Flash 0 1 MarginalPorts 2 1 FaultyPorts 2 1 MissingSFPs 0 0 Note that the value, 0, for a parameter, means that it is NOT used in the calculation. ** In addition, if the range of settable values in the prompt is (0..0), ** the policy parameter is NOT applicable to the switch. ** Simply hit the Return key. The minimum number of Bad PowerSupplies contributing to DOWN status: (0..1) [1] Bad PowerSupplies contributing to MARGINAL status: (0..1) [1] Bad Temperatures contributing to DOWN status: (0..2) [2] Bad Temperatures contributing to MARGINAL status: (0..2) [1] Bad Fans contributing to DOWN status: (0..3) [2] Bad Fans contributing to MARGINAL status: (0..3) [1] Out of range Flash contributing to DOWN status: (0..1) [0] Out of range Flash contributing to MARGINAL status: (0..1) [1] MarginalPorts contributing to DOWN status: (0..16) [2] MarginalPorts contributing to MARGINAL status: (0..16) [1] FaultyPorts contributing to DOWN status: (0..16) [2] FaultyPorts contributing to MARGINAL status: (0..16) [1] MissingSFPs contributing to DOWN status: (0..16) [0] MissingSFPs contributing to MARGINAL status: (0..16) [0] No change
Revision 1008
4 - 48
Port Status
Port name command
portname [slotnumber/]portnumber [name]
Port enable/disable commands

portdisable [slotnumber/]portnumber portenable [slotnumber/]portnumber portcfgpersistentdisable [slotnumber/]portnumber portcfgpersistentenable [slotnumber/]portnumber
Port status command

portshow [slotnumber/]portnumber
49
B51:admin> portshow 2 portName: Bay1 portHealth: No Fabric Watch License
Port Name
Authentication: None portDisableReason: None portCFlags: 0x1 portFlags: 0x20b03 PRESENT ACTIVE F_PORT G_PORT LOGICAL_ONLINE LOGIN NOELP ACCEPT FLOGI portType: 11.0 POD Port: Port is licensed portState: 1 Online portPhys: 6 In_Sync Port initialization from right to left and portScn: 32 F_Port current port type port generation number: 0 portId: 010100 portIfId: 4302000d portWwn: 20:01:00:05:1e:04:88:7e Port WWNs of Device portWwn of device(s) connected: 10:00:00:00:c9:59:45:9d Distance: normal Distance: Normal buffering portSpeed: N4Gbps LE domain: 0 FC Fastwrite: OFF Interrupts: Unknown: Lli: Proc_rqrd: Timed_out: Rx_flushed: Tx_unavail: Free_buffer: Overrun: Suspended: Parity_err: 2_parity_err: CMI_bus_err:
Port speed
9 0 9 14 0 0 0 0 0 0 0 0 0 Link_failure: Loss_of_sync: Loss_of_sig: Protocol_err: Invalid_word: Invalid_crc: Delim_err: Address_err: Lr_in: Lr_out: Ols_in: Ols_out: 0 3 0 0 115 0 0 0 3 0 0 3 Frjt: Fbsy: 0 0
Port part of other ADs: No
Revision 1008
4 - 49
Port Speeds
Individual port speeds can be set by the administrator portcfgspeed <slot/port>,<speed_level> Set the speed level for all ports on a switch switchcfgspeed <speed_level> Valid speeds 0: auto-negotiated 1, 2, 4 or 8 Gbit/sec 1: 1 Gbit/sec1 2: 2 Gbit/sec 4: 4 Gbit/sec 8: 8 Gbit/sec The SFP and hard-coded port speed should match, otherwise a Mod_Inv will display in switchshow output
50
Footnote 1: On 8 Gbit/sec switches a Brocade-branded 4 Gbit/sec SFP is required for a port to run at 1 Gbit/sec.
Revision 1008
4 - 50
Port Settings & Port Setting Commands

B30:admin> portcfgshow Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -----------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+-Speed AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AL_PA Offset 13 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Trunk Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. VC Link Init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. RSCN Suppressed .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Persistent Disable.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. NPIV capability ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON QOS E_Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON Mirror Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Rate Limit .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Credit Recovery ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON Ports of Slot 0 16 17 18 19 20 21 22 23 -----------------+--+--+--+--+----+--+--+-Speed AN AN AN AN AN AN AN AN AL_PA Offset 13 .. .. .. .. .. .. .. .. Trunk Port ON ON ON ON ON ON ON ON Long Distance .. .. .. .. .. .. .. .. VC Link Init .. .. .. .. .. .. .. .. Locked L_Port .. .. .. .. .. .. .. .. Locked G_Port .. .. .. .. .. .. .. .. Disabled E_Port .. .. .. .. .. .. .. .. ISL R_RDY Mode .. .. .. .. .. .. .. .. RSCN Suppressed .. .. .. .. .. .. .. .. Persistent Disable.. .. .. .. .. .. .. .. NPIV capability ON ON ON ON ON ON ON ON QOS E_Port ON ON ON ON ON ON ON ON Mirror Port .. .. .. .. .. .. .. .. Rate Limit .. .. .. .. .. .. .. .. Credit Recovery ON ON ON ON ON ON ON ON where AN:AutoNegotiate, ..:OFF, ??:INVALID, LM:L0.5
51
Speed is displayed as 1G, 2G, 4G, 8G, or AN (when in Auto Speed Negotiation mode). This value is set by the portcfgspeed command. Trunk Port is displayed as ON (when port is set for trunking) or for OFF (when trunking is disabled on the port) as set by the portcfgtrunkport command. Long Distance setting of the port is shown as blank when long distance mode is L0 (normal) and will display modes depending on the distance mode setting: LE (<= 10km), L0.5 (<=25km), L1 (<= 50km), L2 (<= 100km), LD (auto), LS (static). This value is set by the portcfglongdistance command. VC link init setting of the port is shown as blank when VC link init mode is off or ON when VC link init mode is on. This value is set by the portcfglongdistance command. Locked L_Port is displayed as ON when port is locked to L_Port only or when L_Port lock mode is disabled (and it behaves as a U_Port). This value is set by the portcfglport command. Locked G_Port is displayed as ON when port is locked to G_Port only) or blank when G_Port lock mode is disabled (and it behaves as a U_Port). This command is set by the portcfggport command.
Revision 1008
4 - 51
Disabled E_Port is displayed as ON when port is not allowed to be an E_Port. This command is set by the portcfgeport command. Persistent disable is displayed as ON when the port is disabled across reboots or power cycles or when the port is allowed to function normally. This mode is set by the portcfgpersistentdisable command. ISL R_RDY is displayed as ON when the port is set to R_RDY flow control. This mode is set by the portcfgislmode command. RSCN Suppressed is displayed as ON when RSCNs have been suppressed on the port. This mode is set by the portcfg rscnsupr command. Persistent Disable is displayed as ON when the port has been persistently disabled. This mode is set by the portcfgpersistentdisable command NPIV capability mode is displayed as ON when the port is configured to perform N_port virtualization. This mode is set by the portcfgnpivport command. QOS E_Port Displays (..)/OFF when Quality of Service (QoS) is disabled on the port. Displays ON when QoS is enabled. By default, QoS is enabled by best effort based on availability of buffers. This value is set by the portcfgqos command. Mirror Port Displays ON when Mirror Port is enabled on the port or (..)/OFF when disabled. This value is set by the portCfg mirrorport command. FC Rate Limit Displays ON when ingress rate limit is set on the port or (..)/OFF when the ingress rate limiting feature is disabled. This value is set by the portCfgQos --setratelimit command. The default is OFF. Credit Recovery Displays ON when Credit Recovery is enabled on the port or (..)/OFF when disabled. This value is set by the portCfgCreditRecovery command. The credit recovery feature is enabled by default, but only ports configured as long distance ports can utilize this feature.
Revision 1008
4 - 52
Verifying Switch Operation

B30:admin> switchshow switchName: B30 switchType: 71.2 switchState: Online switchMode: Native switchRole: Principal switchDomain: 51 switchId: fffc33 switchWwn: 10:00:00:05:1e:07:7a:e3 zoning: ON (cfg1) switchBeacon: OFF Area Port Media Speed State Proto ========================================= 0 0 id N1 Online F-Port 10:00:00:00:c9:24:76:16 1 1 id N4 Online F-Port 10:00:00:00:c9:29:06:4d 2 2 id N2 Online L-Port 10 public 3 3 id N8 Online Loopback->3 4 4 id N8 Online E-Port (Trunk port, master is port #5) 5 5 id N8 Online E-Port 10:00:00:60:69:90:04:f0 "SWT3850_53" (downstream) (Trunk master) 6 6 id N8 Online E-Port (Trunk port, master is port #5) 7 7 id N8 Online E-Port (Trunk port, master is port #5) 8 8 -N8 No_Module (No POD License) Disabled 9 9 -N8 No_Module (No POD License) Disabled 10 10 -N8 No_Module (No POD License) Disabled 11 11 -N8 No_Module (No POD License) Disabled 12 12 -N8 No_Module (No POD License) Disabled 13 13 -N8 No_Module (No POD License) Disabled 14 14 -N8 No_Module (No POD License) Disabled 15 15 -N8 No_Module (No POD License) Disabled 16 16 -N8 No_Module (No POD License) Disabled 17 17 -N8 No_Module (No POD License) Disabled 18 18 -N8 No_Module (No POD License) Disabled 19 19 -N8 No_Module (No POD License) Disabled 20 20 -N8 No_Module (No POD License) Disabled 21 21 -N8 No_Module (No POD License) Disabled 22 22 -N8 No_Module (No POD License) Disabled 23 23 -N8 No_Module (No POD License) Disabled
53
The switchshow command can be used to verify the switch is operating correctly and display information about the switch status.
switchName - The switchs name switchType - model.motherboard-rev, where the model number is as follows: 1=Brocade 1000 2=Brocade 2800 3=Brocade 2400 4=Brocade 20x0 5=Brocade 22x0 9=Brocade 3800 10=Brocade 12000 12=Brocade 3900 16=Brocade 3200 21=Brocade 24000 26=Brocade 3850 27=Brocade 3250 32=Brocade 4100 34=Brocade 200E 38=AP 7420 42=Brocade 48000 44=Brocade 4900 46=Brocade 7500 58=Brocade 5000 64=Brocade 5300 62=Brocade DCX 66=Brocade 5100 71=Brocade 300 switchState - The state of this switch: Online, Offline, Testing, or Faulty switchMode - The switch mode, Native or InterOp switchRole - The switch role: Principal, Subordinate, or disabled switchDomain - The domain ID of this switch: 0 to 31 or 1 to 239. switchID - The 24-bit address of this switch's embedded port: hex fffc00 to fffcef. switchWwn - The World Wide Name of this switch switchBeacon - Indicates if the beacon is turned on or not Zoning - zoning status Port Number - Each line shows the port number: 0 to 23, the GBIC type, the port state and a comment field Port module type - The GBIC/SFP or other type follows the port number. The four types include (--= none; sw=short wave; lw long wave; cu copper; id - intelligent) Port speed - The speed of the port (1G, 2G, 4G, 8G, N1, N2, N4, N8, AN) Long distance level - L0 (default), LE, LD, LS Port state - The possible port states include:
No_Card - no card present in this switch slot No_Module - no SFP module in this port No_Light - the module is not receiving light No_Sync - the module is receiving light but is out of sync In_Sync - the module is receiving light and is in sync (copper displays Sync, fiber, Online) Laser_Flt - the module is signaling a laser fault (defective GBIC) Port_Flt - the port has been marked faulty (defective GBIC, cable, or device) Diag_Flt - the port failed diagnostics (defective G_Port or FL_Port card or motherboard) Online - the port is up and running Lock_Ref - the port is locking to the reference signal Testing - running diagnostics
Comment field - Some possible comments include: Disabled, Loopback
Revision 1008
4 - 53
Port Status LEDs

Port status LED behavior may vary by switch type Check the Hardware Reference Guide for your particular switch Example for the Brocade 5100:
Port Status LED State Flickering Green Steady Green Slow Green
(2 second intervals)
Meaning Port is online and frames are passing through the port Port is connected to another device, but has no traffic Port is online but segmented Port is disabled Port is faulty
Slow Flashing Amber

(2 second intervals)
Fast Flashing Amber

( second intervals)
54
Port Speed and Serial Port LEDs

Location of LED
Below serial port
Purpose of LED
Indicates switch power
Color of LED
No light
Status of Hardware
Recommended Action
Boot not complete or failed; switch may be off
Verify boot completed; contact switch vendor
Steady green Slow green Above each port on right Indicates port speed No light
Switch on and boot completed
None
One or more ports failed POST
Check error log
POST is running, or port is Tx/Rx at 1 Gbit/sec
None
Steady green Steady amber
Port is Tx/Rx at 2 Gbit/sec
None
Port is Tx/Rx at 4 Gbit/sec
None
Port Speed LED for 3900/12000: upper LED
Revision 1008
4 - 54
Fabric Parameters
Revision 1008
4 - 55
Configuration Parameters
Dictate the way the switch will behave Set using the configure command
Some configure parameters can be changed online
Disable the switch before setting fabric configuration parameters

B51:admin> switchdisable; configure
Reset some factory defaults by using the configdefault command

B51:admin> switchdisable; configdefault
Parameters not reset by a configdefault

World Wide Name Ethernet settings (MAC address, IP address, subnetmask) IP gateway address SNMP configuration Zoning configuration Switch name License keys
56
An example of some of the configuration parameters: B51:admin> switchdisable; configure Configure... Fabric parameters (yes, y, no, n): [no] y Domain: (1..239) [1] R_A_TOV: (4000..120000) [10000] E_D_TOV: (1000..5000) [2000] WAN_TOV: (0..30000) [0] MAX_HOPS: (7..19) [7] Data field size: (256..2112) [2112] Sequence Level Switching: (0..1) [0] Disable Device Probing: (0..1) [0] Suppress Class F Traffic: (0..1) [0] Per-frame Route Priority: (0..1) [0] Long Distance Fabric: (0..1) [0] BB credit: (1..27) [16]
Revision 1008
4 - 56
Consistent Fabric Parameters

The configshow command will display parameter settings
B30:admin> configshow fabric.ops fabric.ops.BBCredit:16 fabric.ops.E_D_TOV:2000 fabric.ops.R_A_TOV:10000 fabric.ops.dataFieldSize:2112 fabric.ops.max_hops:7 fabric.ops.mode.fcpProbeDisable:0 fabric.ops.mode.isolate:0 fabric.ops.mode.longDistance:0 fabric.ops.mode.noClassF:0 fabric.ops.mode.tachyonCompat:0 fabric.ops.mode.unicastOnly:0 fabric.ops.mode.useCsCtl:0 fabric.ops.vc.class.2:2 fabric.ops.vc.class.3:3 fabric.ops.vc.config:0xc0 fabric.ops.vc.linkCtrl:0 fabric.ops.vc.multicast:7 fabric.ops.wan_tov:0
fabric.ops parameters must be consistent on all switches throughout the fabric, otherwise switches cannot join together
57
Other fabric.ops parameters: fabric.ops.mode.noClassF: Class F frames will not be used for inter-switch communications - Class 2 is used. Toggle using Suppress Class F Traffic under Fabric Parameters. fabric.ops.mode.sync: Used to prevent time out delays in remote Fabrics. fabric.ops.mode.useCsCtl: Type configure then yes to Fabric parameters and toggle Per-frame Route Priority: (0..1) [0] to change fabric.ops.mode.useCsCtl. Creates additional Virtual Channel ID for per-frame based prioritization, using existing VCs plus frame header information. fabric.ops.mode.vcEncode: The output relates to Virtual Channel (vc) settings for establishing communication priority over ISLs and are configurable only when VC Encoded Address Mode is set. Like all fabric.ops parameters, they must be the same on all fabric switches. Other fabric.ops parameters are explained in the Fabric OS Reference Guide. Fabric operating mode parameters include: Disable Device Probing fabric.ops.mode.fcpProbeDisable Isolated Operation fabric.ops.mode.isolate Long Distance Fabric fabric.ops.mode.longDistance Suppress Class F Traffic fabric.ops.mode.noClassF Sequence Level Switching fabric.ops.mode.tachyonCompat Unicast-only Operation fabric.ops.mode.unicastOnly Per-frame Route Priority fabric.ops.mode.useCsCtl
Revision 1008
4 - 57
Booting a Switch
fastboot boots the switch bypassing POST reboot boots the switch and includes POST hareboot for single CP non-disruptive reboot
58
The fastboot command reboots the Control Processor (CP) and bypasses the Power On Self Tests (POST). Footnote 1: The reboot command reboots the Control Processor (CP) and includes POST unless diagdisablepost is configured. The hareboot command will reboot the Control Processor (CP). This command is useful to reboot the standby CP blade in a director or backbone.
Revision 1008
4 - 58
Summary
Installation and configuration of Brocade switches includes the following steps:
Initial Configuration Initial Security Configuration Verify Switch Status Fabric Parameters Administrative Tasks
59
Revision 1008
4 - 59

End of Instructor-Led Module 4 Installation and Configuration
Revision 1008
Revision 1008
4 - 60
Appendix - Installation and Configuration
Revision 1008
4 - 61
Initial Interoperability Configuration

Feature Mode 0 Yes Yes Yes Yes No No No Yes Yes3 Yes Yes Yes5 Yes5 Yes Yes Yes Yes Mode 2 Yes1 Yes No* No No Yes Yes Yes2 No Yes4 Yes4 No No Yes No No* No Mode 3 No* No No No No No No Yes2 No Yes4 Yes4 No No No* No No* No
62
Fabric OS v6.1 interoperability software support

Check the latest release notes for an up-to-date list of qualified interoperability features
L2 Fabric OS Hot Code Load Zone Activation Support Traffic Isolation Zones Frame Redirection (Fabric OS) Frame Redirection (mEOS) FCR Fabric Binding L2 Fabric Binding SCC policies E_Port Authentication Port Based Trunking VC_RDYs FICON Cascaded CUP FICON MIHPTO5 Full Scalability Qos - Flow Control Qos Front Port Rate Limiting Admin Domains
Features denoted with * are implemented but not officially qualified/supported nor are they disabled. Footnote 1: Once the supported switch has Fabric OS v6.0.0 loaded, all subsequent upgrades will support Hot Code Loads. Hot Code Loads from Fabric OS v5.2.1_NI to v6.0.0 are also supported on Brocade 5000s. Footnote 2: SCC policies are only supported in conjunction with L2 Fabric Binding support. Footnote 3: Fabric OS v5.3 introduced an E_Port authentication called Fabric Element Authentication; authentication can be configured for both E_Ports and/or F_Ports (only supported in fabrics with all B-Series switches). Footnote 4: Supported on ISLs between B-Series switches but ISLs between BSeries and M-Series will continue to use R_RDY with no frame level trunking. Footnote 5: FICON features are supported on qualified B-Series hardware platforms. Recall: Mode 0 is Fabric OS Brocade Native Mode Mode 2 is Fabric OS McDATA Fabric Mode Mode 3 is Fabric OS McDATA Open Fabric Mode MIHPTO stands for Missing Interrupt Handler Primary Timeout.
Revision 1008
4 - 62
Initial Interoperability Configuration (cont.)

Fabric OS v6.0 & v6.1 interoperability hardware support
M-EOS Interop Support Matrix Fabric OS v6.0 Interopmode 2 & 3 Support No No
1 1 1
Switch Type Brocade 5300 Brocade 5100 Brocade 300 Brocade 7500 Brocade 7600 Brocade 200E Embedded Switch Blades 4012/4014/4018/4020/4024 Brocade 5000 Brocade 4900 Brocade 4100
Fabric OS v6.1 Interopmode 2 & 3 Support Yes Yes Yes Yes Yes Yes Yes Yes Yes No
2
No
Yes Yes Yes Yes Yes Yes No

2
63
Footnote 1: Switch requires Fabric OS v6.1+. Footnote 2: The Brocade 4100 can load Fabric OS v6.0, but does not support interopmode 2 or 3.
Revision 1008
4 - 63
Initial Interoperability Configuration (cont.)

Chassis Type Blade Type FC4-16/32/48 4 Gbit/sec port cards FC10-6 10 Gbit/sec port card FC8-16/32/48 8 Gbit/sec port cards 48000 FC4-16IP iSCSI port card FR4-18i FCR/FCIP port card FA4-18 Application Blade port card FC8-16/32/48 8 Gbit/sec port cards FC10-6 10 Gbit/sec port card DCX FC4-16IP iSCSI port card FR4-18i FCR/FCIP port card FA4-18 Application Blade port card No Yes Yes Yes (6.0.0a) No No No No Yes Yes Yes Yes Yes No Yes Yes FOS v6.0 FOS v6.1 Interopmode 2 & 3 Interopmode 2 & 3 Support Support Yes No No Yes Yes Yes
64
Revision 1008
4 - 64

End of Instructor-Led Module 4 Appendix - Installation and Configuration Revision 1008
Revision 1008
4 - 65
This page intentionally left blank.
Revision 1008
4 - 66
Zoning

Instructor-Led Module 5 Zoning
Revision 1008
Revision 1008
5-1
Zoning
Objectives
Understand the basic concepts associated with Zoning Implement a Zoning scheme using the command line syntax Activate or deactivate a Default Zone Differentiate between Hardware and Session enforcement Add a new switch to an existing fabric with Zoning enabled State the best practices that should be considered when implementing Zoning
Revision 1008
5-2
Zoning
Zoning Overview
Server in the Red zone sees the disks in Loop 1 Server in the Blue zone sees the two disk Arrays Server in the Green zone sees the disks in Loop 1 and one disk Array Server 4 sees no disks; no servers see the disks in Loop 2
A zone is a specified group of fabric-connected devices, also called zone members. Any device, or zone member, connected to the fabric can be included in one or more zones. Devices can communicate only with devices that are in the same zone. After zoning has been enabled, if a device is not explicitly defined in a zone that device is considered not to exist. In the example above both Server 4 and Loop 2 are not defined. When Server 4 queries the fabric to discover what devices it can see, Zoning rejects the request because it is not defined in any zone. Likewise, when the Servers in the Red, Blue and Green zones query the fabric, none of them will see the disk in Loop 2 because it is not defined in any zone. The device will be isolated and will be inaccessible by other devices in the fabric. Devices that attach to the fabric need to be added to a new or existing zone before their ability to communicate is enabled. After the zone members are grouped into zones, zones are grouped into a zone configuration and the zone configuration can then be enabled. When enabled, the zone configuration is distributed to all switches in the fabric and an RSCN is delivered by each switch to its local nodes that are effected by changes in the enabled zone configuration. The Fabric OS Administrators Guide describes zoning concepts in more detail.
Revision 1008
5-3
Zoning
Process to Implement Zoning

Prepare
Create a detailed diagram of the fabric
Define
Establish a naming convention Identify members by Dom,PortIndex or WWN Create aliases or nicknames, zones, zone configuration Exclude E_Ports
Analyze zone configuration

Can be done with CLI, Web Tools, EFCM, Fabric Manager or Health SAN
Enable the zone configuration Verify there is accessibility between zone members
Create a detailed switch diagram of the fabric showing ISL connectivity. This will help account for every switch in the fabric and the E_Ports that are in use. Expand each switch diagram to show every port (F_Port, FL_Port). Switch ports that are not in use should remain disabled with a portcfgpersistentdisable command. Define a naming convention to help identify and reference devices in the fabric. Naming conventions can also be used when creating zones and zone configurations. The zoning syntax when creating a zoning set ultimately defines what zoning scheme will be enforced as the frame is delivered to the destination port. More information on this will follow. Analyze the zones to ensure that all nodes are members of the correct zone(s). When the aliases have been added to zones and the zones are added to the zone configuration, enable the zone configuration and test from the host that each target can be accessed. For fabrics with multiple zones enabled, it is generally best to configure one zone at a time and then test it with the Zone Analyzer available in Web Tools. If you create all the zones without testing each zone as it is created, it is difficult to debug. After the first zone is set up in the fabric, the user may plug in devices and then test the connections to confirm that everything is functioning properly.
Revision 1008
5-4
Zoning
Hierarchy of Objects
Member: Alias is given a name, e.g. Server_1, Disk_Array_2. In EFCM, these are called nicknames. Physical Fabric port number or area number. Node World Wide Name - Obtained using nsshow or switchshow. Port World Wide Name Obtained using nsshow or portloginshow. 64 characters maximum: A-Z, a-z, 0-9 and the _ are allowed. Zone: Is given a name, e.g. Red_Zone. Contains two or more members and uses a ; as a separator. The same member can be in multiple zones. Zone definition is persistent; it remains until deleted or changed by an administrator. Configuration: Is given a name, e.g. Production_Cfg. Is one or more zones. Configuration may be disabled or one configuration may be in effect from any switch in the fabric. An administrator selects which configuration is currently enabled. A configuration is saved when enabled and then distributed to the remaining switches in the fabric where it is enabled and saved.
Revision 1008
5-5
Zoning
Zone Management
Zoning can be managed using:
Command Line Interface (CLI) Web Tools Fabric Manager EFCM
Use the zonehelp command to display help information

Create Alias alicreate Delete alidelete Add aliadd Remove aliremove Show alishow
Zone
zonecreate
zonedelete
zoneadd
zoneremove
zoneshow
Zone Config
cfgcreate
cfgdelete
cfgadd
cfgremove
cfgshow
Fabric OS Zone Management Commands
The following commands are used to create/modify the defined zone configuration: *create *delete *add *remove *show Creates a new alias, zone, or configuration Deletes the entire alias, zone, or configuration Adds a member to an existing alias, zone, or configuration Removes one or more members from an existing alias, zone, or configuration Displays alias, zone, and/or configuration information
Web Tools and Fabric Manager, EFCM, provide a GUI simplifies the administration of zoning.
Revision 1008
5-6
Zoning
Zone Aliases
The use of aliases is optional but aids in the understanding of the zoning structure and content:
Naming
May be up to 64 characters Case sensitive

Members
<domain, port> or <domain, area> Node World Wide Name - from nsshow Port World Wide Name - from nsshow or portloginshow
Sample naming conventions
SRV for Server STO for Storage TPE for Tape VRA for Virtual Appliance
SRV_SunHost1 STO_Entprise TPE_Drive1 VRA_Prod2

Zone objects identified by port number or area number are specified as a pair of decimal numbers d,area, where d is the Domain ID of the switch and area is the area number on that switch. If the switch is replaced that is referenced by <domain, port> or <domain, area>, the new switch should be configured with the predecessors Domain ID. If a Domain ID is changed to a new value, all zones that referenced the predecessors domain number will need to be updated with the successors value. Worldwide Names are specified as a 16 digit hexadecimal number separated by colons, for example 10:00:00:90:69:00:00:8a. When node name is used to specify a zone object, all ports on that device are in the zone. When port name is used to specify a zone object, only that single port is in the zone. Zone aliases simplify repetitive entry of zone objects such as port numbers or PWWN. For example, the name Eng could be used as an alias for 10:00:00:80:33:3f:aa:11. An alias is a name assigned to a device or group of devices. By creating an alias, you can assign a familiar name to a device, or you can group multiple devices into a single name. This can simplify cumbersome entries and it allows an intuitive naming structure such as using NT_Storage to define all NT storage ports in the fabric. When a zoned host received the list of network targets (referenced by <domain,port> or PWWN or NWWN) from the Name Server, the host will send a PLOGI request to the destination addresses. If the PLOGI frame is allowed to pass at the egress port and the target at the destination address replies with an accept to the PLOGI request, the Brocade switch and Zoning has completed its responsibility of networking the source and destination. Limiting the amount of LUNs and target IDs that the host can access when the SCSI inquiry command is sent, is the responsibility by the storage provisioning software located at the storage device.
Revision 1008
5-7
Zoning
Zoning Example
1. Plan for your zoning scheme to meet objectives 2. Create Aliases
>alicreate Eng_Host,1,0 >alicreate Eng_Stor,s1wwn; s2wwn >alicreate Mkt_Host,1,16 >alicreate Mkt_Stor,s3wwn; s4wwn; s5wwn
3. Create Zones
>zonecreate Zone_Eng,Eng_Host; Eng_Stor >zonecreate Zone_Mkt,Mkt_Host; Mkt_Stor
4. Create Configuration
>cfgcreate Cfg_EngMkt,Zone_Eng; Zone_Mkt
This example should not be viewed as a best practice but rather an example that shows how a domain, port and WWN would be coded. The CLI is used to illustrate the zoning structure. Once this is understood, the Web Tools GUI would be a better tool to use. Zoning has a very systematic yet simple approach for implementation. Zoning requires prior planning. What are your goals? How will you achieve them? Create members using aliases Create zones using alias members Create a configuration using zones Enable the zone configuration throughout the fabric
Revision 1008
5-8
Zoning
Zoning Example (cont.)

sw300:admin> cfgshow Defined configuration: cfg: Cfg_EngMkt Zone_Eng; Zone_Mkt zone: Zone_Eng Eng_Host; Eng_Stor zone: Zone_Mkt Mkt_Host; Mkt_Stor alias: Eng_Stor 21:00:00:20:37:87:48:e7; 21:00:00:20:37:87:23:e2 alias: Eng_Host 1,0 alias: Mkt_Stor 21:00:00:20:37:87:49:29; 21:00:00:20:37:87:e5:20 ; 21:00:00:20:37:87:20:c5 alias: Mkt_Host 1,16 Effective configuration: no configuration in effect
The command cfgshow displays the defined configuration and since zoning has not been enabled, there is no effective configuration. Zoning is fabric-wide, thus any switch can be used to display the current zoning configurations. The defined configuration is the Zoning Database and contains all zone objects that have been created. It is possible to have several zone configurations but only one can be enabled.
Revision 1008
5-9
Zoning

5. Enable Zoning Configuration >cfgsave >cfgenable "Cfg_EngMkt"
10
Note: cfgenable also saves the defined configuration and the name of the effective configuration to flash memory.
Revision 1008
5 - 10
Zoning

sw300:admin> cfgshow Defined configuration: cfg: Cfg_EngMkt Zone_Eng; Zone_Mkt zone: Zone_Eng Eng_Host; Eng_Stor zone: Zone_Mkt Mkt_Host; Mkt_Stor alias: Eng_Stor 21:00:00:20:37:87:48:e7; 21:00:00:20:37:87:23:e2 alias: Eng_Host 1,0 alias: Mkt_Stor 21:00:00:20:37:87:49:29; 21:00:00:20:37:87:e5:20; 21:00:00:20:37:87:20:c5 alias: Mkt_Host 1,16 Effective configuration: cfg: Cfg_EngMkt zone: Zone_Eng 1,0; 21:00:00:20:37:87:23:e2; 21:00:00:20:37:87:48:e7 zone: Zone_Mkt 1,16; 21:00:00:20:37:87:e5:20; 21:00:00:20:37:87:49:29; 21:00:00:20:37:87:20:c5
11
Since zoning is now enabled, the configuration in effect is displayed.
Revision 1008
5 - 11
Zoning
Enabling Zoning
Only one active zone configuration for entire fabric Enabled with cfgenable
You do not have to disable one zone configuration to enable another Enable one configuration over another Saves the zone configuration (no subsequent cfgsave needed)
Saved across power cycles, reboots Effective zone configuration displayed in switchshow
sw300:admin> switchshow switchName: sw300 switchType: 71.2.0 switchState: Online switchMode: Native switchRole: Principal switchDomain: 1 switchId: fffc01 switchWwn: 10:00:00:05:1e:07:7a:e6 Zoning: ON (Cfg_EngMkt) switchBeacon: OFF
12
A zone configuration is a group of zones that are enforced whenever that zone configuration is enabled. A zone can be included in more than one zone configuration. To define a zone configuration, specify the list of zones to be included and assign a zone configuration name. Zoning may be disabled at any time. When a zone configuration is in effect, all zones that are members of that configuration are in effect. Defined configuration: The complete set of all zone objects that have been defined in the fabric. Effective configuration: A single zone configuration that is currently in effect. The effective configuration is built when an administrator enables a specified zone configuration. This configuration is compiled by checking for undefined zone names, or zone alias names, or other issues. Saved configuration: A copy of the defined configuration plus the name of the effective configuration which is saved in flash memory by the cfgsave command. There may be differences between the saved configuration and the defined configuration if the system administrator has modified any of the zone definitions and has not saved them.
Revision 1008
5 - 12
Zoning
Enabling Zoning (cont.)

sw300:admin> cfgenable "Cfg_EngMkt"
1 2 3
SIDE NOTE
Issuing cfgenable performs an implicit cfgsave Cfg_EngMkt becomes effective configuration Defined configuration is written to flash memory Name of effective configuration is written to flash memory (Cfg_EngMkt)
DEFINED CONFIGURATION EFFECTIVE CONFIGURATION
Cfg_EngMkt Zone_Eng Zone_Mkt
2 3
RAM
Domain 1
Flash Memory
13
Use the cfgenable command to enable a zone configuration. The specified zone configuration is built by checking for undefined zone names, zone alias names, or other inconsistencies by expanding zone aliases, removing duplicate entries, and then installing the current configuration. If the build fails, the previous state is preserved (zoning remains disabled, or the previous configuration remains in effect). If the build succeeds, the new configuration replaces the previous configuration.
Revision 1008
5 - 13
Zoning
Disabling Zoning
sw300:admin> cfgdisable
1 2 3
Effective configuration is disabled Defined configuration is written to flash memory Name of effective configuration set to none in flash memory
RAM
Domain 1
Flash Memory
14
Use the cfgdisable command to disable the current zone configuration. The fabric returns to non-zoning mode, in which all devices see each other. This command clears and commits the current zoning transaction buffer to both volatile and flash memory. If a transaction is open on a different switch in the fabric when this command is run, the transaction on the other switch is automatically aborted. A message is displayed on the other switches to indicate the aborting of the transaction.
Revision 1008
5 - 14
Zoning
Clearing Zoning
sw300:admin> cfgclear
Defined configuration is cleared from RAM
Cfg_EngMkt 1 Zone_Eng Zone_Mkt
SIDE NOTE
cfgclear does not disable the effective configuration and does not save anything to flash memory
.
RAM
Domain 1
Flash Memory
15
Use the cfgclear command to clear all zone information in the defined configuration. All defined zone objects are deleted. If an attempt is made to clear the defined configuration while a zone configuration is enabled, you are warned to first disable the enabled zone configuration. After using the cfgclear command, use the cfgsave command to commit the defined and effective configuration to flash memory for all the switches in the fabric.
Revision 1008
5 - 15
Zoning
Saving Zoning
sw300:admin> cfgsave
Any changes made to the defined configuration before issuing cfgsave Defined configuration is written to flash memory Name of effective configuration is written to flash memory (Cfg_EngMkt)
Cfg_EngMkt Zone_Eng Zone_Mkt xxxx xxxx
2 3
2 1
RAM
Domain 1
Flash Memory
Note: cfgsave does not do a cfgenable
16
Use the cfgsave command to save the current zone configuration. The defined configuration and the name of the enabled configuration are written to flash memory in all switches in the fabric. This allows changes to be made to the defined configuration without immediately enabling of them. The saved configuration is automatically reloaded by the switch on power on and, if a configuration was in effect at the time it was saved, the same configuration is reinstalled with an automatic cfgenable command. Because the saved configuration is reloaded at power on, only valid configurations are saved. The cfgsave command verifies that the enabled configuration is valid by performing the same tests as cfgenable. If the tests fail, an error is displayed and the configuration is not saved. Tests might fail if a configuration has been modified since the last cfgenable. This command ends and commits the current transaction. If a transaction is open on a different switch in the fabric when this command is run, the transaction on the other switch is automatically aborted. A message is displayed on the other switches to indicate the aborting of the transaction. If the defined configuration is larger than the supported maximum zoning database size, the following message is issued: Commit zone DB larger than supported - <zone db size> greater than <max zone db size> Note: A cfgsave does not make any changes to the effective configuration. A cfgenable command is still needed to enable any changes made in the defined configuration.
Revision 1008
5 - 16
Zoning
Maximum Zoning Database Size

Determined by the amount of Flash Memory available for storing the defined configuration Amount varies by Fabric OS release Size displayed with cfgsize command in bytes
Zone DB max size Committed size Transaction size
DEFINED CONFIGURATION
EFFECTIVE CONFIGURATION
RAM
Domain 1
Flash Memory
17
The switch with the lowest max zone database size, typically the switch with the lowest version of Fabric OS, will determine the maximum zoning database size in a fabric . Use the cfgsize command to display the size details of the zone database. The size details include the Zone DB maximum size, the committed size, and the transaction size. All sizes are in bytes. Zone DB max size is the upper limit for the defined configuration, determined by the amount of flash memory available for storing the defined configuration. Committed size is the size of the defined configuration currently stored in flash memory. Transaction size is the size of the uncommitted defined configuration. This value will be nonzero if the defined configuration is being modified, otherwise it is 0. sw300:admin> cfgsize Zone DB max size - 1045274 bytes Available Zone DB size - 1044056 bytes committed - 206 transaction - 0
Revision 1008
5 - 17
Zoning
Maximum Zoning Database Size (cont.)

The switch with the lowest maximum determines the maximum zoning database size for the fabric
18
If a switch attempts to join a fabric that has a zone database size greater than the supported maximum size of the switch, a segmentation error will occur (the request to join the fabric will be rejected) preventing the switch from joining the fabric. FOS levels and Interop modes also effect the maximum zoning database size. Example: if you have a four switch fabric and three of the switches are running FOS v5.2.x and one is running FOS v5.1.x, then the maximum zoning database is 256 KB.
Revision 1008
5 - 18
Zoning
Zone Object Commands

You can use these commands for all zone object types, configuration, zone and alias:
zoneobjectcopy
Copies a zone object to a new zone object

>zoneobjectcopy Cfg_EngMkt, Cfg_Test
zoneobjectrename
Renames a zone object

>zoneobjectrename Zone_Redd, Zone_Red
zoneobjectexpunge
Deletes the zone object and removes it from the member list of all other
objects >zoneobjectexpunge Mkt_Host
19
sw300:admin> cfgshow "*" cfg: USA_cfg Red_zone; White_zone; Blue_zone sw300:admin> zoneobjectcopy "USA_cfg", "UK_cfg" sw300:admin> cfgshow "*" cfg: UK_cfg Red_zone; White_zone; Blue_zone cfg: USA_cfg Red_zone; White_zone; Blue_zone sw300:admin> cfgshow Defined configuration: cfg: USA_cfg Red_zone; White_zone; Blue_zone zone: Blue_zone 1,0; 1,1 zone: Red_zone 1,2; 1,3 zone: White_zone 1,4; 1,5 sw300:admin> zoneobjectexpunge Blue_zone" sw300:admin> cfgshow Defined configuration: cfg: USA_cfg Red_zone; White_zone zone: Red_zone 1,2; 1,3 zone: White_zone 1,4; 1,5
Revision 1008
5 - 19
Zoning
Zoning Display Commands

nsaliasshow
Displays local name server information and the defined configuration aliases to which the device belongs
sw300:admin> nsaliasshow Type Pid COS PortName NodeName TTL(sec) NL 0110e8; 3;21:00:00:20:37:15:15:a2;20:00:00:20:37:15:15:a2; na FC4s: FCP [SEAGATE ST336605FC 0003] Fabric Port Name: 20:10:00:05:1e:04:88:7e Permanent Port Name: 21:00:00:20:37:15:15:a2 Port Index: 16 Share Area: No Device Shared in Other AD: No Redirect: No Aliases: Disk_1 NL 0110ef; 3;21:00:00:20:37:42:66:35;20:00:00:20:37:42:66:35; FC4s: FCP [SEAGATE ST336605FC 0003] Fabric Port Name: 20:10:00:05:1e:04:88:7e Permanent Port Name: 21:00:00:20:37:42:66:35 Port Index: 16 Share Area: No Device Shared in Other AD: No Redirect: No Aliases: The Local Name Server has 2 entries } na
This node has not been defined in any alias
20
This command is a duplicate of the nsshow command with the added feature of displaying the defined configuration aliases that the device belongs to. The message There is no entry in the Local Name Server is displayed if there is no information in this switch, but there still may be devices connected to other switches in the fabric. The command nsallshow shows information from all switches.
Revision 1008
5 - 20
Zoning
Zoning Display Commands (cont.)

nodefind
Displays all the name server entries matching a given WWN, PID (in hex) or alias
sw300:admin> nodefind Disk_1 Local: Type Pid COS PortName NodeName SCR NL 0110e8; 3;21:00:00:20:37:15:15:a2;20:00:00:20:37:15:15:a2; 0 FC4s: FCP [SEAGATE ST336605FC 0003] Fabric Port Name: 20:10:00:05:1e:04:88:7e Permanent Port Name: 21:00:00:20:37:15:15:a2 Device type: Physical Target Port Index: 16 Share Area: No Device Shared in Other AD: No Redirect: No Aliases: Disk_1
21
sw300:admin> nodefind 21:00:00:20:37:15:15:a2 Local: Type Pid SCR COS PortName NodeName
NL 0110e8; 3;21:00:00:20:37:15:15:a2;20:00:00:20:37:15:15:a2; 0 FC4s: FCP [SEAGATE ST336605FC 0003] Fabric Port Name: 20:10:00:05:1e:04:88:7e Permanent Port Name: 21:00:00:20:37:15:15:a2 Device type: Physical Target Port Index: 16 Share Area: No Device Shared in Other AD: No Redirect: No Aliases: Disk_1
Revision 1008
5 - 21
Zoning

nszonemember
Displays the information of all online devices that are zoned with the given device (WWN or PID)
sw300:admin> nszonemember 0x010100 2 local zoned members: Type Pid COS PortName NodeName SCR N 010100; 2,3;10:00:00:00:c9:59:45:9d;20:00:00:00:c9:59:45:9d; 3 FC4s: FCP NodeSymb: [33] "Emulex LP11000 FV2.10A10 DV8.0.16" Fabric Port Name: 20:01:00:05:1e:04:88:7e Permanent Port Name: 10:00:00:00:c9:59:45:9d Device type: Physical Initiator Port Index: 1 Share Area: No Device Shared in Other AD: No Redirect: No NL 0110e8; 3;21:00:00:20:37:15:15:a2;20:00:00:20:37:15:15:a2; 0 FC4s: FCP [SEAGATE ST336605FC 0003] Fabric Port Name: 20:10:00:05:1e:04:88:7e Permanent Port Name: 21:00:00:20:37:15:15:a2 Device type: Physical Target Port Index: 16 Share Area: No Device Shared in Other AD: No Redirect: No No remote zoned members
22
You can also use the WWN: sw300:admin> nszonemember 20:00:00:00:c9:59:45:9d 2 local zoned members: Type Pid COS PortName NodeName SCR N 010100; 2,3;10:00:00:00:c9:59:45:9d;20:00:00:00:c9:59:45:9d; 3 FC4s: FCP NodeSymb: [33] "Emulex LP11000 FV2.10A10 DV8.0.16" Fabric Port Name: 20:01:00:05:1e:04:88:7e Permanent Port Name: 10:00:00:00:c9:59:45:9d Device type: Physical Initiator Port Index: 1 Share Area: No Device Shared in Other AD: No Redirect: No NL 0110e8; 3;21:00:00:20:37:15:15:a2;20:00:00:20:37:15:15:a2; 0 FC4s: FCP [SEAGATE ST336605FC 0003] Fabric Port Name: 20:10:00:05:1e:04:88:7e Permanent Port Name: 21:00:00:20:37:15:15:a2 Device type: Physical Target Port Index: 16 Share Area: No Device Shared in Other AD: No Redirect: No No remote zoned members
Revision 1008
5 - 22
Zoning

nszonemember -u
Displays all unzoned devices in the entire fabric
sw300:admin> nszonemember -u Pid: 0x010200; Pid: 0x010300; Pid: 0x010400; Pid: 0x0110d1; Pid: 0x0110d2; Pid: 0x0110d3; Pid: 0x0110d4; Pid: 0x0110d5; Pid: 0x0110d6; Pid: 0x0110d9; Aliases: Aliases: Aliases: Aliases: Aliases: Aliases: Aliases: Aliases: Aliases: Aliases:
Total of 10 unzoned device(s) in the fabric.
23
Some useful options with nszonemember: -a: Displays each local devices online zoned data, including PID and zone alias. -u: Displays all unzoned devices in the entire fabric.
Revision 1008
5 - 23
Zoning
Additional Zone Management Commands

cfgtransshow
Displays the current zoning transaction information
cfgtransabort
Aborts the current zoning transaction (anything since the last save)
cfgactvshow
Displays the zoning effective configuration
24
sw300:admin> cfgtransshow There is no outstanding zone transactions sw300:admin> cfgclear Do you really want to clear all configurations? (yes, y, no, n): [no] y Clearing All zoning configurations... sw300:admin> cfgtransshow Current transaction token is 271010736 It is abortable sw300:admin> cfgtransabort sw300:admin> cfgactvshow Effective configuration: cfg: Cfg_EngMkt zone: Zone_Eng 1,0; zone: Zone_Mkt 1,16;
21:00:00:20:37:87:23:e2; 21:00:00:20:37:87:48:e7 21:00:00:20:37:87:e5:20; 21:00:00:20:37:87:49:29; 21:00:00:20:37:87:20:c5
Revision 1008
5 - 24
Zoning
Default Zoning
In early versions of Fabric OS, when zoning was not implemented or a cfgdisable command was issued, all devices in the fabric could access each other In Fabric OS v5.1.0+, you can now create a default zone:
Controls what device access is allowed within a fabric when zoning is not enabled Enable all device access with defzone --allaccess (default) Disable all device access with defzone -noaccess
How it works:
When a user-specified zone configuration is not enabled, defzone is in effect When a user-specified zone configuration is enabled, the defzone is overridden
25
The new default zone feature can enable or disable device access within a fabric. Default zones are based on the FC-GS standard, but are not supported when the switch or Director is in interop mode. The defzone -allaccess is the default because it matches how zoning worked prior to Fabric OS v5.1.0.
Revision 1008
5 - 25
Zoning
Default Zoning
defzone
To create a no-access default zone
defzone --noaccess Creates the following (hidden) zone configuration cfgcreate d__efault__Cfg, d__efault__Zone zonecreate d__efault__Zone,00:00:00:00:00:00:00:01
Two underscore characters used in all instances
To create an all-access default zone

defzone --allaccess Does the equivalent of the following zoning commands: cfgdelete d__efault__Cfg zonedelete d__efault__Zone
Changes must be committed to the fabric

Normally a cfgsave will be used A cfgenable or cfgdisable can be used since each includes an implied save
26
The new defzone command configures a default zone configuration and displays the current configuration. The command has no optional parameters, and takes one of three required arguments: --allaccess: Create a default zone that enables all device-to-device access within the fabric. This is the default behavior in Fabric OS v5.1, and matches the default behavior in a non-zoned fabric. --noaccess: Create a default zone that disables all device-to-device access within the fabric. --show: Display the current default zone. Names beginning with d__efault__ are reserved for default zoning use (note: two underscore characters are used in each instance.) Note: The setting of the defzone command is stored in the zoning transaction buffer. Normally, a cfgsave is used to commit the zoning transaction to the entire fabric. A cfgenable or cfgdisable will do the commit since each command does an implied cfgsave. Because the setting is stored in the zoning transaction buffer, a cfgtransabort could be used to abort the defzone command.
Revision 1008
5 - 26
Zoning
Default Zoning
defzone (cont.)
Display the current default zone
sw300:admin> defzone --show Default Zone Access Mode committed - No Access transaction - No Transaction
If Zoning is not enabled, devices in the fabric cannot access each other
27
Revision 1008
5 - 27
Zoning
Default Zoning
defzone (cont.)
On a Fabric OS v5.1.0+ switch, the cfgactvshow and cfgshow commands do not display the default zone or zone configuration On switches running releases earlier than Fabric OS v5.1.0, the d__efault__Cfg and d__efault__Zone can be seen, but not managed. It may be deleted however, so be careful. With defzone set to noaccess, perform all zoning tasks from a switch running Fabric OS v5.1+
A cfgdisable issued from a switch running an earlier version is rejected
28
From a switch running earlier versions of Fabric OS, the zone* commands cannot manage the default zone, and the cfg* commands cannot manage the default zone configuration. You may delete the default zone configuration. For example, attempting to disable d__efault__Cfg on a Fabric OS v5.0.1 switch results in the following error message: RCSRCA_SFC_REJECTED Sfc Was Rejected: Remote Switch Unable To Process.
Revision 1008
5 - 28
Zoning
Default Zoning
defzone (cont.)
When the defzone is configured as noaccess and zoning is disabled, then the cfgshow output on a Fabric OS v5.1.0+ switch is different from a switch with an earlier release
v5.0.1
v6.1.0
sw200E:admin> cfgshow Defined configuration: cfg: d__efault__Cfg d__efault__Zone zone: d__efault__Zone 00:00:00:00:00:00:00:01 Effective configuration: cfg: d__efault__Cfg zone: d__efault__Zone 00:00:00:00:00:00:00:01
sw5100:admin> cfgshow Defined configuration: Effective configuration: no configuration in effect: (No Access)
29
When zoning is not enabled and the default zone is set to no access, the cfgshow output for the v6.1.0 switch will be different from a switch running a release prior to v5.1. See slide above. Use the defzone --show command to determine which mode the default zone is set to (Access or No Access).
sw5100:admin> defzone --show Default Zone Access Mode committed - No Access transaction - No Transaction sw5100:admin> switchshow switchName: sw5100 switchType: 66.0 switchState: Online switchMode: Native switchRole: Subordinate switchDomain: 2 switchId: fffc02 switchWwn: 10:00:00:05:1e:03:a6:6d zoning: ON (No Access) switchBeacon: OFF <truncated output>
Revision 1008
5 - 29
Zoning
Web Tools Zoning Administration
Click here to display the Zone Admin window
30
This screen allows for the creation and modification of aliases, zones and configuration.
Revision 1008
5 - 30
Zoning
Zoning Enforcement
Session Enforcement
Name Server restricts PLOGIs
Hardware Enforcement
Available through ASIC hardware logic checking Denies illegal access from bad citizens1 More secure than session
Enforcement is based on how members in a zone are defined
31
Devices that are Session enforced cause any PLOGIs to the device to be rejected. Devices that are Hardware enforced cause any frames that do not comply with the effective zone configuration to be rejected. This blocking is performed at the transmit side of the port where the destination device is located. This is the highest level of protection for a device. Footnote 1: A bad citizen is best explained by defining good citizens. Good citizens are defined as fabric devices that support RSCNs, query a name server when they receive RSCNs, and only communicate with devices that the name server gives them when they query. Bad citizens do not do one or more of these things. The decision for what enforcement a device receives is based on how the members in a given zone are defined. The table on the next slide describes this process.
Revision 1008
5 - 31
Zoning
Zoning Enforcement (cont.)

Hardware Enforcement
Frame Filter
Session Enforcement
Trap PLOGI Issues reject to initiator
Zone Members All PORTS
2, 4 and 8 Gbit/sec ASICs Hardware
Z1=dom2,port1; dom2,port2 All WWNs Hardware Z2=wwn1; wwn2; wwn3 MIXED Session Z3=dom2,port3; wwn4
32
Hardware Enforced Zoning: Hardware Enforced zoning is used by zones with all members defined by their <domain ID, port> or all members defined by their WWN. This the strongest form of enforcement and will block all frames that compromise the zone from a device that is not a member of a zone, such as a bad citizen. Destination ASIC checks SID on every frame against CAM table entries. Overlapping zones (zone members that appear in two or more zones) are permitted and hardware enforcement will continue as long as the overlapping zones have either all WWNs or <domain ID, port> entries. Using all WWNs in a zone allows for the node to attach to any port in the fabric and have hardware enforcement. Using all <domain, port>/<domain, area> members restricts the movement of devices in the fabric until a zone update is made. Session Enforced Zoning: A session enforced zone is a zoning protection that guarantees that only members of the zone can complete PLOGI/ADISC/PDISC which prevents any unauthorized access by devices that are not a member of the zone. Enforcement to a zone with WWN members and <domain, port> will change from hardware to session enforcement. The ASIC will perform authentication using the name server to compare the SID/DID in the primitive commands with the current zone configuration. If the current zone configuration does not permit the devices to communicate, the switch issues a reject to the SID, effectively blocking communications.
Revision 1008
5 - 32
Zoning
33
Blue Zone: This zone is Hardware enforced because all devices have been specified by WWN. Green Zone: This zone is Hardware enforced because all devices have been specified by Port. Red Zone: This zone is Session enforced because a mix of port and WWN have been specified in the zone. Orange Zone: This zone is also Session enforced because of a mix of port and WWN in the same zone. Note: The Red and Purple Zones also illustrate that the type of device (initiator vs. target) has no bearing on the type of enforcement.
Revision 1008
5 - 33
Zoning
34
The diagram above shows the results of Hardware and Session enforced overlapping zones. The Blue zone is defined with all WWNs (WWN1 and WWN2) and meets the rules for Hardware enforcement. The Purple zone is defined with a mix of port and WWNs and meets the rules for Session enforcement. The target device WWN1 is defined in both zones. When a device is defined in overlapping zones, where one is Hardware enforced and the other is Session enforced, the device will become Session enforced in all zones. What is important to note is the host (WWN2) is still Hardware enforced, even though the target device (WWN1) is now Session enforced. Under these conditions, zoning enforcement is determined at the device level, not the zone level.
Revision 1008
5 - 34
Zoning
Zoning Enforcement Command

portzoneshow
Displays zoning enforcement for each online device port on the local switch
B300:admin> portzoneshow PORT: 0 (0) F-Port Enforcement: HARD PORT PORT: 1 (1) F-Port Enforcement: HARD PORT PORT: 2 (2) Offline <truncated output> B300:admin> portzoneshow PORT: 0 (0) F-Port Enforcement: HARD WWN PORT: 1 (1) F-Port Enforcement: HARD WWN PORT: 2 (2) Offline <truncated output> defaultHard: 0 defaultHard: 1 IFID: 0x43020000 IFID: 0x43020001
defaultHard: 0 defaultHard: 0
IFID: 0x43020000 IFID: 0x43020001
B300:admin> portzoneshow PORT: 0 (0) F-Port Enforcement: SESSION BASED HARD defaultHard: 0 PORT: 1 (1) F-Port Enforcement: SESSION BASED HARD defaultHard: 0 PORT: 2 (2) Offline <truncated output>
IFID: 0x43020000 IFID: 0x43020001
35
Revision 1008
5 - 35
Zoning
Implementation Considerations
Define all members in a zone with <domain,port> or <domain,area>
Provides hardware enforcement Allows devices to communicate that are connected to the ports defined within the zone Requires a zoning change if a device is moved to a port outside the zone No zoning change if the devices WWN changes
Define all members in a zone with their device WWN

Provides hardware enforcement Allows devices to communicate that have their WWN in the same zone Requires a zoning change if the devices WWN changes No zoning change if a device is moved to another port in the fabric
36
These implementation considerations focus on creating zones to achieve Hardware enforcement and identify when zoning changes are needed. Note: Fibre Channel Fast Write zones must use WWN zoning.
Revision 1008
5 - 36
Zoning
Zoning Best Practices

Make all names meaningful Create aliases to easily identify devices Define each zone with a single HBA initiator1 Define zone members with either all domain, area (port number) or all WWNs for hardware enforcement2 Zone from the switch with the highest Fabric OS level Zone from a core switch rather than an edge switch Consider setting default zone to noaccess to prevent any device access when zoning is not enabled Monitor zone database size
37
Footnote 1: Zoning by single Host Bus Adapter (HBA) most closely recreates the original SCSI bus. Each zone created has only one HBA (initiator) in the zone and all the target nodes are members of that zone. Longer than normal boot time can be caused when single initiator to multiple target or multiple initiator to multiple target zones are used. Footnote 2: Defining zone members with either all port numbers or all WWNs provides Hardware enforcement. Setting the default zone to no access when the fabric is first built allows devices to connect to the fabric, do their FLOGI and Name Server update but not access any other device connected to the fabric. This permits the physical connection to be done in one phase and the enabling of a zone configuration to allow access to be done in another phase. Monitor the zone database sizing as new switches are added to the fabric. Newer switches will tend to have a larger maximum size but the fabric may not be able to take advantage of it due to an older FOS running on an existing switch with a lower maximum. With zoning enabled, check the servers to verify they have access to the desired target devices. Also, use the nszonemember command and SAN Health as tools to discover devices that are online but not defined in a zone, etc.
Revision 1008
5 - 37
Zoning
Zoning Best Practices (cont.)

Analyze zones to verify correct devices are communicating
nszonemember fcping Web Tools zone analysis EFCM Fabric Manager SAN Health
Backup with configupload
38
Revision 1008
5 - 38
Zoning
Adding a New Switch to a Zoned Fabric

1. Ensure new switch has no zoning
cfgshow cfgdisable; cfgclear; cfgsave
2. Connect switch to existing fabric 3. Defined and effective configurations are propagated to the new switch
39
A new switch is one that has not previously been connected to a fabric. Before connecting the new switch, check to see if any zoning data exists with the cfgshow command. If it exists, use the cfgdisable, cfgclear, and cfgsave commands to sanitize it. When a new switch is connected to a zoned fabric, all zone configuration data is immediately copied from the zoned fabric into the new switch. If a zone configuration is enabled in the fabric, then the same configuration becomes enabled in the new switch. After this operation, the cfgshow command displays the same output on all switches in the fabric, including the new switch.
Revision 1008
5 - 39
Zoning
Ensure New Switch Has No Zoning
sw300:admin> cfgshow Defined configuration: no configuration defined Effective configuration: no configuration in effect
RAM
Domain 2
Flash Memory
40
The cfgshow command displays the status of the defined and effective configurations on a new switch.
Revision 1008
5 - 40
Zoning
Connect New Switch to Existing Fabric
EFECTIVE CONFIGURATION
ISL
Propagate Definitions
RAM
Domain 1
Flash Memory
RAM
Domain 2
Flash Memory
41
The defined and effective configurations from the existing fabric are propagated to the new switch.
Revision 1008
5 - 41
Zoning
Merging Two Zoned Fabrics: Segmentation

Segmentation due to: Configuration mismatch Description Occurs when zoning is enabled in both fabrics and the effective configurations are different. Occurs when the name of a zone object in one fabric is also used for a different type of zone object in the other fabric. Fabric A: alias: Mkt_Host 1,16 Fabric B: zone: Mkt_Host 1,16 Occurs when the name and type of a zone object in one fabric is also used in the other fabric but the content or order is different. Fabric A: alias: Eng_Stor wwn2; wwn1 Fabric B: alias: Eng_Stor wwn1; wwn2
Type mismatch
Content mismatch
An [FABR-1001] message will occur stating there is a zoning conflict
42
If the zoning changes are not done correctly, it is possible to have the merging of the fabrics fail due to a segmentation error. The table above shows the three possible mismatches that would cause this condition. The actual error would be an FABR-1001 error.
Revision 1008
5 - 42
Zoning
Other Merge Fabric Considerations

Use Fabric Manager Fabric Merge Check
Checks each fabric for:
Duplicate Domain IDs Incompatible fabric.ops switch configuration settings Any zoning mismatch conditions
Check before you connect!
43
Use Fabric Manager to invoke the Fabric Merge Check. This function allows the comparing of two fabrics and their settings that could cause a fabric segmentation error. It is best to check before you connect.
Revision 1008
5 - 43
Zoning
Fabric Manager Fabric Merge Check
44
The Fabric Merge Check is under the Tools pull-down menu.
Revision 1008
5 - 44
Zoning
Fabric Selection
Fabric Compare
Retrieving and comparing configuration Info
Retrieving and Comparing Zoning Data
45
In preparation for the merge check, two fabrics will be selected. In the example above, fabric-sw51 and fabric-RSL1_BRCD47 have been selected for a check. Once the fabrics have been selected you can select the Check button to extract the elements from each fabric for comparison.
Revision 1008
5 - 45
Zoning
Merging Check Results Successful
46
At the end of the process, a Merge Check Results pop-up window will be displayed. To validate all of the compared results, you can select the up and down buttons to the right to display any identified mismatches.
Revision 1008
5 - 46
Zoning
Other Zoning Tools

SAN Health
Creates zoning tables to quickly compare for differences Highlights hanging zones (zones with defined devices that arent logged into the Name Server)
Highlights unzoned devices (devices logged into the Name Server that arent defined in a zone) Quick check of zoning metrics on Summary tab to see if one is nearing the capacity of zone database
47
SAN Health is a very good tool for cleaning up a zoning database.
Revision 1008
5 - 47
Zoning
SAN Health Sample Zoning Spreadsheet
48
Revision 1008
5 - 48
Zoning
Summary
Zoning logically separates the Fabric into subsets Single HBA zoning is a good practice Hardware enforcement denies illegal access from bad citizen HBAs Session enforcement restricts PLOGIs A default zone can be set to control what device access is allowed within a fabric when zoning is not enabled Sanitize new switch before connecting to existing fabric
49
Revision 1008
5 - 49
Zoning

End of Instructor-Led Module 5 Zoning
Revision 1008
Revision 1008
5 - 50
Fibre Channel Routing

Instructor-Led Module 6 FSPF Routing
Revision 1008
Revision 1008
6-1
Objectives
After completing this module, attendees will be able to describe:
The concept of Fibre Channel (Layer Two) routing ISL functionally The FSPF routing protocol
Describe Routing Metrics Describe Dynamic Load Sharing Describe In Order Delivery
Exchange-based Routing Port-based Routing Requirements for the Trunking feature Routing and Trunking interoperability features on the B-Series and M-Series products
Revision 1008
6-2
Routing Overview
Fabric Routing is logic used by a switch to pass frames from the source domain toward the destination domain This module focuses on Fabric Shortest Path First (FSPF) routing
Also referred to as Layer 2 routing
There is a separate discussion on FC-to-FC routing

Also referred to as Layer 3 routing
The SAN A is comprised of many interconnected switches. Each switch has a unique domain ID within the fabric. Hosts (represented by the computer icon) are able to connect to storage (represented by the disk icon) across switches within SAN A. FC-FC Routing is performed by a Fibre Channel router, to pass frames from a source device in one fabric toward a destination device in another fabric without merging the fabrics. FC-FC Routing will be described in another course.
Revision 1008
6-3
Fabric Terminology
Inter-Switch Links (ISLs)
E_Port-to-E_Port links Communicates using Class F service
Inter-Chassis Link (ICL)

A licensed feature that provides specialized grouping of ISLs that connects two DCX Backbones Provides up to 512 Gbit/sec bandwidth Uses special cabling
Revision 1008
6-4
Fabric Terminology (cont.)

Principal Switch
Selected when the fabric initializes, before routing is established Manages the assignment of unique Domain IDs Provides time synchronization to all other switches in the fabric
Principal ISL
ISL used to communicate between the Principal Switch and other switches in the fabric
Revision 1008
6-5
Principal Switch Path
Ports in the example depicted on this slide are numbered as depicted here: These are the functions of a Principal Switch: Ensure unique domain IDs throughout the fabric Synchronize time throughout the fabric
Port 1 Port 0 Port 4 Port 5 Port 7
FSPF uses several frames to perform its functions. Since it may run before fabric routing is set up, FSPF does not use the routing tables to propagate the frames, but floods the frames throughout the fabric hop-by-hop. At the beginning, frames are flooded on all the Inter-Switch Links (ISLs); as the protocol progresses, it builds a spanning tree rooted on the Principal Switch. Frames are then sent only on the ISLs that belong to the spanning tree. These ISLs are called Principal ISLs. Where there are multiple ISLs between switches, the first ISL to respond to connection requests becomes the Principal ISL. Only one ISL from each switch will be used as the Principal ISL. For example, there is an ISL between Domain 3 and Domain 6, but it was not chosen as part of the principal ISL paths. This ISL is still used for frames routed through the fabric. Issuing a switchshow command from Domain 3 would indicate the following summarized output: Port 1 Port 3 Port 6 Port 7 E_Port E_Port E_Port E_Port (Downstream) (Downstream) (Upstream)
Upstream means going out that E_Port is going toward the Principal Switch. Downstream means going out that E_Port is going away from the Principal Switch.
Revision 1008
6-6
Principal Switch Commands

Set the preferred Principal Switch priority using: fabricprincipal [-f] 1 (v4.x+)
-f forces a fabric rebuild after enabling on a switch
The selection process is as follows:

Check the priority setting on all switches If only one is set, it becomes the Principal Switch If more than one is set, the switch with the lowest WWN of these will be the Principal Switch If priority is equal, the switch with the lowest WWN becomes Principal Switch
RSL_SWT121:admin> fabricshow Switch ID Worldwide Name Enet IP Addr FC IP Addr Name ------------------------------------------------------------------------2: fffc02 10:00:00:60:69:80:04:5e 10.255.255.121 0.0.0.0 "RSL_SWT121" 129: fffc81 10:00:00:60:69:80:05:1c 10.255.255.129 0.0.0.0 "RSL_SWT129" 153: fffc99 10:00:00:60:69:50:0d:d6 10.255.255.153 0.0.0.0 >"RSL_SWT153" 157: fffc9d 10:00:00:60:69:51:2d:57 10.255.255.157 0.0.0.0 "RSL_SWT157"
Fabric OS v4.1+ gives the SAN administrator the ability to set which switch will be the principal with the fabricprincipal command. However, if fabricprincipal is set on more than one switch, the one with the lowest WWN will become the Principal. fabricprincipal [-fhq][mode] Use this command to set Principal Switch selection mode for the switch. The implementation of the fabricprincipal command is based solely on mechanisms specified in the Fibre Channel standards. These mechanisms provide a preference for a switch requesting to be the Principal Switch in a fabric, but they do not provide an absolute guarantee that a switch requesting to be the Principal Switch will actually achieve this status. When dealing with larger fabrics, the selection of the Principal Switch is less deterministic. In these cases, to help ensure that the desired switch is selected as the Principal Switch, a small selection of switches should be connected together first, followed by the addition of the rest of the fabric. Operands This command has the following operands: -f: force a fabric rebuild. This option is required when enabling Principal Switch mode. This option is not valid with the disabling Principal Switch mode. -h: display command usage summary. -q: display the current mode state. mode: Specify 1 to enable, or 0 to disable Principal Switch mode (the mode activates when the fabric rebuilds). This operand is optional.
Revision 1008
6-7
Routing Terminology
Over-subscription
The possible contention for bandwidth by devices through an ISL
Congestion
The actual contention for bandwidth by devices through an ISL
ISL over-subscription ratio

By port count
Easier to calculate, less accurate (# of Device ports) to the (# of E_Ports) expressed as 15:1, 7:1, 3:1
By bandwidth
More difficult to calculate, more accurate (Sum of Ingress ports bandwidth) to the (Sum of ISL Egress route
ports bandwidth) expressed as 7:1, 3:1
Inter-Switch Links (ISLs) can seamlessly extend a single switch fabric, and they can provide resiliency and failover. When interconnecting two switches, a normal cable is used. When an ISL is used to connect a switch, an E_Port and buffer allocation are configured for inter-switch communication depending on the long distance setting. Locality: Reducing transmission latency by physically placing hosts and storage that communicate frequently such that no routing (same switch) or minimal routing (adjacent switches) is required Easier to manage in small SANs than large or growing SANs Easier to manage in Directors
Revision 1008
6-8
Over-Subscribing ISLs
What is the over-subscription ratio?
In the example above: Six servers have negotiated a total of 40 Gb/s and can share the one 8 Gb/s ISL. This becomes a 40:8 oversubscription ratio which is placed on that ISL. This value can be displayed with the TopologyShow command on SW1 switch. Several reasons why over-subscription is a valid topology are due to certain considerations: 1. The servers or targets do not request and respond at the same time. 2. When they do communicate, they generally do not transmit at full bandwidth. 3. Most important, Brocade implements a frame prioritization scheme into the hardware, known as Virtual Channels, that acts like a traffic manager. This will allow all frame types to get a chance to send the frame through the ISL. We will discuss VCs in the next few slides. Depending on the inter-switch communications load, the ISL over-subscription ratio can vary. ISL over-subscription is the ratio of source device ports to E_Ports. Calculated as a fraction, and displayed as a percentage of demand placed on the ISL. Thirty-two hosts negotiating 4 Gb/s per port and one 8 Gb/s E_Port is 15:1 The over-subscription ratio can vary, depending on the communications on the switch. When the performance expectations are demanding, then Brocade recommends: 3:1 over-subscription ratio 7:1 and even 15:1 are common
Revision 1008
6-9
Virtual Channels
Virtual Channels (VC) are unique to Brocade and are not covered by the T11 Fibre Channel specification
Virtual Channels are buffer credit queues used per ISL It is the use of virtual channels that allow interleaving of frames on inter-switch links for non-blocking routes
Virtual Channels are divided into 3 priority groups:

P1 is the highest which is used for Class F; F_RJT and ACK P2 is the next highest which is used for Data frames
The data VC channels can be further prioritized to provide higher

levels of Quality of Service1 P3 is the lowest which is used for broadcast and multicast traffic
10
Footnote 1: Applications used to help provide Quality of Service on ISLs include: QoS Zoning Ingress Rate Limiting These application will be discussed in other modules.
Revision 1008
6 - 10
Virtual Channels 1/2/4 Gbit/s ASICs

Eight Virtual Channels (queues) are used per ISL
VC0 Link Control Frames (Class F) VC1 Class Two ACKS and Link Control VC2 thru VC5 Data Frames VC6 Multicast Traffic VC7 Broadcast Traffic
11
An ISL is divided into 8 virtual channels (0 - 7) which are in 3 groups, defined by priority: P1 P3 P1, is the highest within VC 0 and VC 1 P2 has VC 2,3,4,5 P3 has VC 6 and 7 The four data VC channels are used to multiplex data based upon the destination port number on a switch. Using the lowest two bits, port numbers that have a binary value of 0000 are sent across VC2. Port numbers that have a binary value of 0001 are sent across VC3. Port numbers that have a binary value of 0010 are sent across VC4. Port numbers that have a binary value of 0011 are sent across VC5.
Revision 1008
6 - 11
Virtual Channels 8 Gbit/sec ASICs

VC0 VC1 VC2 thru VC5 VC6 VC7 VC8 and VC9 VC10 thru VC14 VC15 Link Control Frames Class Two ACKS and Link Control Medium Priority QoS Data Frames Multicast Traffic Broadcast Traffic Low Priority QoS Data Frames High Priority QoS Data Frames Reserved
12
An ISL is divided into 16 virtual channels (0 - 15) which are in 5 groups, defined by priority: P1 P5 P1 is the highest within VC 0 and VC 1 P2 has VC 10,11,12,13,14 P3 has VC 2,3,4,5 P4 has VC 8,9 P5 has VC 6 and 7 The seven data VC channels (VC8-14) are used to multiplex data frames based upon QoS Zones. High is identified as a QOSH_* zone name Low is identified as a QOSL_* zone name When QoS zones are not configured devices will use VC2-5 for data transfers.
Revision 1008
6 - 12
Fabric Shortest Path First (FSPF)

Authored by Brocade and now a FC-SW-2 standard Calculates minimum cost path from switch-to-switch Downloads route tables to the ASICs ASIC routes switch-to-switch frames using lowest cumulative costs of all available traversed links Path vs. Route
Paths (possible ways to get from one switch to another)
Each ISL has a metric cost Cumulative cost based upon sum of all costs of all traversed ISLs
Routes (the path chosen)
Paths selected for routing database are the ones with the lowest cost Dynamic Load Sharing (DLS) will assign data across equal cost routes in relation to
the ratio of available bandwidth across ISLs
In-order Delivery (IOD) will insure proper delivery of frames if ingress ports have to be
re-routed through a different ISL due to a topology change
13
The FC-SW-2 standard for Storage Area Networks (SANs) uses an algorithm called Fabric Shortest Path First (FSPF). FSPF is a link state path selection protocol and directs traffic along the shortest path between the source and destination, based upon the link cost, and makes it possible to detect link failures, determine shortest route for traffic, update the routing table, provide fixed routing paths within a fabric, and maintains correct ordering of frames. FSPF keeps track of the state of the links on all switches in the Fabric and associates a cost with each link. The protocol computes paths from a switch to all the other switches in the fabric by adding the cost of all links traversed by the path, and chooses the path that minimizes the costs. This collection of the link states (including costs) of all the switches in the fabric constitutes the topology database (or link state database). Once established, FSPF programs the hardware routing tables for all active ports on the switch. FSPF is not involved in frame switching. There are two types of primary routing protocols in intranet networks, Distance Vector and Link State: Distance Vector is based on hop count. This is the number of switches you traverse through to get from the source domain (switch) to the destination domain (switch). Link State is based on a metric value based on a cost. The cost could be based on bandwidth. FSPF makes minimal use of the ISLs bandwidth, leaving virtually all of it available for traffic. In a stable fabric, a Brocade switch will transmit 64 bytes every 20 seconds in each direction. FSPF frames have the highest priority in the fabric. This guarantees that a control frame is not delayed by user data and that FSPF routing decisions occur very quickly during convergence. FSPF guarantees a routing loop free topology at all times. Why is this important? It is essential for a fabric to include many physical loops, because without loops there would be no multiple path between switches, and therefore no redundancy. Without multiple paths, if a links goes down part of the fabric becomes isolated. FSPF ensures that the topology is loop free and that the frame will never be forwarded over the same ISL more than once. Brocade recommends no more than 7 hops between two switches. This limit is not required or enforced by FSPF. Its purpose is to ensure that a frame will never be delivered to a destination after E_D_TOV has expired. Brocade switches support unicast, multicast and broadcast traffic. For unicast traffic, the Brocade supports both Class 2 and Class 3, whereas broadcast and multicast, as defined in Fibre Channel standards, are supported in Class 3 only. The Brocade ASIC does all frame forwarding (unicast, multicast, and broadcast) in hardware. Revision 1008 6 - 13
FSPF Link Cost

Metric value assigned to the Tx sides on each ISL The Brocade default Link Cost value is 500 for 2, 4, 8, and 10 Gbit/sec links1 Also known as a Hop Ports 2 and 5 have a cost of 500 from Domain 1 to Domain 3 Port 6 has a cost of 1000 from D1 to D3 Shortest routes are Ports 2 and 5 FSPF will set routing table in Domain 1 to only use these routes for frames with a destination of Domain 3
14
Fabric Shortest Path First calculates paths based on the destination domain ID. The Fabric Protocol must complete domain ID assignments before routing can begin. ISLs provide the physical pathway when the Source ID (SID) address has a frame destined to a port on a remote switch Destination ID (DID). When an ISL is attached/removed to a switch, Fabric Shortest Path First (FSPF) will update the route tables to reflect the addition/deletion of the new routes. As each host transmits a frame to the switch, the switch will read the SID and DID in the frame header. If the domain ID of the destination address is the same as the switch (intra-switch communications), the frame buffer is copied to the destination port and a credit (R_RDY) is sent to the host. Brocade only needs to read word zero and word one of the Fibre Channel frame to perform what is known as cut-through routing. A frame may begin to emerge from the output port before it has been entirely received by the input port. The entire frame does not need to be buffered in the switch. If the destination domain ID is different than the source domain ID (inter-switch communications), then the switch will consult the FSPF route table to identify which local E_Port will provide the Fabric Shortest Path First to the remote domain, as discussed on the previous slide. In the example above, looking at FSPF from Domain 1, all ports set to 8 Gbit/sec: D1 to D3 (Paths) Port2 = 500* Port5 = 500* Port6 = 1000 D1 to D2 (Paths) Port2 = 1000 Port5 = 1000 Port6 = 500* Routes = * Dynamic Load Sharing can be managed with the LinkCost command. Static routes can be assigned using the UrouteConfig command or Traffic Isolation application. Footnote 1: Any ISL set to 1G/s will force a metric cost of 1000.
Revision 1008
6 - 14
FSPF Dynamic Load Sharing (DLS)

In this example, all ports are set to 8 Gbit/sec and the three servers will communicate to Domain 3
As the server ports come online, FSPF will channel devices equally across available routes
Domain 1 1 3 4 5
2
50 0
500
50 0
1 11
500
6 1
7 8 9 10
Domain 2
Domain 3
15
Dynamic Load Sharing is a standard in Fibre Channel to share multiple available routes to a destination domain. If multiple routes exist in the routing table, FSPF will DLS according to the ratio of bandwidth available on the routes. The Exchange-based Routing Policy depends on the Fabric OS Dynamic Load Sharing feature (DLS) for dynamic routing path selection. When using the Exchange-based Routing Policy, DLS is by default enabled and cannot be disabled. In other words, you cannot enable or disable DLS when the Exchange-based Routing Policy is in effect. When the Port-based Policy is in force, you can enable DLS to optimize routing. DLS recomputes load sharing when a switch boots up, an E_Port/EX-port goes offline and online, an Nx_Port goes offline. In the example above, looking at FSPF from Domain 1: Paths and routes are created for both Domain 2 and Domain 3 using ports 2, 5, and 6. D1 to D3 (Paths) Port2 = 500* Port5 = 500* Port6 = 1000 Routes have a 1:1 DLS ratio D1 to D2 (Paths) Port2 = 1000 Port5 = 1000 Port6 = 500* Routes = *
Revision 1008
6 - 15
FSPF In-Order Delivery

In this example, Server 2 is removed from the fabric FSPF determines there are two devices on port 5 vs. port 2 has none IOD insures the routing tables is current in 150ms, holds Server 3 for 500ms and re-channels Server 3 to port 2 Holding for 500ms insures old frames on port 5, from Server 3, reach the target before new frames on port 2. If more time is needed for IOD, the timing settings can be managed
Domain 1 1 3 4 5
2
50 0
500
50 0
1 11
500
6 1
7 8 9 10
Domain 2
Domain 3
16
In a stable fabric, frames are always delivered in order, even when the traffic between switches is shared among multiple paths. However, when topology changes occur in the fabric and DLS is on (DLSSet), traffic is rerouted around the change, and some frames could be delivered out of order. Most destination devices tolerate out- of-order delivery, but some do not. By default, out-of-order framebased delivery is allowed, (IODreset) to minimize the number of frames dropped. IODreset = 650ms Convergence time Enabling in-order delivery (IODset) guarantees that frames are either delivered in order or dropped. You should only force in-order frame delivery across topology changes if the fabric contains destination devices that cannot tolerate occasional out-of-order frame delivery. IODset = e_d_TOV parameter (2000ms) This value can be changed by issuing the switchdisable;configure command. This allows you to increase the value of the e_d_TOV parameter. Be aware if you change this value, you must do the same to the rest of the switches, as this is considered a fabric.ops parameter. By default IOD is = IODreset or off You can disable the switches ability to perform this re-routing mechanism by using DLSreset.
Revision 1008
6 - 16
Routing Policies
The routing policy is unidirectional and responsible for selecting a route based on one of two user-selected routing policies:
Exchange-based Routing Port-based Routing
Each switch has its own routing policy Different policies can exist in the same fabric 2 Gbit/sec ASIC routing is handled by the Fabric Shortest Path First (FSPF) protocol and uses only the Port-based Routing Policy 4/8 Gbit/sec ASICs use the FSPF protocol and either Port-based Routing or Exchange-based Routing
Exchange-based is default
17
Revision 1008
6 - 17
Exchange-based Routing
With this policy, the path chosen for an ingress frame is based on:
1. FC address of the source fabric device (SID) for this frame 2. FC address of the destination fabric device (DID) for this frame 3. FC Originator Exchange ID (OXID) for this frame
18
Revision 1008
6 - 18
Exchange-based Routing and DLS

DLS allocates a route from shortest equivalent paths based on link cost and bandwidth
19
The Link State Database calculates the cost of each link and determines the lowest cost path within each switch. The input port from the source is assigned to all output ports toward the destination (Dynamic Route Selection). Exchanges are allocated via round-robin assignment Chosen routes are used regardless of whether or not other devices in the fabric go offline or fabric changes occur Changes in fabric, when Dynamic Load Sharing is enabled (DLSset), causes FSPF to recalculate the Dynamic distribution of exchanges to the remaining output ports to continue to distribute devices across equal cost routes. DLS and In-Order Delivery (IOD) options DLS cannot be disabled when using Exchange-based Routing. When DLS is enabled (DLSSet), routes are assigned as devices come online, but if something in the fabric changes (local E_Port goes up/down, local Fx_Port goes up/down) and DLS is set, exchanges are re-hashed across the remaining routes.
Revision 1008
6 - 19
Exchange-based Routing and DLS (cont.)

DLS allocates a route from shortest equivalent paths based on link cost and bandwidth A single device shares two equal cost Paths established through the Fabric:
Black Path is a 8 Gbit/sec ISL Blue Path is a 4 Gbit/sec ISL Ratio for DLS Distribution is 2:1
20
Exchange-based Routing Policy With this policy, the path chosen for an ingress frame is based on: FC address of the source fabric device (SID) for this frame FC address of the destination fabric device (DID) for this frame FC Originator Exchange ID (OXID) for this frame. This policy allows for optimal utilization of the available paths as I/O traffic between different (SID, DID, OXID) pairs can use different paths. All frames received on a ingress port (in port) with the same (SID, DID, OXID) parameters will take the same path unless there is a fabric event. Just like port-based frame allocation when there are multiple paths to a destination, the ingress traffic will be distributed across the different paths in proportion to the bandwidth available on each of the paths. This improves utilization of the available paths, thus reducing possible congestion on the paths. Every time there is a change in the network (which changes the available paths), the ingress traffic can be redistributed across the available paths. This is a very easy and nondisruptive process when exchange-based policies are engaged.
Revision 1008
6 - 20
Port-based Routing
The Link State algorithm calculates the cost of each link and determines the lowest cost path within each switch The input port from the source is assigned to an output port toward the destination switch (a route) Routes are allocated via round-robin assignment Chosen routes are used until one of the devices in the fabric goes offline or the fabric changes Changes in fabric, when Dynamic Load Sharing is enabled (DLSset), cause FSPF to recalculate the routes and may reassign the output port to better distribute devices across equal cost routes
21
With Port-based Routing, DLS is an option. With Exchanged-based Routing, DLS must be enabled.
Revision 1008
6 - 21
Port-based Routing and DLS

DLS allocates a route from shortest equivalent paths based on link cost and bandwidth 9 devices need to share 2 Paths established through the Fabric:
Blue Path is a 8 Gbit/sec ISL w/ 500 metric cost Red Path is a 4 Gbit/sec ISL w/ 500 metric cost Ratio for DLS Distribution is 2:1
22
Source ID and destination domain are used to allocate routes Devices are round-robin allocated to available equal cost routes DLS will round-robin according to aggregate bandwidth assigned to the route It is possible to have congestion, if too many high I/O requiring devices are allocated to a single route. It is possible with round-robin assignment of the in-ports to overload a particular route. If the aggregate throughput does not exceed the bandwidth of any one ISL, then this is not an issue.
Revision 1008
6 - 22
Routing Policy Selection

The aptpolicy1 command is used to change routing policies
r7-st01-b30-1:admin> aptpolicy Current Policy: 3 3: Default Policy 1: Port Based Routing Policy 3: Exchange Based Routing Policy
The switch must be disabled before changing the Routing policies
23
Footnote 1: apt stands for advanced performance tuning The default routing policy (Exchange-based Routing) can be changed to Port-based Routing Policy. When the switch is enabled, aptpolicy can be used to display the new current policy:
Revision 1008
6 - 23
Display Routing Information Overview

Paths and link connections can be verified using:
switchshow, fcping, and pathinfo
The FSPF protocol determines routing on a local basis. Fabric topology information is known at every switch and displayed with:
urouteshow, topologyshow, and other routehelp commands
No global, edge-to-edge routing table is maintained For a listing of route-related commands use routehelp
24
routehelp: bcastShow disreset dlsSet dlsShow fspfShow interfaceShow iodReset iodSet iodShow linkCost LSDbShow nbrStateShow nbrStatsClear topologyShow uRouteConfig uRouteRemove uRouteShow Print broadcast tree information Turn off Dynamic Load Sharing Turn on Dynamic Load Sharing Print state of Dynamic Load Sharing Print FSPF global information Print FSPF interface information Turn off In-Order Delivery Turn on In-Order Delivery Print state of In-Order Delivery Set or print the FSPF cost of a link Print Link State Database entry Print neighbor's summary information Reset FSPF neighbor's counters Print paths to domain(s) Configure static unicast route Remove static unicast route Print port's unicast routing info
Revision 1008
6 - 24
Display Routing Information fcping

Used in Fibre Channel networks to verify connectivity Sends a Fibre Channel Extended Link Service (ELS) Echo request to two N_Ports/NL_Ports (source and destination) Source and destination port identifiers can be specified as:
24-bit Fibre Channel address (0x0a0100) Port World Wide Names Node World Wide Names
Can use this command to perform a zoning check between the source and destination
25
The ELS Echo request elicits an ELS Echo response from a port identifier in the fabric and is useful for validating link connectivity. The two port identifiers are then used to determine if the identifiers are zoned together. The ELS Echo requests comprise a 24-byte Fibre Channel frame header, a 4-byte ELS Echo request header, an 8-byte timestamp from gettimeofday, and an arbitrary number of bytes as specified by l length to fill out the request frame. The source identifier in the ELS Echo request is the domain controller and the destination identifier is either source or destination. Note: The ELS Echo might not be supported on all devices. If this is due to the device not supporting this ELS Echo request, the message returned may not point directly to the source or destination device. In such cases, the response could be either an ELS reject or a request timeout. Do not assume that the device is not FC connected until you have looked at the results from pathinfo and nsallshow. By default, fcping sends five ELS Echo requests to each port.
Revision 1008
6 - 25
fcping Example
Sends Fibre Channel Extended Link Service (ELS) Echo requests to a pair of ports. This checks zoning and link connectivity. Example from source 1 to destination 4 using 24-bit address port identifier (PID): SW10:admin>fcping 0x0a0100 0x140800
26
b41_1:admin> fcping 10:00:00:00:c9:2a:a3:9e 21:00:00:20:37:e1:42:40 Source: Destination: Zone Check: 10:00:00:00:c9:2a:a3:9e 21:00:00:20:37:e1:42:40 Zoned
Pinging 10:00:00:00:c9:2a:a3:9e [0xa0100] with 12 bytes of data: received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:694 usec received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:664 usec received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:665 usec received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:662 usec received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:531 usec 5 frames sent, 5 frames received, 0 frames rejected, 0 frames timeout Round-trip min/avg/max = 531/643/694 usec Pinging 21:00:00:20:37:e1:42:40 [0x1400e2] with 12 bytes of data: Request timed out Request timed out Request timed out Request timed out Request timed out 5 frames sent, 0 frames received, 0 frames rejected, 5 frames timeout Round-trip min/avg/max = 0/0/0 usec
This device may be offline or may not support ELS Echo frames; nsallshow will display the device if it is online. The device could also reject the request.
Revision 1008
6 - 26
Display Routing Information pathinfo

You can display routing and statistics information between a source port and a destination port with the pathinfo command Gathers information on a specific data path, not the entire fabric Can track the forward path or the reverse path Can specify a sequence of domain,port pairs through which the path must pass (may not be the same path as the route) For each hop, can display the following information:
Routing data - Hop count, input port, switch domain ID and name, output port, bandwidth of output ISL, link cost Basic statistics data - Bytes/second, % of time with zero TX credits Extended statistics - Frames/second, total words, frames, and errors
27
Provides traceroute for SAN functionality. pathinfo determines edge-to-edge routing between source port and destination port. pathinfo provides information in addition to routing: Destination port state Link statistics for every hop from source to destination Link utilization for each hop from source to destination pathinfo is intended to gather information on a specific data stream, not the entire fabric. pathinfo is most useful when all the switches in the fabric are configured for Portbased Routing Policy. The pathinfo command displays routing and statistics information that can aid the optimization of edge-to-edge routing. The pathinfo command is supported in Fabric OS v2.6.2, v3.1.2, and v4.2+. All switches in the desired data path must support the pathinfo command. The percentage of time with zero Tx credits value (Txcrdz) are displayed as a basic statistic to indicate downstream congestion.
If Exchanged-based Policy is in effect, subsequent exchanges might not take the same path as displayed in the pathinfo output.
For more information on pathinfo, see the Fabric OS Command Reference Manual.
Revision 1008
6 - 27
Display Routing Information pathinfo (cont.)

Example pathinfo output showing forward and reverse paths from local domain 1, port 1(Initator) to domain 2, port 6 (Target)
r7-st01-b30-1:admin> pathinfo 2 1 6 r Target port is F_Port Hop In Port Domain ID (Name) Out Port BW Cost ----------------------------------------------------------------------------0 1 1 (r7-st01-b30-1) 16 8G 500 1 3 2 (r7-st01-b53-1) 6 2G 500 Reverse path 2 6 2 (r7-st01-b53-1) 3 8G 500 3 16 1 (r7-st01-b30-1) 1 4G 500
28
Command syntax: pathinfo [domain [src_portnumber [dst_portnumber]] [-r]]
Revision 1008
6 - 28
Display Routing Information topologyshow

Exchange-based Routing
rsl1_st11_b41_1:admin> topologyshow 2 domains in the fabric; Local Domain ID: 20 Domain: 10 Metric: 500 Name: rsl1_st11_b20_1 Path Count: 2 Hops: 1 Out Port: 0 In Ports: 8,9,10,11,12,13,14 Total Bandwidth: 8.000 Gbps Bandwidth Demand: 700% Flags: D 2 domains in the fabric; Local Domain ID: 20 Domain: 10 Metric: 500 Name: rsl1_st11_b20_1 Path Count: 2 Hops: 1 Out Port: 1 In Ports: 8,9,10,11,12,13,14 Total Bandwidth: 8.000 Gbps Bandwidth Demand: 700% Flags: D
29
topologyshow will list all domains that are part of the fabric, and to each of those domains, all possible paths from the local switch. For each path - cost, the number of hops from the local switch to the destination switch, and a summary of all ports are routed through that path. A path is described by the output port that a frame addressed to a certain domain will be forwarded to by the switchs routing hardware, in order to reach the domain. With the Domain ID specified, this command displays the topology information for the specified destination domain. Total Bandwidth specifies the total available bandwidth. Bandwidth Demand displays the possible maximum utilization. In the above example you have two equal cost paths with 8G total available bandwidth for each path: The available bandwidth of each input device is 8 Gbit/sec. The Bandwidth Demand percentage is Bandwidth In/Bandwidth Out. In this example: Path1=56 Gbit/sec over 8 Gbit/sec is 56/8 = 7 which equals 700% Path2=56 Gbit/sec over 8 Gbit/sec is 56/8 = 7 which equals 700% The D flag indicates that Dynamic Load Sharing (DLS) is enabled, this is the default and only available setting for Exchanged-based Routing. Under Exchange-based Routing, FSPF will program all In-port to all available Out Port routes. In this scenario, the Bandwidth Demand on each ISL is equal to each. Over time each path will be utilized equally by spreading exchanges across each path. Theoretically you can divide the 700% by 2, which gives you a Bandwidth Demand of 350% or 3.5:1 for each link. Revision 1008 6 - 29
Display Routing Information topologyshow

Port-based Routing
rsl1_st11_b41_1:admin> topologyshow 2 domains in the fabric; Local Domain ID: 20 Domain: 10 Metric: 500 Name: rsl1_st11_b20_1 Path Count: 2 Hops: 1 Out Port: 0 In Ports: 8,10,12,14 Total Bandwidth: 8.000 Gbps Bandwidth Demand: 400% Flags: D 2 domains in the fabric; Local Domain ID: 20 Domain: 10 Metric: 500 Name: rsl1_st11_b20_1 Path Count: 2 Hops: 1 Out Port: 1 In Ports: 9,11,13 Total Bandwidth: 8.000 Gbps Bandwidth Demand: 300% Flags: D
30
In the above example, you have two equal cost paths with 8 G total available bandwidth for each path. The available bandwidth of each input device is 8 Gbit/sec. The Bandwidth Demand percentage is Bandwidth In/Bandwidth Out. In this example: Path1=32 Gbit/sec over 8 Gbit/sec is 32/8 = 4 which equals 400% Path1=24 Gbit/sec over 8 Gbit/sec is 24/8 = 3 which equals 300% The D flag indicates that Dynamic Load Sharing (DLS) is enabled, this is the default and only available setting for Exchanged-based Routing.
Revision 1008
6 - 30
Interop Mode Considerations Routing

Mode 2 (McDATA Fabric Mode) and Mode 3 (McDATA Open Fabric Mode) support Fabric OS Exchange-based Routing (default) Both modes support frame-level Trunking in an M-EOS fabric between Fabric OS switches with Trunking licenses Routing from B-Series to the M-Series will use configured routing policy
31
Revision 1008
6 - 31
What is Brocade B-Series ISL Trunking?

Trunking combines two or more physical ISLs into a single logical link Trunking Goals:
Reduces individual ISL congestion Forms a fault-tolerant, high bandwidth logical ISL (called a trunk or trunk group) that withstands the failure of individual ISLs
Trunk group characteristics:

Frames are multiplexed across ISLs in the trunk group One port in the trunk group represents the link in the routing data base ASICs preserve in-order delivery
32
The routing data base determines how frames are routed from input port to output port when going to the next destination. Fabric Shortest Path First (FSPF) puts available equal cost routes in the routing data base. One output port in the trunk group is put into the routing data base. When a communication between two end devices in a fabric is assigned a route through a trunk, the ASIC of the assigned trunk group port will be the same ASIC as all ports in the trunk group. This ASIC will multiplex frames across ISLs in the trunk group and maintain in-order delivery. The ASIC will send a frame down each link to determine the links latency, these individual link latency calculations will be used to maintain in-order delivery.
Revision 1008
6 - 32
Trunking Requirements
Trunking license required for all switches participating in trunking
Available when the license is installed and ports are reinitialized
Trunking is enabled by default; if previously disabled, it must be reenabled (portcfgtrunkport) on the trunk ports Trunk ports must operate at a common port speed Trunk ports must originate and end in a valid port group When trunking criteria is met, the trunk forms automatically
33
To use trunking, you must first install the Brocade trunking license. Trunking is enabled automatically when the trunking license is activated and ports are reinitialized. Trunks are easily managed using either Fabric OS CLI commands or Web Tools. Switches are shipped with trunking enabled. Trunk port-groups are ASIC specific and will be discussed in more detail in subsequent slides. 2 GBit/s port groups have historically been called quads. Bloom port groups include: ports 0-3; 4-7; 8-11; and so on. Bloom ASICs include the Bloom I ASIC which is the foundation for the following Brocade models: Brocade 3000, 3200, 3800, 3900, and 12000. The Bloom II ASIC is the foundation for the following Brocade switches: Brocade 3250, 3850, and 24000. 4 and 8 GBit/s port groups include: ports 0-7; 8-15; and so on The Condor (4 GBit/s) ASIC is the foundation for the following Brocade models: Brocade 5000, 4900, 48000 The Condor2 (8 GBit/s) ASIC is the foundation for the following Brocade models: Brocade DCX Backbone GoldenEye port groups include: ports 0-3; 4-7; 8-11; and so on The GoldenEye ASIC is the foundation for the following Brocade switch: Brocade 200E. The GoldenEye2 ASIC is the foundation for the following Brocade switches: Brocade 300, 5100, 5300 Additional ASIC specific and advanced trunking criteria include: Interoperability is not supported with M-series and third-party switch vendors. The port ISL mode must be disabled (use the portcfgislmode command). This parameter is related to using third-party gateways to extend fabric distances. This will be further discussed later in this course. Long Distance Trunks must be set to the same distance. When using QoS, all ports in the Trunk Group must be QoS enabled or they will not join the Trunk. See the Brocade Fabric OS Administrators Guide for additional information.
Revision 1008
6 - 33
Condor 4 Gbit/sec Trunking Overview

Automatically aggregates 2-8 ISLs when switches are connected1 Supports multiple 2/4 Gbit/sec trunks between switches
Ports in a group must operate at a common port speed
Trunks formed between 4 Gbit/sec switches can provide a maximum aggregate bandwidth of up to 32 Gbit/sec Trunking port groups include: ports 0-7, 8-15, and so on
34
The Brocade 4100, 5000, 4900, and 48000 use Brocade Condor ASICs. Footnote 1: Automatically creates 2-8 ISL trunks when switches are connected to form 1 logical link. Condor logical links can provide aggregate speeds up to 32 Gbit/sec.
Revision 1008
6 - 34
GoldenEye 4 Gbit/sec Trunking Overview

The GoldenEye ASIC trunking behavior is the same as Condors with some implementation differences: Automatically aggregates 2-4 ISLs when switches are connected Trunks formed between 4 Gbit/sec GoldenEye switches can provide maximum aggregate bandwidths up to 16 Gbit/sec GoldenEye trunk port groups include: ports 0-3, 4-7, 8-11, and 12-15
35
The Brocade 200E use Brocade GoldenEye ASICs.
Revision 1008
6 - 35
8 Gbit/sec Trunking Overview

8 Gbit/sec trunking behavior is similar to 4 Gbit/sec
Automatically aggregates 2-8 ISLs when switches are connected1 Supports multiple 2/4/8 Gbit/sec trunks between switches Trunks formed between 8 Gbit/sec switches can provide a maximum aggregate bandwidth of up to 64 Gbit/sec
Trunking port groups include: ports 0-7, 8-15, and so on
36
The Brocade 300, 5100, 5300 and DCX use 8 Gbit/sec ASICs (Condor2 and GoldenEye2). Footnote 1: Automatically creates 2-8 ISL trunks when switches are connected if requirements are met. Fibre Channel links of between 2 and 8 ISLs are connected to form 1 logical link. Condor/Condor2 logical links can provide aggregate speeds up to 32 Gbit/sec.
Revision 1008
6 - 36
Port-based Routing without Trunking:

Unpredictable Performance
ISL utilization in topologies before trunking:
HBA Traffic Round-robin across links In this example, the 5th HBA shares the 1st ISL with the 1st HBA Could have single ISL bandwidth congestion (hot spot)
8G 6G 4G 2G 8G
Four ISLs in use in this example
3 4 3 4 2 2 3
4G
6G
4G 2G 4G
Congestion
1 2 2 1 1 1 1
28 Gb In
20 Gb Out
37
Frame Traffic on ISLs before Trunking

When the allocation to ISL becomes saturated, congestion may occur on an independent ISL creating a hot spot.
Revision 1008
6 - 37
Trunking Frame Allocation

ASICs evenly distribute frames when bandwidth of trunk is fully utilized Frames at low bandwidth will not appear to be evenly distributed Frames use predetermined ports which results in no fabric configurations when pseudo-master ports go offline
Exchanges
Trunks can be 28 links wide
Exchanges
7 5 4
3 5 5 6 4
4 3 3 4 5 4
2 3 2 3 3 2 2
1 1 2 1 1 1 2 1
Frames prefer predetermined ports
6 8
Additional Bandwidth
Frames arrive in-order
Frames on the trunk

38
With 4+ Gbit/sec ASIC trunking in effect, 8 ISLs are aggregated into a single logical ISL capable of up to 64 Gbit/set aggregate bandwidth. Frames entering the logical pipe will be allocated to a predetermined port which just happens to be the lowest back ASIC port in the port group. If that port is busy, they will then be allocated another predetermined port. When the bandwidth of the pipe is fully utilized the frames are evenly distributed. When the aggregate bandwidth of the logical pipe is not fully utilized and the 1st predetermined port is free, the frames do not need to be evenly distributed. Calculated time differentials are needed to maintain in-order delivery of frames. If traffic patterns involve large sequences of frames and high bandwidth utilization, then the frame allocation across the ISLs in the trunk will begin to evenly distribute frames across the ISLs. The predictable way frames are allocated to 4+ Gbit/sec ASIC trunk groups enables non-disruptive loss of trunk master behavior which is also referred to as PseudoMaster Trunking. This is sometimes called Masterless Trunking. Bloom trunks stripe frames across participating links regardless of total traffic load, no ISL in trunk group has priority over another. With Bloom trunking in effect, the 4 ISLs are treated as a single logical ISL capable of 8 Gbit/set aggregate bandwidth. The sequences of frames are distributed evenly on the frame level across all four physical paths. Since the source and destination quads use the same buffer pool, frames received on the ports will be reassembled in the same sequence preserving in-order delivery.
Revision 1008
6 - 38
Port-based Routing over Trunks

Port-based Routing is supported on all switches A trunk group is considered one logical ISL identified by the trunk master The load assigned to a trunk group is primarily based on the ISL link cost and secondarily on total bandwidth metrics of all the ISLs in the group
39
Revision 1008
6 - 39
Exchange-based Routing over Trunks

Exchange-based Routing is only supported on 4/8 Gbit/sec ASICs The exchanges assigned to a trunk group/ISL are based primarily on the link cost and secondarily on total bandwidth metrics of all the ISLs
40
Dynamic Path Selection (DPS) is exchange-base routing where exchanges or communications between end devices in a fabric are assigned to egress ports in ratios proportional to the potential bandwidth of the ISL or trunk group. When there are multiple paths to a destination, the input traffic will be distributed across the different paths in proportion to the bandwidth available on each of the paths. This improves utilization of the available paths, thus reducing possible congestion on the paths. Every time there is a change in the network (which changes the available paths), the input traffic can be redistributed across the available paths. This is a very easy and non-disruptive process when the Exchangebased Routing Policy is engaged. Exchanges in the example depicted on this slide are allocated based on the primary criteria: link cost and secondary criteria: potential bandwidth. The potential bandwidth allocation depicted in this example yields flow allocations of 3:1.
Revision 1008
6 - 40
One Port Group with Multiple ISL Trunks

One port group can simultaneously have different speed trunks
B300:admin> switchshow switchName: B300 switchType: 71.2 switchState: Online <output truncated> Area Port Media Speed State ============================== 0 0 id N4 Online 1 1 id N4 Online 2 2 id N4 Online 3 3 id N4 Online 4 4 id N4 Online (upstream)(Trunk master) 5 5 id N4 Online 6 6 id N8 Online 7 7 id N8 Online master) <output truncated>
A six-ISL 4 Gbit/sec trunk group
A two-ISL 8 Gbit/sec trunk group
E-Port E-Port E-Port E-Port E-Port
(Trunk port, master is Port 4 ) (Trunk port, master is Port 4 ) (Trunk port, master is Port 4 ) (Trunk port, master is Port 4 ) 10:00:00:05:1e:34:01:e6 "toist04b41"
E-Port (Trunk port, master is Port 4 ) E-Port (Trunk port, master is Port 7 ) E-Port 10:00:00:05:1e:34:3b:8b "toist03b41 (Trunk
41
Revision 1008
6 - 41
ASIC Trunk Pseudo-Master

4+ Gbit/sec ASICs changed the role of the Master ISL to a role referred to as Pseudo-Master The Trunk Master represents the group for routing There is no fabric disruption when trunk master goes offline! The trunkshow output still depicts the port as MASTER
b5100:admin> trunkshow 1: 4 -> 8 10:00:00:05:1e:02:12:b1 0 -> 9 10:00:00:05:1e:02:12:b1 1 -> 10 10:00:00:05:1e:02:12:b1 5 -> 11 10:00:00:05:1e:02:12:b1 . . .<truncated output> . . . deskew deskew deskew deskew 15 15 16 16 MASTER
When the MASTER is disabled, trunkshow immediately depicts the new MASTER NO fabric disruption
B5100:admin> trunkshow 1: 1 -> 10 10:00:00:05:1e:02:12:b1 5 -> 11 10:00:00:05:1e:02:12:b1 0 -> 9 10:00:00:05:1e:02:12:b1 deskew 16 deskew 16 deskew 15 MASTER
42
When the Trunk Master is disabled, another pre-determined port takes over the role without fabric disruption.
Revision 1008
6 - 42
The Deskew Counter

Deskew values are related to distance and link quality
Deskew units represent the time difference for traffic to travel over each ISL as compared to the shortest ISL in the group The system automatically sets the minimum deskew value of the shortest ISL to 15 deskew units
Differences > than 30 m could introduce performance degradation

A 30 m difference = 15 deskew unit difference Since shortest ISL is set to a deskew of 15, an ISL with a difference of 30 m will have a deskew of 30 An ISL with a deskew above 30 indicates possible degradation
Cable length between shortest ISL and longest ISL in a trunk

Max. cable difference between ISLs is 400 m remember to consider this when creating long distance trunks over WDMs!
Deskew values are displayed in the trunkshow command output
43
Light in a vacuum travels much faster, but in optical cable the rate is about 5 ns/meter. 5ns/meter times 30 meters is equal to 150 ns. The difference in cable lengths between the ISLs in a trunk determines the deskew value. This is needed for timing purposes so that delivery of frames across the trunk can be ensured. The shortest ISL is selected as the base and is assigned a deskew value of 150 nsec. The deskew values are expressed (shown in all command displays) by dividing the time value by 10. Example: A deskew value of 150 nanoseconds is shown as 15 (150/10). The first ISL in the Bloom trunk to initialize is selected as the trunk master. On 4+ Gbit/sec ASICs the ISL attached to the lowest backport will become the groups trunk master. The length of the cable is not a consideration when selecting the master. The deskew values for the other ISLs in the trunk will be calculated from the base ISL and will have a higher value. Each switch connected by the ISL will have a deskew value since each has a separate transmit line to the other. Due to the signal quality/optical media, cables that are identified as the same length may have a different deskew value. For example, one cable may have a deskew value of 16 and a cable of the same length may calculate to be 17. This is not a problem since deskew is a true measurement of its transmission capabilities.
Revision 1008
6 - 43
Trunking Related Commands Overview

Commands that enable you to view trunk information:
trunkshow switchshow islshow portcfgshow
Commands that allow you to configure trunk parameters:

portcfgtrunkport switchcfgtrunk
Command that allows you to test trunking:

trunkdebug <start port> <end port>
44
trunkshow Displays each trunk group Displays which local port is connected to which remote port Displays WWN of the other switch Displays deskew values and identifies the trunk master port switchshow Displays the master port (trunk master) and each non-master port Displays the WWN of connected switch to trunked ports islshow Displays bandwidth information associated with each trunk group switchcfgtrunk Used to configure trunking to be ON or OFF on all ports on switch portcfgtrunkport Used to configure trunking per port as either ON or OFF, default is ON portcfgshow Displays port configuration information including trunk ON/OFF status trunkdebug <start port>, <end port> Tests specified ports in trunk group and reports trunking status. Possible trunkdebug outputs include: Switch does not support trunking; Trunking license required; Trunking not supported in switch interop mode; port<port_id> is not E_Port; port<port_id> trunking is disabled; port<port_id> speed is not 2 Gbit/sec; port<port_id> and port <port_id> are not on the same quad; port<port_id> and port <port_id> are connected to different switches; port<port_id> is not a trunk port due to E_port being disabled, or trunking may be disabled at remote port; port<port_id> and port<port_id> cannot trunk. Check the link length to verify that the difference is less than 400 m.
trunkshow
B300:admin> trunkshow 1:16 -> 0 10:00:00:05:1e:34:55:fa 17 -> 1 10:00:00:05:1e:34:55:fa 18 -> 2 10:00:00:05:1e:34:55:fa 19 -> 3 10:00:00:05:1e:34:55:fa 2:20 -> 4 10:00:00:05:1e:34:55:fa 21 -> 5 10:00:00:05:1e:34:55:fa 3:22 -> 6 10:00:00:05:1e:34:55:fa 23 -> 7 10:00:00:05:1e:34:55:fa
Source port Destination port on attached switch
Trunk Groups
deskew deskew deskew deskew deskew deskew deskew deskew
15 16 16 17 16 15 16 15
MASTER
MASTER MASTER
Destination switch WWN
45
Trunk master selection is not related to deskew. Trunk master selection is based on the ASICs internal port values, it is predictable. It is dependent on distance and quality of the link and its associated connection points. The switches in this example have multiple trunk groups between them. Notice that the trunk master is not always the lowest port number in the group nor is it related to the deskew value.
Revision 1008
6 - 45
switchshow
B300:admin> switchshow switchName: B300 switchType: 71.2 switchState: Online switchMode: Native switchRole: Principal switchDomain: 1 switchId: fffc01 switchWwn: 10:00:00:05:1e:34:be:5a zoning: OFF (No Access) switchBeacon: OFF [* Output Truncated *] Port Media Speed State ========================= *** <Truncated Output> *** 16 16 id N8 Online E-Port 10:00:00:05:1e:34:55:fa "B5300 master) 17 17 id N8 Online E-Port (Trunk port, master is Port 16 18 18 id N8 Online E-Port (Trunk port, master is Port 16 19 19 id N8 Online E-Port (Trunk port, master is Port 16 20 20 id N8 Online E-Port 10:00:00:05:1e:34:55:fa "B5300" 21 21 id N8 Online E-Port (Trunk port, master is Port 20 22 22 id 4G Online E-Port (Trunk port, master is Port 23 23 23 id 4G Online E-Port 10:00:00:05:1e:34:55:fa "B5300
(downstream) (Trunk ) ) ) (Trunk master) ) ) (Trunk master)
46
The switchshow commands shows trunk groups associated with ports 12-15 with the master port of the individual trunk groups noted. The remaining ports in the trunk groups are the non-master ports. It should also be noted that only one trunk group represents the principal ISL path. Here is the Brocade 300 switchshow truncated command output:
B300:admin> switchshow switchName: B300 *truncated output* Area Port Media Speed State ============================== *truncated output* 8 8 id N8 Online E-Port 10:00:00:05:1e:34:55:fa "B4100" (Trunk master) 9 9 id N8 Online E-Port (Trunk port, master is Port 8 ) 10 10 id N8 Online E-Port (Trunk port, master is Port 8 ) 11 11 id N8 Online E-Port (Trunk port, master is Port 8 ) 12 12 id N8 Online E-Port (Trunk port, master is Port 13 ) 13 13 id N8 Online E-Port 10:00:00:05:1e:34:55:fa "B4100" (Trunk master) 14 14 id 4G Online E-Port (Trunk port, master is Port 15 ) 15 15 id 4G Online E-Port 10:00:00:05:1e:34:55:fa "B4100" (upstream)(Trunk master)
Revision 1008
6 - 46
islshow
B300:admin> islshow 1: 16-> QoS 2: 20-> QoS 3: 22-> QoS 0 10:00:00:05:1e:34:55:fa 4 10:00:00:05:1e:34:55:fa 6 10:00:00:05:1e:34:55:fa 2 B5300 2 B5300 2 B5300 sp: sp: sp: 8.000G bw: 32.000G TRUNK 8.000G bw: 16.000G TRUNK 4.000G bw: 8.000G TRUNK
47
islshow displays the speeds for each ISL in the trunk groups, the aggregate bandwidth for trunk groups, and the WWN of the other switch (in this example, we are connected to one other switch with three trunk groups). Each trunk group is defined by the trunk master of each trunk. Port 8 of trunk group 1 points to port 8 of a trunk group on the attached switch; port 8 is the trunk master of this trunk group on both of these switches. Here is the Brocade 300 islshow truncated command output:
B200E:admin> islshow 1: 8-> 8 10:00:00:05:1e:34:55:fa 2: 13-> 13 10:00:00:05:1e:34:55:fa 3: 15-> 15 10:00:00:05:1e:34:55:fa B300 sp: B300 sp: B300 sp: 8G bw: 8G bw: 4G bw: 32G TRUNK 16G TRUNK 8G TRUNK
Revision 1008
6 - 47
portcfgtrunkport/portcfgshow
B200E:admin> portcfgtrunkport 8,0 Trunking turned off for port 8 r7-st01-b30-1:admin> portcfgshow Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -----------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+-Speed AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AL_PA Offset 13 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Trunk Port ON ON ON ON ON ON ON ON .. ON ON ON ON ON ON ON Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. VC Link Init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. RSCN Suppressed .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Persistent Disable.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. NPIV capability ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON QOS E_Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON Mirror Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Rate Limit .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Credit Recovery ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON
48
portcfgtrunkport is used to turn on or off trunking on one port, specified by port number. The last parameter indicates on or off (1 for on and 0 for off). Usage: portCfgTrunkPort [SlotNumber/]PortNumber Mode Mode: 1 - Configure port to be Trunking capable 0 - Configure port to be Trunking incapable switchcfgtrunk is used to turn on or off trunking capability for the whole switch Usage: switchCfgTrunk Mode Mode: 0 - Configure ports to be Trunking incapable 1 - Configure ports to be Trunking capable
Revision 1008
6 - 48
trunkdebug
B300:admin:> trunkdebug <start port #>, <end port #>
Used to debug a trunk link failure, operands required The command reports one of the following:
Switch does not support trunking Trunking license required Trunking not supported in switch interop mode port<port_id> is not E_Port port<port_id> trunking is disabled port<port_id> speed is not 2 Gbit/sec, 4 Gbit/sec or 8 Gbit/sec port<port_id> and port <port_id> are not on the same port group port<port_id> and port <port_id> are connected to different switches port<port_id> is not a trunk port due to E_port being disabled, or trunking may be disabled at remote port port<port_id> and port<port_id> cannot trunk. Check the link length to verify that the difference is less than 400 m B300:admin> trunkdebug 11, 12 local or remote ports are not in the same port group
49
Usage: trunkdebug port1, port2 port1 - Specify the first port number where you want to debug a trunk link error. This operand is required. port2 - Specify the last port number where you want to debug a trunk link error. This operand is required. Example 1: To debug a trunk connection for ports 1 through 4: sw2:admin> trunkdebug 1, 4 port 4 is not E port Example 2: To debug a trunk for ports 8 11 when only 8 and 11 are trunked: sw2:admin> trunkdebug 8, 11 ports 8 and 11 are trunked together Director command syntax: Director:admin> trunkdebug trunkdebug: area_number1 area_number2 Note: spinfab was not designed to do performance testing over trunks. Trunk performance testing will have to be done without spinfab. spinfab was designed to run a functional test of individual switch-to-switch ISL cabling Optionally use porttest with a loopback plug at the other end of the link What is porttest? Test online ports (F/FL/E or loopback ports); looks for port type and runs appropriate test can optionally select port types Runs in the background so does not affect traffic Use porttestshow to retrieve information from porttest Use stopporttest to stop porttest porttestshow will return porttest data when successful or failed reason and error code if not successful.
Revision 1008
6 - 49
Brocade M-series Open Trunking

Open Trunking is a licensed-based feature
Once installed, the user must activate it. It operates on a per-switch basis, but it is not necessary to purchase the Open Trunking license for every switch in the fabric
Open Trunking reduces congestion by actively monitoring the utilization thresholds on all ISLs
It will automatically recalculate the routing table if the utilization of bandwidth between domains on multiple ISLs is not in relative balance for 60 seconds Nodes assigned to a particular ISL could be reassigned to a new one
50
Open Trunking actively monitors the utilization on all ISLs. It is designed to minimize dropped frames by reducing over congestion on ISLs. It will automatically recalculate the routing table if the utilization of bandwidth between domains on multiple ISLs is not in relative balance. The entire route table is not recalculated, but nodes assigned to a particular ISL can be reassigned to a new one.
Revision 1008
6 - 50
Brocade M-series Open Trunking (cont.)

Modes 2 and 3 support M-EOS Open Trunking and Exchange-based Routing between M-series switches and B-series switches
Routing from M-Series to a B-Series will use Open Trunking (if licensed) B-series to M-series switches will utilize Exchange-based Routing
In order to maximize utilization on the ISL, Open Trunking must be enabled on both switches Any E_Port on the M-series switch may participate in Open Trunking
There is no limitation on port groups or port cards Open Trunking has no distance limitation It operates with short-wave or long-wave optics and over DWDM/CWDM
51
Nodes are assigned exit ports from one domain to another domain. There may be multiple exit ports for a node when one switch is connected to several other switches. Open Trunking balances the transmission of frames to a connected switch. That is, Open Trunking on an Open Trunking-enabled switch balances the transmit and not the receive on the link.
Revision 1008
6 - 51
Summary
Fabric Routing is logic used by a switch to pass frames from the source domain toward the destination domain Inter-Switch Links (ISLs) are E_Port-to-E_Port links that communicate Class F services DLS is used to share multiple available routes to a destination domain FSPF implements IOD to preserve in-order delivery of frames Port-based Routing maps a single ingress port to a single outbound route Exchange-based Routing maps a single ingress port to all available outbound routes Modes 2 and 3 support M-EOS Open Trunking and Exchangebased Routing between M-series switches and B-series switches
52
Revision 1008
6 - 52

End of Instructor-Led Module 6 Fibre Channel Routing
Revision 1008
Revision 1008
6 - 53
Revision 1008
6 - 54
Fibre Channel Long Distance

Instructor-Led Module 7 Fibre Channel Long Distance
Revision 1008
Revision 1008
7-1
Objectives
Describe the concepts of long distance Fibre Channel Understand the use of long distance components
Cabling Connectivity options Understand B-Series licensing requirements

Describe the different long distance settings for B-Series products
Understand how to calculate payload size, distance and speed to add

the appropriate number of required credits to a port
Understand how to interpret important command outputs such as portbuffershow, portcfglongdistance, portshow
Revision 1008
7-2
Long Distance Fabrics Overview

The most common reason for extending storage area networks beyond 10Km geographical distances is to safeguard critical business data and provide near-continuous access to applications and services in the event of a localized disaster Configuring a distance connectivity solution involves a number of considerations
Recovery Point Objectives (RPO) Recovery Time Objectives (RTO)
Options for replication generally fall into one of several categories

A high-end business continuity solution with strict RTO and RPO A medium level Disaster Recovery (DR) solution with an RTO from several minutes to a few hours Backup of non-critical application data that does not require immediate access
Recovery from tape has the longest RTO

The most common reason for extending storage area networks over geographical distances is to safeguard critical business data and provide near-continuous access to applications and services in the event of a localized disaster. Designing a distance connectivity solution involves a number of considerations. The enterprise must classify stored data and determine how important it is for business operation, how often it must be backed up, and how quickly it needs to be recovered in the event of failure. Two important objectives in the designing process are the Recovery Point Objective (RPO) and the Recovery Time Objective (RTO). The RPO is the time period between backup points and describes the acceptable age of the data that must be restored after a failure has occurred. For example, if a remote backup occurs every day at midnight and a site failure occurs at 11 pm, changes to data made within the last 23 hours will not be recoverable. The RTO describes the time needed to recover from the disaster. The RTO determines the acceptable length of time a break in continuity can occur with minimal or no impact to business services. Options for replication generally fall into one of several categories. A business continuity solution with strict RTO and RPO may require highspeed synchronous or near-synchronous replication between sites as well as application clustering for immediate service recovery. A medium level Disaster Recovery (DR) solution may require high-speed replication that could be synchronous or asynchronous with an RTO from several minutes to a few hours. Backup of non-critical application data that does not require immediate access after a failure can be accomplished via tape vaulting. Recovery from tape has the longest RTO.
Revision 1008
7-3
Long Distance Cabling

As light travels through fiber, the intensity of the signal degrades, called attenuation When using optical cabling, two attributes to measure attenuation is Power Budget and Signal Loss
Power budget identifies how much attenuation can occur across a fiber span while still maintaining sufficient output power for the receiver Signal loss is the total sum of all losses due to attenuation across the fiber span
The two main transmission wavelength/windows in which attenuation is minimal for long distance:
1310nm and 1550nm
Single-mode Fiber is the main cable type that will support these transmission wavelength/windows
For specific Brocade transceiver part numbers and transmission measurements, refer to the appendix in this module
There are two basic types of optical fiber: Multimode Fiber (MMF) and Single-Mode Fiber (SMF). Multimode fiber has a larger core diameter of 50 m or 62.5 m (the latter was common for Fiber Distributed Data Interface (FDDI)) and carries numerous modes of light through the waveguide. It is less expensive than singlemode fiber, but its characteristics make it unsuitable for distances greater than several hundred meters. Because of this, multimode fiber is generally used for short distance spans and is common for interconnecting SAN equipment within a data center. Single-mode fiber has a smaller core diameter of 9 m and carries only a single mode of light through the waveguide. It is better at retaining the fidelity of each light pulse over long distances and thus results in lower attenuation. Single mode fiber is always used for long-distance extension over optical networks and often used even within the data center for FICON installations. Optical Small Form-Factor Pluggable (SFP) transceivers are available in short- and long-wavelength versions. Short-wavelength transceivers transmit at 850 nm and are used with 50 or 62.5 m multimode fiber cabling. For fiber spans greater than several hundred meters without regeneration, use long-wavelength transceivers with 9 m single-mode fiber. Long-wavelength SFP transceivers typically operate in the 1310 or 1550 nm range. Refer to the Appendix in this module for specific Brocade Transceiver models.
Revision 1008
7-4
Long Distance Cabling (cont.)

Links less than 100m can use less expensive multimode transceivers with OM3 fiber
50m glass core size OM3 2000MHz Transceivers provide Short-Wave Transmission (850nm)
For links greater than 150m (OM3 8 Gbits/sec), use single-mode fiber
9m glass core size Transceivers provide Long-Wave Transmission (1310nm, 1550nm)
Revision 1008
7-5
Long Distance Connectivity Options

There are a number of methods in which FC SANs can be extended over long-distance optical networks:
Native FC over Dark Fiber Wave Division Multiplexing
Transponder-Based Solutions SFP-Based Solutions

Time Division Multiplexing FC-SONET/SDH Extended Distance Solutions
Native Fibre Channel over dark fiber is highly reliable with high performance and does not require a protocol conversion. It can provide distances up to 100 km at 4 Gbits/sec and 500 km at 1 Gbit/sec. Native FC connections can also be made to a course or dense wave division multiplexer. The data is carried on a dedicated wave-length. Transponder-based solutions use a transponder to convert the different data wavelengths (850 nm, 1310 nm, 1550 nm) to a common wave-length. SFP-based solutions have the wavelength set by whichever SFP interface is installed. Fibre Channel can be carried over Synchronous Optical Networks using carriers such as OC3 (155 Mbits/sec) or OC12 (622 Mbits/sec).
Revision 1008
7-6

Native FC over Dark Fiber
The term dark fiber typically refers to fiber optic cabling that has been deployed, but remains unlit or unused The simplest is to connect FC switches directly to the dark fiber using
Long-wavelength SFPs Extended long-wavelength SFPs FC Extenders with LWL SFP transceivers
Use Extended Fabric to allocate sufficient buffers to the long distance E_Ports
Configure Brocade Extended Fabrics for the appropriate distance
Refer to the Appendix in this module for specific Brocade Transceiver models.
Revision 1008
7-7

WDM
Wave Division Multiplexing (WDM) describes the concept of combining several streams of data onto the same physical fiber-optic cabling
Light of different wavelengths does not interfere
Dense Wavelength Division Multiplexing (DWDM) is optimized for highspeed, high-capacity networks and longer distances Coarse Wavelength Division Multiplexing (CWDM) provides the same optical transport and features of DWDM, but at a lower capacity, which allows for lower cost There are two basic types of Wavelength Division Multiplexing (WDM) solutions:
Transponder-Based Solutions
Using 850nm or 1310nm, it converts these signals using

Optical-to-Electrical-to-Optical (OE-O) conversion WDM frequencies for transport across a single fiber
SFP-Based Solutions
These eliminate the need for transponders by requiring
switch equipment to utilize special WDM transceivers (also known as colored optics), reducing the overall cost
Dense Wavelength Division Multiplexing (DWDM) is optimized for high-speed, highcapacity networks and long distances. DWDM is suitable for large enterprises and service providers who lease wavelengths to customers. Most equipment vendors can support 32, 64, or more channels over a fiber pair with each running at speeds up to 10 Gbit/sec. Fiber distances between nodes can generally extend up to 100 km or farther. DWDM equipment can be configured to provide a path protection scheme in case of link failure or in ring topologies that also provide protection. Switching from the active path to the protected path typically occurs in less than 50 ms. Coarse Wavelength Division Multiplexing (CWDM) provides the same optical transport and features of DWDM, but at a lower capacity, which allows for lower cost. CWDM is generally designed for shorter distances (typically 50 to 80 km) and thus does not require specialized amplifiers and high-precision lasers (lower cost). Most CWDM devices will support up to 8 or 16 channels. CWDM generally operates at a lower bit rate than higher-end DWDM systemstypically up to 4 Gbit/sec. There are two basic types of Wavelength Division Multiplexing (WDM) solutions: Transponder-Based Solutions. Allows connectivity to switches with standard 850 or 1310 nm optical SFP transceivers. A transponder is used to convert these signals using Optical-to-Electrical-to-Optical (OE-O) conversion WDM frequencies for transport across a single fiber. By converting each input to a different frequency, multiple signals can be carried over the same fiber. SFP-Based Solutions. These eliminate the need for transponders by requiring switch equipment to utilize special WDM transceivers (also known as colored optics), reducing the overall cost. CWDM SFPs are like any standard transceiver used in Fibre Channel switches, except that they transmit on a particular CWDM frequency.
Revision 1008 7-8

Time Division Multiplexing (TDM)
Time Division Multiplexing (TDM) takes multiple client-side data channels, such as FC, and maps them onto a single higher-bit-rate channel for transmission on a single wavelength TDM is used in conjunction with a WDM solution to provide additional scalability and bandwidth utilization Because TDM sometimes relies on certain FC primitives to maintain synchronization, it may require special configuration on BSeries switches when Extended Fabrics is enabled
By default, Extended Fabrics E_Ports use ARB primitives (specific to Virtual Channels) as fill words between frames Most TDM devices require IDLEs as fill words Configure a B-Series switch to use R_RDY flow control
Revision 1008
7-9

SONET/SDH
Synchronous Optical Network (SONET) and Synchronous Digital Hierarchy (SDH) are standards for transmission of digital information over optical networks and are often the underlying transport protocols that carry enterprise voice, video, data and storage traffic across metropolitan and area networks FC-SONET/SDH is the protocol that provides the means for transporting FC frames over SONET/SDH networks FC frames are commonly mapped onto a SONET or SDH payload using an International Telecommunications Union (ITU) standard called Generic Framing Procedure (GFP) Like TDM, FC-SONET devices typically require enabling R_RDY flow control on FOS-based switches
10
Revision 1008
7 - 10

Extended Distance Solutions
Some FC distance extension equipment (FC-SONET, FC-IP, etc.) can participate in FC buffer-to-buffer flow control to increase the distance to be greater than what is possible with Extended Fabrics Such devices typically participate in E_Port link initialization with an FC switch by querying the receive-buffer field in the Exchange Link Parameters (ELP) payload These devices are not aware of Brocade VC_RDY flow control, R_RDY flow control must be enabled on a B-Series switch
These devices return R_RDY credit to the switch in order to maintain performance over hundreds or even thousands of kilometers Flow control and error correction between distance extension nodes are performed independent of the switch and are usually dependent on the long haul network protocols
Additional considerations must be taken to configure these types of networks

Effective Bandwidth, Delay, Packet Loss (FC-IP), MTU sizes, etc.
11
Effective bandwidth is a measure of how much of the available bandwidth can actually be used, taking into consideration dropped packets and retransmission due to congestion and protocol inefficiency. Delay or latency (synonym) is the amount of time that it takes a packet of data to get from one point to another. Delay is often measured by sending a packet that is returned to the sender; the packets round-trip time (RTT) is considered the delay in the network. In an IP network, delay depends not only on distance, but also upon router hop count, router processing, and packet size. A reasonable rule of thumb to estimate delay is driving distance * 1.5. Packet Loss: The Internet Standards treat packet loss and congestion as synonyms. Congestion is the prime cause of packet loss. Congestion occurs when Routers discard incoming packets that cant be stored or transmitted because the average sum of the inputs to a router exceeds the capacity of its output. Slightly reworded: anytime the average feed from the Ethernet exceeds the capability of a router, packets will be lost. When the output connection is a costly nation-to-nation or satellite link, it becomes very expensive to make the pipe big enough so packets wont be lost. If the pipe is too big, then you become concerned with effective bandwidth. Other contributors that cause packet loss include bit errors, deliberate discards, and router in/out times. See http://www.gigabytex.com/whttcp.htm for additional information. TCP assumes that all packet loss is caused by congestion and responds by reducing the transmission rate. Different applications use protocols with different block sizes to transfer data. Block access protocols access "blocks" of data in portions that are a multiple of the OS system block. Consider using the following guidelines to determine block sizes: Transaction data (4-8k block size); Office automation (16-32k block size); Data warehousing (64 - 256k block size); CAD/Design (64-128k block size); Multimedia (512k - 4M block size). Small block sizes of contiguous data mean more I/O especially if the data is spread across the disk; large block sizes that don't use all the space read the whole block just to get a small piece of data. Applications can be configured to allow multiple outstanding I/Os to occur before requiring an acknowledgement. The # of outstanding I/Os is typically 1 to 16. FOS 5.3.x and later support FC FastWrite to improve SCSI write operations over long-distance ISLs. This is accomplished by the creation of a Proxy Target (PT) local to the initiator host, and a Proxy Initiator (PI) local to the target storage device. When a SCSI write is received, the PT responds with XFER_RDY, enabling the host to begin transmission of data over the network without having to wait for response from the target device, thus eliminating latency caused by the long-distance link. Once the PI intercepts the XFER_RDY from the target device, data is sent to the target. FastWrite is supported on the Brocade 7500 SAN Router and FR4-18i Routing Blade.
Revision 1008
7 - 11
FOS Extended Fabrics Feature

The Extended Fabrics feature extends the distance inter-switch links (ISLs) can reach over a dark fiber or wave division-multiplexing (WDM) connection. This is accomplished by providing enough buffer credits on each side of the link to compensate for latency introduced by the extended distance. Bookend Switches are Extended Fabric licensed switches that connect the two remote ends of the fabrics over a long distance (greater than 10 km)
As the distance between switches and the link speed increases, additional buffer credits are required for the ports used for long distance connections Distance levels (LD or LS) define how these buffer credits are allocated and managed for extended ISLs. Buffer credits are managed from a common pool available to a group of ports on a switch.
12
Data traffic Virtual Channels (VCs) are collapsed to optimize performance over long distances using the portcfglongdistance command, as shown in the diagram below. Information about switch characteristics and capacity in terms of buffers per port group, port speed, and distances supported is contained in FOS Fabric OS Administrator's Guide and the appropriate Hardware Reference manual specific to the switch you are configuring.
VC2 VC3 VC4 VC5
VC2
VC 2 only
VC3 VC4 VC5
Revision 1008
7 - 12
Extended Fabric Configuration

Connecting ports on bookend switches need to be set to the same long distance parameters Use Web Tools Switch Admin Extended Fabric View panel Use the portcfglongdistance command to specify an Extended Fabric Distance Level
Level 0 static mode (L0) is the normal mode for a port Level E static mode (LE) reserves a static number of buffer credits that supports distances up to 10 km
The number reserved depends on the port speed

Dynamic Mode (LD) calculates buffer credits based on the distance measured during port initialization
An upper limit is placed on the calculation by providing a desired distance value

Static long distance mode (LS) calculates a static number of buffer credits based on a desired distance value
13
LE does not require a license key. LS mode is for Fabric OS v5.1+ ONLY. The maximum Extended Fabric distance depends on the version of Brocade switch ASIC installed in the switch. Extended Fabric distance levels (L0, LE, LD, LS) persist across switch reboots and power cycles: Can not be set or removed by configure or configdefault Can be cleared by portcfgdefault Saved in a switch configuration file (configupload ) as portcfg parameter Level 0 static mode (L0) - L0 is the normal mode for a port. An extended ISL port competes with other ports for buffers. No buffer credits are reserved for extended ISLs. Level E static mode (LE) - LE reserves a static number of buffer credits that supports distances up to 10 km. The number reserved depends on the port speed. The baseline for the calculation is one credit per km at 2 Gbps. This yields the following values for 10 km: 5 credits per port at 1 Gbps 10 credits per port at 2 Gbps 20 credits per port at 4 Gbps 40 credits per port at 8 Gbps 510 credits for one port only at 120 km for 10Gbps (specific to the FC10-6 blade) Dynamic Mode (LD) - LD calculates buffer credits based on the distance measured during port initialization. An upper limit is placed on the calculation by providing a desired distance value. When the measured distance is more than desired distance, the desired distance is used in the calculation. This is a mechanism for controlling the number of reserved buffer credits ensuring buffer availability for other ports in the same group. Static long distance mode (LS) - LS calculates a static number of buffer credits based on a desired distance value. For both LD and LS distance levels, you can use the following formula to get an approximation of the calculated number of buffer credits: buffer credits = [(distance in Km) * (data rate) * 1000] / (payload size) For LD, distance in Km is the smaller of the distance measured during port initialization versus the desired distance value. For LS, distance in Km is always the desired distance value. data rate is: 1.0625 for 1 Gbps 2.125 for 2 Gbps 4.25 for 4 Gbps 8.5 for 8 Gbps
Revision 1008
7 - 13
Extended Fabric Configuration (cont.)

Use the portcfglongdistance command to configure bookend switch ports used to connect the fabrics
SW5300:admin> portcfglongdistance 3 LD 1 75
Distance level LS or LD must be the same at each end of the link Desired Distance represents the user-estimated (LD) maximum distance in km or a user-specified (LS) distance in km Example: setting port 3 to a dynamic mode with a maximum Desired distance of 75 km The 1 is associated with a VC Link Init setting that needs to be manually configured from CLI when an LD or LS desired distance parameter is supplied The VC Link Init value is automatically set to 1 for all other Lx configurations engaged with the portcfglongdistance command
14
Syntax portcfglongdistance [Slot/]<port_number> , distance level, [vc_translative_init ] , <desired_distance> The vc_translative_init value of 1 in the example on this slide is needed as a syntax position holder. If a desired_distance is not specified then vc_translative_init is enabled without explicitly calling it out. Best Practice When VC Translation Link Initialization is enabled (set to ON), the port uses an extended link initialization sequence, which is an enhanced link reset protocol designed to avoid excessive resetting of ports used on a long distance connection. As a best practice, Brocade recommends this feature be enabled. It is enabled by default on Fabric OS v4.4+ switches. Use the portcfgshow command to verify that vc_translaltion_link_init is enabled at both ends of the link.
Revision 1008
7 - 14
portshow
The portshow [slot/]port command can be run to see the distance setting on the port
SW5300:admin> portshow 3 <truncated output> Distance: auto (desired = 75 Km) portSpeed: N8Gbps <truncated output>
15
Revision 1008
7 - 15
Distance Availability and Performance

Many distance extension designs are capable of providing several options for maintaining fault tolerance, availability and performance of Inter-Switch Links (ISLs) spanning the optical network In order to maintain fault tolerance and availability in the FC switch environment
Trunking Dynamic Path Selection Traffic Isolation
To maintain Flow control and performance, focus on:

Buffer Credit Allocation QoS Buffer Credit Recovery
16
Revision 1008
7 - 16
Extended Distance Availability

Trunking
Criteria for trunking an Extended Fabric:
All trunking criteria must be met Extended Fabric license on supported switches at both sides of the ISL All trunk ports in the same trunk group must be configured to the same long distance mode
Example: LELE, LDLD, LSLS LD modes must specify the same maximum desired distance parameter LS modes must specify the same user-defined distance parameter
Long distance ISLs within trunk groups are subject to distance limitations Refer to Appendix and/or your FOS Administrators Guide for these limitations
17
It is also important to note that when frame-based trunking is used, the deskew value is calculated when the trunk is formed. (For more about the deskew value, see the Fabric OS Administrators Guide.) If a protection switch results in a change of latency, then these deskew values are no longer valid and unpredictable behavior could result. Therefore a key design principle is to ensure that if any rerouting is possible, then entire trunks must be rerouted as homogenous groups.
Revision 1008
7 - 17
Extended Distance Availability

Protection Switching
Optical extension products often provide protection switching in the case of optical fiber failure on the long haul side of the network
Most WDM and SONET/SDH equipment has the ability to perform a protection switch from the active path to the standby path in less than 50 ms in order to maintain connectivity between sites Brocade has set the loss-of-sync timer to 100 ms
For most distance extension devices, an ISL remains online during a protection switch Frames in flight during a protection switch will be lost and must be resent by the initiating end device
If your active and passive paths are of different lengths, use the LS option and configure your user-defined distance in relation to the longer path
18
Revision 1008
7 - 18
Extended Distance Performance

Buffer-to-Buffer Credit Allocation
While exact calculations are possible, a simple rule of thumb is used in the calculation of the BB credit requirement of a given link
The use of QoS can also affect the distance
Rule of thumb: Based on the speed of light in an optical cable (5ns/m), a full-size FC frame spans approximately
4 km at 1 Gbit/sec 2 km at 2 Gbit/sec 1 km at 4 Gbit/sec 0.5 km at 8 Gbit/sec 400m at 10 Gbit/sec 5 credits per port at 1 Gbps 10 credits per port at 2 Gbps 20 credits per port at 4 Gbps 40 credits per port at 8 Gbps
For example, to keep a link at full speed that spans a distance of 10 km:
If your payload size is smaller than 2112 bytes, use the following formula to calculate exact number of minimum buffer credit requirements
buffer credits = [(distance in Km) * (data rate in Gbits) * 1000] / (payload size) Refer to the Appendix for distances on specific platforms, speeds, and use of QoS
19
Before discussing FC-level buffer allocation, it is important to note that the availability of sufficient FC-level buffering is not itself sufficient to guarantee bandwidth utilization. Specifically other limitations, particularly at the SCSI level of the storage initiator and/or target, are often the limiting factor. The I/O size, IO per second limit, and concurrent/outstanding IO capability at the SCSI level of the initiators/targets can be and often are gating factors. While exact calculations are possible, a simple rule of thumb is used in the calculation of the BB credit requirement of a given link. Based on the speed of light in an optical cable, a full-size FC frame spans approximately 4 km at 1 Gbit/sec, 2 km at 2 Gbit/sec, 1 km at 4 Gbit/sec, or 400m at 10 Gbit/sec. To calculate the BB credit requirement, simply take the round-trip cable distance and divide by the frame length. For example, at 1 Gbit/sec at 10km, 5 BB credits are required to allow wire speed data transfer. As the FC speed or distance increases the number of BB credits increases in a linear fashion. In addition, as a rule of thumb, a link with a round-trip latency of 1 ms spans a distance of approximately 100 km. With a round-trip distance of 200 km, 200 buffer credits would be required at 4 Gbit/sec. Having insufficient BB credits will not cause link failure, but will reduce the maximum throughput. So the aforementioned 1 ms link running at 4 Gbit/sec with only 100 BB credits can achieve a maximum throughput of only approximately 2Gbit/sec. Finally, these calculations use approximate metrics. The most important thing to remember is that if a given implementation had in a high percentage of FC frames significantly less than maximum size, then more BB credits may be required. Refer to the Appendix in this module for distances on specific products.
Revision 1008
7 - 19
E_Port Credit Recovery

E_Port Credit Recovery is a new enhancement for Fabric OS v6.1 Provides a more robust credit recovery for lost buffer credits or frames on long distance E_Ports
If a lost buffer credit or frame is detected, switch will perform Link Reset protocol to recover the lost credit1
Tracks both buffer credits sent and frames sent Only supported on long distance E_Port links
Only LE, LD, and LS long distance modes are supported R_RDY mode supported EX, VE, and VEX ports are not supported
E_Port Credit Recovery is enabled by default
20
Footnote 1: This is done by sending an LR (Link Reset) and the target switch sends back an LRR (Link Reset Response). Because this happens on an E_Port, the link does not reset, it just resets the BB counters. Footnote 2: The default port mode L0 is not supported. Supported on 300, 5100, 5300 and DCX Backbone (GE2 and Condor2 ASICs only) Footnote 3: To disable this feature run: portcfgcreditrecovery --disable [slot/]port
Revision 1008
7 - 20
E_Port Credit Recovery (cont.)

Prior to this enhancement, a long distance port had to lose all transmit buffer credits before a Link Reset would occur to recover the lost credits
Until all credits are lost, port would operate in a buffer starvation state In this example, two frames are lost causing the receiving switch to only send two VC_RDYs to replenish the transmitters credit count
21
Credit Recovery is defined in FC-FS BB_SC_N (Buffer-to-Buffer State Changer Number) in ELP is defined in FC-SW4. BB_SC_N: This field indicates the Buffer-to-Buffer State Change number. The BB_SC_N field is valid only if the R_RDY_Flow Control mode is specified in the ISL Flow Control Mode field. A value between 0 and 15 indicates that the sender of the ELP frame is requesting a 2BB_SC_N number of frames to be sent between two consecutive BB_SCs Primitive Signals, and a 2BB_SC_N number of R_RDY Primitive Signals be sent between two consecutive BB_SCr Primitive Signals. When the two ports exchanging link parameters specify different non-zero values of BB_SC_N, the larger value shall be used. If either port specifies a BB_SC_N value of zero, then the BB_Credit recovery process shall not be performed and no BB_SCx Primitive Signals shall be sent. If a port specifies a non-zero BB_SC_N value it shall support the BB_SCs and BB_SCr Primitive Signals. See the FC-FS-2 standard on the www.t11.org website for a description of the BB_Credit recovery process.
Revision 1008
7 - 21

With the new enhancement, lost credits and frames are identified and recovered much quicker
Extremely beneficial to long distance ports which suffer from performance drains due to lost buffer credits
Once a lost buffer is detected, switch will perform the following:

Hardware notifies software that a lost credit occurred Software generates a Link Reset to recover credits immediately portlogdump records an LR2 event RASlog is updated with an error message
22
Revision 1008
7 - 22

E_Port Credit recovery is negotiated during ELP phase Each side includes a Buffer-to-Buffer State Change Number (BB_SC_N) in the ELP The BB_SC_N has a value of 0-15
The value in the BB_SC_N is used to determine how often each side will check to verify all frames/credits are accounted for If value is 0, then new credit recovery is not used and standard credit recovery is used This ensures backward compatibility with older Fabric OS versions
Switch selects BB_SC_N based on link distance If both sides agree on the same BB_SC_N, then E_Port Credit Recovery will be used
23
Revision 1008
7 - 23

Two new primitives are used to track frames and credits sent
Both types are always used, one for frames and one for credit count
BB_SCs Primitive: credit recovery primitive for frames

Is generated every 2^BB_SC_N frames sent When a BB_SCs is received, switch will determine if 2^BB_SC_N frames have been received
BB_SCr Primitive: credit recovery primitive for credits

Is generated every 2^BB_SC_N VC_RDYs or R_RDYs sent When a BB_SCr is received, switch will determine if 2^BB_SC_N VC_RDYs or R_RDYs have been received
24
Revision 1008
7 - 24

In this example, both ends have agreed to send BB_SCs after every four frames and BB_SCr after every 4 VC_RDYs Two of the four frames are lost
As only two frames are received before receiving the BB_SCs, the receiver port issues a Link Reset to recover the lost credits
25
Revision 1008
7 - 25
portbuffershow
Display buffer allocation information with the portbuffershow command
sw2:admin> portbuffershow User Port Lx Max/Resv Buffer Needed Link Remaining Port Type Mode Buffers Usage Buffers Distance Buffers Remaining Buffers ---- ---- ---- ------- ------ ------- --------- ---------will be calculated 0 L 8 8 ... <truncated output> ... by subtracting 3 E 8 26 26 5km 4 E 8 26 26 2km the greater value: 5 E 8 26 26 2km Buffer Usage 6 8 0 7 8 0 or Max/Resv 8 E LD 56 31 31 25km 9 E LD 56 31 31 25km Buffers from 10 E 8 26 26 2km the total available 11 E 8 26 26 2km 12 E 8 26 26 2km in buffer allocation group 13 E 8 26 26 2km 14 E 8 26 26 2km 15 E 8 26 26 2km 16 8 0 ... <truncated output, all truncated ports the same as port 16>... 31 8 0 486 Remaining Buffers from 1000 available = 486
26
For each user port, the following information is displayed: Possible Port Type values are E (E_Port), F (F_Port), G (G_Port), L (L_Port), or U (U_Port). Possible Lx Mode (level) values are (Mode L0), LE (mode LE), LM (mode L0.5), L1 (mode L1), L2 (mode L2), LD (mode LD) and LS (mode LS). Max/Resv Buffers displays the maximum number of buffers that will be allocated to the port, based on the maximum possible speed for the port and the specified LX mode. In the example on this slide, port 8 is configured as mode L0.5. To support the maximum possible speed of 4 Gbit/sec, the Condor reserves 56 buffers for port 8. Buffer Usage displays the actual number of buffers the system was able to allocate to the port. In the example above, port 8 is configured as mode L0.5 but is operating at 2 Gbit/sec. To support the actual speed, the Condor allocates 31 buffers (at 2 Gbit/sec) for port 8 but reserves 56 (for 4 Gbit/sec potential) which will not be available in the Remaining Buffers. If port speed is locked to 2 Gbit/sec the Max/Resv Buffers will equal the Buffer Usage. Needed Buffers displays the estimated number of buffers that are needed for the port to operate at full bandwidth (depending on the port configuration). In the example above, 31 buffers are needed for port 8, to support a full-bandwidth, 2 Gbit/sec, mode L0.5 port. Link Distance displays (in kilometers) either the physical distance of an LD/LS link, or the maximum distance for a non-LD/LS port. In the example above, the link distance value for port 8 is 25 km, which matches the 2 Gbit/sec, mode L0.5 configuration of the port. On a Condor/GoldenEye-based switch, the Remaining Buffers value is determined by subtracting the larger of the per-port Max/Resv Buffers or Buffer Usage values from the total buffers available. Since the example above is taken from a Brocade 4100 with a Condor ASIC, there are a total of 1000 buffers available to the ports (there are no internal buffers). These values are: There are 21 ports with 8 buffers (ports 0-2, 6,7, and 16-31): 8 x 21 = 168 There are 9 ports with 26 buffers (ports 3-5, and 10-15): 26 x 9 = 234 There are 2 ports with 56 buffers (ports 8 and 9): 56 x 2 = 112 Remaining Buffers is 1000 168 234 112 = 486 which is the value shown on the last line in the slide above. Revision 1008 7 - 26
Summary
The most common reason for extending storage networks beyond 10Km geographical distances is to safeguard critical business data Long distance components include:
MMF (SWL)/SMF (LWL) Cabling ELWL SFP, Repeaters, WDM
The Extended Fabric license allows buffer credit management to extended Fibre Channel Different Long Distance settings for B-Series products are:
Payload sizes, distance and speed to add the appropriate number of required credits to a port
27
Revision 1008
7 - 27

End of Instructor-Led Module 7 Fibre Channel Long Distance
Revision 1008
Revision 1008
7 - 28
Appendix - Fibre Channel Long Distance
Revision 1008
7 - 29
Appendix A FC-PI-4 LINK NOMENCLATURE
Revision 1008
7 - 30
Appendix B Brocade Transceiver Part Numbers
Revision 1008
7 - 31
Appendix C Cable types and distances
Revision 1008
7 - 32
Appendix D Link Loss based on 8G
Revision 1008
7 - 33
Appendix E 4G Switches Extended Fabrics
Revision 1008
7 - 34
Appendix E 48k Extended Fabrics
Revision 1008
7 - 35
Appendix E 5300 Extended Fabrics
Revision 1008
7 - 36
Revision 1008
7 - 37
Revision 1008
7 - 38
Appendix E DCX Extended Fabrics
Revision 1008
7 - 39
Appendix F Trunking and Extended Fabrics
Revision 1008
7 - 40
Appendix G FC Frame Length over Fiber Optics

The given we have is it takes light 5 nsec to traverse 1 meter over a fiber optic medium, thus in 1 second light traverses 200,000,000 meters (1/.000000005). If the speed is 1 Gbit/sec, thats 100 Mbytes/sec, which is 100,000,000 bytes/sec. 200,000,000 meters / 100,000,000 bytes = 2 meters per byte. Using 2K as the given frame size, a frame is 4 km in length (2K x 2 meters per byte) when the speed is 1 Gbit/sec. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ If the speed is 2 Gbits/sec: 200,000,000 meters / 200,000,000 bytes = 1 meter per byte 2K frame x 1 meter per byte = 2 km per frame +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ If the speed is 4 Gbits/sec: 200,000,000 meters / 400,000,000 bytes = .5 meter per byte 2K frame x .5 meter per byte = 1 km per frame +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ If the speed is 8 Gbits: 200,000,000 meters / 800,000,000 bytes = .25 meter per byte 2K frame x .25 meter per byte = .5 km per frame +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Revision 1008
7 - 41

End of Instructor-Led Module 7 Appendix - Fibre Channel Long Distance Revision 1008
Revision 1008
7 - 42
Administration and Maintenance

Instructor-Led Module 8 Administration and Maintenance
Revision 1008
Revision 1008
8-1
Objectives
Describe management tools and interfaces used to manage Brocade fabrics including:
EFCM 9.7 Fabric Manager 5.5 Web Tools/Element Manager Telnet/SSH/HTTP/SSL/SNMP

Identify how ESCM is used to manage Brocade 4x0 HBAs Describe how to upgrade firmware
Standalone switch Director/DCX Backbone
Appendix Additional information on firmware downloads
Revision 1008
8-2
Centralized Management
EFCM & Fabric Manager
With today's demand for compliance and continuous data access, storage networks are growing in size and complexity. Without the right tools, management costs can skyrocket, and make business objectives and government mandates extremely difficult to achieve. Brocade SAN management solutions help organizations optimize their storage network resources, maximizing availability while improving efficiencies and reducing costs. Brocade's comprehensive management software enables centralized management of SAN resources - regardless of vendor, protocol, or geography. Whether a SAN is small or very large, straightforward or extremely complex, Brocade management software can reduce both complexity and overall storage costs. EFCM stands for Enterprise Fabric Connectivity Manager.
Revision 1008
8-3
Which Application to Use?

Installation All Brocade All Legacy McDATA Mixed Legacy McDATA & Brocade New SANs with >100 ports New customer with 48000, 5000 New customer with i10K, 6140 X X X X X X EFCM FM X
Revision 1008
8-4
Fabric Manager Overview

GUI-based application that allows monitoring and management of an entire SAN from one central location Has the ability to get high-level or detailed information about fabrics, switches and ports Launches Brocade Web Tools and related services as needed Assists by centralizing the configuration, monitoring and management tasks of SANs Enables firmware download and change management across multiple switches and fabrics Provides rapid access to SAN information across B-Series SANs
Fabric Manager lets you manage your Storage Area Network (SAN) from a single user interface. With this software, you can configure any aspect of your SAN. Fabric Manager encompasses multiple fabrics, all switch types, and all firmware versions. Use Fabric Manager to configure multiple switches simultaneously from one location. View the status of multiple devices in one window. Perform SAN-level maintenance without accessing switch after switch. Fabric Manager lets you administer all of the elements of your SAN from one point of entry. Fabric Manager features include: Firmware Downloads, Management/Monitoring Groups, Topology ISL Monitoring, Sequenced Reboots, At-a-glance Views, Subnet Scanning, License Management, Fabric Checking, Configuration Handling, Multi Fabric Administration, Fabric Merge Tool, FDMI, FICON/FICON CUP, Change Management, Fibre Channel Routing management, supportshow capturing, Performance monitoring, Alerts view, Secure Fabric OS (SFOS) Management, Reports - link to SAN Health, RADIUS, configuration and replication, Server Management Console (SMC), and FCIP Tunneling. See Brocades Fabric Manager Administrators Guide for additional information. You can additionally search for Brocade Fabric Manager courseware from Brocades Education Services web site. You should have the following information available prior to starting the installation: Fabric Manager serial number and license key (if you are installing the full version) Type of authentication to use: Windows authentication: You need to know the Windows Domain or Workgroup name. RADIUS server authentication: For each RADIUS server, you need to know the IP or DNS name, authentication port and type (PAP or CHAP), and shared secret. Switch-based authentication: You need to know the switch IP addresses. Solaris and Linux only: NIS authentication. You need to know the host name or IP address of your NIS server and NIS domain name.
Revision 1008 8-5
EFCM Offerings
Basic*
Features Single switch SANs or up to 100 ports
Standard
Up to 140 ports
Enterprise
EFCM supported scalability X X
Directors/mixed Mainframe Device Management Fabric Management Routing/extension Options Advanced Module Option (sold in port-increments) MOM (Manager of Managers) X X X X
X X X
* Standalone embedded Web server similar to Web Tools for basic device configuration zoning, port, switch
EFCM V9.7 comes in the following levels of service: EFCM Basic is the embedded web server within the M-Series products and does not require a license. The Mi10K does not have EFCM basic, use CLI or EFCM to manage. EFCM Standard Edition is intended for small business SANs, up to a maximum of 140 ports. EFCM Enterprise Edition is intended for all environments, including the largest Enterprise-class data centers. EFCM Enterprise is the only option for those customer who wish to license the optional EFCM V9.7 Advanced Module or Software Bundles. EFCM Enterprise (required for MEOS Directors) supports up to 2500 ports and 25 simultaneous clients. It also gives you the option to export configurations to flat-files and to databases (mySQL and DB2). It also allows auto discovery via subnets. Each M-Series Switch in the fabric requires an Element Manager license installed in order to be managed by EFCM. Like Fabric OS Switches/Directors, M-EOS Switch/Director license keys are linked to serial numbers. Directors with M-EOS automatically add license features for all previously purchased software. M-EOS Switches with M-EOS 6.0 or later need a feature key to enable Element Manger or any other Switch feature. When you purchase additional software license keys you receive a new feature key that includes existing features. The Advanced SAN Routing option gives you the equivalent functionality to manage classic McDATA routers by linking to SANvergence Enterprise, and is available with either the EFCM Enterprise Edition or EFCM Standard Edition. The Advanced Module enables four EFCM software modules: Performance, Event Management, Group Configuration, and Security Center. Bundling options include the Advanced modules and PFE (Product Feature Enhancement) keys for firmware-based features like Open Trunking, and FICON Management Server. Binding and Authentication are also available a la carte for individual switches or combined into Software Bundles with the Advanced Module. These software bundles are sold on a perproduct basis and are designed to provide all the advanced capabilities necessary for a particular type of environment. They are sold separately and include device-specific firmware enhancements, such as FICON, large-scale mixed fabrics, enhanced encryption and performance tuning. An FTP server can be configured and installed during the EFCM installation. Revision 1008 8-6
Managing Individual Switches

Web Tools and EFCM Basic
Used for managing individual switches or small SANs Client connects to an embedded web server on the switch
Web Tools - A browser-based application, Web Tools simplifies management by configuring, monitoring, and managing individual switches and small SANs. Available for SANs running Brocade Fabric OS. EFCM Basic - A simple, straightforward management tool for configuring, monitoring, and managing individual switches and small SANs up to 100 ports. Available for SANs running Brocade M-EOS.
Revision 1008
8-7
EFCM Basic Edition

EFCM Basic is an embedded web server that provides a webbased graphical user interface (GUI) for M-Series switches, enabling you to:
Manage and administer a single switch Monitor switches in a simple Storage Area Network (SAN) Perform troubleshooting tasks Upgrade switch firmware Manage PFE keys Perform configuration tasks Monitor switch statistics
Users can perform the following tasks from the EFCM Basic interface: Display the properties and operational status of the product, FRUs, and Fibre Channel ports, display product operating parameters, and display fabric parameters. Configure the director or switch, including: Fibre Channel port parameters, port types, and data transmission speeds Product identification, date and time, operating domain parameters, fabric parameters, and network addresses Parameters for product management through Simple Network Management Protocol (SNMP), the Command Line Interface (CLI), the Open System Management Server (OSMS) feature, or the Fibre Connection (FICON) management server (FMS) feature Security options which include authorized users, settings, port binding, switch binding, and fabric binding. Additional security includes setting access to IP ACL, Authorization, and RADIUS functions. Zones and zone sets Authentication for users Monitor ports, port optics, and port statistics, and display the event log and node list. Perform product operations and maintenance tasks, including: Enable unit beaconing, turn off the system error light, set the product online or offline, and perform a configuration reset Enable port beaconing, perform port diagnostics, and reset ports Retrieve dump files and retrieve product information files Install optional feature keys Configure product Internet Protocol (IP) addresses, names, and SNMP settings Install new versions of product firmware Manage user access to features Control product ports on an individual basis Troubleshoot problems using event log and error status indicators. Administrators and operators can access real-time information about the product and fabric. The interface requires an industry-standard web browser. EFCM Basic Edition does not require any licensing. Revision 1008 8-8
Web Tools
Enables monitoring and managing single or small fabrics, switches, and ports Provides administrative control point for optionally licensed Brocade features, including:
Advanced Zoning, ISL Trunking, Advanced Performance Monitoring, Fabric Watch, FCRS, FCIP, and a telnet interface
SSL can be used to enable secure Web Tools access to the fabric
SSL certificates are based on the switch IP address or fully qualified domain name
The Web Tools access into a fabric should be through a switch with the latest Fabric OS version
For some switch models, Web Tools provides a simplified interface, Web Tool EZ, that allows less experienced users to perform basic management tasks. Web Tools EZ is a simplified version of Web Tools. It simplifies switch management by providing an easy-to-use subset of basic switch-management tasks. Web Tools works for a single switch fabric only. It displays only the launch switch and associated tasks, without fabric information. You can use Web Tools to do the following: monitor the switch, including port and FRU status manage basic zoning perform basic switch configurations. Web Tools is part of the Fabric OS of a switch, and Web Tools functionality is engaged with a Web Tools license. When you launch Web Tools on a switch, you can manage other switches in the fabric that have lower or higher firmware versions. It is important to note that when accessing these switches you are opening the remote switchs version of Web Tools, and the functionality available for those switches might vary. From the Port Management application, you can: Swap ports Enable/disable ports Configure port type/speed
Revision 1008
8-9
ESCM (Enterprise Server Connectivity Manager)

HBA management tool loaded on the host that contains the HBAs Access:
GUI CLI
10
The GUI can be launched by itself or from EFCM (right-click on the HBA and select ESCM. To access the CLI commands run escm_cli. CLI options are:
1. 2. 4. 8. 13. 16. 20. 22. 23. 30. 31. 42. 60. 68. 69. 98. 99. p w Identify firmware, BIOS, and/or FCode Download firmware (update the FLASH) Download/erase BIOS and/or FCode (update the FLASH) Scan for devices10. Change IOC settings (interrupt coalescing, EEDP) Change FC Port settings Display logged-in devices Diagnostics Reset bus Reset target Beacon on Beacon off Display operating system names for devices Show non-default settings61. Restore default settings Show port state summary Show board manufacturing information Reset FC link Reset port Enable paged mode in menus Enable logging
Main menu, select an option: [1-99 or p/w or 0 to quit]
Revision 1008
8 - 10
ESCM (cont.)
With ESCM you can:
Collect event logs Manage HBA firmware upgrades Access the Boot BIOS View port statistics including error statistics Manage remote hosts (Import HBAs from other hosts) Perform Diagnostics Configure device persistence
11
Boot BIOS Can be accessed by interrupting boot of the host. See Brocade 400 Host Bus Adapters Users Guide for more information. Remote host HBAs can be discovered and managed. Diagnostics: Requires HBA to have access to a target device. The diagnostics will then test the path from the HBA to the target to make sure there is a reliable connection. Device Persistence: Locks the association of an FC target with a specific SCSI ID. By marking a device persistent, the Brocade adapter will remember the target mappings between power cycles. This will ensure that the OS will not accidentally move a target to a new drive letter or mount point.
Revision 1008
8 - 11
SNMP Overview
SNMP is a standard method for monitoring and managing network devices (both Ethernet and Fibre Channel) SNMP has the following components:
SNMP Entities: Network Management Stations (NMS) and Agents Management Information Bases (MIBs) Every Brocade switch runs an SNMP agent and Management Information Base (MIB) This agent accesses MIB information about the switch and makes it available to a network manager station
12
Simple Network Management Protocol (SNMP) is a standard method for monitoring and managing network devices. Using SNMP components, you can program tools to view, browse, and manipulate Brocade switch variables and set up enterpriselevel management processes. Every Brocade switch runs an SNMP agent and Management Information Base (MIB). The agent accesses MIB information about a device and makes it available to a network manager station. You can manipulate information of your choice by trapping MIB elements using the Fabric OS CLI, Web Tools, or Fabric Manager. The SNMP Access Control List (ACL) provides a way for the administrator to restrict SNMP get/set operations to certain hosts/IP addresses. This is used for enhanced management security in the storage area network. Network management is all about keeping the network up and running, monitoring and controlling network devices remotely using conventional network technology. Local management and remote management are two ways of managing a device. Enterprise management software, like HP OpenView, Tivoli, and CA Unicenter, monitor thousands of devices in an enterprise. Many of these packages have addon components that are tailored to manage Brocade SANs. There are also commercial and freely available packages that can be run separately by the SAN administrator so that they can get alerts, trend performance and capture details of errors of switches separately from large network management station.
Revision 1008
8 - 12
SNMP Components
Involves communication between a Network Management Station (NMS) and Agent (SNMP entities) SNMP uses the User Datagram Protocol (UDP) as the transport protocol for passing data between managers and agents Minimum of one Network Management Station (NMS) One or more managed nodes - each containing an agent Station and agents/entities use SNMP to exchange information Management Information Bases (MIBs) get consulted for lookups
13
Within the SNMP model, a manageable network consists of one or more manager systems (or network management stations), and a collection of agent systems (or network elements). SNMPv3 calls both manager and agent systems entities. A Manager or Network Management System (NMS) is a separate entity that manages the agent entity from a remote place. This is typically a computer that is used to run one or more network management systems. The NMS is responsible for managing the SNMP agent/entity; it is also responsible for polling and receiving traps from agents. The manager communicates with an agent using the Simple Network Management Protocol (SNMP). The Brocade agent supports both SNMP version 1 (SNMPv1) and Community-based SNMP version 2 (SNMPv2C). SNMP allows a manager to retrieve management information, such as its identification, from an agent. A manager can change management information on the agent. This operation is termed SNMP-SET. An agent can send information to the manager without being explicitly polled. This operation is termed a trap in SNMPv1 or a notification in SNMPv2C/v3. Traps or notifications alert the manager to events that occur on the agent system, such as a reboot. For the rest of the document, the term trap is used. An agent entity is a mediator between the manager entity and an end device in a SAN. The agent entity resides in a SAN device and makes the management information available to the manager entity. It acts as a mediator between management entity and an end device in the SAN. It resides in a SAN devicein the Brocade world it resides in the switch. It makes information available to the SNMP Management Entity. Brocades agent entity does the following: implements SNMP protocol, stores and retrieves management information defined in supported SNMP MIBs, and collects and maintains information about the switches local environment. MIB is an acronym for Management Information Base. A MIB is a database of network management information. The information is a combination of switch settings, hardware configuration, status and statistical data (information about the agent). Brocade MIBs can be compiled in the management station.
SAN Monitoring with SNMP

The Network Management Station (NMS) uses MIBs to poll or GET/SET information from Agents
The Agent (switch) can be configured to send traps (notifications) to the management station Brocade switches support both MIBs and traps Traps are defined in MIBs and must be configured on both agents (switches) and NMS
14
Every Brocade switch carries an Agent and Management Information Base (MIB), as illustrated above. The agent accesses information about a device and makes it available to an SNMP NMS. When active, the NMS can get information or set information when it queries an agent. SNMP commands, such as: get, set, getnext, setnext, and getresponse are sent from the NMS, and the agent replies once the value is obtained or modified. Agents use variables to report such data as the number of bytes and packets in and out of the device, or the number of broadcast messages sent and received. These variables are also known as managed objects. All managed objects are contained in the MIB. The NMS can also receive traps, unsolicited messages, from the switch (agent) if an unusual event occurs. NMS SNMP application must be configured to respond to traps. To receive SNMP traps ensure proper MIBs are enabled and the trap recipient(s) on the switch have been set.
Revision 1008
8 - 14
SNMP Version Authentication

SNMPv1 - uses community strings to authenticate SNMPv3 - changes 1st-level authentication to user-name and provides authentication and privacy services Change SNMP authentication with the smnpconfig command
15
SNMPv1 Authentication between the NMS and the switch is performed using unsecure clear text community strings only. SNMPv3 Authentication between the NMS and the switch is performed using an MD5 or SHA cryptographic hash of the Username stored on both the switch and the NMS station. In addition privacy is also supported in SNMPv3 by encrypting packets using DES. SNMPv3 supports thee security levels: No authentication and no privacy Username is not hashed and data is not encrypted. Authentication and no privacy Username is hashed but data is not encrypted. Authentication and privacy Username is hashed and data is encrypted. The security subsystem can contain more than one security model. In the case of Brocade switches, for example, it contains: SNMPv1 with community strings; SNMP v3 user-names, authentication, and privacy; and Access Control Lists (ACLs). Brocade enabled SNMP ACLs in all Fabric OS versions. Authentication occurs using either MD5 or SHA algorithm checks. MD5, message-digest algorithm, is an extension of the MD4 algorithm. The MD5 algorithm takes an input a message of arbitrary length and produces an output 128-bit "fingerprint" or "message-digest". It is designed for circumstances where a large file must be "compressed" in a secure manner before being encrypted with a private (secret) key under a public-key cryptosystem such as RSA. RSA is a public-key encryption technology created by Ron Rivest, Adi Shamir, and Leonard Adleman. RSA was built from the fact that, given that it is pretty easy to multiply two very large prime numbers together to get their product, it is extremely difficult to go the reverse direction: to find the two prime factors of a composite number. This one-way nature of RSA allows an encryption key to be generated and shared with the world, and still not allow a message to be decrypted. SHA is an acronym which stands for Secure Hash Standard. SHA is modeled after MD4. Like MD5, SHA takes a message and produces a message-digest, its output is 160-bit. The message-digest can then be put into the Digital Signature Algorithm (DSA) to generate or verify the signature for the message. This signing of the message-digest rather than the message is designed to improve the efficiency. The same algorithm must be used at both ends of the communication. DES is a Digital Encryption Standard that uses a secret key to encrypt/decrypt messages. Example configure output SNMP authentication section: RSL1_ST07_B41:admin> configure Not all options will be available on an enabled switch. To disable the switch, use the "switchDisable" command. Configure... System services (yes, y, no, n): [no] ssl attributes (yes, y, no, n): [no] http attributes (yes, y, no, n): [no] snmp attributes (yes, y, no, n): [no] y Select SNMP Security Level: (0 = No security, 1 = Authentication only, 2 = Authentication and Privacy): (0..2) [0]
Revision 1008
8 - 15
Brocade SNMP Fabric OS Commands

Commands to configure SNMP are related to Brocade Fabric OS versions Use snmpconfig to set the agent and traps and management of features such as enable SNMPv3
b5100:admin> snmpconfig --set seclevel Select SNMP GET Security Level (0 = No security, 1 = Authentication only, 2 = Authentication and Privacy, 3 = No Access): (0..3) [0]
16
To configure an external management application to receive traps: Verify that the community string matches the value used in the monitoring tools. Ensure the proper MIBs are enabled.
Revision 1008
8 - 16
Brocade Single-CP Switch

Firmware Download Internal Process
1. The firmwaredownload command is entered 2. Firmware is downloaded to Secondary Partition 3. Primary and Secondary boot pointers are swapped 4. CP boots from firmware in new Primary Partition 5. Firmware in Primary Partition is committed to Secondary 6. Download complete!
17
Revision 1008
8 - 17

Current Firmware Status
Running in Switch RAM Primary Partition Secondary Partition
FOS A
FOS A
FOS A
Compact Flash
18
The firmware download process for a single-CP switch is less complex than for the Brocade directors. The flash is divided into two equal size partitions. The version of the firmware running on the switch is in the Primary Partition and another version of firmware is kept in the Secondary Partition. The version in the Secondary can be the same as the Primary or different. The following slides will illustrate the individual steps that occur when a firmwaredownload command is issued.
Revision 1008
8 - 18

Issue firmwaredownload Command
>firmwaredownload 10.255.23.43,admin,/v6.x/6.1.0c,password
FOS A
FOS A
FOS A
Compact Flash
19
The firmwaredownload command is issued to download Fabric OS version v6.1.0c. The default firmwaredownload options are automatically executed. The default settings are to auto-reboot and auto-commit. If the -s option is used and auto-boot mode is not enabled, a reboot command must be issued to activate the downloaded firmware. If the -s option is used and auto-commit mode is not enabled, a firmwarecommit command must be issued to replicate the downloaded firmware to the Secondary Partition. Current status: Firmware FOS A is in the Primary Partition and has been committed to the Secondary Partition.
Revision 1008
8 - 19

Firmware Downloaded to Secondary Partition
FOS A
FOS A
FOS A FOS B
Compact Flash
20
The firmware is downloaded into the Secondary Partition.
Revision 1008
8 - 20

Primary and Secondary Pointers are Swapped
FOS A
FOS A
FOS A FOS B
Compact Flash
21
Upon completion of the download, the Secondary Partition becomes the Primary and the Primary Partition becomes the Secondary.
Revision 1008
8 - 21

New Firmware Reboots
FOS A
FOS A FOS B
FOS B
Compact Flash
22
After rebooting, the new version is now running on the switch.
Revision 1008
8 - 22

New Firmware Commits
FOS B
FOS A FOS B
FOS B
Compact Flash
23
The firmware in the Primary Partition is committed to the Secondary Partition.
Revision 1008
8 - 23

Firmware Download is Complete
Success!
FOS B
FOS B
FOS B
Compact Flash
24
The results of firmwaredownload are: r7-st04-b51-1:admin> firmwareshow Appl FOS Primary/Secondary Versions v6.1.0c v6.1.0c ------------------------------------------
Revision 1008
8 - 24
CP Differences for Directors and Backbones

CP4 Blade One CPU per blade Each compact flash contains a primary and secondary partition CP8 Blade Two CPUs per blade (DCX contains 4 CPUs) The main-CPU and co-CPU are connected through an internal Ethernet No need to do a separate download to the co-CPU Each compact flash contains a primary and secondary partition
25
Revision 1008
8 - 25
Non-Disruptive CP Card Failover

The first CP Card to boot is the Active CP, which runs Fabric OS and presents the switch IP address The second CP is the Standby CP Active/Standby status is controlled by the HA Monitor (HAM) running on both CPs
Standby CP will become Active CP if the HAM detects a failure
CP Card failover is non-disruptive - switch port LED stay lit during the CP card failover
Nx_Port traffic continues - not interrupted
26
The Active CP: manages POST for all FRUs runs Fabric OS; configures all port cards for data transfers, hard-zoning, and routing interacts with all attached devices presents the IP addresses for the switch and the CP card issues all SNMP-based messages illuminates the Active CP LED synchronizes the Standby CP to the Active CP through the High Availability Monitor. The Standby CP runs only those parts of Fabric OS associated with CP failover, so only a limited set of management tasks can be performed on the Standby CP. The High Availability Monitor (HAM) tracks several failover mechanisms, including the CP Error Line and private IP-based heartbeat between the CP Cards; the Watchdog Timer running on each CP Card; and the Health Monitor running on the Standby CP card is ready to become active at any time. The failover to the Standby CP is a hot failover that is, all I/Os continue uninterrupted through the switch ports. The flashing of LEDs is driven by Fabric OS. Thus, during the CP card failover, the software that would normally flash the port LEDs cannot do so, and the port LEDs will remain solidly lit. To ensure stable functionality during a failover, you must install the same version of Fabric OS on both CP cards, and have the same version of Fabric OS running on both CP cards. The procedures that you will perform later in this class conform to this requirement. Use the hashow command to determine the current High Availability (HA) status: B48000:admin> hashow Local CP (Slot 5, CP0): Active Remote CP (Slot 6, CP1): Standby, Healthy HA enabled, Heartbeat Up, HA State synchronized
High Availability Commands

For a non-disruptive CP Card failover:
Two CP Cards must be installed, with the same firmware on each Both CP Cards must be healthy The HA Monitor synchronized between the CP Cards
To monitor the current high-availability status on a Brocade Director, use the hashow command, and look for:
Which CP is Active? Which CP is Standby? Is the Standby CP Healthy? (ready to become the Active CP) Is HA enabled between the CP Cards? Is the Heartbeat up and working? Are the HA Monitors on the CPs synchronized?
r6-st01-dcx-1:admin> hashow Local CP (Slot 6, CP0): Active, Cold Recovered Remote CP (Slot 7, CP1): Standby, Healthy HA enabled, Heartbeat Up, HA State synchronized
27
The hashow command displays many details about the current high-availability status of the Brocade Director including: Local CP state (slot number, CP ID): Active or Standby Remote CP state (slot number, CP ID): Active or Standby High Availability: HA Enabled or HA Disabled Heartbeat: Up or Down Health of standby CP: Healthy (standby CP is running, and the health monitor has not detected any errors), Failed (standby CP is running, but the health monitor has discovered a problem with the blade), and Unknown (the standby CP does not exist, the heartbeat is down, or the health monitor detects a configuration file error). HA synchronization status: HA State Synchronized (the system is currently fully synchronized; a CP Card failover would be non-disruptive) and HA State Not In Sync (the system is unable to synchronize the two CPs, due to the standby CP being faulty or another system error; a CP Card failover would be disruptive). In the example, CP0 is the Active CP, and CP1 is the Standby CP, which is healthy. HA is enabled on this Director, the Heartbeat is up, and the HA Monitor is synchronized; thus, a non-disruptive failover would occur.
Revision 1008
8 - 27
Other HA Commands
Other commands related to managing high availability include:
hasyncstart hasyncstop Enable HA synchronization Disable HA synchronization
haenable Enable the HA feature hadisable hafailover Disable the HA feature Force a manual CP Card failover from the Active CP
28
These commands are rarely used by most system administrators.
Revision 1008
8 - 28
Firmware Maintenance CP Cards

Compact Flash managed as two partitions Switch boots from the primary partition Firmware downloads to the secondary partition and is used to boot from if the primary becomes corrupt After downloading firmware, you can commit the new firmware, or restore the old firmware A firmware download will be nondisruptive if:
Two CP Cards are installed, with the same firmware on each Both CP Cards are healthy HA Monitor is synchronized between the CP Cards
29
In the example, P2 is currently the primary partition and holds the version of firmware currently running on this CP card (FW vC). P1 is the secondary partition and holds an older revision of Fabric OS firmware (FW vB).
Revision 1008
8 - 29
Firmware Download Process

The firmware download process in general is identical for both the DCX and 48000:
1. 2. 3. 4. 5. 6. 7. 8. firmwaredownload command is entered on Active CP Firmware is downloaded to Standby CP Standby CP reboots Failover causes Active and Standby CPs to reverse roles Firmware is downloaded to new Standby CP New Standby CP reboots New Active CP and New Standby CP commit firmware Download complete!
30
RSL_48K8:admin> firmwaredownload This command will upgrade both CPs in the switch. If you want to upgrade a single CP only, please use -s option. You can run firmwareDownloadStatus to get the status of this command. This command will cause the active CP to reset and will require that existing telnet, secure telnet or SSH sessions be restarted. Do you want to continue [Y]: y Server Name or IP Address: 10.255.253.43 User Name: root File Name: /training/firmware/v6.0.0 Password: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. Firmwaredownload has completed successfully on Standby CP. Standby CP reboots. Standby CP booted up. Standby CP booted up with new firmware.
Revision 1008
8 - 30
48000 Firmware Upgrade

.
Running on CP0
Active CP
Primary Partition Secondary Partition
Standby CP
Running on CP1
FOS A
FOS A
FOS A
FOS A
FOS A
FOS A
CP0 CompactFlash
CP1 CompactFlash
31
The firmware download process for a Brocade Director is more complex than for other Brocade switches. It has dual Control Processor cards (CP0 and CP1) with one performing an active role and the other a standby role. Each CP card has a Compact Flash card that holds the Fabric OS firmware. The flash is divided into two equal size partitions. The version of the firmware controlling the switch is in the Primary Partition and another version of firmware is kept in the Secondary Partition. The version in the Secondary can be the same as the Primary or different. It is these factors that make the firmware process more complex. The following slides will illustrate the individual steps that occur when a firmwaredownload command is issued.
Revision 1008
8 - 31
Perform firmwaredownload to Active CP

>firmwaredownload 10.255.23.43,admin,/v6.x/6.0.0/,password
Running on CP0
Active CP
Standby CP
Running on CP1
FOS A
FOS A
FOS A
FOS A
FOS A
FOS A
CP0 CompactFlash
CP1 CompactFlash
32
The firmwaredownload command is issued from the Active CP to download Fabric OS version v6.0.0. The default firmwaredownload options are automatically executed. The default settings are to auto-reboot and auto-commit. If the -s option is used and auto-boot mode is not enabled, a reboot command must be issued to activate the downloaded firmware. If the -s option is used and auto-commit mode is not enabled, a firmwarecommit command must be issued to replicate the downloaded firmware to the Secondary Partition. Current status: The Active CP is CP0; the Standby CP is CP1. Firmware FOS A is in the Primary Partition and is committed to the Secondary Partition on both CP cards. The firmwareshow command issued from the current standby CP will have the following output (notice that the Secondary Partition is being written):
RSL_48K8:admin> firmwareshow Local CP (Slot 6, CP1): Standby Primary partition: Secondary Partition: Remote CP (Slot 5, CP0): Active Primary partition: Secondary Partition:
v5.3 Version is not available v5.3 v5.3
Revision 1008
8 - 32
Active CP Instructs Standby CP to Download
Running on CP0
Active CP
Primary Partition
Firmware Download
Standby CP
Running on CP1
Secondary Partition
FOS A
FOS A
FOS A
FOS A
FOS A
FOS A FOS B
CP0 CompactFlash
CP1 CompactFlash
33
The Active CP instructs the Standby CP to perform a firmware download of version v6.0.0. The firmware is downloaded into the Secondary Partition. Upon completion, the Secondary Partition becomes the Primary and the Primary Partition becomes the Standby. If the Standby CP does not respond within 10 minutes, the Active CP will time out and abort the command. As mentioned above, we perform all Brocade Directors firmware downloads to the Standby CP card. Before the firmware download process begins, Fabric OS checks to see that both CP cards are running the same version of Fabric OS. If they are running different versions, you will have to update each CP card separately. Before proceeding from this step, new firmware should be downloaded to the standby CP. The Standby CP should send a status=OK to the Active CP, Active CP will respond with reboot instructions and then the Standby will begin the reboot. The results of the firmwaredownloadstatus command, issued from the standby CP appear as follows:
RSL_48K8:admin> firmwaredownloadstatus [0]: Thu Aug 28 20:01:53 2005 cp0: Firmwaredownload has started on Standby CP. It may take up to 30 minutes.
Revision 1008
8 - 33
Standby CP Notifies Download Successful
Running on CP0
Active CP
Primary Partition
Status = OK
Standby CP
Secondary Partition
Running on CP1
Secondary Partition
Primary Partition
FOS A
FOS A FOS A
FOS A FOS A FOS A FOS B FOS B
CP0 CompactFlash
CP1 CompactFlash
34
Upon successful completion of the firmware download, the Standby CP returns status to the Active CP. The results of the firmwaredownloadstatus command issued from current Standby CP are: RSL_48K8:admin> firmwaredownloadstatus [0]: Thu Aug 28 20:01:53 2005 cp0: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [1]: Thu Aug 28 20:07:15 2005 cp0: Firmwaredownload has completed successfully on Standby CP.
Revision 1008
8 - 34
Active CP Instructs Standby CP to Reboot
Running on CP0
Active CP
Primary Partition
Reboot
Standby CP
Running on CP1
Secondary Partition
FOS A
FOS A FOSA
FOS A
FOS A
FOS A
FOS B FOS B
FOS B
CP0 CompactFlash
CP1 CompactFlash
35
If the return status is OK, the Active CP instructs the Standby CP to reboot. After reboot, version 6.0.0 is now running on CP1 (and is still the Standby). If it doesnt receive a heartbeat from the Standby CP within 4 minutes, the Active CP will time out and abort the command. The results of the firmwaredownloadstatus command issued from current Standby CP are:
RSL_48K8:admin> firmwaredownloadstatus [0]: Thu Aug 28 20:01:53 2005 cp0: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [1]: Thu Aug 28 20:07:15 2005 cp0: Firmwaredownload has completed successfully on Standby CP. [2]: Thu Aug 28 20:07:18 2005 cp0: Standby CP reboots.
Revision 1008
8 - 35
Standby CP Synchronizes with Active CP
Running on CP0
Active CP
Primary Partition
Heartbeat
Standby CP
Running on CP1
Secondary Partition
FOS A
FOS B
FOS A
FOS A
FOS A
FOS B
CP0 CompactFlash
CP1 CompactFlash
36
After reboot, the Standby CP synchronizes with the Active CP by a heartbeat signal. The output of the firmwaredownloadstatus command is: RSL_48K8:admin> firmwaredownloadstatus [0]: Thu Aug 28 20:01:54 2005 cp0: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [1]: Thu Aug 28 20:07:15 2005 cp0: Firmwaredownload has completed successfully on Standby CP. [2]: Thu Aug 28 20:07:18 2005 cp0: Standby CP reboots. [3]: Thu Aug 28 20:10:13 2005 cp0: Standby CP booted up
Revision 1008
8 - 36
Active CP Initiates Failover
Running on CP0
Active CP
Primary Partition
Failover
Standby CP
Running on CP1
Secondary Partition
FOS A
FOS B
FOS A
FOS A
FOS A
FOS B
CP0 CompactFlash
CP1 CompactFlash
37
The Active CP is ready to initiate a failover to the Standby CP.
Revision 1008
8 - 37
Standby CP and Active CP Exchange Roles
Running on CP0
Standby CP Active CP
Active CP Standby CP
Running on CP1
FOS A
FOS B
FOS A
FOS A
FOS A
FOS B
CP0 CompactFlash
CP1 CompactFlash
38
Failover will cause the Standby CP to become the Active and the Active CP to become the Standby. Notice the hashow, still synchronized output:
RSL_48K8:admin> hashow Local CP (Slot 5, CP0): Standby Remote CP (Slot 6, CP1): Active HA enabled, Heartbeat Up, HA State synchronized
Notice firmwareshow, issued from standby CP shows the new Active CP, Primary Partition and Secondary Partition firmware status:
RSL_48K8:admin> firmwareshow Local CP (Slot 5, CP0): Standby Primary partition: Secondary Partition: Remote CP (Slot 6, CP1): Active Primary partition: Secondary Partition: v6.0.0 v5.3 v5.3 Version is not available
Here is the same command issued from Active CP:

RSL_48K8:admin> firmwareshow Local CP (Slot 6, CP1): Active Primary partition: Secondary Partition: Primary partition: Secondary Partition: v6.0.0 v5.3 v5.3 Version is not available
Remote CP (Slot 5, CP0): Standby
Revision 1008
8 - 38
Standby CP Synchronizes with Active CP
Running on CP0
Standby CP
Primary Partition
Heartbeat
Active CP
Secondary Partition
Running on CP1
Secondary Partition
Primary Partition
FOS A
FOS B
FOS A
FOS A
FOS A
FOS B
CP0 CompactFlash
CP1 CompactFlash
39
Note: From this slide forward, the Active CP refers to CP1 and the Standby CP refers to CP0. The Active CP waits for the Standby CP to synchronize with a heartbeat signal. This indicates failover has completed. Notice the firmware download status at this time:
RSL_48K8:admin> firmwaredownloadstatus [0]: Thu Aug 28 20:01:53 2005 cp0: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [1]: Thu Aug 28 20:07:15 2005 cp0: Firmwaredownload has completed successfully on Standby CP. [2]: Thu Aug 28 20:07:18 2005 cp0: Standby CP reboots. [3]: Thu Aug 28 20:10:14 2005 cp0: Standby CP booted up. [4]: Thu Aug 28 20:13:04 2005 cp1: Active CP forced failover succeeded. Now this CP becomes Active.
Revision 1008
8 - 39
Active CP Instructs Standby CP to Download
Running on CP0
Standby CP
Primary Partition
Firmware Download
Active CP
Secondary Partition
Running on CP1
Secondary Partition
Primary Partition
FOS A
FOS B
FOS A
FOS A FOS B
FOS A
FOS B
CP0 CompactFlash
CP1 CompactFlash
40
The Active CP instructs the Standby CP to perform a firmware download of version v6.0.0. The firmware is downloaded into the Secondary Partition. Upon completion, the Secondary Partition becomes the Primary and the Primary Partition becomes the Secondary.
RSL_48K8:admin> firmwaredownloadstatus [0]: Thu Aug 28 20:01:54 2005 cp0: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [1]: Thu Aug 28 20:07:15 2005 cp0: Firmwaredownload has completed successfully on Standby CP. [2]: Thu Aug 28 20:07:18 2005 cp0: Standby CP reboots. [3]: Thu Aug 28 20:10:13 2005 cp0: Standby CP booted up. [4]: Thu Aug 28 20:13:04 2005 cp1: Active CP forced failover succeeded. Now this CP becomes Active. [5]: Thu Aug 28 20:13:07 2005 cp1: Firmwaredownload has started on Standby CP. It may take up to 30 minutes.
Revision 1008
8 - 40
Standby CP Notifies Download Successful
Running on CP0
Standby CP
Primary Partition
Status = OK
Active CP
Secondary Partition
Running on CP1
Secondary Partition
Primary Partition
FOS A
FOS B
FOS A
FOS B
FOS A
FOS B
CP0 CompactFlash
CP1 CompactFlash
41
Upon successful completion of the firmware download, the Standby CP returns status to the Active CP. Current outputs as this process occurs: RSL_48K8:admin> hashow Local CP (Slot 6, CP1): Active Remote CP (Slot 5, CP0): Standby, Healthy HA enabled, Heartbeat Up, HA State not in sync RSL_48K8:admin> hashow Local CP (Slot 6, CP1): Active Remote CP (Slot 5, CP0): Standby, Healthy HA enabled, Heartbeat Up, HA State synchronized
Revision 1008
8 - 41
Active CP Instructs Standby CP to Reboot
Running on CP0
Standby CP
Reboot
Active CP
Running on CP1
FOS A
FOS B
FOS A FOS B
FOS B
FOS A
FOS B
CP0 CompactFlash
CP1 CompactFlash
42
After reboot, version v6.0.0 is now running on CP0.

RSL_48K8:admin> firmwaredownloadstatus [0]: Thu Aug 28 20:01:54 2005 cp0: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [1]: Thu Aug 28 20:07:15 2005 cp0: Firmwaredownload has completed successfully on Standby CP. [2]: Thu Aug 28 20:07:18 2005 cp0: Standby CP reboots. [3]: Thu Aug 28 20:10:13 2005 cp0: Standby CP booted up. [4]: Thu Aug 28 20:13:04 2005 cp1: Active CP forced failover succeeded. Now this CP becomes Active. [5]: Thu Aug 28 20:13:07 2005 cp1: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [6]: Thu Aug 28 20:18:41 2005 cp1: Firmwaredownload has completed successfully on Standby CP. [7]: Thu Aug 28 20:18:43 2005 cp1: Standby CP reboots. [8]: Thu Aug 28 20:22:02 2005 cp1: Standby CP booted up with new firmware.
Revision 1008
8 - 42
Active and Standby CP Commit Firmware
Running on CP0
Standby CP
Active CP
Running on CP1
FOS B
FOS B
FOS A FOS B
FOS B
FOS A FOS B
FOS B
CP0 CompactFlash
CP1 CompactFlash
43
The firmware in the Primary Partition on the Standby CP is committed to the Secondary Partition.
RSL_48K8:admin> firmwaredownloadstatus [0]: Thu Aug 28 20:01:54 2005 cp0: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [1]: Thu Aug 28 20:07:15 2005 cp0: Firmwaredownload has completed successfully on Standby CP. [2]: Thu Aug 28 20:07:18 2005 cp0: Standby CP reboots. [3]: Thu Aug 28 20:10:13 2005 cp0: Standby CP booted up. [4]: Thu Aug 28 20:13:04 2005 cp1: Active CP forced failover succeeded. Now this CP becomes Active. [5]: Thu Aug 28 20:13:07 2005 cp1: Firmwaredownload has started on Standby CP. It may take up to 30 minutes. [6]: Thu Aug 28 20:18:41 2005 cp1: Firmwaredownload has completed successfully on Standby CP. [7]: Thu Aug 28 20:18:43 2005 cp1: Standby CP reboots. [8]: Thu Aug 28 20:22:02 2005 cp1: Standby CP booted up with new firmware. [9]: Thu Aug 28 20:22:05 2005 cp1: Firmwarecommit has started on both Active and Standby CPs.
Revision 1008
8 - 43
Firmware Download is Complete
Success!
Running on CP0
Standby CP
Secondary Partition Primary Partition
Active CP
Secondary Partition Primary Partition
Running on CP1
FOS B
FOS B
FOS B
FOS B
FOS B
FOS B
CP0 CompactFlash
CP1 CompactFlash
44
The results of firmwaredownload are: RSL_48K8:admin> firmwareshow -v Slot Name 6 CP0 Appl FOS Co-FOS 7 CP1 FOS Co-FOS Primary/Secondary Versions v6.0.0a v6.0.0a v6.0.0a v6.0.0a v6.0.0a v6.0.0a v6.0.0a v6.0.0a Standby Status Active -------------------------------------------------------
Revision 1008
8 - 44
DCX Firmware Upgrade

Firmware download process has been improved for the DCX Only downloads the code from the source once instead of twice like on the 48000 With FOS v6.0.0+, firmware download now shows percentage of completion during the download:
Completed download of 4/104 packages (3%). Please wait
If replacing CP, you still must check firmware and upgrade using firmwaredownload s if required It will upgrade the main-CPU and co-CPU on that blade
45
Revision 1008
8 - 45
DCX Firmware Upgrade (cont.)

Active CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: A Sec Partition FW: A Compact Flash Standby CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: A Sec Partition FW: A
In this example, we will upgrade firmware on both CPs from version A to version B using the firmwaredownload command.
46
Prm Part = Primary Partition Sec Part = Secondary Partition FW A = Old firmware FW B = New firmware The total time to upgrade the firmware is up to 30 minutes (about the same as the 48000).
Revision 1008
8 - 46

Active CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: A Sec Partition FW: A Standby CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A
Download firmware
Sec Partition FW: B
Sec Partition FW: A
Step 1: Download the new firmware to the secondary partition of the main CPU on the standby CP from the server. This is the only access to the FTP Server.
47
Revision 1008
8 - 47

Active CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: A Sec Partition FW: A Standby CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: B Sec Partition FW: B
Step 2: The firmware is copied via the internal network from the standby CP main CPU to the Co-CPU.
48
Revision 1008
8 - 48

Active CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: A Sec Partition FW: A Standby CP Main CPU Primary Part FW: B Co-CPU Primary Part FW: B Sec Partition FW: A Sec Partition FW: A
Step 3: Swap the primary and secondary partition pointers on the Standby CP. Now the primary partitions on the Standby CP contain the new firmware. Step 4: Reboot the standby CP; now running the new B version code on the standby CP.
49
Revision 1008
8 - 49

Standby CP Active CP Standby CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: A Sec Partition FW: A CP Failover Active CP Standby CP MainActive CP CPU Primary Part FW: B Co-CPU Primary Part FW: B Sec Partition FW: A Sec Partition FW: A
Step 5: Force the active CP to failover. Now the old standby becomes the active which is running the new B version of the code.
50
Revision 1008
8 - 50

Standby CP Standby CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: A Sec Partition FW: B Active CP Active CP Main CPU Primary Part
Firmware is FW: B pulled from Active CP Co-CPU
Sec Partition FW: A
Primary Part FW: B
Sec Partition FW: A
Step 6: Copy the firmware from the active CP main CPU to the standby CP main CPU secondary partition. This is done via the internal network.
51
Revision 1008
8 - 51

Standby CP Standby CP Main CPU Primary Part FW: A Co-CPU Primary Part FW: A Sec Partition FW: B Sec Partition FW: B Active CP Active CP Main CPU Primary Part FW: B Co-CPU Primary Part FW: B Sec Partition FW: A Sec Partition FW: A
Step 7: The firmware is copied via the internal network from the standby CP main CPU to the Co-CPU.
52
Revision 1008
8 - 52

Standby CP Standby CP Main CPU Primary Part FW: B Co-CPU Primary Part FW: B Sec Partition FW: A Sec Partition FW: A Active CP Active CP Main CPU Primary Part FW: B Co-CPU Primary Part FW: B Sec Partition FW: A Sec Partition FW: A
Step 8: Swap the primary and secondary partition pointers on the Standby CP. Now the primary partitions on the Standby CP contain the new firmware. Step 9: Reboot the standby CP; now both the active and standby CPs are running the new B version code.
53
Revision 1008
8 - 53

Standby CP Standby CP Main CPU Primary Part FW: B Co-CPU Primary Part FW: B Sec Partition FW: B Sec Partition FW: B Active CP Active CP Main CPU Primary Part FW: B Co-CPU Primary Part FW: B Sec Partition FW: B Sec Partition FW: B
Step 10: Commit both CPUs on both CPs to the new firmware. Now all four partitions on both CPs are at the B version of code.
54
Revision 1008
8 - 54
Firmware Maintenance Single-CP Mode

For greater control of the firmware download, use the single-CP mode (firmwaredownload -s), which enables auto-commit and reboot options:
B48000:admin> firmwaredownload -s Server Name or IP Address: 10.255.252.50 User Name: anonymous File Name: /v6.x/v6.0.0c Network Protocol(1-auto-select, 2-FTP, 3-SCP) [1]: 2 Password: Do Auto-Commit after Reboot [Y]: N Reboot system after download [N]: Checking system settings for firmwaredownload... Protocol selected: FTP Trying address-->AF_INET IP: 10.255.252.50, flags : 2 System settings check passed. You are running firmwaredownload with auto-reboot disabled. After firmware is downloaded, please reboot the system to activate the new firmware. Do you want to continue [Y]: Y Firmwaredownload has started.
55
When the s argument is added to the firmwaredownload command, two additional options are available: -b: Autoreboot mode; if included, switch reboots automatically after the firmware download completes. -n: Disable autocommit mode, so firmware is downloaded only to the secondary partition. The example above was recorded on a dual-CP Brocade 48000, running Fabric OS v6.0.0a. The only argument included at the command line is s, so you are prompted to provide the FTP server IP address, user name, firmware filename, and password, as well as the two single-CP options available in Fabric OS v5.x+.
Revision 1008
8 - 55
Firmware Maintenance Other Commands

Other firmware-related commands include:
firmwaredownloadstatus Real-time status of a firmware download firmwarecommit Copy firmware from the primary partition to the secondary partition (no reboot) firmwarerestore Copy firmware from the secondary partition to the primary partition (reboot occurs) firmwareshow Display the firmware in both partitions of all CP Cards and any AP blades
B48000:admin> firmwareshow Slot Name Appl Primary/Secondary Versions Status --------------------------------------------------------------2 FR4-18i FOS v6.0.0 v6.0.0 4 FC4-16IP FOS v6.0.0 v6.0.0 5 CP0 FOS v6.0.0 ACTIVE * v6.0.0 6 CP1 FOS v6.0.0 STANDBY v6.0.0
56
The example above is on a dual-CP Brocade 48000. Both CP cards have Fabric OS v6.0.0 installed on both memory partitions. The auto-leveling process automatically ensures compatibility between the version of Fabric OS running on the Active CP and all of the AP blades installed in the Director chassis. When the Brocade 48000 boots up or an advanced blade (FR418i, FA4-18 and FC4-16IP) is inserted or powered on, the Active CP determines if the installed advanced blades are running the same version of Fabric OS as the active CP. If the Fabric OS versions differ, the Active CP will install the BFOS image on all installed advanced blades. Auto-leveling occurs automatically, without administrator intervention. Auto-leveling occurs: After CP failover (CP error), including during a dual-CP firmware download (e.g. firmwaredownload -s) After hot-plug of an FR4-18i, or FC4-16IP blade or when slotpoweron is invoked on a slot with these blades The active CP always updates the FC4-16IP blade and MAY update the FR418i blade Auto-leveling is always enabled, and cannot be disabled
Revision 1008
8 - 56
Displaying Control Processor Status

firmwareshow
Before invoking a firmwaredownload on a dual CP director, verify that the CPs are in sync and the firmware versions match Issue the firmwareshow and hashow commands
SW48000:admin> firmwareshow Local CP (Slot 5, CP0): Active Primary partition: v6.0.0 Secondary Partition: v6.0.0 Remote CP (Slot 6, CP1): Standby Primary partition: v6.0.0 Secondary Partition: v6.0.0 Note: If Local CP and Remote CP have different versions of firmware, please retry firmwaredownload command.
57
Revision 1008
8 - 57
Downloading Firmware for M-Series

Firmware can be downloaded to a M-Series product by:
Serial Console Element Manager EFCM - Group Manager EFCM Basic (not Mi10k) Mi10k can use CLI
58
Revision 1008
8 - 58
Archiving Switch Configuration

configupload
Saves the switch configuration as an ASCII text file to an FTP server Perform this anytime before and after the configuration changes:
Zoning or Fabric Watch is modified License keys are added New switch is added to the fabric Changes are made to switch configuration Remove from FTP server to a secure location for security reasons
Ways to perform
CLI Web Tools Fabric Manager
59
This command uploads a current switch configuration to a host file on a FTP server. The upload process uses either FTP (File Transfer Protocol) or the RSHD protocol (TCP service 514). Both of these services are widely available on Unix hosts, but less so on Windows hosts. For security reasons it is recommended to remove the switch configuration file from the FTP server to a secure location. The command may be invoked without any parameters, in which case the user is prompted for input, including choice or FTP or RSHD (VxWorks switches only). RSHD is not an option on Linux based switches. In the case of VxWorks switches, if invoked from one command line session (non-interactive) with three parameters then RSHD is used, the presence of the fourth parameter (FTP password) selects FTP. Here is an example from an interactive session on a VxWorks switch:
rsl1_st07_b38_1:admin> configupload Server Name or IP Address [host1] : 10.255.252.50 User Name [none] : anonymous File Name [config.txt] : config.txt Protocol (RSHD or FTP) [FTP] : ftp Password : upload complete
The upload may fail for many reasons, including: the host name is not known to the switch the host IP address can't be contacted the user doesn't have permission on the host the user runs a script that prints something at login the RSHD (VxWorks only) or FTP server isn't running on the destination host Note: Opening the ASCII txt files in Microsoft Word could change formatting and cause the file to become corrupt. WordPad is a better option.
Revision 1008
8 - 59
Archiving Switch Configuration

configupload Example
RSL1_ST05_B41:admin> configupload Protocol (scp or ftp) [ftp]: ftp Server Name or IP Address [host]: 10.255.252.50 User Name [user]: configs File Name [config.txt]: sw4100_st05_date.txt Password: Upload complete
Management Server FTP Server
configupload
txt file
60
FOS switches: configupload / configdownload License information Port configuration Security policies SNMP configuration Switch configuration parameters Zoning information
Revision 1008
8 - 60
Restoring Switch Configuration

configdownload Downloads ASCII text file from an FTP server to the switch When to perform
Switch replacement Restore switch settings Replicate Fabric Watch profiles Restore SNMP settings Restore zoning
txt file configdownload Management Server FTP Server
Boot parameters are not restored (switch name, IP address, etc.) A license key is only accepted if boot.mac line = WWN
61
This command downloads the switch configuration from a host file. The configuration file may have been generated previously using configupload or may have been created by a user to download only specific configuration changes.
rsl1_st07_b41_1:admin> switchdisable rsl1_st07_b41_1:admin> configdownload Protocol (scp or ftp) [ftp]: ftp Server Name or IP Address [host]: 10.255.252.50 User Name [user]: configs File Name [config.txt]: config.txt Password: *** CAUTION *** This command is used to download a backed-up configuration for a specific switch. If using a file from a different switch, this file's configuration settings will override any current switch settings. Downloading a configuration file, which was uploaded from a different type of switch, may cause this switch to fail. A switch reboot might be required for some parameter changes to take effect. Do you want to continue [y/n]: y download completed.
Note: You must re-enable the switch to see the effects of the configuration changes.
Revision 1008
8 - 61
File Copy Protocols

Brocade supports a couple of options for transferring files during:
Configuration uploads and downloads Firmware downloads
Brocade supports the following protocols for file transfer:

USB - Must be supported for the switch (8 Gbit/sec switches) FTP SCP
62
Revision 1008
8 - 62
Summary
Management tools and interfaces used to manage Brocade fabrics include:
EFCM 9.7 Fabric Manager 5.5 Web Tools/Element Manager Telnet/SSH/HTTP/SSL/SNMP
ESCM is used to manage Brocade 4x0 HBAs Upgrade firmware

Standalone switch Director/DCX Backbone
63
Revision 1008
8 - 63

End of Instructor-Led Module 8 Administration and Maintenance
Revision 1008
Revision 1008
8 - 64
Appendix - Administration and Maintenance

More information on firmware download
Revision 1008
8 - 65
Appendix A Error Handling

Provides information on how the DCX recovers if an error occurs during the firmware upgrade process
66
Revision 1008
8 - 66
No
1
When
During step 1
Scenario of Error
During downloading to the main CPU of the standby CP, if an error occurs and the main CPU reboots. During downloading to the co CPU of the standby CP, if an error occurs and the co CPU reboots. During downloading to any of the CPUs on the standby CP, if the downloading takes too long and exceeds the 30 minute timeout. Active CP fails over during the downloading to any of the CPUs on the standby CP. If the standby CP failed to reboot or unable to synchronize with the active CP.
Error Handling
1)When the main CPU boots up, firmwaredownload is aborted. 2)firmwarecommit will be initiated on the main CPU and the original firmware is restored on that CPU. 3)Both CPUs on both CPs will have the original firmware. 1)When the co CPU boots up, firmwaredownload is aborted. 2)firmwarecommit will be initiated on both CPUs and the original firmware is stored on both CPUs. 3)Both CPUs on both CPs will have the original firmware. 1)firmwarecommit will be initiated on both CPU on the standby CP and the original firmware is restored on both CPUs. 2)Both CPUs on both CPs will have the original firmware.
User action
Restart firmwaredownload after the repair is done.
During step 2
During step 1 or 2
During step 1,2
1)firmwarecommit will be initiated on both CPU on the new active CP and the original firmware is restored on both CPUs. 2)Both CPUs on both CPs will have the original firmware. 1)Active CP will wait for 10 minutes and abort firmwaredownload. 2)If the standby CP boots up, firmwarecommit will start on both CPUs on the standby CP. 3)Both CPUs on the standby CP will have the new firmware, and both CPUs on the active CP will have the old firmware. 1)When the main CPU on the standby CP boots up, firmwaredownload is aborted. 2)firmwarecommit will be initiated on both CPUs on both CPs. 3)Both CPUs on the standby CP will have the old firmware and both CPUs on the active CP will have the new firmware.
During step 4
Determine why the CPs fail to gain HA sync and remedy it before restarting firmwaredownload
During step 6
When downloading to the main CPU of the standby CP, if an error occurs and the main CPU reboots.
Revision 1008
8 - 67
No
7
When
During step 7
Scenario of Error
When downloading to the co CPU of the standby CP, if an error occurs and the co CPU reboots. When downloading to any of the CPUs on the standby CP, downloading takes too long and exceeds the 30 minute timeout. The active CP fails over during the downloading to any of the CPUs on the standby CP.
Error Handling
1)When the co CPU on the standby CP boots up, firmwaredownload is aborted. 2)firmwarecommit will be initiated on both CPUs on both CPs. 3)Both CPUs on the standby CP will have the old firmware and both CPUs on the active CP will have the new firmware. 1)firmwarecommit will be initiated on both CPU on both CPs. 2)Both CPUs on the standby CP will have the old firmware and both CPUs on the active CP will have the new firmware.
User action
During step 6 or 7
During step 6 or 7
1)The standby CP will become the new active CP. 2)firmwarecommit will be initiated on both CPUs on the active CP and the original firmware is restored on both CPUs. 3)When the standby CP boots, firmwarecommit will be initiated on both CPUs on the standby CP and they will have the new firmware. 4)Both CPUs on the standby CP will have the new firmware and both CPUs on the active CP will have the old firmware. 1)Active CP will wait for 10 minutes and abort firmwaredownload. 2)firmwarecommit will be initiated on both CPUs on the active CP. 3)If the standby CP boots up, firmwarecommit will start on both CPUs on the standby CP. 4)Both CPUs on the both CPs will have the new firmware The affect CPUs will have different versions of firmware on its partitions. An error message is logged.
10
During step 8
If the standby CP failed to reboot or unable to synchronize with the active CP.
Determine why the CPs fail to gain HA sync and remedy it before restarting firmwaredownload
11
During step 10
If commit fails
It requires fixing the switch manually. It rarely happens.
Revision 1008
8 - 68
Appendix B DCX Firmware Upgrade

Sample outputs for the following commands:
firmwaredownload firmwaredownloadstatus (No change in this output) errdump firmwareshow -v
Note: The step number you will see in the next three slides refer back to the DCX Firmware Upgrade steps covered earlier in this module.
69
Revision 1008
8 - 69
DCX10:admin> firmwaredownload Server Name or IP Address: 10.255.252.50 User Name: upd207 File Name: /firmware/v6.0.0 Network Protocol(1-auto-select, 2-FTP, 3-SCP) [1]: 2 Password: xyz Checking system settings for firmwaredownload... Protocol selected: FTP Trying address-->AF_INET IP: 10.255.252.50, flags : 2 System settings check passed. This command will upgrade the firmware on both CP blades. If you want to upgrade firmware on a single CP only, please use -s option. You may run firmwaredownloadstatus to get the status of this command. This command will cause a warm/non-disruptive boot on the active CP, but will require that existing telnet, secure telnet or SSH sessions be restarted.
Do you want to continue [Y]: Y Firmware is being downloaded to standby CP. This step may take up to 30 minutes. Firmware is being downloaded to Standby CP. Please wait... Completed download of 2/104 packages (1%). Please wait... Completed download of 4/104 packages (3%). Please wait... Completed download of 8/104 packages (7%). Please wait... <Trunked Output>. Completion . . percentage . is displayed . <Trunked Output> Completed download of 101/104 packages (97%). Please wait... Completed download of 101/104 packages (97%). Please wait... Completed download of 104/104 packages (100%). Please wait... Firmware has been downloaded successfully to Standby CP. Standby CP is going to reboot with new firmware. Standby CP booted successfully with new firmware.
Revision 1008
8 - 70
DCX10:admin> firmwaredownloadstatus [1]: Wed Oct 24 22:13:41 2007 (Step 1: Only access to the FTP Server). Slot 6 (CP0, active): Firmware is being downloaded to standby CP. This step may take up to 30 minutes. [2]: Wed Oct 24 22:18:56 2007 Slot 6 (CP0, active): Firmware has been downloaded successfully to Standby CP. [3]: Wed Oct 24 22:19:01 2007 (Step 3 and 4) Slot 6 (CP0, active): Standby CP is going to reboot with new firmware. [4]: Wed Oct 24 22:20:19 2007 Slot 6 (CP0, active): Standby CP booted successfully with new firmware. [5]: Wed Oct 24 22:21:23 2007 (Step 5) Slot 7 (CP1, active): Forced failover succeeded. New Active CP is running new firmware [6]: Wed Oct 24 22:31:30 2007 (Step 6: This is done via the internal network) Slot 7 (CP1, active): Firmware is being downloaded to standby CP. This step may take up to 30 minutes. [7]: Slot [8]: Slot [9]: Slot Wed Oct 7 (CP1, Wed Oct 7 (CP1, Wed Oct 7 (CP1, 24 22:35:07 2007 active): Firmware has been downloaded successfully on Standby CP. 24 22:35:11 2007 (Steps 8 and 9) active): Standby CP reboots. 24 22:36:27 2007 active): Standby CP booted successfully with new firmware.
[10]: Wed Oct 24 22:36:30 2007 (Step 10) Slot 7 (CP1, active): Firmware commit operation has started on both active and standby CPs. [11]: Wed Oct 24 22:36:30 2007 Slot 7 (CP1, active): The firmware commit operation has started. This may take up to 10 minutes. [12]: Wed Oct 24 22:39:04 2007 Slot 7 (CP1, active): The commit operation has completed successfully. [13]: Wed Oct 24 22:39:05 2007 Slot 7 (CP1, active): Firmware commit operation has completed successfully on both CPs. [14]: Wed Oct 24 22:39:05 2007 Slot 7 (CP1, active): Firmwaredownload command has completed successfully. Use firmwareshow to verify the firmware versions.
Revision 1008
8 - 71
DCX10:admin>errdump <Trunked Output> 2007/10/25-01:08:09, [SULB-1001], 9349,, WARNING, ?, Firmwaredownload command has started. 2007/10/25-01:08:09, [SULB-1036], 9350,, INFO, ?, The current Version: Fabric OS v6.0.0_beta1 2007/10/25-01:10:48, [FSSM-1002], 9398,, INFO, ?, HA State is in sync. 2007/10/25-01:11:38, [SULB-1007], 9399,, INFO, ?, Standby CP reboots. (Step 4) 2007/10/25-01:11:38, [FSSM-1003], 9400,, WARNING, ?, HA State out of sync. 2007/10/25-01:11:47, [HAM-1005], 9401,, INFO, ?, HeartBeat Miss reached threshold. 2007/10/25-01:11:47, [HAMK-1004], 9402,, INFO, ?, Resetting standby CP (double reset may occur) 2007/10/25-01:11:47, [EM-1033], 9403,, ERROR, ?, CP in Slot 7 set to faulty because CP ERROR asserted. (Message is normal, this is the CP failover step 5.) 2007/10/25-01:12:02, [EM-1047], 9404,, INFO, ?, CP in slot 7 not faulty, CP ERROR deasserted. 2007/10/25-01:12:50, [FSSM-1002], 9405,, INFO, ?, HA State is in sync. 2007/10/25-01:12:56, [SULB-1008], 9406,, INFO, ?, Standby CP booted successfully with new firmware. (Step 9) 2007/10/25-01:12:59, [SULB-1003], 9407,, INFO, ?, Firmwarecommit has started. (Start of step 10) 2007/10/25-01:15:25, [SULB-1004], 9408,, INFO, ?, Firmwarecommit has completed. (Completion of step 10) 2007/10/25-01:15:25, [SULB-1002], 9409,, INFO, ?, Firmwaredownload command has completed successfully. 2007/10/25-01:15:25, [SULB-1036], 9410,, INFO, ?, The new Version: Fabric OS v6.0.0 <Trunked Output>
Revision 1008
8 - 72
firmwareshow -v
Must use v option to see co-CPU
DCX10:admin> firmwareshow -v Slot Name Appl Primary/Secondary Versions Status -----------------------------------------------------------------------6 CP0 FOS v6.0.0 ACTIVE * v6.0.0 Co-FOS v6.0.0 v6.0.0 7 CP1 FOS v6.0.0 STANDBY v6.0.0 Co-FOS v6.0.0 v6.0.0 * Local CP
73
Note if the firmware versions do not match you will see the following message at the bottom of this output:
WARNING: The local CP and remote CP have different versions of firmware, please retry firmwaredownload command.
If a firmware download was just done suggest waiting several minutes for the firmware to be copied to all partitions. Suggest waiting for about 30 minutes: If the firmware still has not been copied to all partitions then I would look at the firmwaredownloadstatus and errdump output to see what happened.
Revision 1008
8 - 73

End of Instructor-Led Module 8 Appendix - Administration and Maintenance Revision 1008
Revision 1008
8 - 74

CFP270 SG 1008

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CFP270 SG 1008

Uploaded by

Copyright:

Available Formats

CFP270: Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Course Overview

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Brocade Certification Program

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

What is a Brocade Certification Worth?

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Course Objectives

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Course Objectives (cont.)

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Course Prerequisites

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Instructor-Led Course Agenda

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Registering for the BCFP Exam

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

The Fulfillment Process

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Training Facility and Training Policies

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

This page is intentionally blank.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Hardware Product Features

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Hardware Product Features

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Hardware Product Features

The Brocade Family of Products

200E, 300, 4900, 5000, 5100 and 5300

48000, Mi10K, M6140 and DCX

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Hardware Product Features

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Hardware Product Features

Condor2 / Condor ASIC Comparison

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory

Hardware Product Features

Virtual Channels (VCs)1

2008 Brocade Communications Systems, Inc. All Rights Reserved.

CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory