You are on page 1of 144

DellTMBladeServerConfigurationand

Troubleshooting

Contents
Introduction..................................................................................................................................................8
CourseSafety............................................................................................................................................8
ElectrostaticDischarge..............................................................................................................................8
Module1:BladeSystemTechnology..........................................................................................................10
Introduction............................................................................................................................................10
OverviewofBladeTechnology...............................................................................................................10
BladeorBladeServer..........................................................................................................................11
BladeSystem.......................................................................................................................................11
CommonTerms.......................................................................................................................................12
Fabric...................................................................................................................................................12
Lane.....................................................................................................................................................12
Link......................................................................................................................................................12
Port......................................................................................................................................................13
ProductLines...........................................................................................................................................13
Summary.................................................................................................................................................13
Module2:DellBladeSystemEnclosures....................................................................................................14
ModuleObjectives..................................................................................................................................14
M1000eModularBladeEnclosure:EnclosureBasics.........................................................................14
EnclosureOverview................................................................................................................................14
BasicConfigurationandExpansion.....................................................................................................15
TheFrontoftheEnclosure..................................................................................................................16
TheBackoftheEnclosure...................................................................................................................16
CablingOrder......................................................................................................................................17
Cooling................................................................................................................................................18
EnclosureComponents...........................................................................................................................19
1|P a g e

BladeServerConfiguration&Troubleshooting

ChassisManagementController(CMC)..............................................................................................19
iDRAC..................................................................................................................................................20
iKVM....................................................................................................................................................20
IOMs....................................................................................................................................................21
Fans.....................................................................................................................................................21
Midplane.............................................................................................................................................23
PSUs....................................................................................................................................................24
PowerDistributionUnits.....................................................................................................................24
HotSwappableComponents..............................................................................................................25
I/OModules............................................................................................................................................26
TopicObjectives..................................................................................................................................26
Fabrics.................................................................................................................................................26
PassthroughModules........................................................................................................................27
ChoosingIOMs....................................................................................................................................28
FabricConsistencyChecking...............................................................................................................28
StackingIOMs.....................................................................................................................................29
PowerConnectTMM6220Switch..........................................................................................................30
CiscoSwitch........................................................................................................................................31
Redundancy............................................................................................................................................32
CMCRedundancy................................................................................................................................32
FabricRedundancy..............................................................................................................................32
FanRedundancy..................................................................................................................................33
PSURedundancy.................................................................................................................................33
Summary.................................................................................................................................................34
Module3:KeyComponents.......................................................................................................................37
ModuleObjectives..................................................................................................................................37
CommonFeatures...................................................................................................................................37
Blades..................................................................................................................................................37
Features..............................................................................................................................................37
MezzanineCards.....................................................................................................................................40
HardDriveOptions..................................................................................................................................40
SerialAdvancedTechnologyAttachment(SATA)II............................................................................40
2|P a g e

BladeServerConfiguration&Troubleshooting

SerialAttachedSCSI(SAS)...................................................................................................................40
EmbeddedEthernetControllers.........................................................................................................41
iDRAC..................................................................................................................................................41
Differences..............................................................................................................................................42
M600UniqueFeatures.......................................................................................................................43
M805andM905UniqueFeatures......................................................................................................44
DriveControllers.....................................................................................................................................45
DriveControllerOptions.....................................................................................................................45
CERC6i................................................................................................................................................46
CERC6iCachePolicy...........................................................................................................................46
CERC6iConfigurationUtility...............................................................................................................47
CreatinganArrayontheSAS6/iR.......................................................................................................48
SAS6/iRVirtualDiskManagement......................................................................................................53
Viewvirtualdiskproperties................................................................................................................53
Synchronizeavirtualdisk....................................................................................................................54
Activateavirtualdisk..........................................................................................................................54
Deleteavirtualdisk............................................................................................................................54
InternalStorage......................................................................................................................................55
DisklessConfigurations.......................................................................................................................55
SATARepeaterDaughterCard............................................................................................................55
SAS6/iRModularDaughterCard.........................................................................................................56
CERC6iModularDaughterCard.........................................................................................................57
StorageDaughterCardInstallationandRemoval...................................................................................58
BladeStorageDaughterCardInstallation...........................................................................................59
BladeStorageDaughterCardRemoval...............................................................................................59
SAS/SATAHDDBackplaneComponents.................................................................................................60
SAS/SATAHDDBackplane...................................................................................................................60
HDDIndicatorLEDs.............................................................................................................................61
MechanicalIssues...............................................................................................................................61
ClusteringandVirtualization...................................................................................................................62
ClusterComputing..............................................................................................................................62
Virtualization.......................................................................................................................................62
3|P a g e

BladeServerConfiguration&Troubleshooting

Summary.............................................................................................................................................63
Module4:SystemsManagementHardware..............................................................................................65
ModuleObjectives..................................................................................................................................65
ManagementBasics................................................................................................................................65
ManagementConnections..................................................................................................................65
ManagementFeatures........................................................................................................................66
OperatorPanelandLCD......................................................................................................................68
InterpretingtheOperatorPanelLED..................................................................................................68
LCDMenus..........................................................................................................................................69
ChassisManagementController.............................................................................................................69
FeaturesandCapabilities....................................................................................................................70
CMCCabling........................................................................................................................................71
DaisychainConnection......................................................................................................................71
Redundancy........................................................................................................................................72
InitialConfiguration............................................................................................................................72
IntegratedKVM.......................................................................................................................................73
Features..............................................................................................................................................73
Capabilities..........................................................................................................................................74
OSCARConfiguration..........................................................................................................................75
IntegratedDRAC......................................................................................................................................75
EmbeddedbutSeparate.....................................................................................................................76
ManagementFeatures........................................................................................................................76
ManagementInterface.......................................................................................................................77
SecurityFeatures................................................................................................................................79
OutofbandCommunication..............................................................................................................79
ConfigurationInterfaces.....................................................................................................................80
iDRACConfigurationUtility.................................................................................................................81
iDRACWebinterface...........................................................................................................................81
CMCWebInterface.............................................................................................................................81
ChassisLCDPanel................................................................................................................................81
Localracadm.......................................................................................................................................81
OpenManageIntegration........................................................................................................................82
4|P a g e

BladeServerConfiguration&Troubleshooting

OpenManageServerAdministrator....................................................................................................82
ITAssistantIntegrationnowreferredtoasDMCprovidesPowerMonitoringforservers
supportingPMBusreportswiththefollowingtypesofdata:.............................................................83
Summary.................................................................................................................................................84
Module5:DellOpenManageSoftware......................................................................................................87
ModuleObjectives..................................................................................................................................87
SystemManagementBasics...................................................................................................................87
WhatIsManagement?........................................................................................................................88
DellOpenManageSystemOverview..................................................................................................88
TheDellOpenManageModel.............................................................................................................89
SystemsBuildandUpdateUtility........................................................................................................90
Deploymenttoolkit.............................................................................................................................91
ServerManagement:RemoteManagement......................................................................................91
ServerManagement:InbandInterfaces............................................................................................92
SystemsManagementProtocols........................................................................................................93
Review.....................................................................................................................................................93
DellOpenManageServerAdministrator.................................................................................................93
OMSA:InstrumentationService.........................................................................................................94
OMSAStorageManagementService..................................................................................................94
OMSA:RemoteAccessController.......................................................................................................96
MonitoringTools:ITAssistant...........................................................................................................97
MonitoringTools:BaseboardManagementController......................................................................97
MaintenanceTools:DellServerUpdateUtility..................................................................................98
MaintenanceTools:DellOnlineDiagnostics......................................................................................98
Summary.................................................................................................................................................99
Module6:iDRAC.......................................................................................................................................100
iDRACManagementandSecurityFeatures..........................................................................................100
iDRACManagementFeatures...........................................................................................................101
iDRACSecurityFeatures....................................................................................................................101
iDRACSecurityFeatures....................................................................................................................102
ConfigurationTasks...............................................................................................................................103
ConfiguringtheManagementStation..............................................................................................103
5|P a g e

BladeServerConfiguration&Troubleshooting

ConfiguringtheiDRAC.......................................................................................................................105
ConfiguretheManagedServer.........................................................................................................108
ConfiguringtheiDRACUsingtheWebInterface..............................................................................110
ConfiguringtheiDRACUsingtheWebInterface..............................................................................110
UsingConsoleRedirection....................................................................................................................112
ConsoleRedirectionConsiderations.................................................................................................112
ConfiguringConsoleRedirectionintheiDRACWebInterface.........................................................113
UsingtheVideoViewer.....................................................................................................................113
UsingtheVideoViewer.....................................................................................................................113
Summary...............................................................................................................................................114
Module7:NetworkFabricsandIOMs......................................................................................................116
ChassisOverview..................................................................................................................................116
FabricABroadcomEthernetControllers..............................................................................................117
FabricAFeatures..............................................................................................................................117
DellLOMTOELicenseKey.................................................................................................................117
ExpansionCardsOverview....................................................................................................................118
MezzanineCards...................................................................................................................................118
DellBCM57086S................................................................................................................................118
EmulexLPe1105M4FCHBA.............................................................................................................118
QLogicQME2472FCHBA..................................................................................................................119
QlogicSANBladeQNE247MFibreChannelHBATechnicalSpecifications........................................120
IOMs......................................................................................................................................................121
Dell/EmulexPT1016FibreChannelPassThrough...........................................................................121
Dell/EmulexPT1016.........................................................................................................................121
Dell/EmulexPT1016FibreChannelPassThrough...........................................................................122
KeyFeatures......................................................................................................................................122
TechnicalSpecifications....................................................................................................................122
Brocade4424FibreChannelSwitch.................................................................................................123
InfiniBandTechnology..........................................................................................................................124
AboutInfiniBand...............................................................................................................................124
HowInfiniBandWorks......................................................................................................................124
AdvantagesofInfiniBand..................................................................................................................125
6|P a g e

BladeServerConfiguration&Troubleshooting

Cabling...............................................................................................................................................126
LinkSpeeds........................................................................................................................................126
InfiniBandPhysicalLayerCharacteristics.........................................................................................127
ParallelTransmissioninInfiniBandLinks..........................................................................................127
MellanoxConnectXIBMDIInfiniBandHostChannelAdapter(HCA)MezzanineCard....................128
InfiniBandIOM:CiscoSFSM7000EIBSwitch...................................................................................131
TechnicalSpecifications....................................................................................................................133
PowerOnIndicators.........................................................................................................................134
Summary...........................................................................................................................................135
Module8:Troubleshooting......................................................................................................................137
CommonConfigurationIssuesforBlades.............................................................................................137
Objectives..............................................................................................................................................137
ReceiveMessagestatingthattheCMCsarenotredundant...............................................................137
Solutions............................................................................................................................................137
UpdatetheFirmware........................................................................................................................137
MessagereceivedintheiDRACsellog.................................................................................................138
FC(QLGCorEMLX)Mezzaninewouldexperience"F1/F2"duringpost...........................................138
Servernonresponsiveatpowerup......................................................................................................138
TroubleshootingResource....................................................................................................................139
Support.dell.com...................................................................................................................................140
Ftp.dell.com..........................................................................................................................................141
Module9:NavigatingDellInformationandTools....................................................................................142
DellSystemESupportTool(DSET).......................................................................................................142
Support.dell.com...................................................................................................................................142
Learndell.com........................................................................................................................................143
EnterpriseSupportServices..................................................................................................................143
ThankYou!............................................................................................................................................143

7|P a g e

BladeServerConfiguration&Troubleshooting

Introduction

CourseSafety
Throughoutthecourse,youwillbeexpectedtoobservestrictsafetyproceduresatalltimes.These
include(butarenotlimitedto):

Alwaysuseandcarrytoolsinthecorrectmanner.
Neverliftanythingthatyoufeelistooheavy.
Neverleavepartsinplaceswhereothersmaytripoverthem.
Whendisassemblingsystems,alwaysthinkaboutotherpeoplenearyou.
Ifyouseeanythingthatyoufeelisapossiblesafetyconcern,pleasealertyourinstructor
immediately.

Alotoftheequipmentusedduringthiscourseisextremelyheavy.Ifyoufeelthatanypieceof
equipmentistooheavyforyoutoeasilycarry,pleaseseekassistance.

ElectrostaticDischarge
Electrostaticdischarge(ESD)isamajorconcernwhenhandlingcomponents,especiallyexpansioncards
andsystemboards.Veryslightchargescandamagecircuitsinwaysthatmaynotbeobvious.
Intermittentproblemsorashortenedproductlifespancanalsoresult.

StaticElectricityAchargestoredinanybody.
ElectrostaticDischargeAsuddentransferofelectrostaticchargebetweenbodiesatdifferent
electrostaticpotential,usuallyasasparkasthebodiesapproachoneanother.
ESDcandamageelectroniccomponentsimmediately,orwithdelayedeffect.
YoumusttakeprecautionstopreventESD.

Adischargeofstaticelectricitycandamagethecomponentsofyourcomputer.Electrostaticdischarge
occurswhen,bytouchingacomponent,youcauseasuddenvoltagesurgethatequalizestheelectricity
betweenyouandthecomponent.ByusingthewristgroundingstrapthatcomesintheESDkitand
takingseveralprecautions,youcanminimizethechancesofadischargeoccurring.
Ifawristgroundingstrapisnotavailable,youcandischargethestaticelectricityinyourbodyby
touchinganunpaintedmetalsurface,suchasonthecomputerchassis.
Beforeyouunplugthemachine,performthefollowingsteps:whilethesystemispluggedintotheEarth
circuitviathepowersocket,attachtheESDwriststraptoyourwristandcliptheotherendtoaground
object.
Thenyoucanunplugthemachine.Staticsensitivecomponentsarrivewrappedinantistaticpacking
material.
Dothefollowingwhenhandlingstaticsensitivecomponents:

8|P a g e

BladeServerConfiguration&Troubleshooting

UseanESDwristgroundingstrap.
Handleallsensitivecomponentsinastaticsafearea.
Ifpossible,useantistaticfloormatsandworkbenchpads.

Whenunpackingastaticsensitivecomponentfromitsshippingcarton,donotremovethecomponent
fromtheantistaticpackingmaterialuntilyouarereadytoinstallthecomponentintoyoursystem.

9|P a g e

BladeServerConfiguration&Troubleshooting

Module1:BladeSystem
Technology
Introduction
Thismoduleintroducesthebasic
termsandconceptsthatare
developedinthiscourse.After
completingthismodule,youwill
beableto:

Definethebasictermsand
conceptsusedinthis
course.
Explainthebenefitsof
bladetechnology.
Listthecomponentsofthe
Mseriesbladeservers.

OverviewofBladeTechnology
Bladeserverscanbeviewedasacompleteserverchassis,acomponentonasystemboard,atypeof
computerconfiguration,orsimplyasatypeofserver:

Abladeserverisaserverchassishousingmultiplethin,modularelectroniccircuitboards,known
asserverblades.Eachbladeisaserverinitsownright,oftendedicatedtoasingleapplication.
Thebladesareliterallyserversonacard,containingprocessors,memory,integratednetwork
controllers,anoptionalFibrechannelhostbusadaptor(HBA)andotherinput/output(IO)ports.
AbladeserverisacomputeronaSystemboard,whichincludesprocessor,memory,and
sometimesstorage.Thebladeserverisintendedtoaddresstheneedsoflargescalecomputing
centerstoreducespacerequirementsandlowercosts.
Bladeserversarepartofacomputerconfigurationwherepower,cooling,storageand
connectivityarelargelyprovidedbyanouterhousingorchassis.Thechassiscontainsand
servicesanumberofspecialized,strippeddownSystemboardunitsthebladeserverseach
oneacompletecomputerorservicedevicecontainingonlyvitalprocessingandstorage
elements.
Bladeserversareselfcontainedcomputerservers,designedforhighdensity.

10|P a g e

BladeServerConfiguration&Troubleshooting

BladeorBladeServer
BladeorBladeServerreferstotheindividualserversina
bladeenclosure.Dellbladesincludebutarenotlimited
to:

PowerEdge19559Gblade
PowerEdgeM60010GIntelbasedblade
PowerEdgeM60510GAMDbasedblade
PowerEdgeM80510GAMDbasedblade
PowerEdgeM90510GAMDbasedblade
PowerEdgeM61011GIntelbasedblade
PowerEdgeM71011GIntelbasedblade

BladeEnclosure
Bladeenclosurereferstotheenclosurethatprovides
power,cooling,andconnectivitytotheblades.Dell
bladeenclosuresinclude:
PowerEdge1955enclosure
PowerEdgeM1000eenclosure

BladeSystem
Abladesystemreferstothegestaltofthebladeenclosure
andthebladesitcontains.
Theenclosureprovidespower,cooling,andconnectivityfor
theblades.Withintheenclosure,theindividualbladeservers
canbeclustered(likediskarrays)orfunctionindependently.

11|P a g e

BladeServerConfiguration&Troubleshooting

CommonTerms
Fabric
Amethodofencoding,transporting,andsynchronizingdatabetweendevices.Examplesoffabrics
include:

FibreChannel(FC)
GigabitEthernet(GE)
InfiniBand(IB)

FabricsarecarriedinsidethePowerEdgeM1000eenclosure.Theycarryinformationbetweenbladesand
I/Omodules(IOMs)throughthemidplane.
FabricsarealsocarriedoutsidetheM1000ethroughthephysicalinterfacesontheIOMs.
Lane
AsinglefabricdatatransportpathbetweenIOMsandenddevices.Eachlanecomprisesonetransmit
andonereceivecircuit.Differentialsignalingreducesnoiseinterference.
Physically,alaneisfourwiresinacableorfourcoppertracesonaprintedcircuitboard,eachof
whichcarriesoneofthefollowingsignals:

Apositivetransmitsignal
Anegativetransmitsignal
Apositivereceivesignal
Anegativereceivesignal

Specifictermsforlanesvarybytechnology:

Ethernet:Link
FibreChannel:Link
InfiniBand:PhysicalLane
PCIe:Lane
Inthiscourse,linkhasadifferentdefinition.

Link
AcollectionofmultiplefabriclanesusedtoformasinglecommunicationtransportpathbetweenlOMs
andenddevices.Examplesinclude:

Two,four,oreightlanePCIe
Fourlane10GbKX4PCIe

12|P a g e

BladeServerConfiguration&Troubleshooting

AlthoughEthernetandFibrechannelusethistermforbothsingleandmultiplelanes,thiscourseuses
thetermsdistinctlytoavoidconfusioninthelearningprocess.Alink,asusedinthiscourse,provides
synchronizationacrossmultiplelanessothattheyactasasingletransport.
Port
ThephysicalI/Oendinterfaceofadevicetoalinkorlane.Aportcanconnecttomultiplelanes.

ProductLines
Dellcurrentlyofferstwobladeproductlines.

ThePowerEdge1955line,whichpackagesboth
bladesandabladeenclosure
TheMseries,whichcomprises:
o TheM1000ebladeenclosure
o TheM600Intelbasedblade
o TheM605AMDbasedblade
o TheM805AMDbasedblade
o TheM905AMDbasedblade

Summary

Reviewthesummaryofthismodulebeforetakingtheselfchecktoseehowwellyoumastered
thetopic.
Certainbladeterminologycanhavedifferentmeaningsintheindustry.Forthepurposesofthis
course,wedefinethesetermsasfollows:
o BladeorBladeServerreferstotheindividualserversinabladeenclosure.
o Bladeenclosurereferstotheenclosurethatprovidespower,cooling,andconnectivity
totheblades.
o Bladesystemisadatacenterinachassis.
o Afabricisamethodofencoding,transporting,andsynchronizingdatabetweendevices.
o AlaneisasinglefabricdatatransportpathbetweenIOMsandenddevices.
o Alinkisacollectionofmultiplefabriclanesusedtoformasinglecommunication
transportpathbetweenIOMsandenddevices.
o AportisthephysicalI/Oendinterfaceofadevicetoalinkorlane.

13|P a g e

BladeServerConfiguration&Troubleshooting

Module2:DellBladeSystemEnclosures
ModuleObjectives
M1000eModularBladeEnclosure:EnclosureBasics
ThismoduledescribesthePowerEdgeM1000eModular
BladeEnclosureandthecomponentsthatmakeitwork.
TheM1000eintegratesthelatestinmanagement,I/O,
power,andcoolingtechnologiesinaconvenient
modularpackage.Designedtosupportcurrentand
futuregenerationsofserver,storage,networking,and
managementtechnologies,theM1000eincludesthe
headroomnecessarytoscaleyourenvironmenttomeet
yourfutureneeds.
Aftercompletingthistopic,youwillbeableto:

ListtheM1000efeaturesandcomponents.
ListtheminimalconfigurationoftheM1000e.
ExplainhowmezzaninecardsandIOMsinteract
infabrics.
ListtheorderinwhichyoushouldconnectcableswheninstallingtheM1000e.
DescribethecoolingpathsoftheM1000e.

EnclosureOverview
TheM1000eisa10Uenclosurethatholdsupto16blades.Thatmeansyoucanhave64bladesina40U
space.
TheM1000esolutioncomprises:

Network,storage,andclusterinterconnectmodules(switchesandpassthroughmodules)
Ahighperformanceandhighlyavailablepassivemidplanethatconnectsbladeservermodules
totheinfrastructurecomponents
Powersupplies
Fans
IntegratedKVM(iKVM)andChassisManagementControllers(CMCs)

ThePowerEdgeM1000eusesredundantandhotpluggablecomponentsthroughouttoprovide
maximumuptime.
Benefits:

Simplifiedoperations,withexpandedmanagementfeatures

14|P a g e

BladeServerConfiguration&Troubleshooting

Energysavingsandincreasedcomputecapacitywithefficientpowersupplies,fansanddynamic
powermanagement
Increasedcomputingdensity:64blades,512processorcoresina42Urack(2Uremainfree)

ComparisonofthePowerEdge1955andMl000eBladeEnclosures
1955
Ml000e
FormFactor
7U
10U
BladeCapacity
10(60in42Us)
16(64in4OUs)
MultislotBlades
No
Yes
I/OBays
4/2fabrics
6/3
Fibrechannel
Switchorpassthrough
Switchorpassthrough
Ethernet
Switchorpassthrough
Switchorpassthrough
InfiniBand
No
Yes
PowerSupplies
4
6(3+3)
Fans
4(2fansin2bays)
9(3fansin3zones)
ChassisManagement
No
Upto2
Modules
KVM
Yes
Optional
SimultaneousVirtual
No
Yes
KVM
InteractiveLCD
No
Yes
PowerManagement
No
Yes
PowerMonitoring
No
OnCMC
InternalPersistent
Bladelevel
Bladeandchassislevel
Storage

BasicConfigurationandExpansion
TheminimalconfigurationoftheM1000eis:

OneCMC:AddingasecondCMCprovidesredundancy.
Threepowersupplyunits(PSUs):Tomakeiteasiertoupgrade,Dellrecommendsaddingan
additionalpackageofthreePSUs.
Ninefans:Withallninefanspopulated,youneverhavetoworryabouthavingsufficientcooling
forexpansion.

Youalsoneedatleastonebladetomaketheenclosureuseful.

15|P a g e

BladeServerConfiguration&Troubleshooting

TheFrontoftheEnclosure

Upto16bladesareaccessiblefromthefrontofthe
M1000e.Onthebottomfrontleftcornerofthe
chassisthereisacontrolpaneltoconnecttothe
integratedKVM(iKVM)onthebackoftheenclosure.
Thisconnectionallowsyoutoconnectacrashcart,if
needed.

TheBackoftheEnclosure

Fromthebackoftheenclosure,youcanaccess:

OneortwoCMCs(oneoneachupper
rightandleftcorner)
AnintegratedKVM(iKVM)
UptosixPSUs
Ninefans
Fabricconnections(A,B,andCintwo
banks)

16|P a g e

BladeServerConfiguration&Troubleshooting

CablingOrder
ForoptimalcablemanagementwhensettinguptheM1000e,connectthecablesinthisorder:
1.
2.
3.
4.
5.
6.
7.

CMCA
CMCB(ifpopulated)
iKVM
PowerSupplies
FabricAIOMs
FabricBIOMs
FabricCIOMs

17|P a g e

BladeServerConfiguration&Troubleshooting

Cooling
TheM1000econtainsmany
devices,allofwhichgenerateheat
andrequirepower.Eachofthese
devicesreportsitstemperature
andpowerstatustotheCMC,
whichmanagesthePSUsandfans
toconserveenergyandmaximize
performance.
Airentersthefrontofthechassis
andexitsthroughtherearofthe
chassis.Onitsway,itpasses:

Throughtheblades
Throughtheholesinthe
midplane
PasttheIOMsandother
rearcomponents
Outofthebackofthechassis

Plenumsdistributetheairamongthree
columnsoffans.Thisarrangementbreaks
theairflowintothreezones.TheCMC
directsthefansineachzonetopullasmuch
airasneededtocoolthatzone.

Input/outputmodules(IOMs),CMCs,and
theiKVMuseabypassducttodrawambient
airfromthetopfrontoftheenclosure.It
passesfromthetopoftheIOMtothe
bottomandexitsintotheplenumbetween
fans.

18|P a g e

BladeServerConfiguration&Troubleshooting

EachPSUalsohasitsownfan.Airflows
throughthePSUfromfronttoback,
keepingtheunitcool.
TheCMCalsothrottlestheutilizationof
eachPSUanddistributestheworkload
amongallofthePSUs.WhentheCMC
throttlesbacktheutilizationofaPSU,the
PSUdrawslesscurrentandgeneratesless
heat.

EnclosureComponents
NowthatyouknowwhattheM1000econtains,wecanexplorethecomponentsingreaterdepth.This
topicprovidesmoreinformationabouttheM1000ecomponents,including:

OneortwoCMCs(forredundancy)
IntegratedDellRemoteAccessController(iDRAC)
Ninefans
Three,four,orsixPSUs
OptionaliKVM

Aftercompletingthistopic,youwillbeableto:

DescribethecomponentsintheM1000e.
Listthehotswappableenclosurecomponents.

ChassisManagementController(CMC)
TheCMCisahotswappablesystemsmanagementdevice
thathasahardwareandasoftwareelement.Itprovides
remotemanagementandpowercontrolfortheM1000e
andinstalledcomponents.YoucanconfiguretheCMCto
sendemailalertsorSNMPtrapalertsforwarningsor
errorsrelatedto:

Temperature
Hardwaremisconfiguration
Poweroutage
Fanspeed

EachCMCincludesanintegrated24portEthernetswitch
foroutofbandinternaluseonly.Theportsareall

19|P a g e

BladeServerConfiguration&Troubleshooting

dedicatedtospecificdevices:

iDRAContheblades(16)
CMCs(2)
FabricIOMs(6)
Gbconnectors(2)

TheM1000emusthaveatleastoneCMC.YoucanaddasecondCMCforredundancy.EachCMChasthe
equivalentprocessingpowerofasmallserver.
YoucaninitiallyconfigureyourCMCsvia:

Aserialconnection
TheoperatorpanelandLCDonthefrontoftheenclosure

iDRAC
TheMseriesenhancesmanagementwithmultipleDRACs:

DRACfunctionalityisbuiltintotheM1000eenclosuresCMC.
AnintegratedDRAC(iDRAC)ineachbladehasfullhardwaremanagementdevicefunctionalityat
thebladelevel.

TheMseriessupportstheSimplifyITmodel:

TheiDRACrollsuptotheCMC.
TheCMCrollsuptotheDellOpenManage
systemsmanagementsoftware.
TheDellOpenManagesystems
managementsoftwarerollsuptoother
products.

iKVM
TheKVMontheM1000eisoptional.Itisacard
thatmountsinaspecialslotintherearofthe
enclosure,ratherthanadiscreteunit.Dellcallsit
anintegratedKVM(iKVM)because,when
installed,itfunctionsaspartoftheenclosure.The
connectionsareaccessiblefromtherearofthe
enclosuretoprovideaccesstoallinstalledbladesandCMCs.Therearenocablesbeyondthosethat
connectthemouse,keyboard,andmonitortotheKVM.
Wheninstalledandconfigured,theiKVMactivatestheVGAandtwoUSBportsonthefrontofthe
enclosure.Theconnectionsprovidecrashcartaccesstotheenclosure.
FeaturesoftheiKVMinclude:
20|P a g e

BladeServerConfiguration&Troubleshooting

Faultindicator
Powerindicator
ACIport
TwoUSBconnectorsforkeyboardandmouse
Videoconnector

IOMs
IOMsprovideaccesstotheinternalandexternalnetworksintheM1000e.Theenclosurecanhouseup
tosixIOMs,twooneachofthreefabrics.Thisflexibilityfacilitatesgreaterrolediversityintheblades.

FabricA
o Dual1Gbfabricfortheenclosuresinternalnetwork
FabricB&C
o Dual1Gbfabricfortheenclosuresinternalnetwork
o Upto2x10Gblinksperfabric
o DedicatedEthernetandSerialManagementconnections
o SupportforFibreChannelandInfiniBand

EthernetandFibreChannelpass
throughmodulesarealsoavailable.
Moreinformationisavailableinthe
IOMtopicofthismodule.
Fans
EachM1000eshipswithallninefans
installed.Thisprovidesthefollowing
benefits:

Fullypopulatedenclosures
havearedundantfan.
Coolingisalreadyinplace
whenupgradingan
enclosurethatisnotfully
populated.

Thefansinstallinthreebanksof
threefanstoprovideforzoned
cooling.TheCMCcontrolsfansineachzoneto:

Maximizecoolingeffectiveness
Reducepowerconsumption
Reducenoise

21|P a g e

BladeServerConfiguration&Troubleshooting

22|P a g e

BladeServerConfiguration&Troubleshooting

Midplane
ThenervoussystemoftheM1000eisitsmidplane.Themidplaneconnectsallofthevarious
componentsandmodulestogetherandfacilitatescommunicationamongthem.Portsontheblades
connecttothemidplane.SodotheIOMsintherearoftheenclosure.Signalspassfromthebladesto
themidplanetotheIOMStotheexternalnetworkandbackagain.
ThemidplaneroutestrafficfromthethreefabricsthroughtheIOMsonthebackoftheenclosure.
WhendesigningtheM1000esmidplane,Dellwasthinkingbeyondthenextgenerationoftechnology.
Themidplaneisalreadycertifiedfor:

FibreChannel8(FC8)
InfiniBand

Bladesconnecttothefrontofthemidplane.Signal
andpowerconnectorsonthebladesslideinto
receptorsonthemidplane.

23|P a g e

BladeServerConfiguration&Troubleshooting

Becauseofthediversityof
devices,therearofthemidplane
looksmorecomplexthanthe
front.AllsixPSUsconnecttothe
midplanetoprovidepowerforthe
enclosure.Inturn,themidplane
provides:

Powertoninefans
Powerand
communicationsforthe:
o CMCs
o iKVM
o LCDandoperator
panel
o IOMs

PSUs
TheM1000eshipswithaminimum
ofthreePSUsinanonredundant
configuration.Forredundancy,you
needaminimumoffourPSUs,but
Dellrecommendsfullypopulating
theenclosurewithsixPSUs.Ifyou
donotwanttouseallsixpower
supplies,youcan:
Manuallyturnoffpowersupplies
notinuse
AutomaticallyturnoffunusedPSUs
withdynamicpowerengagement
o ThreePSUsrunning
o Oneisredundantandnotrunning
o Twoareinfullstandby
WhenusingmorethanthreePSUs,Dellrecommendsusingaseparatepowerdistributionunit(PDU)for
eachPSU.Formoreinformation,seePSURedundancyandGridRedundancylaterinthismodule.
PowerDistributionUnits
Thepowersupplyunits(PSUs)donotplugdirectlyintoawalloutlet.Insteadavarietyofpower
distributionunits(PDUs)provide220VpowertothePSUs.

24|P a g e

BladeServerConfiguration&Troubleshooting


Type

RequiredOutlet

Singlephase20amp(20A)
(16ADeRatedinUS)

NEMAL630R

Singlephase30A
(24ADeRatedinUS)

NEMAL630R

Threephase30A
(24ADeRatedinUS)

NEMAL1530R

Singlephase60A
(48ADeRatedinUS)

Hubbell363R6W

Description

ExampleInstallation

Providesone20Acircuitper
PSU.
Therecommended
configurationisonePDU
perPSU.
TwoPDUsprovidegrid
redundancy,thoughthe
recommended
configurationisonePDU
perPSU.
Totalpowerconsumption
fortheenclosuremustnot
exceed24ampsinthenon
recommended
configuration.
Littleheadroomremainsfor
otherdevices.
OnePDUsupportsupto
threePSUs.
TwoPDUsprovidegrid
redundancy.
EachPDUbreaksoutthree
20ampsinglephase
connectionsforthePSUs.
Eachconnectorhasitsown
20ampbreaker.
OnePDUsupportsupto
threePSUs.
TwoPDUsprovidegrid
redundancy.
EachPDUbreaksoutthree
20ampsinglephase
connectionsforthePSUs.
Eachconnectorhasitsown
20ampbreaker.

HotSwappableComponents
ComponentsthatyoucanaddorremovewithoutpoweringdowntheM1000einclude:

CMC(ifredundantwithtwoCMCs)
iKVM
Individualblades
IOMs
Fans
PSUs

25|P a g e

BladeServerConfiguration&Troubleshooting


Beforeremovingthebladeorthe
drives,youmustshutdownthe
operatingsystemontheblade.

I/OModules
TopicObjectives
I/OModules(IOMs)handlealloftheirfabrictrafficbetweenthebladesintheenclosureanddevices
outsidetheenclosure.
Aftercompletingthistopic,youwillbeableto:

ExplainthefunctionofthethreefabricssupportedbytheM1000e.
ListfabricoptionsfortheM1000e.
ExplainhowmezzaninecardsandIOMsinteractinfabrics.

Fabrics
TheM1000esupportsthreehighspeedfabrics,labeledA,B,andC.Bladesintheenclosure
communicatewitheachfabricusing:

Dedicatedconnectionsthroughthemidplane
Mezzaninecardsinstalledintheblade
IOMsinstalledinthebackoftheenclosure

FabricA
FabricAsupportsGigabitEthernetforcommunicationbetweendevicesintheenclosure.Itcanalso
connecttoexternalnetworks.

EthernetcontrollersonthebladesarebasedonBroadcomtechnologyandsupport:
o TCP/IPOffloadEngine(TOE)
Inaddition,fullSCSIHBAcapabilitywithfullprotocoloffloadandabroaderarrayofoperating
systembootsupportisavailable.

WhilebladesreleasedinthisseriesinitiallyhaveadualGbELOMconfiguration,thesystemmidplaneis
designedtoallowfuturesupportforuptofourGbELOMsinthehalfheightformfactor.
FabricsBandC
YoucanconfigureFabricsBandCtosupportyouruniqueneedsbyaddinguptotwodualport
mezzaninecardstoeachblade.Theseoptionalmezzaninecardsinclude:

26|P a g e

BladeServerConfiguration&Troubleshooting

o
o
o

Ethernet(includingiSCSIand10GbEthernet)
FibreChannel
InfiniBand

Liketheconnector,mezzaninecardformfactorsarecommontobothfabricB(card1)andC(card2).
Theoptionalmezzaninecardsconnecttothebladeschipsetvia8lanePCIe.
NomatterwhichtechnologyyouchooseforFabricsBandC,themezzaninecardsinstalledintheblades
andthelOMsinstalledintheenclosuremustbecompatible.
PassthroughModules
ApassthroughmoduleprovidesatransparentconnectionbetweentheLANandindividualblades.Dell
currentlyoffersEthernetandFibreChannelpassthroughmodules.

27|P a g e

BladeServerConfiguration&Troubleshooting

Ethernet
TheEthernetpassthroughmodulesupports10/100/1000Mb
connectionsasdeterminedbytheLOMfirmwareorblade
operatingsystem.

FibreChannel
TheFibreChannelpassthroughmodulesupports1,2,or4
Gigabitpersecond(GBs)connections.

ChoosingIOMs
TheIOMinstalledinafabricdeterminesthetypesof
mezzaninecardsthatyoucaninstallinthebladesthat
attachtothatfabric.Forexample,youcannotinstallan
EthernetpassthroughmoduleinA1andanEthernet
switchinA2.FabricsexpectthesametypeofIOMin
bothslots.Forbestresults,youshouldinstalltwo
modulesofthesamebrandandmodel.
So,ifyouinstallaFibreChannelIOMinFabricB,blades
thatattachtoFabricBmusthaveaFibreChannel
mezzaninecardinstalledinthefirstmezzanine.Ifa
mezzaninecardforadifferenttechnologywereinstalled
inthatslot,thebladewouldnotboot.
Youcanmixthemezzaninecardsintheblades,butnot
theIOMs.SoyoucouldinstallaQLogicFibreChannel
cardinonebladeandanEmulexFibreChannel
controllerinanother,butyoushouldinstallmatching
IOMsinthefabric.
Also,EthernetIOMsdesignedforFabricsBandCarenotinterchangeablewithEthernetIOMsfor
FabricA.
FabricConsistencyChecking
TheM1000esystemmanagementhardwareandsoftwareincludesFabricConsistencyChecking,which
preventsaccidentalactivationofanymisconfiguredFabricdeviceonablade.
SincemezzaninetoIOMconnectivityishardwiredyetfullyflexible,itispossibletoinadvertentlyhot
plugabladeservermodulewiththewrongmezzanineintothesystem.SoifFibreChannelIOMsare
installedinFabricCI/Oslots,allbladeservermodulesmusthaveeither:
28|P a g e

BladeServerConfiguration&Troubleshooting

NomezzanineinFabricC.
FibreChannelcardsinFabricC.

Forexample:IfanEthernetmezzaninecardwereinaCslotofabladebeingswappedintothe
enclosure,theFabricConsistencyCheckwouldautomaticallydetectthismisconfiguration,andtheCMC
wouldalertyouoftheerror.
Inthissituation,nodamageoccurstothesystem,andyoucanreconfigurethefaultedmodule.
BladeMezzanine
CardSlotB
None

Ethernet

None

Ethernet

FibreChannel

None

FibreChannel

Ethernet

FibreChannel

BladeMezzanine
CardSlotC
None

I/OBayA1/A2

I/OBayB1/B2

I/OBayC1/C2

Ethernetswitchor None
None
passthrough
module
None
Ethernetswitchor Ethernetswitchor None
passthrough
passthrough
module
module
Ethernet
Ethernetswitchor None
Ethernetswitchor
passthrough
passthrough
module
module
Ethernet
Ethernetswitchor Ethernetswitchor Ethernetswitchor
passthrough
passthrough
passthrough
module
module
module
None
None
Ethernetswitchor FibreChannel
passthrough
switchorpass
module
throughmodule
FibreChannel
Ethernetswitchor None
FibreChannel
passthrough
switchorpass
module
throughmodule
FibreChannel
Ethernetswitchor FibreChannel
FibreChannel
passthrough
switchorpass
switchorpass
module
throughmodule
throughmodule
FibreChannel
Ethernetswitchor Ethernetswitchor FibreChannel
passthrough
passthrough
switchorpass
module
module
throughmodule
Ethernet
Ethernetswitchor FibreChannel
Ethernetswitchor
passthrough
switchorpass
passthrough
module
throughmodule
module
Therelationshipoftheblademezzaninecardstothechassisfabrics

StackingIOMs
Youstackswitchesdifferently,dependingonwhatyouwanttoaccomplish.
SwitchesfortheM1000ehaveoptional10GbEuplinksand/orstackingconnectors.Theseletyou:

Manage/configuremultipleswitchesasonewithstacking
Consolidateuplinksfrommultipleenclosuresinto24x10GbEports

29|P a g e

BladeServerConfiguration&Troubleshooting

Reduceportconsumptiononexternalswitches
Simplifycabling

StackingIOMs
PowerConnectTMM6220Switch
TheDellPowerConnectM6220isa
keycomponentoftheFlexIO
architectureoftheM1000e.
ThePowerConnectM6220packagesa
highperformance24portstackable
Ethernetswitchwithmodularbays
thatcanbepopulatedwith10GbEor
stackingmodules.Stackingswitches
createsasinglelogicalswitchthatcanbemanagedandconfigured.Thisstackingcapabilityiscombined
withtheabilitytouplinktotheEthernetcoreviahighbandwidth10GbEconnections.
ThePowerConnectM6220switchhasfourexternalEthernetportsandtwoexternaloptionmodulebays
for10GbEuplinkorstackingmodules.ThefourexternalgigabitEthernetportshavestandardRJ45
connectors,buttheexternaloptionmodulebaysrequirea10GbEorstackingmodule.Eachmodulebay
acceptsadualport10GbEuplinkmodule.Modulebay1willalsoacceptadualportstackingmodule.
30|P a g e

BladeServerConfiguration&Troubleshooting

Youcanstackupto12PowerConnectM6220switchesfor2401GbEthernetportsand2410Gbports.
TostackthePowerConnectM6220,youmust:

Installanoptional48Gbstackingmoduleinbay1ofeachstackedswitch.
AssignaseparateuniqueIDforthestack.
Interconnectallthestackedswitches.

ThePowerConnectM6220canbeupgradedinplacewithstackingand10GbEmodulesinstalledinthe
optionmodules.
CiscoSwitch
CiscoEthernetswitchescanbepurchasedinoneof
threebaseconfigurations:

CiscoCatalyst3130X10GRackSwitch
CiscoCatalyst3130GGbERackSwitch
CiscoCatalyst3032EntryLevelGbESwitch

Thesethreeswitchescannotbedifferentiatedbyappearance.
Upgradabilitydependsontheconfigurationoption.Forexample,ifyouwanttostackCiscoswitches,
youneedtheCiscoCatalyst3130Gor3130X.
YoucanstackuptonineCBS3130switchestocreateasinglelogicalswitchthat:

Simplifiesmanageability
Consolidatesuplinks
Lowerstotalcostofownership(TCO)
RequiresoneofthefollowingswitchesthatallowsVirtualBladeSwitchinterconnect:
o CiscoCatalyst3130X10GRackSwitch
o CiscoCatalyst3130GGbERackSwitch
RequiresaCiscoproprietarycable

Someconditionsapply:

YoucanstackPowerConnectswitcheswithstandalonePowerConnectswitches.
YoucannotstackCiscoEthernetswitcheswithstandaloneCiscoEthernetswitches.
YoucanstackbothPowerConnectandCiscoswitchesbetweenenclosures.

Ciscoprovidesthreebaseconfigurations.Upgradingrequiresaripandreplaceoftheexistingbase
configurationifitwasnotpurchasedwithstackingand/or10GbEoptions.

31|P a g e

BladeServerConfiguration&Troubleshooting

Redundancy
Redundancyistheinclusionofmultipledevices
thatperformthesamefunctiontoreducethe
likelihoodofdowntimewhenoneofthedevices
fails.TheM1000esupportsthefollowing
redundancies:

CMC
Fabric
Fans
PSU

Fabric redundancy typically relies on additional configuration,


software, and technologies such as NIC teaming or clustering.
CMCRedundancy
EachM1000ecanhousetwoCMCs,eventhoughonlyoneCMCisactiveat
anygiventime,andonlyoneisrequired.ThestandbyCMCmirrorstheactiveone.IftheactiveCMC
fails,thestandbyCMCtakesover.YoucanconfiguretheCMCstoalertyouwhenafailoveroccurs.
Onboot,theCMCschecktoseeifanotherCMCisactiveintheenclosure.ThefirstCMCtoboot
becomesactive.ThesecondoneautomaticallybecomesthestandbyCMC
FabricRedundancy
I/Oportredundancy
withinafabric

EachmezzaninecardorLOMhastwoportson
eachblade.Ifoneoftheportsfails,trafficcan
bereroutedbysoftwarethroughtheother
port.
IOMredundancywithin EachofthethreefabricshastwoIOMsineach
afabric
enclosure.IfeitheroftheIOMsfails,traffic
canbereroutedbysoftwarethroughthe
alternateIOM,dependingonhowthefabricis
configured.
Redundancybetween
Tohavefabricredundancy,FabricsBandC
fabrics
mustusethesameprotocol.
IfFabricsBandCareEthernet,failureofany
ofthethreefabricsiscoveredbythe
remainingfabrics.
IfFabricsBandCusethesameprotocol,
failureofeitherfabriciscoveredbythe
remainingone.

32|P a g e

BladeServerConfiguration&Troubleshooting

FanRedundancy
AllninefansarerequiredwithintheM1000eenclosuretopowerupthechassis.Ifanyofthefansfails,
theremainingeightfanscankeepafullypopulatedenclosurecooluntilthefailedfancanbereplaced.
FansareN+1redundant,meaningthatanysinglefancanfailwithoutimpactingsystemuptimeor
reliability.Ifafanfails,systembehaviorisdependentontheresultanttemperaturesofthesystem,as
monitoredbytheServerModuleiDRACandI/OModules.TheCMCcontinuestointerprettheairflow
needsofeachserverandI/Omoduletocontrolthefanspeedsappropriately.Thesystemwillnotramp
thefanstofullspeedintheeventofafanfailureunlessdeemednecessarybyonboardmonitoring.
TheM1000eisengineeredforgoodsoundqualityinaccordancewiththeDellEnterpriseacoustical
specification.Comparedtopreviousgenerationsofproducts,thefanshavemorelevelsofcontrol,
allowingmuchfinertuningofthefanbehavior.Firmwareisoptimizedtochoosethelowestfanspeeds
andthereforethelowestacousticaloutputforanyconditionandconfiguration.

Whenthefansarerunningatnormalrpmtheyfalljustwithinunprotectedhearingsafetyparameters.
However,whenstartingup,orduringacomponentsfailure,theincreaseinfanspeedsbringthenoise
beyondunprotectedregulationsfornoise,thereforehearingprotectionmustbeused.
Iftheenclosureisnotfullypopulated,fewerfansmayberequired,butyoushouldstillkeepallnine
working.TheCMCcanmanagethefanssothatonlytherequisitenumberrunsatanytime.
PSURedundancy
TheM1000ecanhouseuptosixPSUsbutonlyrequiresaminimumofthree.Soinstallinganynumber
greaterthanthreePSUsprovidessomedegreeofredundancy.

FourPSUscancoverthefailureofanyonePSU.

SixPSUsprovidesmaximumredundancy.

Noredundancy(N+0)

PowerfromallthreepowersuppliesononeACcircuit(grid)isusedto
powerontheentirechassis.

33|P a g e

BladeServerConfiguration&Troubleshooting

Theadditionalpowersupplyiskeptasaspare,ensuringthatafailure
ofanyonepowersupplywillnotcausetheserversorchassistopower
down.Allsixpowersuppliesareneverinuseatthesametime.Based
onloadandavailability,fromtwotofouroftheavailablepower
suppliesareusedatanygiventime.Therestareheldreadyforfailover.

Powersupplyredundancy
(N+1)

Gridredundancy(N+N)

Allsixpowersuppliesareusedat50%capacity.Eachbankofthree
powersuppliesconnectstoaseparateACgrid.

Summary
Reviewthesummaryofthismodulebeforetakingtheselfchecktoseehowwellyoumasteredthe
topic.

ThePowerEdgeM1000ecomprisesallofthecomponentsneededtosupportbladenetworking
andclustering.
o TheminimumconfigurationoftheM1000eincludesaCMC,threePSUs,andninefans.
Fortheenclosuretobeusable,youmustaddatleastoneMseriesblade.
o Youcanaccesstheblades,crashcartconnection,andoperatorpanelfromthefrontof
theM1000e.
o Thefollowingcomponentsareaccessiblefromthebackoftheenclosure:
OneortwoCMCs(oneoneachupperrightandleftcorner)
AnintegratedKVM(iKVM)
UptosixPSUs
Ninefans
Fabricconnections(A,B,andCintwobanks)

34|P a g e

BladeServerConfiguration&Troubleshooting

YoushouldconnectthecablesontheM1000einaspecificorder.
TheM1000eisanaircooledenclosure.Airmovesfromfronttobackthroughthreedistinctive
paths:
o Throughtheblades,pastthemidplane,intoaplenum,andoutanexhaustfan
o PasttheCMC,iKVM,andIOMs,intoaplenum,andoutanexhaustfan
o ThroughthePSUs
ThecomponentsoftheM1000einclude:
o AtleastoneCMCcommunicateswithalloftheenclosurecomponentsandwiththe
iDRACsbuiltintoeachblade.YoucanconfiguretheCMCtosendemailalertsorSNMP
trapalertsforwarningsorerrorsrelatedto:
Temperature
Hardwaremisconfiguration
Poweroutage
Fanspeed
o TheM1000esoptionaliKVMintegratesintotheenclosuretoprovideaccesstoall
blades.
o IOMscommunicatewiththethreefabricssupportedbytheM1000e:
FabricAisalwaysEthernetandsupportstheenclosuresprivatenetwork.
FabricsBandCcanbe:
o Ethernet
o FibreChannel
o InfiniBand
TheM1000eshipswithallninefanspopulated.Thefansinstallinthreebanksofthreefans.
Themidplanefacilitatescommunicationbetweenallofthedevicesintheenclosure.
o FabricAisadual1Gbfabricthatsupportstheenclosuresinternalnetwork.
o FabricsBandCcansupportvariousfabrics,includingFibreChanelandInfiniBand.
YoucaninstallthreetosixPSUsintheM1000e,butyouneedatleastfourforredundancy.
TheM1000esupportsbothPDUsavailablefor9Gserversandtwo10GserverPDUs.
HotswappablecomponentsintheM1000einclude:
o CMC(ifredundantwithtwoCMCs)
o iKVM
o Individualblades
o IOMs
o Fans
o PSUs
IOMsandmezzaninecardsonafabricmustusethesametechnology.Whileyoucanmix
mezzaninecardsofthesametechnologyonafabric,bothIOMsshouldbeidentical.
o TheFabricConsistencyCheckpreventsaccidentalactivationofmisconfiguredfabric
devicesonablade.
o TheM6220isastackable24portEthernetswitchdesignedforFabricA,but16ofthe
portsarereservedforblades.

35|P a g e

BladeServerConfiguration&Troubleshooting

Redundancyistheinclusionofmultipledevicesthatperformthesamefunctiontoreducethe
likelihoodofdowntimewhenoneofthedevicesfails.
o WhentwoCMCsareinstalledintheM1000e,thefirsttobootbecomestheactiveCMC.
TheothermirrorstheactiveoneandstandsreadytotakeoveriftheactiveCMCfails.
o TheM1000esupports:
I/Oportredundancywithinafabric
IOMredundancywithinafabric
Redundancybetweenfabrics
o TheM1000eshipswithallninefansinstalled.
o FourormorePSUsprovidesomedegreeofredundancy.Forfullredundancy,youshould
installallsixPSUsandattachthemtotwoseparategrids.

36|P a g e

BladeServerConfiguration&Troubleshooting

Module3:KeyComponents
ModuleObjectives
Thismodulediscussesthemodularservers(blades)that
arecompatiblewiththeM1000eModularBlade
Enclosure(M1000e).
Aftercompletingthismodule,youwillbeableto:

IdentifyMseriesblades.
ListthefeaturesofMseriesblades.
DescribethedifferencesbetweentheM600,M605,M805,andM905.
ExplainthebenchmarksusedtomeasureMseriesblades.
Explainthedifferencebetweenclusteringandvirtualization.

CommonFeatures
Bladesarethereasonyouhaveabladeenclosure.Thistopicintroducesthebladesthatarecompatible
withtheM1000e.
Aftercompletingthistopic,youwillbeableto:

IdentifyMseriesblades.
DescribethefeaturesandoptionsofMseriesblades.

Blades
TheM1000esupportsthefollowingblades:

M600,anIntelbasedsingleheightblade
M605,anAMDbasedsingleheightblade
M805,anAMDbasedsingleheightblade
M905,anAMDbasedsingleheightblade
M610,anIntelbasedsingleheightblade
M710,anIntelbasedfullheightblade

Features
Mseriesbladessharethefollowingfeatures:

Twomezzaninecardslotswithdualporting
o Ethernet
o FibreChannel
o InfiniBand

37|P a g e

BladeServerConfiguration&Troubleshooting

Midplaneinterface
IntegratedDellRemoteAccessController(iDRAC)
o IPMI2.0
o Virtualmedia
o VirtualKVM
o SerialoverLAN(SOL)
o OutofbandGUIandCLI
SixI/Oports(withbothmezzaninecardsinstalled)
PCIeI/Otechnology
(M805andM905)Upto32GBDDR2SDRAMor64GBwithqualifiedDIMMs
o MemoryECC,SDDC,SpareBank,Mirroring
Uptotwohotpluggableharddrives
o StaticDiskDrivesupport
o SAS
o SATA
IntegratedRAID
o CERC6iRAIDoptionsforlocaldrives
o SAS6/iR
DualportembeddednetworkinterfacecardwithTOE

Thechartonthisscreen,showsthefeaturesofsomeoftheDellMSeriesBladeServers

FeaturesofDellMseriesBladeServers
Feature
FormFactor
Processors
Processorwattage
options
RAMslots
TotalRAM
Harddrivetypes
LOMs
Mezzaninecard
slots
RAIDcard

M600
Singleheight
Intel
StandardandLV

M605
Singleheight
AMD
StandardandLV
8
To64GB
SASandSATA
2
2

M805
Fullheight
AMD
Standard,HEand
SE
16
To128GB
SAS
4
4

M905
Fullheight
AMD
Standard,HEand
SE
24
To128GB
SAS
4
4

8
To64GB
SASandSATA
2
2
SAS/iRCERC6i

SAS/iRCERC6i

SAS/iRCERC6i

SAS/iRCERC6i

Inaddition,bothbladescan:

BootfromiSCSIusingEthernetconnectionsembeddedonthesystemboard

38|P a g e

BladeServerConfiguration&Troubleshooting

BootfromFibreChannelSAN
Supportenterprisesolutions:
o OracleandSQLdatabaseprogramintegration
o VMwareprogramintegration
o Dell|EMCintegration
o Dell|EqualLogiciSCSIstorage
o DellPowerVaultstorage
o Highperformancecomputingcluster(HPCC)

39|P a g e

BladeServerConfiguration&Troubleshooting

MezzanineCards
InadditiontothetwoEthernet
controllersembeddedonthe
systemboard,Mseriesblades*
havetwomezzaninecardslots.
Theseslotsenableyoutoinstall
mezzaninecardstocommunicate
withthefabrics.
Forthebladetoboot,themezzaninecardmustmatchthefabricasdeterminedbytheI/Omodule
(IOM)installedintheenclosure.
Ifthecarddoesnotmatchthefabric,theChassisManagementConsole(CMC)canbeprogrammedto
sendyouanalert.Becausethebladedoesnotbootinthissituation,youdonthavetoworryabout
causinganydamagetothebladeortheenclosure.
*TheM710,M805andM905bladeservershavefourmezzanineslots.

HardDriveOptions
Althoughusinglocalharddrivesisoptional,Mseriesbladessupportthefollowinginternalstorage
buses:
SerialAdvancedTechnologyAttachment(SATA)II
SATAII(SerialAdvancedTechnologyAttachment),ahotswappable,serialrevisionofthepopularlegacy
ATAbus,features:

Arawdatatransferrateof3.0gigabitspersecond(Gb)
Significantlysmallerconnectorsandcables
Dedicatedbandwidthandcableforeachdevice.

SATAIIdrivesarelesscostlythantheirSAScounterpartsandoffercosteffectivealternativesfor
customerswhohavestrictbudgetsandarelessconcernedwithmaximummeantimebeforefailure
(MTBF)orperformance.
SATAIIdrivesoperateat7.2RPM,considerablyfasterthanpreviousgenerations.SATAIIdeliversa
blendofhighperformanceandconfigurationconvenience.
SerialAttachedSCSI(SAS)
SAScombineselementsofSCSIandSATA.ItsupportsthetraditionalSCSIcommandstructure,soSCSI
softwaretoolscontinuetowork.
LikeSATAII,SASsrawdatatransferrateis3.0Gb.SAS,however,isafullduplexbus,sodatacanbe
routedinbothdirections,yieldinghighersimultaneousdatatransferstoandfrommultipledevicesthan
SATAII.
40|P a g e

BladeServerConfiguration&Troubleshooting

Finally,SASsupportsmoredevicesandlongerexternalcablelengthsthanSATAII.SASisrecommended
forthemosthighend,highdemandenterpriseserverdeployments.
Botharchitecturesimprovespeedandreliabilityoverpreviousgenerationsofharddrivetechnology.
EmbeddedEthernetControllers
TwoembeddedBroadcomEthernetcontrollersattachtoFabricA.ThesecontrollerssupporttheTCP
offloadengine(TOE).
ThecontrollerscircuitryisembeddedintheplanarasindependentEthernetinterfacedevices.The
followinginformationdetailsthefeaturesoftheLANdevices.

PCIex4interface
SMBfornetworkmanageabilityonlywithLOM1:
o IPMI
o ASF2.0
64KBonchippacketbuffer
MACandSERDESinterfaceintegrated
WakeonLAN(WOL)10/100speeds
PXEissupported
iSCSIFirmwareAssistedBootcapability
iSCSIprotocoloffloadoptionenabledviahardwarelicensekey

AllNICsarecapableofrunningat10/100/1000Mbevenwiththepassthroughmodules.Thisspeedwas
notpossibleinpreviouspassthroughtechnology.
iDRAC
TheIntegratedDellTMRemoteAccessController(iDRAC)isasystemsmanagementhardwareand
softwaresolutionthatprovides:

Remotemanagementcapabilities(outofband)
Crashedsystemrecovery
Powercontrolfunctions

iDRACsareintegratedinMseriesblades.TheiDRAChardwarecomponentprovidessupportfor:

VirtualMedia(vMedia)
VirtualKVM(vKVM)
Outofbandgraphicaluserinterface(GUI)
Outofbandinventory
Outofbandstatusmessaging

TheiDRACusesanintegratedsystemonchipfortheremotemonitoringandsystemcontrol.TheiDRAC
coexistsonthesystemboardwiththemanagedMseriesbladeservers.
41|P a g e

BladeServerConfiguration&Troubleshooting

AllnetworkconnectionstotheiDRACarethroughtheCMCGb1port.TheCMCroutestraffictothe
iDRACsonitsserversthroughtheenclosuresprivatenetwork.Thisprivatemanagementnetworkis
betweentheCMCandtheiDRAC.PrivatenetworkconnectsallthenetworksandtheCMCsplusthe
iKVM.

Differences
AlthoughMseriesbladeshavemanycommonfeaturesandcanbeusedinterchangeablyassuming
theyhavethesamemezzaninecardssignificantdifferencesbetweenmodelsexist.Thistopicexplores
thosedifferences.
Aftercompletingthistopic,youwillbeableto:

IdentifydifferencesbetweentheM600,M605,M805,andM905.

42|P a g e

BladeServerConfiguration&Troubleshooting

M600UniqueFeatures
TheM600hasthefollowingfeatures:

Uptotwodualorquadcore
Intelprocessors
Intel5000Pchipset
Fullybuffered567MHzRAM
To64GBwithqualified8GB
DIMMs

M605UniqueFeatures
TheM605hasthefollowingfeatures:

Uptotwodualorquadcore
AMDprocessors
NVIDIA MPC55 chipset
667MHz,single/dual/quad
rankedDIMMs
To64GBwithqualified9GB
DIMMs

43|P a g e

BladeServerConfiguration&Troubleshooting

M805andM905UniqueFeatures
LeveragedDesign

ThesystemboardonthePowerEdgeM905isthesame
systemboardthatisusedonthePowerEdgeM805.The
numberofprocessorsinstalledduringtheinitialP.O.S.T.
determinesthesystemnamedisplayeduponP.O.S.T.
Theonlydifferencebetweenthetwosystems
cosmeticallyisthattheM805hastwoprocessorsand
twohypertransportbridgecardsinprocessorsockets3and4.Also,becauseoftheprocessorreduction
from4to2processors,thetotalamountofmemorysupportedis128GBofRAM onthePowerEdge
M805versusthe196GBsupportedonthePowerEdgeM905.
Feature
Chipset
Processor

Memory
Harddrives
SAS/RAIDcontrollers

EmbeddedLOMs

OtherI/O
EmbeddedManagement
44|P a g e

PowerEdgeM805
(FullHeightAMDBlade)
DualNvidiaMCP55
Opteron2000series
(splitplane)Dualorquad
core
Dualcore(nonsplitplane)
16XDDR2667/800MhzDIMMS
(128GBtotal)
Hotpluggable2x2.5SAS
DaughtercardOptionsfor:
1. LSI1068ew/IRfunctionality
(SAS6/ir)
2. CERC6
4xGBEthernetw/hardware
TCP/IPOffloadEngine,iSCSI
OffloadacceleratorandiSCSI
FirmwareBoot
Fourx8PCIemezzaninecards
iDRAC

PowerEdgeM905
(FullHeightAMDBlade)
DualNvidiaMCP55
AMD

24XDDR2667/800MhzDIMMS
(192GBtotal)
Hotpluggable2x2.5SAS
DaughtercardOptionsfor:
1. LSI1068ew/IR
functionality(SAS6/ir)
2. CERC6
4xGBEthernetw/hardware
TCP/IPOffloadEngine,iSCSI
OffloadacceleratorandiSCSI
FirmwareBoot
Fourx8PCIemezzaninecards
iDRAC

BladeServerConfiguration&Troubleshooting

ExternalPorts
InternalPersistentStorage

3xUSB2.0ports
1. CMCbased
2. DedicatedInternalSCSlot
perbladeforFIembedded
hypervisor(same
implementationas805)

3xUSB2.0ports
1. CMCbased
2. DedicatedInternalSCSlotper
bladeforF1embedded
hypervisor(same
implementationas805)

DriveControllers
TheMseriesbladeserverssupportuptotwooptionalharddrives.Thetypeofdrivesyoucaninstall
dependsonthecontroller.
Aftercompletingthistopic,youwillbeableto:

ListthedrivecontrollersavailableforMseriesblades.
IdentifythefeaturesoftheMseriesdrivecontrollers.
DescribehowtoconfigureRAIDoptionsonthedrivecontrollers.

DriveControllerOptions
OptionalstoragecardsprovideinternalRAIDcapabilitiesfortheMseriesbladeserversasdescribedin
thefollowingtable:

Thesecardsattachintoacardslotundertheharddrivebaysinthefrontoftheblade,notintooneof
theI/Omezzanine(cardslots).

CERC6iisforspecificapplicationsforoptimalperformance(forexample,Citrix).Readonlyapplications
arebestbecausethe128MBcacheisnotcurrentlybatterybacked.
Dellalsosupportstheoptionfornoharddrives.Inthisconfiguration,nocardwouldbeinstalled.

45|P a g e

BladeServerConfiguration&Troubleshooting

CERC6i
TheCERC6iisaninternalRAIDcardforcustomerswhorequirebasicRAIDfunctionality.Features
include:

Hostbaseddaughtercardtoconnecttointernaldrives
PCIExpress(PCIe)HostInterface
128MBcache
SupportforRAID0andRAID1

ThisversionoftheCERC6icarddoesnotprovideabatterytobackupthecache.So,itispossibletolose
dataduringapoweroutage.
CERC6iCachePolicy
Thecachecontrollerwritesablockofdatatocachememory,whichismuchfasterthanwritingtothe
physicaldisk.Thecachecontrollersendsanacknowledgementofdatatransfercompletiontothehost
system.
TheCERC6icanusetwotypesofcaching:

Withwritebackcaching,thecontrollersendsadatatransfercompletionsignaltothehostwhen
thecontrollercachehasreceivedallthedataina
transaction.Thecontrollerthenwritesthecacheddata
tothestoragedevicewhensystemactivityislowor
whenthewritebufferapproachescapacity.Thecached
dataisnotwrittentothestoragedeviceimmediately.
Withwritethroughcaching,thecontrollersendsadata
transfercompletionsignaltothehostsystemwhenthe
disksubsystemhasreceivedallthedataina
transaction.

Thedefaultcachesettingiswritethroughcaching.
Theriskofusingwritebackcacheisthatthecacheddatacan
belostifthereisapowerfailurebeforeitiswrittentothe
storagedevice.Writebackcachinghasaperformance
advantageoverwritethroughcaching.
WriteBackmodeisavailablewhentheuserselectsForceWBwithnobatteryintheCERC6iinterface
whilebootingtheserver.WhenForcedWriteBackmodeisselected,thevirtualdiskisinWriteBack
modeevenifthebatteryisnotpresent.

46|P a g e

BladeServerConfiguration&Troubleshooting


TheCERC6idoesnothaveabatterybackupasofthiswriting.Becauseofthepossibilityofdataloss,Dell
recommendsthatyouusewritethroughcachingwiththeCERC6i.
CERC6iConfigurationUtility
TheCERC6i'sBIOSconfigurationutilitycanbeaccessedbypressingthe<Ctrl><R>keyswhenbooting
theblade.Itisastoragemanagementapplicationembeddedinthecontrollerfirmwarethatperforms
thefollowingactionsindependentlyoftheoperatingsystem:

ConfiguresandmaintainsRAIDdiskgroupsandvirtualdisks
ManagestheRAIDsystem

CTRL+Ristobeusedforinitialsetupanddisasterrecovery.Advancedfeaturescanbesetthrough
OpenManageTMStorageManager.
YoucanapplyCTRL+Rusingonlythekeyboard.Otherkeystrokefunctionality:

Thearrowkeysmovethecursorinaspecificdirection.
Pressing<Alt>andanunderlinedletteractsasamenushortcut.
Pressing<Enter>activatesthehighlightedoption.
Pressing<Tab>movestothenextcontrolonthepage.
Pressing<Esc>closesapopup.

47|P a g e

BladeServerConfiguration&Troubleshooting

CreatinganArrayontheSAS6/iR
ThisprocedureconfigurestheSAS6/iRtouseRAID0integratedstriping(IS)orRAID1integrated
mirroring(IM)onthebladesharddrives.
Step1
Whenthesystemisbooting,press
<Ctrl><C>toentertheSAS6/iR
controllerBIOS.
Thesystempromptsyoutopress
<Ctrl><C>,thenthetextchangesto
indicatethattheSAS6/iR user
interfaceisloading.
Ifthesystemdidnotrecognizethe
<Ctrl><C>keystroke,itdisplaysalist
ofattacheddevices.

48|P a g e

BladeServerConfiguration&Troubleshooting

Step2
Whentheuserinterfaceloads,use
theArrowkeystohighlightandthen
press<Enter>toselectthedesired
controller.
Inthisexample,boththeSAS5and
SAS6arelisted.NoticethattheSAS6
islistedwithitschipset1068E.
Thecontrollerindicatesthatitis
readingnonvolatilesettings.

49|P a g e

BladeServerConfiguration&Troubleshooting

Step3
TheAdapterPropertiesscreencanbe
manipulatedbyusingtheArrowkeysto
highlightRAIDPropertiesandpress
<Enter>.

Step4
OntheSelectNewArrayTypescreen,
usethearrowkeystohighlightthetype
ofarrayyouwanttocreateandpress
<Enter>.

ChooseCreateIMVolumefora
mirrororRAID1 array.

ChooseCreateISVolumefora
stripedorRAID0array.

50|P a g e

BladeServerConfiguration&Troubleshooting

Step5
Createthestripedvolume.
1) Foreachdrivetobeincluded:
a) Usearrowkeystohighlightthe
Yellowtext.
Disksthatdonotdisplayyellow
textarenotselectable.They
mayalreadybeusedin
anotherconfigurationeither
anothervolumeorahotspare
configuration.
b) Pressthespacebartochange
theRAIDDiskselectiontoYes.
2) Press<C>tocreatethestriped
volume.
Ifthespacebarispressedonanyother
columnthantheRAIDdiskcolumnthe
followingmessageshoulddisplay.
Afterselectingthediskdrivesand
pressing<C>,theSAS6/iRBIOSasksyou
tocreateandsavethenewarray.

Step6
ChooseSaveChanges,thenexitby
navigatingwiththearrowkeysand
pressing<Enter>.
Theinterfacegivesyouonelastchance
toabandonthechanges.

51|P a g e

BladeServerConfiguration&Troubleshooting

Step7
Press<F3>tofinalizethenewarray.

Thevolumeisnowcreated.

TheinterfacereturnstotheAdapter
Propertiesscreen.
Step8
UsethearrowkeystohighlightSAS
Topologyandpress<Enter>.

52|P a g e

BladeServerConfiguration&Troubleshooting

Step9
Makesurethenewvirtualdiskappearsin
thelist.

Step10
NoticethatthedrivesinBay0andBay1
arelistedasRAIDphysicaldisks,because
theyarepartofanarray.Therestofthe
baysshowthedrivemanufacturerand
productnumberaswellasversion
number.Thisinformationcanbeobtained
forthedrivesinBay0andBay1by
expandingtheIMVolumetoshowthe
disksthatareinthatvolume.
Step11
Press<ESC>toexittheuserinterface.

SAS6/iRVirtualDiskManagement
YoucanperformthefollowingvirtualdiskmanagementtaskswiththeSAS6/iR:
Viewvirtualdiskproperties
FollowthesestepstoviewthepropertiesofISandIMvirtualdisks:
1. SelectacontrollerfromtheAdapterListintheConfigurationUtility.
2. SelecttheRAIDPropertiesoption.
Iftherearenoexistingvirtualdisks,youwillbepromptedtocreateanIMoranISvirtual
disk.
Ifthereisoneexistingvirtualdisk,selectViewExistingArray.
Iftherearetwoexistingvirtualdisks,press<Alt><N>toviewthenextvirtualdisk.
3. Press<Enter>whentheManageVirtualDiskitemisselectedtomanagethecurrentvirtualdisk.

53|P a g e

BladeServerConfiguration&Troubleshooting

Synchronizeavirtualdisk
Synchronizingavirtualdiskmeansthatthefirmwaresynchronizesthedataonthesecondarydisk(s)of
themirror.FollowthesestepstostartsynchronizationforanIMvirtualdisk:
1. SelectSynchronizeMirror.
2. Press<Y>tostartthesynchronizationor<N>tocancelit.

Thisprocesscantakeseveralhourstocomplete.
Activateavirtualdisk
Avirtualdiskcanbecomeinactiveif,forexample,itisremovedfromoneSASS/iRcontrollerorcomputer
andmovedtoanotherone.TheActivateMirroroptionletsyoureactivateaninactivevirtualdiskthat
hasbeenaddedtoasystem.Thisoptionisonlyavailablewhentheselectedvirtualdiskiscurrently
inactive.
1.SelectActivateMirror.
2.Press<Y>toproceedwiththeactivationorpress<N>toabandonit.
Afterapause,thevirtualdiskwillbecomeactive.

Activationofmigratedvirtualdisksisonlysupportedwhenthemigratedvirtualdisk(s)isinanoptimal
stateandcontainsallthephysicaldisks.
Deleteavirtualdisk

Beforedeletingavirtualdisk,besuretobackupalldataonthevirtualdiskthatthecustomerwantsto
keep.
Followthesestepstodeleteaselectedvirtualdisk:
1. SelectDeleteVirtualDisk.
2. Press<Y>todeletethevirtualdiskorpress<N>toabandonthedeletion.
3. Press<F3>toconfirmdeletionofthevirtualdisk.Afterapause,thefirmwaredeletesthevirtual
disk.

Ifthephysicaldisksofavirtualdiskareremovedandthevirtualdisk'sconfigurationissubsequently
deletedfromtheSASS/iRcontroller,thephysicaldisksshowuponlyassimplediskswithnoRAID
associationiftheyareplacedbackontothesameSASS/iRcontroller.Oncethevirtualdiskisremoved
fromaSAS6/iRcontrollerusingtheBIOSConfigurationUtility(regardlessofwhetherthephysicaldisks
arepresent),thevirtualdiskcannotberestored.

54|P a g e

BladeServerConfiguration&Troubleshooting

InternalStorage
Dellbladeserverstoragecanbeconfiguredinthefollowingways:

DisklessConfiguration
SATARepeaterDaughterCard
SAS6/iRModularDaughterCard
CERC6iModularDaughterCard

Aftercompletingthistopic,youwillbeableto:

Enumeratethetypesofbladeserverstoragedevices.
Listthefeaturesofeachtypeofstorage.

DisklessConfigurations
Characteristicsofdisklessconfigurationsinclude:

Nostoragedaughtercardpopulated
TwoHDDblankspopulated
SAS/SATAbackplanepresent(forcoolingpurposes)

SATARepeaterDaughterCard
FeaturesoftheSATARepeaterDaughterCardinclude:

Providesstandaloneharddiskdrivesupport
Nohotplugsupport
NoRAIDconfigurationssupported
TwoSATAHDDssupported
SystemBIOSconfigurable(PressF2duringPOST)
BIOSOption:SATAConfigurationispresentonlyifaSATAcontrollercardisinstalled

55|P a g e

BladeServerConfiguration&Troubleshooting


SAS6/iRModularDaughterCard
FeaturesoftheSAS6/iRModularDaughterCardinclude:

UptotwoSASorSATAhotpluggableHDDssupported
SASandSATAHDDmixingisnotsupported
RAID0orRAID1supported
Press<Ctrl><C>toentertheSAS6/iRConfigurationUtility

56|P a g e

BladeServerConfiguration&Troubleshooting

CERC6iModularDaughterCard
FeaturesoftheCERC6iModularDaughterCardinclude:

UptotwoSAShotpluggableHDDssupported
RAID0orRAID1supported
Readandwritecachememoryforimprovedperformance
Cacheisnotbatterybackedup

57|P a g e

BladeServerConfiguration&Troubleshooting

StorageDaughterCardInstallationandRemoval
ThistopiccoversinstallingandremovingbladestoragedaughtercardsinDellPowerEdgebladeservers.
Aftercompletingthistopic,youwillbeableto:

Explainhowtoaddastoragecardtoabladeenclosure.
Explainhowtoremoveastoragecardfromabladeenclosure.

58|P a g e

BladeServerConfiguration&Troubleshooting

BladeStorageDaughterCard
Installation
Installationandremovalofblade
storagedaughtercardsis
straightforward.
Toinstallabladestoragedaughter
card:
1.Alignthecontrollerboardsothat
thetabsonthemetalsystemboard
trayfitthroughthenotchesonthe
edgesofthecontrollerboard.
2.Slidetheboardtowardsthe
connectoronthesystemboard
untiltheboard'sedgeconnectoris
fullyseated.
BladeStorageDaughterCard
Removal
Thebladestoragedaughtercardis
locatedunderneaththeharddrive
baysofthebladeenclosure.To
removethecard:
1. Openthereleaseleverto
disengagethecontroller
boardedgeconnector
fromthesystemboard
connector.
2. Liftthecontrollerboard
straightupfromthe
systemboard.

59|P a g e

BladeServerConfiguration&Troubleshooting

SAS/SATAHDDBackplaneComponents
ThissectiondescribescomponentsoftheSAS/SATAHDDbackplaneandthefunctionsofHDDblanksin
Dellbladeservers.
Aftercompletingthistopic,youwillbeableto:

ListthecomponentsoftheSAS/SATAHDDbackplane.
DescribethefunctionalityoftheSAS/SATAHDDIndicatorLEDs.
ExplainthefunctionandconveniencefactorsofHDDblanksinDellbladeservers.
ListrequirementsforpopulatingtheHDDslotsinaDellbladeserver.

SAS/SATAHDDBackplane
Componentsofthebackplaneincludethefollowing:

TwoSAS/SATAHDDConnectorsforsupportoftwoSAS/SATA2.5HDDs
Cablelesscardedgeinterfacetobladeplanar
ToollessHDDbackplaneinstallation/removalvialatchandhookretentionmethod
CommonHDDbackplanemechanical/electrical/boardsolutionsharedbetweentheMseries
bladeservers
AcommonHDDbackplanemechanical/electrical/boardsolutionsharedbetweentheM805and
M905

Forpropersystemcooling,theblademusthaveanHDDoranHDDblankinstalledineachHDDbay.

60|P a g e

BladeServerConfiguration&Troubleshooting

HDDIndicatorLEDs
FeaturesoftheHDDstatusindicator
LEDs,foundonthefrontofthe
enclosure,include:

TheHDDstatusindicator
LEDisonlyfunctionalfor
RAIDHDDconfigurations.
FornonRAID
configurations,onlythe
driveactivityindicatoris
active.
o TheHDDstatus
indicatorLEDisnot
supportedforSATA
RepeaterSATAHDD
configurations.

MechanicalIssues
ThistopiccoversHDDblanksandcooling
issues.
DellPowerEdgebladeserversleveragethe
existing2.5HDDBlank,commontoalready
shippingplatforms.Tomaintainproper
systemthermals,HDDBlanksmustbe
installedforHDDslotsthatdonothavea
2.5SASorSATAHDDpopulated.

61|P a g e

BladeServerConfiguration&Troubleshooting

ClusteringandVirtualization
Bladeslendthemselvestotwopopularservertechnologies:clustercomputingandvirtualization.Ina
sense,clusteringandvirtualizationareoppositeendsoftheserverspectrum.

Clusteringisgroupingtwoormoreserverstogethersothattheyfunctionasasingleunit.
Virtualizationiscausingasingleservertobehaveasifitweretwoormoreservers.

Aftercompletingthistopic,youwillbeabletoexplainthedifferencebetweenclusteringand
virtualization.
ClusterComputing
Clusteringisgroupingtwoormoreserverstogethersothattheyfunctionasasingleunit.Clustered
computerscommunicatewitheachotheroveraprivatelocalareanetwork.
Differenttypesofclusterarchitectureservedifferentcomputingneeds.

Highperformancecomputingclusters(HPCCs):Ahighperformancecomputingcluster(HPCC)is
apowerfulandeconomicalsolutionforenhancingyourresearchenvironments.AnHPCCisa
groupofcommercialofftheshelfcomputersinterlinkedviahighspeedinterconnectstocreate
arobust,highspeedcomputingresourcethatrivalstheperformanceoftraditionalmainframe
supercomputersatafractionofthecost.
Gridcomputing:GridsworkmuchlikeHPCCs,exceptthatthenodeseachretaintheiridentities.
SETI@homeisoneexampleofaverylargegrid.
Highavailability(HA)clusters:Alsoknownasfailoverclusters,theseclustersimprovethe
availabilityofcriticalresources.MostHAclustersconsistoftwonodes;oneactiveandone
redundant,Theredundantnodemirrorstheactivenodeandtakesoveritsfunctionalityinthe
eventofafailure.
Loadbalancingclusters:Afrontendcontrollerallocatesworkloadamongthenodesinthe
clustertoachievemaximumthroughputandbalancetheworkloadofeachnode.

Bladeserversgiveyoutheadvantageofhighdensitycomputing,reducedcabling,reducedcooling
requirementsandoptimizedpower.
Virtualization
Virtualizationallowsyoutorunmultipleapplicationsandoperatingsystemsindependentlyonasingle
server.Administratorscanquicklymoveworkloadsfromonevirtualworkspacetoanothereasily
prioritizingbusinessneedswhilemaximizingserverresources.Flexibilityinallocatingcomputing
capacityallowstheconsolidationofapplicationsandservers,easingITmanagementrequirementsand
loweringexpenses.
Delloffersarangeofvirtualizationsolutions,including:

MicrosoftVirtualServer
VMwareESXServerTM

62|P a g e

BladeServerConfiguration&Troubleshooting

CitrixXenServerTM

Gettingthemostoutofanyofthesesolutionsrequiresindepthtrainingthatgoesbeyondthescopeof
thiscourse.
Summary
TheM1000esupportsthefollowingblades:
o M600,anIntelbasedsingleheightblade
o M605,anAMDbasedsingleheightblade
o M805andM905,AMDbasedfullheightblades
Mseriesblades:
o Supportthesamemidplaneinterface
o HaveanintegratediDRAC
o HavesixI/Oportswhenbothmezzaninecardsareinstalled
o Havebaysfortwohotswappableharddrives
o IncludeBroadcomEthernetcontrollersthatattachtoFabricA
o SupporttwomezzaninecardstocommunicatewithFabricsBandC
o SupporttwointernalharddrivesthatcanbeconfiguredinatwodriveRAIDarray
DifferencesbetweentheMseriesbladeserversinclude:
o TheM600isanIntelbased,singleheightbladethatisdesignedforheavyuseindata
centers.
o TheM605isanAMDbased,singleheightbladethatsupportsTPM.
Youhaveachoiceof2RAIDdrivecontrollers.
o TheCERC6iprovidesRAID0andRAID1functionalityand128MBRAIDcaching.
AlthoughyoucansettheCERC6itousewritebackcaching,youshouldavoidthis
option,becausethelackofbatterybackupplacesyourdataatrisktopowerloss.The
CERC6isconfigurationutilityiscommonlyknownasCTRL+R,becauseyoupressthose
keyswhenbootingthebladetoaccesstheutility.
o TheSAS6/iRisanentrylevelRAIDcapabledrivecontrollerforcostsensitivecustomers.
AdministratorscanperformRAIDconfigurationontheSAS6/iRbypressing<Ctrl><C>
duringPOST.TheSAS6/iRconfigurationutilityprovidesanoptiontoconfigurethe
Virtualdisks.Theusercanactivateordeletethevirtualdisks.
InternalStorageconsistsofavarietyofdaughtercardconfigurations.
o Typesofbladestorageconfigurations:
DisklessConfiguration,wheretherearenodisks,andHDDblanksareusedto
populatethebackplane
SATARepeaterDaughterCard
SAS6/iRModularDaughterCard
CERC6iModularDaughterCard
o SAS/SATAHDDBackplane:
HasconnectorsfortwoSAS/SATA2.5HDDs
HasCablelesscardedgeinterfacetoBladePlanar
63|P a g e

BladeServerConfiguration&Troubleshooting

HasToollessHDDbackplaneinstallation/removal
CommonHDDBackplanemechanical/electrical/boardsolutionsharedbetween
models
LEDsfordriveactivityandHDDstatus,butHDDstatusLEDisonlyfunctionalin
RAIDHDDconfiguration
The2.5HDDblankiscommontoexistingplatforms
HDDblanksmustbeinstalledforanyslotwithoutaSAS/SATAHDDpopulated
o Bladestoragedaughtercardinstallation/removal:
Toinstall,alignthecontrollerboardsothatthetabsonthemetalsystemboard
trayfitthroughthenotchesintheedgesofthecontrollerboard,thenslidethe
boardtowardstheconnectoronthesystemboarduntiltheboardsedge
connectorisfullyseated.
Toremove,openthereleaselevertodisengagethecontrollerboardedge
connectorfromthesystemboardconnector,thenliftthecontrollerboard
straightupfromthesystemboard.
Differenttypesofclusterarchitectureservedifferentcomputingneeds.
Bladeserversgiveyoutheadvantageofhighdensitycomputing,reducedcabling,reduced
coolingrequirementsandoptimizedpower.

64|P a g e

BladeServerConfiguration&Troubleshooting

Module4:SystemsManagementHardware
ModuleObjectives
ThismoduleoutlinesthesystemsmanagementoptionsavailablewiththeMseriesbladeservers.
Aftercompletingthismodule,youwillbeabletodescribethemanagementfeaturesofM1000eblade
servers.

ManagementBasics
DellsM1000emodularserverdeliversmajorenhancementsinmanagementfeatures.These
enhancementsderivefromdirectcustomerfeedbackandassistcustomersdeployingmodular(blade)
servers.Eachsubsystemhasbeenreviewedandadjustedto:

Optimizeefficiencies
Minimizeimpacttoexistingmanagementtoolsandprocesses
Providefuturegrowthopportunitiestostandardsbasedmanagement

Aftercompletingthistopic,youwillbeableto:

Explainhowmanagementinformationiscommunicatedthroughtheenclosure.
DescribetheM1000eenclosuremanagementfeatures.

ManagementConnections
ManagementconnectionsintheM1000eenclosuretransferhealthandcontroltrafficthroughoutthe
enclosure.Thesystemmanagementfabricuses100MbEthernetoverdifferentialpairsroutedtoeach
module.

Two100Mbinterfaces,oneswitchedandoneunswitched,communicatebetweenChassis
ManagementControllers(CMCs).
EveryIntegratedDRAC(iDRAC)modulehasamanagementnetworklinktoeachCMC.

65|P a g e

BladeServerConfiguration&Troubleshooting

Redundancyisatthecontrollerlevel.Failureofanyindividuallinktriggersfailovertotheredundant
CMC,aslongasthesystemisconfiguredwitharedundantCMC.

ManagementFeatures
Efficiencyisthecoreofthebladephilosophy,andeasy,securemanagementisattheheartofefficiency
inabladeenvironment.ManagementfeaturesavailableonMseriesbladeserversinclude:
ChassisManagementController(CMC)
TheCMCisthecentralpointforinfrastructuremonitoring,inventory,andcontrol.Itprovidesdedicated
EthernetmanagementconnectivitytoiDRACsandI/Oswitches.AddinganoptionalsecondCMC
providesredundancy.FailureoftheactiveCMCwilltriggerfailovertothestandbyCMCifasecondCMC
isinstalled.
A24portEthernetswitchisreservedforinternalcommunicationbetweentheiDR4Conthebladesto
theCMC.ItalsoprovidescommunicationbetweentheCMCandtheexternalmanagementnetwork.
IftwoCMCsareinstalled,theheartbeatforCMCredundancyisalsopresentandCMCredundancyis
supportedovertheinternalnetwork.Theinternalnetworkisseparatefromthenetworksusedbythe
hostLOMsandthemezzaninecardsintheblades.
TheCMCprovidesinternalpersistentstoragewithaflashcard.

66|P a g e

BladeServerConfiguration&Troubleshooting

iDRAC
TheM1000esupportstheIntegratedDellRemoteAccessController(iDRAC)thatisintegratedoneach
Mseriesblade.
AllnetworkconnectionstotheiDRACarethroughtheCMCnetworkinterface(Gb1).TheCMCroutes
traffictotheiDRACsonitsserversthroughaprivate,internalnetwork.Thisprivatemanagement
networkisoutsideoftheserversdatapathandoutsideoftheoperatingsystem'scontrol(outofband).
Themanagedservers'inbandnetworkinterfacesareaccessedthroughI/Omodules(lOMs)installedin
thechassis.
Eachbladeservermodule'siDRAChasitsownIPaddressandcanbeaccessed,ifsecuritysettingsallow,
directlythroughasupportedbrowser,Telnet,SSH,orIPMIclientonthemanagementstation.The
internalnetworkisseparatefromthenetworksusedbythehostLOM.
iKVM

TheiKVM:

SwitchlocalKVM(frontorrear)
SeamlesslytierintoDellandAvocentKVMinfrastructure
Provideslocalconsoleaccesstoallthe
bladesattheCMC(oneatatime)

LCDoperatorpanel
Supportssimplifiedinitialchassissetup
wizard
Providesenhancedlocaltroubleshooting
andinformation
Displaysconfiguration,statusinformation,
andsystemhealthforeachbladesiDRAC
andIOM,respectively

67|P a g e

BladeServerConfiguration&Troubleshooting

OperatorPanelandLCD
Theoperatorpanelcontainsthelocaluser
interfacethatprovidesbladeandenclosure
healthandinformation.Functionsinclude:

ChassisleveldiagnosticLEDs
LCD
Powerbutton

Theoperatorpanelisalwayspowered,evenin
chassisstandbymode.
Theoperatorpanelpivotssothatyoucanadjustitforeasyviewing,wherevertheenclosureisinthe
rack.Italsoretractsfullyintothechassiswhenitisnotneeded.

InterpretingtheOperatorPanelLED
TheLEDintheoperatorpanelchangescolortoindicate
theenclosuresstatus:

BlueAlldevicesintheenclosureare
functioningproperly.
OrangeCheckthestatusofthedevicesusing:
o TheLCDmenus
o Remotechassismanagement

68|P a g e

BladeServerConfiguration&Troubleshooting

LCDMenus
TheLCDshowschassisinformation,chassis
statusandfaults,andmajorbladeserver
modulefaults.Italsoprovidesaninterfacefor
bladeservermoduleandCMCnetwork
configuration.Includedisanerrormessage
whennoCMCispresentinthechassis.Service
Tag,AssetTagandIPAddressareeasilyvisible
oneveryM1000emodularsystemthroughthe
LCDdisplay.
Health:Displaysalerts,statusinformation,andhealthinformationfor:

Enclosure
Blades
IOM

Information:

Modulemodel/typeanduserdefinedname
Hardwareconfigurations
Servicetag
IPAddress

YoucaninitiallysettheCMCsIPaddressusingtheLCDConfigurationWizard.

ChassisManagementController
TheChassisManagementController
(CMC)isahotpluggablesystems
managementhardwareandsoftware
solution.TheM1000ealwayshasatleast
oneCMC,butmayhavetwo.Redundancy
isprovidedinanActiveStandbypairing
ofthemodulesintheenclosure.Failover
occurswhentheactivemodulefailsor
degrades.
Aftercompletingthistopic,youwillbeableto:

DescribethefeaturesandcapabilitiesoftheCMC.
ExplainhowtoaccesstheCMCandhowitcommunicateswithotherdevicesintheenclosure.
ExplainredundancyinMseriesCMCs.

69|P a g e

BladeServerConfiguration&Troubleshooting

FeaturesandCapabilities
TheCMChasitsownmicroprocessorandmemorybutreceivespowerfromtheenclosure.Itinterfaces
throughdualstackingEthernetportsandoneserialport.TheCMCsserialportinterfaceprovides
commonmanagementofuptosixI/Omodules(IOMs)throughasingleconnection.
TheCMCprovidessecureremotemanagementaccessandpowercontrolfunctionsforthechassisand
installedmodules.Itmanagesorfacilitatesthemanagementofthefollowing:

Status,InventoryandAlertingforbladeservermodules,chassisinfrastructureandI/OModules
CentralizedconfigurationforiDRAC,I/OModules,CMC,andiKVM
Powersequencingofmodulesinconjunctionwiththedefinedchassispowerstates
Powerbudgetmanagementandallocation
Configurationoftheembeddedmanagementswitch,whichfacilitatesexternalaccessto
manageablemodules
Remoteusermanagement
Userinterfaceentrypoint(Web,Telnet,SSH,serial)
Monitoringandalertingforchassisenvironmentalconditionsorcomponenthealththresholds
Thisincludesbutisnotlimitedtothefollowing:
o Realtimepowerconsumption
o Powersupplies
o Fans
o Powerallocation
o Temperature
o CMCredundancy
o I/Ofabricconsistency
Ethernettrafficmanagement(firewall)
PrivateVLAN
PrivateDHCPservice
PrivateTFTPservice
VirtualMedia
VirtualKVM(vKVM)interfacebetweentheexternalIPnetworkandthebladeservermodule
iDRAC
FirmwaremanagementofCMC,IOM,iDRAC,iKVM
AllserialportconfigurationsofIOMsgothroughtheserialportoftheCMC

TheCMCcanalsobeconfiguredtosendemailalertsorSNMPtrapalertsforwarningsorerrorsrelated
to:

Temperatures
Hardwareconfigurationerrors
Poweroutages
Fanspeeds

70|P a g e

BladeServerConfiguration&Troubleshooting

ProtocolsandothertoolssupportedbytheCMCinclude:

SMASHCLPfeaturesprovidingchassisbladeservermodulepowercontrolfunctions
WSMAN
CIMXML
SNMP
racadm
SSL/SSH

CMCCabling
AtleastoneCMCmustbeinstalledintheprimary
CMCbayfortheenclosuretopowerup.Ifasecond,
optionalCMCmoduleisinstalled,failoverprotectionandhotplugreplacementisavailable.
EachCMChastwoEthernetports:

Gb1isusedtoconnecttotheexternalmanagementnetwork.
Gb2willallowCMCsinadjacentenclosurestobedaisychained.

Withbasiccabling,youconnecttheGb1porttothemanagementnetworkandleavetheGb2port
unused.DONOTconnectbothportstoyournetwork.Gb2isonlyfor
daisychainingCMCs.
A24portEthernetswitchprovidesinternalcommunicationbetween:

TheiDRAConeachblade
IOMs
OptionalKVM
Optionalsecond,redundantCMC

DaisychainConnection
Ifyouhavemorethanoneenclosureinarack,youcanreducethe
numberofconnectionstothemanagementnetworkbydaisy
chaininguptofourchassistogether.

IfeachoffourchassiscontainsaredundantCMC,bydaisy
chainingyoureducethenumberofmanagementnetwork
connectionsrequiredfromeighttotwo.
IfeachchassishasonlyoneCMC,youreducetheconnections
requiredfromfourtoone.

Whendaisychainingchassistogether,Gb1istheuplinkportandGb2
isthestackingport.AGb1portmustconnecttothemanagement
networkortotheGb2portoftheCMCinanenclosurethatiscloser
tonetwork.TheGb2portmustonlyreceiveaconnectionfromaGb1portfurtherfromthechain.
71|P a g e

BladeServerConfiguration&Troubleshooting

CreateseparatechainsfortheCMCsintheprimaryCMCslotandthesecondCMCslot.
Redundancy
WhenbothCMCslotsinanM1000earepopulated,youhaveCMCredundancy.
ThefirstCMCbootedistheactive(primary)CMC,whiletheotherbecomesthestandbyCMC.The
standbyCMCmirrorstheactiveCMC.ThesettingsforbothCMCsareheldinNVRAMontheM1000e
chassisoperatorpanel.

BothCMCsmustrunthesamefirmwarerevision.Ifthefirmwarerevisionsdiffer,thesystemreportsas
redundancydegraded.
Failoverscanoccurwhen:

TheactiveCMCfails.
ThenetworkcableisremovedfromtheactiveCMC.
TheactiveCMCisremovedfromthechassis.
ACMCfirmwareflashisinitiatedontheactiveCMC.

IfaCMCfailoveroccurs:

AllexternaliDRACsessionsandallactiveCMCsessionsdisconnectandmustbereestablished.
InternaliDRACsessionsmovetothenewactiveCMC.

InitialConfiguration
TherearefourwaystoconfiguretheCMC:

LCDConfigurationWizard:ThiswizardisaccessedviatheLCDcontrolpanelonthefrontofthe
chassis.
CLIusinganullmodemcable:ThisisadirectserialconnectiontotheCMCviaanullmodem
cable.
Syntax:setniccfgs<IPaddress><networkmask><gateway>
Example:setniccfgs192.168.1.60255.255.255.0192.168.1.1
WebGUI:Canbeusedifyousetyourmanagementstation'sIPaddresstobeonthesame
subnetasthedefaultCMCIPaddress.

ThedefaultIPaddressis192.168.0.120.

CLIusingentry17ontheOSCARinterfacefortheiKVM.

72|P a g e

BladeServerConfiguration&Troubleshooting

IntegratedKVM
Theintegratedkeyboard,video,andmouse(iKVM)
switchfortheM1000eisanoptional,hotpluggable
module.Itprovideslocalkeyboard,mouse,andvideo
accesstoanyofthe16bladesintheenclosure.Italso
providesaccesstotheadditionalDellCMCcommand
lineconsole.
Aftercompletingthistopic,youwillbeableto:

ListthefeaturesandcapabilitiesoftheiKVM.
AccesstheOSCARSetupdialogbox.

Features
TheoptionalAvocentIntegratedKVMSwitchModule(iKVM):

Provideslocalkeyboard,mouse,andvideoaccesstoanyofthe16bladesinanM1000e
enclosure
ProvidesaccesstotheadditionalDellCMCcommandlineconsole
Maintainsallbladeconnectionsasinputisswitchedfromeachblade
Occupiesasingleslotaccessiblethroughtherearoftheenclosure
Redirectslocalbladevideo,keyboard,andmouseelectricalinterfacestoeither:
o TheiKVMlocalports
o TheM1000efrontpanelports

LocaliKVMaccesscanberemotelydisabledonaperbladebasis,usingtheblade'siDRACinterface
(accessisenabledbydefault).
TheiKVMalsohasanAnalogConsoleInterface(ACI)compatibleRJ45portthatallowstheiKVMtotie
theinterfacetoaKVMapplianceupstreamoftheiKVMviaCAT5cabling.AlthoughtheACIportisanRJ
45connectorandusesCAT5(orbetter)cabling,itisnotanEthernetnetworkinterfaceport.Itisonly
usedforconnectiontoexternalKVMswitcheswithAnalogRackInterface(ARI)ports.TheACIport
reducescostandcomplexitybygivingaccessforsixteensimultaneousserversusingonlyoneportonan
externalKVMswitch.

TheiKVMavailablefortheM1000Echassisisanalog.TheiDRAChandlesfunctions,suchasvirtualmedia
orremotedesktop,thatdigitalKVMsprovidedinthepreviousblademodels.

73|P a g e

BladeServerConfiguration&Troubleshooting

Capabilities
iKVMuserinterface:OnScreenConfigurationandActivityReporting(OSCAR).TheOSCAR
displayallowsyoutoselect,configureandmanagethebladesintheenclosure.

Security:TheOSCARuserinterfaceallowsyoutoprotectthesystemwithascreensaver
password.
Video:TheiKVMvideoconnectionssupportvideodisplayresolutionsrangingfrom640x480at
60Hzupto1280x1024at60Hz.
Flashupgrades:YoucanupdatetheiKVMfirmwareusing:
o CMCWebbasedinterface(requiresthefirmwarefile)
o racadmfwupdatecommand(requiresaTFTPservertouploadthefileusingtheOSCAR
interface)
CMCaccessthroughentry17:TheCMCcommandlineinterfacecanbeaccessedthroughentry
17ontheOSCARinterface.
Enablinganddisablingaccess:YoucanenableordisableaccessthroughthefrontpanelorCMC
(throughentry17)withtheCMC.

74|P a g e

BladeServerConfiguration&Troubleshooting


OSCARConfiguration
TheOSCARmenuhasacommonfeelfoundthroughouttheDell
rangeofKVMs.
ToaccesstheSetupmenu:
1. Press<PrintScreen>tolaunchOSCAR.
2. ClickSetuptoaccesstheSetupdialogbox.
YoucansoftswitchtheiKVMbetweenblades,usingahotkey
sequence,bypressing<PrintScreen>andthentypingthefirst
fewcharactersofitsnameornumber.Ifyoupreviouslyseta
delaytime(thenumberofsecondsbeforetheMaindialogboxis
displayedafter<PrintScreen>ispressed)andyoupressthekey
sequencesbeforethattimehaselapsed,theOSCARinterface
doesnotdisplay.

IntegratedDRAC
TheIntegratedDellRemoteAccessController(iDRAC)isasystemsmanagementhardwareandsoftware
solutionthatprovides:

Remotemanagementcapabilities
Crashedsystemrecovery

75|P a g e

BladeServerConfiguration&Troubleshooting

Powercontrolfunctions

Aftercompletingthistopic,youwillbeableto:

DescribethefeaturesoftheiDRAC.
ListtheconfigurationutilitiesavailableontheiDRAC.

EmbeddedbutSeparate
iDRACsareembeddedintheMseriesfamilyofblades.TheiDRACcombinesthefunctionalityofthe
baseboardmanagementcontroller(BMC)andDellRemoteAccessController(DRAC)usedontraditional
serversintoasingledeviceonthebladessystemboard.
TheiDRAChasbothsoftwareandhardwarecomponents.Thehardwarecomponentprovides:

Virtualmedia(vMedia)andvirtualKVM(vKVM)functions
Outofbandgraphicaluserinterface(GUI)
Outofbandstatusandinventoryinformation

TheiDRACispoweredbystandbypower.Withabladeinthechassis,theiDRACisalwaysonand
providesmanagementandmonitoringcapabilities.
ManagementFeatures
TheiDRACprovidesthefollowingmanagementfeatures:

ConsoleRedirection:Providesremotesystemkeyboard,video,andmousefunctions.
DellOpenManagesoftwareintegration:EnablesyoutolaunchtheiDRACWebinterfacefrom
DellOpenManageServerAdministratororITAssistant.
DynamicDomainNameSystem(DDNS)registration.
iDRACalerts:AlertsyoutopotentialmanagednodeissuesthroughanemailmessageorSNMP
trap.
IPMIsupport.
MicrosoftActiveDirectoryauthentication:CentralizesiDRACuserIDsandpasswordsinActive
Directoryusingthestandardschemaoranextendedschema.
Monitoring:Providesaccesstosysteminformationandstatusofcomponents.
Passwordlevelsecuritymanagement:Preventsunauthorizedaccesstoaremotesystem.
Remotepowermanagement:Providesremotepowermanagementfunctions,suchas
shutdownandreset,fromamanagementconsole.
Remotesystemmanagementandmonitoring:Using:
o Webinterface.
o Localracadmcommandlineinterface.
o SMCLPcommandlineoveraTelnet/SSHconnection.
Rolebasedauthority:Providesassignablepermissionsfordifferentsystemsmanagementtasks.
SecureSocketsLayer(SSL)encryption:Providessecureremotesystemmanagementthrough
theWebinterface.

76|P a g e

BladeServerConfiguration&Troubleshooting

Systemlogaccess:Providesaccesstothesystemeventlog,theiDRAClog,andthelastcrash
screenofthecrashedorunresponsivesystem.Accessisindependentoftheoperatingsystem
state.
VirtualMedia:Enablesamanagedservertoaccessalocalmediadriveonthemanagement
stationorCD/DVDISOimagesonanetworkshare.

ManagementInterface
TheiDRACnetworkinterfaceisdisabledbydefault.Youmustconfigureitbeforeyoucanaccessthe
iDRAC.AfteraccessingtheiDRAConthenetwork,youcanaccessitatitsassignedIPaddresswith:

TheiDRACWebinterface
Telnet
SSH
Supportednetworkmanagementprotocols,suchas:
o SimpleNetworkManagementProtocol(SNMP)
o IntelligentPlatformManagementInterface(IPMI)

YoucanconfiguretheiDRACtosendyouanemailmessageorSimpleNetworkManagementProtocol
(SNMP)trapalertsforwarningsorerrors.Tohelpyoudiagnosetheprobablecauseofasystemcrash,
iDRACcanlogeventdataandcaptureanimageofthescreenwhenitdetectsthatthesystemhas
crashed.
TheCMCmonitorsandmanagesallcomponentsintheenclosure,includingiDRACs.TheLCDthatispart
oftheenclosuredisplaystheiDRAC'sintegrationinformationandthehealthstatusoftheblade.
AllnetworkconnectionstotheiDRACarethroughthenetworkinterface(Gb1)ontheactiveCMC.The
CMCroutestraffictotheiDRACsonitsserversthroughaprivate,internalnetwork.

77|P a g e

BladeServerConfiguration&Troubleshooting


TraditionalIPMIbasedbaseboardmanagementcontroller(BMC)featuressuchashardwaremonitoring
andpowercontrolaresupported.Withuptotwosimultaneoussessionsperblade,remote
managementsatisfiesvirtuallyanyusagemodel.
VirtualmediaoverIPisstandard.TheiDRACemulates:

DVDROM
CDROM
USBFlashDrive
ISOimage
USB/legacyfloppy
Floppyimage

FullUSBaccessisavailablethroughthebladeservermodulefrontpanel.Connectiontoconsole
redirectionandvirtualmediaisthroughtheCMC,withanoptionavailabletoencryptthosesessions.
HighlightsoftheiDRACsolutioninclude:

Dedicatedmanagementinterface
VirtualMedia
ConsoleredirectionviaVirtualKVM
IPMI2.0OutOfBandmanagement

78|P a g e

BladeServerConfiguration&Troubleshooting

SerialoverLAN
SMASHCLP
Bladestatusandinventory
Activepowermanagement
LocalSecurityandIntegrationwithActiveDirectory

SecurityFeatures
TheiDRACprovidesthefollowingsecurityfeatures:

UserauthenticationthroughMicrosoftActiveDirectory(optional)hardwarestoreduserIDsand
passwords
Rolebasedauthority,whichenablesanadministratortoconfigureprivilegesforeachuser
UserIDandpasswordconfigurationthroughtheWebinterfaceorSMCLP
SMCLPandWebinterfaces,whichsupport128bitSSLencryptionand40bitSSLencryption(for
countrieswhere128bitisnotacceptable)
Sessiontimeoutconfiguration(inseconds)throughtheWebinterfaceorSMCLP
ConfigurableIPports(whereapplicable)

TelnetdoesnotsupportSSLencryption.

SecureShell(SSH),whichusesanencryptedtransportlayerforhighersecurity
LoginfailurelimitsperIPaddress,withloginblockingfromtheIPwhenthelimitisexceeded
LimitedIPaddressrangeforclientsconnectingtotheiDRAC

OutofbandCommunication
AllnetworkconnectionstotheiDRACarethroughtheCMCnetworkinterface(Gb1).TheCMCroutes
traffictotheiDRACsonitsserversthroughaprivate,internalnetwork.Featuresoftheoutofband
networkinclude:

Dedicatedinternal100MbEthernetconnectiontoeachCMC
o TheiDRACconnectstotheCMCviadedicated,fullyredundant100MbpsEthernet
connections.
Passesthroughthemidplanetoadedicated24portEthernetswitchonthe
CMC
ExposedtotheoutsideworldthroughtheCMCsexternalManagement
Ethernetinterface(Gb1)
o ThisconnectionisdistinctfromthethreeredundantdataFabricsA,BandC.
UnlikepreviousgenerationsofDellbladeservers,theiDRACsconnectivityis
independentof,andinadditionto,theonboardGbELOMsontheblade.
ConfigurableWeborcommandlineinterface(CLI)usingVirtualMedia

79|P a g e

BladeServerConfiguration&Troubleshooting

ConfigurationInterfaces
YouhaveseveraloptionsforconfiguringtheiDRAC.However,usingmorethanoneconfiguration
methodatatimecangenerateunpredictableresults.

80|P a g e

BladeServerConfiguration&Troubleshooting

iDRACConfigurationUtility
Accessthisatboottimetosetupthenetworkandbasicsecurityfeaturesandforenablingother
features.Press<Ctrl>+<E>duringPOSTtoaccesstheutility.IftheWebinterfacetotheCMCisnotbeing
used,thisisthebestopportunitytoturnontheEthernetinterfacetotheiDRACandsettheIPaddress
forremoteaccess.
iDRACWebinterface
TheiDRACWebinterfaceisabrowserbasedmanagementapplicationthatyoucanusetointeractively
managetheiDRACandmonitorthemanagedserver.Itistheprimaryinterfacefordaytodaytaskssuch
as:

Monitoringsystemhealth
Viewingthesystemeventlog
ManaginglocaliDRACusers

LaunchingtheCMCwebinterfaceandconsoleredirectionsessions

CMCWebInterface
Inadditiontomonitoringandmanagingthechassis,youcanusetheCMCwebinterfaceto:

Viewthestatusofamanagedserver
ConfigureiDRACnetworksettings
Start,stop,orresetthemanagedserver

Thecurrentfirmwaredoesnotproperlyperformagracefuloperatingsystemshutdown.
ChassisLCDPanel
TheLCDpanelonthechassiscontainingtheiDRACcanbeusedtoviewthehighlevelstatusofthe
serversinthechassis.DuringinitialconfigurationoftheCMC,theconfigurationwizardallowsyouto
enableconfigurationofiDRACnetworking.
Localracadm
Thelocalracadmcommandlineinterfacerunsonthemanagedserver.Theserverisaccessedfrom
eithertheiKVMoraconsoleredirectionsessioninitiatedfromtheiDRACWebinterface.
RacadmisinstalledonthemanagedserverwhenyouinstallDellOpenManageServerAdministrator.
RacadmcommandsprovideaccesstonearlyalliDRACfeatures.Youcan:

Inspectsensordata,systemeventlogsrecords,andthecurrentstatusandconfigurationvalues
maintainedintheiDRAC.
AlteriDRACconfigurationvalues.
Managelocalusers.
Enableanddisablefeatures.
Performpowerfunctionssuchasshuttingdownorrebootingthemanagedserver.

81|P a g e

BladeServerConfiguration&Troubleshooting


TheCMCalsohasaracadminterfaceavailablethroughslot17oftheiKVM.Itcanbeaccessedremotely
viaTelnet,SSH,orfromamanagementstation.

OpenManageIntegration
MseriesbladeserversfullyintegrateDellOpenManage,OpenManageServerAdministrator,andIT
Assistant.Aftercompletingthismodule,youwillbeableto:

DescribehowOpenManageServerAdministratorandITAssistantintegratewiththeM1000e.

OpenManageServerAdministrator
OpenManageServerAdministrator(OMSA)supportsthefollowingM1000emanagementfunctions:

Powermonitoring:Thislistshowssomeofthefeaturesthatyoucanmonitor:
o Ampsperpowersupply
o Aggregatewattsconsumed
o Aggregatecumulativepowerdrawn
o Resettingofpeakpowervalues
o Maximumpeakpower(wattsandamps)
o Powersupplyinputvoltage
DellOpenManagesoftwareintegration:YoucanlaunchtheiDRACwebinterfacefromOMSA.

82|P a g e

BladeServerConfiguration&Troubleshooting

OMSAiscoveredingreaterdepthintheOpenManageServerAdministratormoduleofthiscourse.

ITAssistantIntegrationnowreferredtoasDMCprovidesPowerMonitoringforserverssupporting
PMBusreportswiththefollowingtypesofdata:
Aggregationofpowerandenergyconsumedbyvariousgroupsofservers
Amperageperpowersupply
Energyandpowerconsumptionperblade
Peakpowerandpeakamperageperserver
Powerconsumptionandtotalenergyconsumed
ITAssistantiscoveredingreaterdepthintheITAssistantmoduleofthiscourse.

83|P a g e

BladeServerConfiguration&Troubleshooting


ITAssistant

Summary
Reviewthesummaryofthismodulebeforetakingtheselfchecktoseehowwellyoumasteredthe
topic.

Managementbasicconceptsinclude:
o ManagementinformationfromeachbladeandmoduleintheenclosuretotheCMC
overredundant100BaseTpaths.
o ManagementfeaturesofMseriesbladeserversinclude:
CMC
iDRAC
iKVM
LCDoperatorpanel
o TheretractableoperatorpanelcontainstheLCDpanel,powerbutton,anddiagnostic
LEDs.
o TheLEDontheoperatorpanelisbluewhenalldevicesarefunctioningproperly.Itblinks
orangetoindicatethatoneofthedevicesintheenclosurehasaproblem.
TheMseriesoperatorpanelhastwomenus:
o Healthdisplaysalerts,statusinformation,andhealthinformationforthedevicesin
theenclosure.
o Informationdisplaysmoduleinformation,servicetag,andIPaddress.
TheChassisManagementController(CMC)isahotpluggablesystemsmanagementhardware
andsoftwaresolution.

84|P a g e

BladeServerConfiguration&Troubleshooting

Status,InventoryandAlertingforbladeservermodules,chassisinfrastructureandI/O
Modules
o TheCMCcanbeconfiguredtosendemailalertsorSNMPtrapalertsforwarningsor
errorsrelatedto:
Temperatures
Hardwareconfigurationerrors
Poweroutages
Fanspeeds
o TheCMChastwoEthernetports
Gb1isusedtoconnecttotheexternalmanagementnetwork.
Gb2willallowCMCsinadjacentenclosurestobedaisychained.
o DaisychainingCMCsintheracksignificantlyreducesthenumberofcablesrequired.
o CMCredundancyisattainedwhenbothCMCslotsinanM1000earepopulated.
o Failoverscanoccurwhen:
ThenetworkcableisremovedfromtheactiveCMC.
TheactiveCMCisremovedfromthechassis.
ACMCfirmwareflashisinitiatedontheactiveCMC.
o ThefourwaystoconfiguretheCMCare:
LCDConfigurationWizard
CLIusinganullmodemcable
WebGUI
CLIusingport17ontheiKVM
Theintegratedkeyboard,video,andmouse(iKVM)switchfortheM1000eisanoptional,hot
pluggablemodule.
o TheMseriesiKVMis:
Optional
Hotswappable
Analog
o TheiKVMprovides:
Anonscreenutilitytoselect,configure,andmanagebladesintheenclosure
Ascreensaverpassword
Flashupgrades
AccesstotheCMCCLI
o YoucanconfigureOSCARbypressing<PrintScreen>andclickingSetupintheresulting
menu.
TheIntegratedDellRemoteAccessController(iDRAC)isasystemsmanagementhardwareand
softwaresolution.
o TheiDRACresidesonthebladessystemboardandprovides
Virtualmedia(vMedia)andvirtualKVM(vKVM)functions
Anoutofbandgraphicaluserinterface(GUI)
Outofbandstatusandinventoryinformation
o MseriesiDRACmanagementfeaturesinclude:
o

85|P a g e

BladeServerConfiguration&Troubleshooting

ConsoleRedirection
DellOpenManagesoftwareintegration
DynamicDomainNameSystem(DDNS)registration
iDRACalerts
IPMIsupport
MicrosoftActiveDirectoryauthentication
Monitoring
Passwordlevelsecuritymanagement
Remotepowermanagement
Remotesystemmanagementandmonitoring
Rolebasedauthority
SecureSocketsLayer(SSL)encryption
Systemlogaccess
o TheiDRACcommunicatesthroughtheGb1connectionontheCMC.
o AllcommunicationwiththeiDRACpassesthroughtheCMC.
o ThefollowingconfigurationinterfacesareavailableontheiDRAC:
iDRACConfigurationUtility
iDRACWebinterface
CMCWebinterface
ChassisLCDPanel
Localracadm
iVMCLI
iDRACSMCLP
IPMI
MseriesbladeserversfullyintegrateDellOpenManage.
o YoucanuseOpenManageServerAdministratorwithanM1000eenclosureto:
Monitorpowerusage
LaunchtheiDRACWebinterface
o PowermanagementfeaturesavailablethroughITAssistantinclude:
Aggregationofpowerandenergyconsumedbyvariousgroupsofservers
Amperageperpowersupply
EnergyandPowerconsumptionperblade
PeakpowerandPeakamperageperserver
Powerconsumptionandtotalenergyconsumed

86|P a g e

BladeServerConfiguration&Troubleshooting

Module5:DellOpenManageSoftware

ModuleObjectives
Aftercompletingthismodule,youwillbeableto:

ExplainhowtouseDellOpenManageServer
Administrator(OMSA)tomanage
interactionsbetweenbladeserversanda
storageareanetwork(SAN).

SystemManagementBasics
Thistopicintroducesthebasicconceptsofsystemmanagement.Aftercompletingthistopic,youwillbe
ableto:

Definesystemsmanagement.
DescribeDellOpenManage.
ListthesystemsmanagementprotocolssupportedbyOpenManage.

87|P a g e

BladeServerConfiguration&Troubleshooting

WhatIsManagement?
SystemsManagementusestools(hardwareandsoftware)toperformmundane,simple,orcomplex
tasks.Systemsmanagementcancallonmultiplecomponents:

Hardware
Remotemanagementhardware,inadditiontothebasehardwarethatmakestheserverwork
Software
Agentstomonitorstatusandgeneratealerts
Softwaretomanagesameordifferentsystems

DellOpenManageSystemOverview
DellOpenManagesystemsmanagementsoftwareisasuiteofapplicationprogramsforDellsystemsthat
allowsyoutomanageyoursystemwithproactivemonitoring,diagnosis,notification,andremote
access.EachofyourmanagedsystemswilluseapplicationsthatincludeServerAdministrator,IT
Administrator,andremoteaccesscontroller(RAC)software.Amanagementstationcanbeusedto
remotelymanageoneormoremanagedsystemsfromacentrallocation.ByinstallingITAssistantona
managementstation,youcaneffectivelymanagefromonetothousandsofremotemanagedsystems.
88|P a g e

BladeServerConfiguration&Troubleshooting

Thisgraphicillustratestherelationshipbetweenamanagementstationanditsmanagedsystems.Italso
showstheoperatingsystemsandtheDellOpenManagesoftwareproductsthatmaybeinstalledonthe
managedsystems.

TheDellOpenManageModel
ReviewtheOpenManageModelon
thispage.NotethatOpenManage
consistsofDeploymentTools,
MonitoringTools,andMaintenance
Tools.Reviewthenamesofallthe
individualtoolsintheOpenManage
suite,andthenfocusonthetoolsin
theDeploymentcategory.

89|P a g e

BladeServerConfiguration&Troubleshooting

SystemsBuildandUpdateUtility
TheDellSystemsBuildandUpdate
UtilityisabootableCDthatprovides:

Stepbystepconfigurationfor
anunattendedinstallofthe
operatingsystem.
Installationsupportfor
WindowsandLinuxoperating
systems.
Delloptimizeddevicedrivers,
firmware,andutilitiesforDell
PowerEdgeTMservers.

90|P a g e

BladeServerConfiguration&Troubleshooting

Deploymenttoolkit
DellOpenManageDeploymentToolkit
(DTK)providessomeofthe
componentsthatfacilitateunattended
serverdeployment.DTKincludesaset
ofutilitiesthatsupportsbothCLIand
filebasedparameters,aswellas
documentationthatdescribesits
components.DTKcomponentsare:

Unattendedserverhardware
configuration
Utilitypartition(WP)
BasicInput/OutputSystem
(BIOS)
RedundantArrayof
IndependentDisks(RAID)
NetworkInterfaceCards(NICs)

UnattendedinstallationofDelldriversandmanagementagentsperoperatingsystem.Thisfeature
containsasetoftoolsandscriptsthatcanbeusedbymostthirdpartydeploymentsoftwarethat
hasanAutomatedDeploymentService(ADS).
ServerManagement:RemoteManagement
RemoteManagementHardware
Remotemanagementhardwareisoftenoptional.Itgenerallyenablesyoutoperformextra
managementtasksthatwouldnototherwisebepossible.ForDellsystems,thisadditionalhardwareis
partofaDellRemoteAccessController(DRAC),whichbundlesthehardwareandsoftwareneededfor
remotemanagement.
Remotemanagementhardware:

Oftenworksoutofband
Doesnotrequireoperatingsysteminteraction
Functionsevenwhenthesystemispowereddown

Systemsmanagementsoftwareusesmanagementprotocols(suchasSNMPorCIM)toIMPIcomponents
(suchasBMC)andtochangesettings.Theseprotocolsareembeddedwithintheoperatingsystem.
Therefore,theyareinband,availableonlywhentheoperatingsystemisrunning.

91|P a g e

BladeServerConfiguration&Troubleshooting

OutofBandManagement
Anadditionaladaptercardisrequiredtoperformoutofbandmanagement.DellrecommendstheDRAC
familyofadapters.ThesecardsenableyoutoremotelyaccesstheinformationsuppliedbyIMPIand
controlthemanageablehardware.
DRAChardwaresupplied
interfacesinclude:

Serial
Telnet

Ethernet

InBandManagement
Inbandmanagementis
accomplishedviainstrumentation
intheoperatingsystem.Inband
managementmeansthata
managementsystemisdealing
withamanagednodethroughthe
operatingsystemandthrough
agentsanddrivers.
TheDellRemoteAccesshardware
supportsinbandmanagement,
andallowsyoutoperformthree
majortypesoftasks:

Viewsystemrelated
managementinformation
suchastemperature,voltagesvianewBMC
Performpowermanagement
TakecontroloftheserverGUIremotely

ServerManagement:InbandInterfaces
OpenManageServerAdministrator
ServerAdministratorprovidesacomprehensive,onetoonesystemsmanagementsolutionfroman
integrated,WebbrowserbasedGUI(theServerAdministratorhomepage),andfromacommandline
interface(CLI)throughtheoperatingsystem.ServerAdministratorisdesignedforsystemadministrators
tobothlocallyandremotelymanagesystemsonanetwork.

92|P a g e

BladeServerConfiguration&Troubleshooting

ITAssistant
ITAssistantprovidesaonetomanymanagementsolutionwhenOpenManageServerAdministrator
(OMSA)isalsoinstalledonthemanagedsystems.ServerAdministratorenablesyoutoconfigurethe
RAC.
SystemsManagementProtocols
Protocolscanbedescribedas:

Rulesofcommunicationbetweenagentsandthesystemmanagementapplication
Industrystandardtechnologiesforcosteffectivemanagement

Protocolsgatherandtransmitmanagementinformation.Systemsmanagementprotocolsinclude:

SimpleNetworkManagementProtocol(SNMP)
CommonInformationMethod(CIM)
IntelligentPlatformManagementInterface(IPMI)

Review
SystemsmanagementprotocolssupportedbyDellOpenManagesoftwareinclude:

CommonInformationMethod(CIM)
IntelligentPlatformManagementInterface(IPMI)
SimpleNetworkManagementProtocol(SNMP)

DellOpenManageServerAdministrator
BundledwithDellOpenManagesoftware,DellOpenManageServerAdministrator(OMSA)provides
configurationandmonitoring,andreplacingfunctionality.ThistopiccoversthebasicsofOMSA.
Aftercompletingthistopic,youwillbeableto:

DescribethefunctionsoftheOMSAmanagementservice.
DescribetheOMSAstoragemanagementservice.
DescribetheOMSAremoteaccesscontroller.
DescribeITAssistant.
DescribetheDellOnlineDiagnostics.

Ifyouneedmoreindepthinformation,youmaywanttotaketheDellOpenManagecourse.

93|P a g e

BladeServerConfiguration&Troubleshooting

OMSA:InstrumentationService
TheServerAdministratorInstrumentationServicemonitorsthehealthofasystemandprovidesrapid
accesstodetailedfaultandperformanceinformation.Thereportingandviewingfeaturesallowretrieval
ofoverallhealthstatusforallchassisinthesystem.Atthesubsystemlevel,userscanviewinformation
aboutthevoltages,temperatures,fanspeed,andmemoryinthesystem.Adetailedaccountofevery
relevantcostofownership(COO)detailaboutyoursystemcanbeseeninsummaryview.Version
informationforBIOS,firmware,operatingsystem,andallinstalledsoftwareiseasytoretrieve.
Additionally,systemsadministratorscan
usetheInstrumentationServiceto
performthefollowingessentialtasks:

Specifyminimumandmaximum
valuesforcertaincritical
components.Thevalues,called
thresholds,determinetherange
inwhichawarningeventforthat
componentoccurs.
Specifyhowthesystemresponds
whenawarningorfailureevent
occurs.Userscanconfigurethe
actionsthatasystemtakesin
responsetonotificationsof
warningandfailureevents.
Populatealloftheuserspecific
valuesforthesystem,suchasthe
nameofthesystem,thephone
numberofthesystemsprimary
user,thedepreciationmethod,
whetherthesystemisleasedor
owned,andsoon.

OMSAStorageManagementService
TheDellOpenManageapplicationsusedto
controlandmonitorthestoragesubsystem
areOpenManageStorageManagement
(OMSM).Theapplicationprovidesstorage
managementinformationinanintegrated
graphicalview.FeaturesoftheStorage
ManagementServiceinclude:

TheStorageManagementService
enablesyoutoviewthestatusoflocalandremotestorageattachedtoamonitoredsystem

94|P a g e

BladeServerConfiguration&Troubleshooting

SupportsSCSE,SATA,andSAS
DoesnotsupportFibreChannel
Enablesyoutoperformcontrollerandenclosurefunctionsforallsupportedcontrollersand
enclosuresfromasinglegraphicalorcommandlineinterfacewithoutusingcontrollerBIOS
utilities
Protectsyourdatabyhelpingyou:
o Configuredataredundancy
o Assignhotspares
o Rebuildfaileddrives
Providesagraphicalinterfacethatiswizarddrivenwithfeaturesfornoviceandadvancedusers
anddetailedonlinehelp
Providesacommandlineinterfacethatisfullyfeaturedandscriptable

95|P a g e

BladeServerConfiguration&Troubleshooting

OMSA:RemoteAccessController
TheServerAdministratorRemote
Controller(RAC)providesacomplete
remotesystemmanagementsolutionfor
systemsequippedwithanyofthe
followinginstrumentation:

IntegratedDellRemoteAccess
Controller(iDRAC)
DellRemoteAccessCard(DRAC)5
EmbeddedRemoteAccess(ERA)
controller
ERAOption(ERA/O)card
ERA/MCcontroller

Remoteaccessrequiresinstrumentation
thatsupportseither:

SimpleNetworkManagement
Protocol(SNMP)
CommonInformationMode(CIM)

ThesehardwareandsoftwaresolutionsarecollectivelyknownasRemoteAccessControllers(RACs).
UsersmaychoosetoperformsomeofthesefunctionsthroughtheBMC,butworkingthroughaRACis
moreefficient.
TheRACprovides:

Remoteaccesstoaninoperablesystem,enablingyoutogetthesystemupandrunningas
quicklyaspossible.
Alertnotificationwhenasystemisdown.
Remoterestartofasystem.
Logsoftheprobablecausesofsystemcrashesandthemostrecentcrashscreen.
RemoteaccessthroughtheServerAdministratorhomepageorbydirectlyaccessingthe
controllersIPaddressusingasupportedbrowser.

96|P a g e

BladeServerConfiguration&Troubleshooting

MonitoringTools:ITAssistant
Useemail,paging,orconsole
alertingtomonitorfaultswith:
o Disks
o Fans
o Memory
o Thermalconditions
o Voltage
Inventoryandassetreporting
thatincludes:
o Servicetag
o Costofownership
information
o SpecificsofBIOS,
processor(s),andmemory
Incontextlaunchofdevice
specifictools,suchas:
o StorageManager
o ServerAdministrator
o RemoteAccessController
o PowerConnectSwitchManager
MonitoringTools:BaseboardManagementController
Thebaseboardmanagementcontroller
(BMC)isresponsibleformonitoringthe
healthoftheserverandcomponents
associatedwiththeserverssystemboard.It
doesnotusuallymonitordevicesontheI/O
buses,anditoperatesindependentlyofthe
systemboard.
Themonitoringtechnologyisbasedonthe
IntelligentPlatformManagementInterface
(IPMI)standard.Thisstandardisdefinedin
conjunctionwithIntelandotherOriginal
EquipmentManufacturers,includingDell.
IMPIisbeingusedinthesystems
managementworkinggroupatthe
DistributedManagementTaskForce
(DTMF).

97|P a g e

BladeServerConfiguration&Troubleshooting

TheDellBMCsupportsIPMI1.5andlater.Itdelivers:

Monitoringofserverhardware.
Alertswhenpotentialoractualfaultsareencountered.
Networkandserialaccesstocontrolserverpowerandresettheserverregardlessoftheservers
powerstatus.

MaintenanceTools:DellServerUpdateUtility
TheDellServerUpdateUtility(SUU)
simplifiesthetaskofupdatingBIOS,
firmware,anddriversonservers.Insert
theDellServerUpdatesDVDintoa
serversDVDdrivetoautomatically:

InventorytheDellfirmwareand
driversonthatserver.
Comparetheinstalled
configurationwiththecontenton
theDellServerUpdatesDVD.
Reportondiscrepanciesand
recommendupdates,including
anyprerequisitesthatimpacttheupdatesequence.
Updateand(ifneeded)reboottheserver.

TheDellServerUpdateUtilityisdesignedtosimplifysingleserverupdatesandisavailableontheDell
ServerUpdatesDVDincludedaspartoftheDellOpenManageSubscriptionService.
MaintenanceTools:DellOnlineDiagnostics
TheDellOnlineDiagnostics:

Providestestingofthesystem
withoutbringingitdown:

Thesystemadministratorcanrun
thediagnosticsasapreventative
measure,orafteranonfatal
failure,toverifythestatusofthe
systemhardware.

Helpsdetermineifthecauseofthe
failureisthehardware:
Theonlinediagnosticsenabletheadministratortodetermineiftheproblemishardwareandto
makeamoreaccurateassessmentoftheproblem.

98|P a g e

BladeServerConfiguration&Troubleshooting

ProvideshardwareFieldReplacementUnit(FRU)isolation.

DellOnlineDiagnosticsarenowavailableonthefollowingsources:

DellSystemsServiceandDiagnosticsToolsCD
DellSystemsManagementToolsandDocumentationDVD
TheOpenManageSubscriptionService

Summary
Reviewthesummaryofthismodulebeforetakingtheselfchecktoseehowwellyoumasteredthe
topic.

Basicsystemsmanagementconceptsinclude:
o Systemsmanagementconsistsof:
Usingavarietyoftoolstoremotelymanagecomputerhardware.
o DellOpenManageconsistsof:
Deploymenttools
Monitoringtools
Maintenancetools
o TheSystemsBuildandUpdateUtilityprovides:
Configurationforanunattendedoperatingsysteminstallation
InstallationsupportforWindowsandRedHatLinux
Delloptimizeddevicedrivers,firmware,andutilities
o TheDeploymenttoolkitincludescomponentsthatfacilitateunattendedserver
deployment.
o SystemsmanagementprotocolssupportedbyDellOpenManagesoftwareinclude:
CommonInformationMethod(CIM)
IntelligentPlatformManagementInterface(IPMI)
SimpleNetworkManagementProtocol(SNMP)
OpenManageServerAdministrator(OMSA)providesconfiguration,monitoring,andreporting
functions.
o TheOMSAInstrumentationservicemonitorsthehealthofasystemandprovidesrapid
access.
o TheDellOpenManageStorageManagementapplicationprovidesstoragemanagement
information.
o TheOMSAremoteaccesscontrollerprovidesacompleteremotesystemmanagement
solutioninstrumentation:
DellRemoteAccessCard(DRAC)5
EmbeddedRemoteAccess(ERA)controller

99|P a g e

BladeServerConfiguration&Troubleshooting

ERAOption
(ERA/O)
card
ERA/MC
controller
UsingITAssistant
youcan:
Bealerted
ofspecific
system
faults
Generate
inventory
andasset
reports
Launch
device
specifictools
TheDellOnlineDiagnostics:
Enablesyoutotestasystemwithoutdowningit
Helpsdistinguishbetweenhardwareandsoftwareissues
Provideshardwareisolation

Module6:iDRAC
TheIntegratedDellRemoteAccessController(iDRAC)isafullyfunctionalDRACthatisembeddedonthe
systemboardofMseriesbladeservers.ThismoduleassumesyouarealreadyfamiliarwithDRAC
functionality.Ifnot,youshouldreviewtheDRACchapterofthePowerEdgeservercourse.
Aftercompletingthismodule,youwillbeableto:

DescribethebasicfeaturesoftheiDRAC
ExplainhowtoconfiguretheiDRAC

iDRACManagementandSecurityFeatures
TheIntegratedDellRemoteAccessController(iDRAC)isasystemsmanagementhardwareandsoftware
solutionthatprovidesremotemanagementcapabilities,crashedsystemrecovery,andpowercontrol
functionsforDellPowerEdgesystems.

100|P a g e

BladeServerConfiguration&Troubleshooting

Aftercompletingthistopic,youwillbeableto:

DescribetheiDRACsmanagementfeatures
DescribetheiDRACssecurityfeatures

iDRACManagementFeatures
TheiDRACprovidesthefollowingmanagementfeatures:

DynamicDomainNameSystem(DDNS)Registration:ProvidestheabilitytoregistertheiDRAC
nametotheDomainNameServers(DNS).
Remotesystemmanagementandmonitoring:Providesmanagementandmonitoring
capabilitiesusing:
o AWebinterface
o Thelocalracadmcommandlineinterface(CU)
o TheSMCLPcommandlineoveraTelnet/SSHconnection
SupportforMicrosoftActiveDirectoryauthentication:CentralizesiDRACuserIDsand
passwordsinActiveDirectoryusingthestandardschemaorDellextendedschema.
ConsoleRedirection:Providesremotekeyboard,video,andmousefunctions.
VirtualMedia:Enablesamanagedservertoaccessalocalmediadriveonthemanagement
stationorCD,DVD,floppy,CD/DVDISOimagesorfloppyimagesonanetworkshare.
Monitoring:Providesaccesstosysteminformationandstatusofcomponents.

Accesstosystemlogs:Providesaccesstothesystemeventlog,theiDRAClog,andthelastcrash
screen.

DellOpenManagesoftwareintegration:EnablesyoutolaunchtheiDRACwebinterfacefrom
DellOpenManageServerAdministratororITAssistant.

iDRACalert:Alertsyoutopotentialmanagednodeissuesthroughanemail,IPMI,alertsor
SNMPtrap.

Remotepowermanagement:Providesremotepowermanagementfunctions,suchas
shutdownandpowerup,fromamanagementconsole.

IntelligentPlatformManagementInterface(IPMI)support:ProvidessupportforIPMIfeatures
asdefinedintheIPMISpecificationversion1.5and2.0.

SecureSocketsLayer(SSL)encryption:Providessecureremotesystemmanagementthrough
theWebinterfaceorSSH.

Passwordlevelsecuritymanagement:Preventsunauthorizedaccesstoaremotesystemby
usingDRACdefinedusersandpasswordauthentication.

Rolebasedauthority:Providesassignablepermissionsfordifferentsystemsmanagementtasks.

iDRACSecurityFeatures
TheiDRACprovidesthefollowingsecurityfeatures:

UserauthenticationthroughMicrosoftActiveDirectory(optional)orhardwarestoreduserIDs
andpasswords

101|P a g e

BladeServerConfiguration&Troubleshooting

Rolebasedauthority,whichenablesanadministratortoconfigurespecificprivilegesforeach
user
UserIDandpasswordconfigurationthroughtheWebinterfaceorSMCLP
SMCLPandWebinterfaces,whichsupport128bitSSLencryptionand40bitSSLencryption(for
countrieswhere128bitisnotacceptable)
Sessiontimeoutconfiguration(inseconds)throughtheWebinterface,SMCLP,orracadm
ConfigurableIPports(whereapplicable)
SecureShell(SSH),whichusesanencryptedtransportlayerforhighersecurity

TelnetdoesnotsupportSSLencryption.

LoginfailurelimitsperIPaddress,withloginblockingfromtheIPaddresswhenthelimitis
exceeded
LimitedIPaddressrangeforclientsconnectingtotheiDRAC

iDRACSecurityFeatures
WhenopeningfirewallsforremoteaccesstoaniDRAC,youneedtomakesurethatallofthenecessary
portsareavailable.TheiDRACclientandlisteningportsare:
iDRACclientports
Port Function
25 SMTP
53 DNS
68 DHCPassignedIPaddress
69 TFTP
162 SNMPtraps
636 LDAPS
3269 LDAPS



iDRACserverlisteningports
Port Function
22 SSH
23 Telnet
80 HTTP
443 HTTPS
623 RMCP/RMCP+
3668 VirtualMediaService
3669 VirtualMediaService
3770 VirtualMediaSecureService
3771 VirtualMediaSecureService
5900 Consoleredirectionkeyboardandmouse
5901 Consoleredirectionvideo
Allportsexcept623areconfigurable.

102|P a g e

BladeServerConfiguration&Troubleshooting

ConfigurationTasks
SincetheiDRACispartofthebladessystemboard,noinstallationisrequired.Butyoustillneedtodo
severalconfigurationtaskstomakeituseful.Thistopicdiscussesthesetasksindetail.
Aftercompletingthistopic,youwillbeabletoexplainhowto:

ConfiguretheiDRAC
Configurethemanagementstation
Configurethemanagedsystems
UsetheiDRACWebinterface

ConfiguringtheManagementStation
AmanagementstationisacomputerusedtomonitorandmanagethePowerEdgeserversandother
modulesinthechassis.
Thefollowingarethesoftwareinstallationandconfigurationtasksthatsetupamanagementstationto
workwiththeiDRAC.BeforeyoubeginconfiguringtheiDRAC,followtheproceduresinthissectionto
ensurethatyouhaveinstalledandconfiguredthetoolsyouwillneed.
Step
1.Setupthemanagement
stationnetwork

2.Installandconfigurea
supportedWebbrowser.

103|P a g e

Procedure
ToaccesstheiDRAC,themanagementstationmustbeonthesame
networkastheCMCRJ45connectionportlabeledGB1.Youcan
isolatetheCMCnetworkfromthenetworkthemanagedserveris
on,sothatyourmanagementstationmayhaveLAN accesstothe
iDRACbutnottothemanagedserver.

UsingtheiDRACconsoleredirectionfeature(seeUsingGUIConsole
Redirection),youcanaccessthemanagedserversconsoleevenif
youdonothavenetworkaccesstotheserver'sports.Youcanalso
useiDRACfacilitiestoperformseveralmanagementfunctionsonthe
managedserver,suchasrebootingthecomputer.
ForalistofsupportedWebbrowsers,seeSupportedWebBrowsers.

Note:TheiDRACWebinterfaceisnotsupportedon64bitWeb
browsers.Ifyouopena64bitbrowser,accesstheConsole
Redirectionpage,andattempttoinstalltheplugin,theinstallation
procedurefails.Ifthiserrorwasnotacknowledgedandyourepeat
thisprocedure,theConsoleRedirectPageloadseventhoughthe
plugininstallationfailsduringyourfirstattempt.

ThisissueoccursbecausetheWebbrowserstorestheplugin
informationintheprofiledirectoryeventhoughtheplugin
installationprocedurefailed.Tofixthisissue,installandruna
supported32bitWebbrowserandlogintotheiDRAC.
Ifyouuseaproxyserver,makesurethebrowserbypasses
theproxyforlocaladdresses.
BladeServerConfiguration&Troubleshooting

Step

3.InstallaJavaRuntime
Environment(JRE).

4.InstallSecureShellclients,if
required.

5.InstallaTFTPserver,if
required.

6.InstallDellOpenManageIT
Assistant.

104|P a g e

Procedure
MakesuretheiDRACisintheTrustedDomainslist.
IfyouuseFirefox,disabletheWhitelistfeature.Otherwise
youmustinstallaconsoleredirectionviewerforeachiDRAC
youaccesseventhoughallviewersareidentical.
ThisstepisoptionalforstationsrunningInternetExplorer.Ifyouuse
InternetExplorer,anActiveXcontrolisprovidedfortheconsole
viewer.YoucanalsousetheJavaconsoleviewerwithInternet
ExplorerifyouinstallaJREandconfiguretheconsoleviewerin
iDRACWebinterfacebeforeyoulaunchtheviewer.SeeConfiguring
ConsoleRedirectionintheiDRACwebInterfaceformore
information.

IfyouuseFirefoxyoumustinstallanIRE(oraJavaDevelopmentKit
[JDK])tousetheconsoleredirectionfeature.Theconsoleviewerisa
Javaapplicationthatisdownloadedtothemanagementstationfrom
theiDRACwebinterfaceandthenlaunchedwithJavawebStarton
themanagementstation.Gotojava.sun.comtoinstallaJREorJDK.
Version1.6(Java6.0)orhigherisrecommended.

Note:IntheJavaConsole,settheRuntimeSettingsparametertoa
valueofxmx256M.
Bydefault,theiDRACTelnetserviceisdisabledandtheSSHserviceis
enabled.Sincetelnetisaninsecureprotocol,youshoulduseitonlyif
youcannotinstallanSSHclientoryournetworkconnectionis
otherwisesecured.

Note:TheiDRACsupportsonlyoneactiveSSHconnection.IfanSSH
sessionisalreadyactive,theiDRACdeniesallotherSSHconnection
attempts.
TrivialFileTransferProtocol(TFTP)isasimplifiedformoftheFile
TransferProtocol(FTP).ItisusedtoupdateiDRACfirmwareusing
SMCLP.CertificatescanbetransferredtotheiDRACusingracadam
orSMCLP.IfyouchoosetouseSMCLPorracadmracadmwhenyou
performthesetasks,anactiveTFTPservermustberunningonthe
computerthatcanbeaccessedbytheiDRAC.

Note:NoTFTPserverisrequiredifyouonlyusetheiDRACWeb
interfacetouploadnewiDRACfirmwareorinstallSSLcertificates.
YoursystemincludestheDellOpenManageSystemManagement
SoftwareKit.Thiskitincludes,butisnotlimitedto,thefollowing
components:

DellSystemsServiceandDiagnosticsToolsCD
DellSystemsManagementToolsandDocumentationDVD
TheOpenManageSubscriptionService

BladeServerConfiguration&Troubleshooting

Step

Procedure
UsetheDellSystemManagementConsolesCDtoinstallthe
managementconsolesoftware,includingDellOpenManageIT
Assistant,onthemanagementstation.Forinstructionsoninstalling
thissoftware,seeyourQuickInstallationGuide.

ConfiguringtheiDRAC
BeforeyoucanconfiguretheiDRAC,theblademustbeinstalledinanenclosurewithatleastone
functioningCMCandIOM.Thefollowinglistdescribestheconfigurationtasksyoumayneedtoperform.
Italsopointstoresourcesyoucanusetoaccomplishthesetasks.
Step
1.Configurethemanagement
station.
2.ConfigureiDRACnetworking.

Procedure
SetupamanagementstationbyinstallingDellOpenManage
software,abrowser,andothersoftwareutilitiesasdescribedin
ConfiguringtheManagementStation.
EnabletheiDRACnetworkandconfigureIP,netmask,gateway,and
DNsaddresses.

Note:ChangingtheiDRACnetworksettingsterminatesallcurrent
networkconnectionstotheiDRAC.
TheoptiontoconfiguretheserverusingtheLCDpanelis
availableonlyduringtheCMCinitialconfiguration.Oncethe
chassisisdeployed,theLCDpanelcannotbeusedto
reconfiguretheiDRAC.
TheLCDpanelcanbeusedtoenableDHCPtoconfigurethe
iDRACnetwork.Ifyouwanttoassignstaticaddresses,you
mustusetheiDRACConfigurationUtilityortheCMCWeb
interface.
ChassisLCDPanelseetheDellChassisManagement
ControllerUsersGuide.
DRACConfigurationUtility seeLAN.
CMCWebInterface seeConfiguringNetworkingUsingthe
CMCWebInterface.
racadmracadm seecfgLanNetworking.
SetupthelocaliDRACusersandpermissions.TheiDRACholdsa
tableofsixteenlocalusersinfirmware.Youcansetusernames,
passwords,androlesfortheseusers.
iDRACConfigurationUtility(configuresadministrativeuser
only):seeLANUserConfiguration
iDRACWebinterface:seeAddingandConfiguringiDRAC
Users
racadmracadm:seeAddinganiDRACUser
InadditiontothelocaliDRACusers,youcanuseMicrosoftActive
DirectorytoauthenticateiDRACuserlogins.

SeeUsingtheiDRACwithMicrosoftActiveDirectory.

3.ConfigureiDRACusers.

4.ConfigureActiveDirectory.

105|P a g e

BladeServerConfiguration&Troubleshooting

Step
5.ConfigureIPfilteringand
blocking.

6.ConfigurePlatformEvents.

Procedure
Inadditiontouserauthentication,youcanpreventunauthorized
accessbyrejectingconnectionattemptsfromIPaddressesoutsideof
adefinedrangeandbytemporarilyblockingconnectionsfromIP
addresseswhereauthenticationhasfailedmultipletimeswithina
configurabletimespan.
iDRACWebinterface:seeConfiguringIPFilteringandIP
Blocking
racadmracadm:seeConfiguringIPFiltering(IpRange),
ConfiguringIPBlocking
PlatformeventsoccurwhentheiDRACdetectsawarningorcritical
conditionfromoneofthemanagedserverssensors.Configure
platformEventFilters(PEFs)tochoosetheeventsyouwantto
detect,suchasrebootingthemanagedserver.

iDRACwebinterface seeConfiguringPlatformEventFilters
(PEF)
racadmracadm seeConfiguringPEF

ConfigurePlatformEventTraps(PETs)tosendalertnotificationsto
anIPaddress,suchasamanagementstationwithIPMIsoftware,or
tosendanemailtoaspecifiedemailaddress.

iDRACWebinterface seeConfiguringPlatformEventTraps
(PET)
racadmracadm ConfiguringPET
SerialOverLAN (SOL)isanIPMIfeaturethatallowsyoutoredirect
themanagedserver'sserialportI/Ooverthenetwork.SOLenables
theiDRACconsoleredirectionfeature.
iDRACWebinterface seeConfiguringSerialOverLAN
SeealsoUsingGUIConsoleRedirection.
EnableordisabletheiDRACnetworkservices,suchasTelnet,SSH,
andtheWebserverinterface,andreconfigureportsandother
serviceparameters.
iDRACWebinterface:seeConfiguringiDRACServices
racadm:seeConfiguringiDRACTelnetandSSHServices
UsingLocalracadm
ConfigureSSLfortheiDRACWebserver.
iDRACWebinterface seesecuresocketsLayer(SSL)
racadm:seecfgRacSecurity,sslcsrgen,sslcertupload,
sslcertdownload,sslcertview
Configurethevirtualmediafeaturesothatyoucaninstallthe
operatingsystemonthePowerEdgeserver.Virtualmediaallowsthe
managedservertoaccessmediadevicesonthemanagementstation
orCD/DVD,ISOimages,andfloppyimagesasiftheyweredevices
onthemanagedserver,amongotherfunctions.

7.ConfigureserialoverLAN
(SOL).

8.ConfigureiDRACservices

9.ConfigureSecureSockets
Layer(SSL).

10.Configurevirtualmediafor
loadinganoperatingsystem.

106|P a g e

BladeServerConfiguration&Troubleshooting

Step

Procedure
iDRACWebinterface seeConfiguringandUsingVirtual
Media
iDRACconfigurationutility seeVirtualMedia
Note:UsingtheiDRACconfigurationutility,youcanonlyattachor
detachvirtualmediadevices.
InstalltheMicrosoftWindowsorLinuxoperatingsystemonthe
PowerEdgeserverusingvirtualmediaandtheninstalltheDellOpen
ManagesoftwareonthemanagedPowerEdgeserverandsetupthe
lastcrashscreenfeature.
Consoleredirection seeInstallingtheSoftwareonthe
ManagedServer
iVMCLI:seeUsingtheVirtualMediaCommandLine
InterfaceUtility
SetupthemanagedserversothattheiDRACcancapturethescreen
imageafteranoperatingsystemcrashorfreeze.

SeeConfiguringtheManagedServertoCapturetheLastCrash
Screen,DisablingtheWindowsAutomaticRebootOption.
Note:YoumusthaveChassisConfigurationAdministrator
privilegetosetupiDRACnetworksettingsfromtheCMC.
ThedefaultCMCuserisrootandthedefaultpasswordis
calvin.
TheCMCIPaddresscanbefoundintheiDRACWebinterface
byclickingSystemRemoteAccessCMC.Youcanalsolaunch
theCMCWebinterfacefromthispage.
1.UseyourWebbrowsertologintotheCMCWebuserinterface
usingaWebaddressoftheformhttps://czCMCIPeddress>or
https://<CMCDNSname>.
2.EntertheCMCusernameandpasswordandclickOK.
3.Clicktheplus(+)symbolnexttoChassisintheleftcolumn,then
clickServers.
4.DeployiDRAC.
5.EnabletheLANfortheserverbycheckingthecheckboxnextto
theserverbeneaththeEnableLANheading.
6.EnableordisableIPMIoverLANbycheckingoruncheckingthe
checkboxnexttotheserverbeneaththeEnableIPMIoverLAN
heading.

Note:Bydefault,DHCPisdisabledandadefaultIPaddress,netmask
andgatewayareprovided.

7.ChecktheboxtoenableDHCPforthebladeserver.
8.IfDHCPisdisabled,enterthestaticIPaddress,netmask,and
defaultgatewayfortheserver.
9.ClickApplyatthebottomofthepage.

11.Installthemanagedserver
software.

12.Configurethemanaged
serversLastCrashScreen
feature.

13.Configurenetworkingusing
theCMCWebinterface.

107|P a g e

BladeServerConfiguration&Troubleshooting

Step
14.UpdatetheiDRACfirmware
asneeded.

Procedure
UpdatingtheiDRACfirmwareinstallsanewfirmwareimageinthe
iDRACflashmemory.Youcanupdatethefirmwareusinganyofthe
followingmethods:
SMCLPloadcommand
iDRACWebinterface
DellUpdatePackage(forLinuxorMicrosoftWindows)
DOsiDRACFirmwareupdateutility
CMCWebinterface(onlyifiDRACfirmwareiscorrupted)

ConfiguretheManagedServer
Thefollowingarethetaskstosetupthemanagedservertoenhanceyourremotemanagement
capabilities.
Task
Installthemanagement
softwareonthemanaged
server

Details
TheDellmanagementsoftwareincludesthefollowingfeatures:

LocalracadmCLI: allowsyoutoconfigureandadministerthe
iDRACfromthemanagedsystem.Itisapowerfultoolfor
scriptingconfigurationandmanagementtasks.
ServerAdministrator:aWebinterfacethatallowsyouto
administertheremotesystemfromaremotehostonthe
network.
ServerAdministratorInstrumentationService
o Providesaccesstodetailedfaultandperformance
informationgatheredbyindustrystandardsystems
managementagentsandallowsremote
administrationofmonitoredsystems.
o Displayslogsofcommandsissuedtoorbythe
system,monitoredhardwareevents,POSTevents,
andsystemalerts.Youcanviewlogsonthehome
page,printorsavethemasreports,andsendthem
byemailtoadesignatedservicecontact.
ServerAdministrationStorageManagementService provides
storagemanagementinformationinanintegratedgraphical
view.
UsetheDellSystemsManagementToolsandDocumentationDVDto
installServerAdministrator.Forinstructionsoninstallingthis
software,seeyourQuickInstallationGuide.

108|P a g e

BladeServerConfiguration&Troubleshooting

Task
Details
Configurethemanagedserver
TheiDRACcancapturethelastcrashscreensothatyoucanviewitin
tocapturethelastcrashscreen. theWebinterfacetohelptroubleshootthecauseofthemanaged
systemcrash.Followthesestepstoenablethelastcrashscreen
feature.
1.Installthemanagedserversoftware.Formoreinformationabout
installingthemanagedserversoftware,seetheServer

AdministratorUser'sGuide.
2.IfyouarerunningaMicrosoftwindowsoperatingsystem,ensure
thattheAutomaticallyRebootfeatureisdeselectedintheWindows
StartupandRecoverySettings.SeeDisablingthewindows
AutomaticRebootOption.
3.EnabletheLastCrashscreen(disabledbydefault)intheiDRAC
webinterface.

ToenabletheLastCrashScreenintheiDRACwebinterface,click
SystemRemoteAccess iDRACNetwork/SecurityServices,then
checktheEnablecheckboxundertheAutomaticSystemRecovery
AgentSettingsheading.

ToenabletheLastCrashScreenusinglocalracadm,opena
commandpromptonthemanagedsystemandtypethefollowing
command:
racadmconfiggclgRacTuningocfgRacTuneAsrEnable1

4.IntheServerAdministratorWebbasedinterface,enabletheAuto
RecoverytimerandsettheAutoRecoveryactiontoReset,Power
Off,orPowerCycle.

ForinformationabouthowtoconfiguretheAutoRecoverytimer,
seetheServerAdministratorUsersGuide.Toensurethatthelast
crashscreencanbecaptured,theAutoRecoverytimershouldbeset
to60seconds.Thedefaultsettingis480seconds.

TheLastCrashscreenisnotavailablewhentheAutoRecovery
actionissettoShutdownorPowerCycleifthemanagedserveris
poweredoff.
DisabletheWindowsAutomatic ToensurethattheiDRACcancapturetheLastCrashScreen,disable
Rebootoption.
theAutomaticRebootoptiononmanagedserversrunning
MicrosoftWindowsServer.

1.OpentheWindowsControlPanelanddoubleclicktheSystem
icon.
2.ClicktheAdvancedtab.
3.UnderStartupandRecovery,clickSettings.
4.DeselecttheAutomaticallyRebootcheckbox.
5.ClickOKtwice.

109|P a g e

BladeServerConfiguration&Troubleshooting


ConfiguringtheiDRACUsingtheWebInterface
UsetheiDRACWebinterfaceforeverydaysystemsmanagementtasks.Youcan:

ConfiguretheiDRACpropertiesandusers
Performremotemanagementtasks
Troubleshootaremote(managed)systemforproblems

MostWebinterfaceconfigurationtaskscanalsobeperformedwithlocalracadmcommandsorwithSM
CLPcommands.

Localracadmcommandsareexecutedfromthemanagedserver.Formoreinformationabout
localracadm,seeUsingtheLocalracadmCommandLineInterface.
SMCLPcommandsareexecutedinashellthatcanbeaccessedremotelywithaTelnetorSSH
connection.FormoreinformationaboutSMCLP,seeUsingtheiDRACSMCLPCommandLine
Interface.

ConfiguringtheiDRACUsingtheWebInterface
ThefollowinglistexplainsthetasksyoucanaccomplishwiththeiDRACWebinterface.
Task
Details
AccesstheiDRACWebinterface ToaccesstheDRACWebinterface:

1.Openasupportedwebbrowserwindow.
2.TypethefollowingtextstringintheAddressfield:
Ifthedefaultporthasnotbeenchanged,Type:
https://cziDsQ4cIPaddress>andpress<Enter>.
TheiDRACLoginwindowappears.YoumustlogintotheDRACwith
validcredentials.
ConfiguretheiDRACNIC.
AssumingthattheiDRAChasalreadybeenconfiguredandis
accessibleonthenetwork,youcan:
ConfigurethenetworkandIPMILANsettings
ConfigureIPFilteringandIPBlocking
Configureplatformevents.
Platformeventconfigurationprovidesamechanismforconfiguring
theiDRACtoperformselectedactionsoncertaineventmessages.
Theactionsinclude:
Noaction
Rebootsystem
Powercyclesystem
Poweroffsystem
Generateanalert

110|P a g e

BladeServerConfiguration&Troubleshooting

Task
Configureplatformeventfilters
(PEFs).

Configureplatformeventtraps
(PETs).

111|P a g e

Details
Whenaplatformeventoccurs(forexample,abatterywarning
assert),asystemeventisgeneratedandrecordedintheSystem
EventLog(SEL).

Ifthiseventmatchesaplatformeventfilter(PEF)thatisenabledand
youhaveconfiguredthefiltertogenerateanalert,aPEToremail
alertissenttouptofourconfigureddestinations.

Ifthesameplatformeventfilterisalsoconfiguredtoperforman
action(suchasrebootingthesystem),theactionisperformed.

Note:Configureplatformeventfiltersbeforeyouconfigurethe
platformeventtrapsoremailalertsettings.
ToconfigurePEFs:

1.LogintotheiDRACWebinterface.
2.ClickSystem.
3.ClicktheAlertManagementtab.
4.OnthePlatformEventspage,doeitherofthefollowing:
PlaceachecknexttotheGenerateAlertheadingtogroup
selectalleventsortocheckeachindividualevent
5.Clicktheradiobuttonbelowtheactionyouwouldliketoenable
foreachevent.Onlyoneactioncanbesetforeachevent.
6.ClickApply.
Note:GenerateAlertmustbeenabledforanalerttobesenttoany
valid,configureddestination(PEToremail).
1.Logintotheremotesystemusingasupportedwebbrowser.
2.EnsurethatyoufollowedtheproceduresinConfiguringPlatform
EventFilters(PEFs).
3.ConfigureyourPETdestinationIPaddress:
a.ClicktheEnablecheckboxnexttotheDestinationNumber
youwouldliketoactivate.
b.EnteranIPaddressintheDestinationIPAddressbox.
Note:Thedestinationcommunitystringmustbethesame
astheiDR4Ccommunitystring.
c.ClickApply.
Note: Tosuccessfullysendatrap,configuretheCommunityString
valueontheNetworkConfigurationpage.TheCommunityString
valueindicatesthecommunitystringtouseinaSimpleNetwork
ManagementProtocol(SNMP)alerttrapsentfromtheiDRAC.SNMP
alerttrapsaretransmittedbytheiDRACwhenaplatformevent
occurs.ThedefaultsettingfortheCommunityStringisPublic.
d.ClickSendtotesttheconfiguredalert(ifdesired).
e.Repeatforanyremainingdestinationnumbers.

BladeServerConfiguration&Troubleshooting

Task
Configureemailalerts

Details
1.LogintotheremotesystemusingasupportedWebbrowser.
2.EnsurethatyoufollowedtheproceduresinConfiguringPlatform
EventFilters(PEF).
3.Configureyouremailalertsettings.
a.OntheAlertManagementtab,clickEmailAlertSettings.
4.Configureyouremailalertdestination.
a.IntheEmailAlertNumbercolumn,clickadestination
number.Therearefourpossibledestinationstoreceive
alerts.
b.EnsurethattheEnabledcheckboxisselected.
c.IntheDestinationEmailAddressfield,typeavalidemail
address.
d.ClickApply.
Note:Tosuccessfullysendatestemail,theSMTPServerAddress
mustbeconfiguredontheNetworkConfigurationpage.TheIP
addressoftheSMTPServercommunicateswiththeDRACtosende
mailalertswhenaplatformeventoccurs.
e.ClickSendtotesttheconfiguredemailalert(ifdesired).
f.Repeatstepathroughstepeforanyremainingemailalert
settings.

UsingConsoleRedirection
TheiDRACconsoleredirectionfeatureenablesyoutoaccessthelocalconsoleremotelyineithergraphic
ortextmode.Youdonothavetositinfrontofeachservertoperformalltheroutinemaintenance.You
canmanageserversfromyourdesktoporlaptopcomputerwhereveryouare.
ConsoleRedirectionConsiderations
TheConsoleRedirectionpageenablesyoutomanagetheremotesystembyusingthekeyboard,video,
andmouseonyourlocalmanagementstationtocontrolthecorrespondingdevicesonaremote
managedserver.ThisfeaturecanbeusedinconjunctionwiththeVirtualMediafeaturetoperform
remotesoftwareinstallations.
Thefollowingconditionsapply:

Amaximumoftwosimultaneousconsoleredirectionsessionsaresupported.Bothsessionsview
thesamemanagedserverconsolesimultaneously.

Thefirstloggedinuserwillbenotifiedifaseconduserconnects.

112|P a g e

BladeServerConfiguration&Troubleshooting

AconsoleredirectionsessionshouldnotbelaunchedfromaWebbrowseronthemanaged
systemforitsownconsoleredirection.
Themanagedsystemshouldbesettoascreenresolutionandrefreshratelistedinthetable.
Thescreenresolutiononthemanagementstationshouldbesethigherthantheresolutionof
themanagedsystem.
AsupportedWebbrowsermustbeinstalledonthemanagementstation.
IfyouareusingFirefoxorwanttousetheJavaViewerwhenusingInternetExplorer,youmust
installaJavaRuntimeEnvironment(JRE)onthemanagementstation.

ScreenResolution
640x480
720x400
800x600
1024x768
1280x1024

RefreshRate(Hz)
60,72,75,85
70
60,70,72,75,85
60,70,72,75,85
60

ConfiguringConsoleRedirectionintheiDRACWebInterface
ToconfigureconsoleredirectionintheiDRACWebinterface,performthefollowingsteps:

AccesstheiDRACsWebinterfacefromthemanagementstation.
ClickSystemandthenclicktheConsoletab.
ClickConfigurationtoopentheConsoleRedirectionConfigurationpage.

UsingtheVideoViewer
TheVideoViewerprovidesauserinterfacebetweenthemanagementstationandthemanagedsystem.
Itenablesyoutoseethemanagedserver'sdesktopandcontrolitsmouseandkeyboardfunctionsfrom
yourmanagementstation.TheVideoViewerprovidesvariouscontroladjustments.TheHelpmenu
providesformoreinformationonthesefunctions.
Whenyouconnecttotheremotesystem,theVideoViewerstartsinaseparatewindow.Youmayneed
toadjustthecolormodeandsynchronizethemousepointers.Thebaracrossthetopoftheviewer
containsamenuwhereyoucanadjusttheviewersettings.
UsingtheVideoViewer
Thefollowingtabledescribesthemenuoptionstoadjusttheviewersettings.
View
Pause
Resume
Refresh
CaptureCurrentScreen

FullScreen

113|P a g e

Temporarilypausesconsoleredirection.
Resumesconsoleredirection.
Redrawstheviewerscreenimage.
Capturesthecurrentremotesystemscreentoa.bmpfileonWindowsora
.pngfileonLinux.Adialogboxisdisplayedthatallowsyoutosavethefileto
aspecifiedlocation.
TomaketheVideoViewerexpandintofullscreenmode,selectFullScreen
fromtheVideomenu.
BladeServerConfiguration&Troubleshooting

Exit

WhenyouhavefinishedusingtheConsoleandhaveloggedout(usingthe
remotesystemslogoutprocedure),selectExitfromtheVideomenuto
closetheVideoViewerwindow.

Mouse
SynchronizeCursor
Options
ColorMode

Media
VirtualMediaWizard

Help

TheMousemenuenablesyoutosynchronizethecursorsothatthemouse
ontheclientisredirectedtothemouseontheserver.
Allowsyoutoselectacolordepthtoimproveperformanceoverthe
network.

Forexample,ifyouareinstallingsoftwarefromvirtualmedia,youcan
choosethelowestcolordepth(3bitgray),sothatlessnetworkbandwidth
isusedbytheconsoleviewer,leavingmorebandwidthfortransferringdata
fromthemedia.

Thecolormodecanbesetto:
15bitcolor
7bitcolor
4bitcolor
4bitgray
3bitgray
TheMediamenuprovidesaccesstotheVirtualMediaWizard,whichallows
youtoredirecttoadeviceorimagesuchasa:
Floppydrive
Floppyimage
CD
DVD
ImageinISO format
USBflashdrive

ForinformationabouttheVirtualMediafeature,seeConfiguringandUsing
VirtualMedia.

YoumustkeeptheConsoleViewerwindowactivewhenusingVirtual
Media.
ActivatestheHelpmenu.

Summary
Reviewthesummaryofthismodulebeforetakingtheselfchecktoseehowwellyoumasteredthe
topic.

TheiDRACimplementsseveralmanagementandsecurityfeatures.

114|P a g e

BladeServerConfiguration&Troubleshooting

TheiDRACprovidesbothinbandandoutofbandmanagementsolutionssuchasblade
powermanagement,loggingcriticalevents,andremotemanagement.
o MseriesiDRACsprovidesecurityfeaturessuchas:
ActiveDirectoryauthenticationandauthenticationusinglocalusers
40bitand128bitencryption
IPaddressrestrictionwhenloginfails
SincetheiDRACispartofthebladessystemboard,noinstallationisrequired,butsome
configurationis.
o Tosetupthemanagementstation,theadministratorshouldsetupWebbrowsers,
TFTP,andSSHclients.
o WhenconfiguringtheiDRAC,configure:
Networksettings
Users
ActiveDirectory
Eventfilters
ThevKVM
vMedia
o YoucanalsoenableservicessuchasWeb,SSH,andTelnet.
o TheiDRACnetworkcanalsobeconfiguredusingtheCMCWebinterface.
o Whenconfiguringthemanagedserver,theSystemadministratormust:
InstallOpenManageSystemsoftware
ConfiguretheAutoRecoveryfeature
EnabletheLastCrashScreencapture
o SystemadministratorscanconfigurethesettingsofiDRACNIC,PlatformEventTrap,E
mailalerts,andPlatformEventFiltersthroughiDRACWebinterface.MostWeb
interfaceconfigurationtaskscanalsobeperformedwithlocalracadmcommandsor
withSMCLPcommands.
o

115|P a g e

BladeServerConfiguration&Troubleshooting

Module7:NetworkFabricsandIOMs
ThismoduleprovidesinformationonNetwork
FabricsA,BandC.
Aftercompletingthismodule,youwillbeableto:

DescribethefeaturesofFabricAandtheTOE
key
Describethefeaturesanduserestrictionsof
mezzanineFabricsBandC
Comparethefeaturesofthethreetypesof
mezzaninecardscompatiblewithDell
M1000ebladeservers
ExplainhowInfiniBandtechnologyfunctionsandlistitsadvantages
DescribethefeaturesandfunctionalityofInfiniBandmezzaninecardsandIOM

ChassisOverview
TheM1000echassiscanhouse6xI/O
modules.Thisallowsforagreaterdiversity
ofrolesforalloftheenclosedbladeservers.
The6xI/Oslotsareclassifiedasthree
separatefabrics.Eachfabriccontainstwo
slots,1and2,whichrelatetothetwoports
foundontheserversideI/Ocards.The
fabricscanbeusedindependentlyofeach
other,thougheachfabricmustcontainthe
sametechnology.Forexample,FabricAis
hardwiredtothetwoLANsOnMotherboard
(LOMs)onthebladeservermainboards,so
theI/OmodulesinFabricAmustbe
Ethernet.FabricsBandCcanbeusedfor
Ethernet,FibreChannelandInfiniBand.

116|P a g e

BladeServerConfiguration&Troubleshooting

FabricABroadcomEthernetControllers
FabricAconsistsoftwoembeddedBroadcomEthernetcontrollers.Thecontrollerwasdesignedasa
ConvergenceNetworkInterfaceCard(CNIC)thatcombinesnetworkfunctionalityintoasinglechip.
FabricAincludes:

LANOnMotherboard(LOM)

FabricAFeatures
ThebidirectionaldataforasingleBroadcomGbLOMiscarriedoveraPCIExpressx4bus.

SerDes(serialtoparallelandparalleltoserial)transceiverstransmitdataacrossthemidplaneon
asinglelanetotheI/OmodulebaysA1andA2
Supportfor10/100/1000GbEthernetspeeds
iSCSIboot
WakeOnLAN(WOL)
PXE
TOE(TCPOffloadEngine)
o RequiresoptionalTOElicensekey(allsystemsshipwiththeTOEkeypopulated)

DellLOMTOELicenseKey
OptionalLOMfeaturesareactivatedwith
thepopulationoftheTOELicenseKey.

TOELicenseKey

117|P a g e

BladeServerConfiguration&Troubleshooting

ExpansionCardsOverview
TheMseriesbladeserverssupportoptionalFabricexpansionMezzaninecardsdesignedtosupporthigh
speednetworkcommunicationtechnologiesincluding:

InfiniBand
10GbEthernet
FibreChannel

MezzanineCards
MezzaninecardssupportedonthePowerEdge1000e
BladeServersinclude:

EmulexLPe1105M4FCHBA
QLogicQME2472FCHBA
DellBCM57086SEthernetMezzanine

MezzaninecardsinstalledintheFabricBorFabricCcardslotrequireamatchingI/Omoduleinstalledin
thecorrespondingI/Obaytosupportdataflowforthatfabric.
TheAMDChipsetconnectiontoMezzaninecardslotCsupportsasinglePCIex8link,whichcannotbe
splitintotwoPCIex4links.
DellBCM57086S
TheDellBCM57086SEthernetMezzaninecardrequirestwoPCIex4links.
Asaresult,theM605,M805,andM905bladeserverscannotsupporttheDellBCM57086SEthernet
MezzaninecardinMezzaninecardslotC(slotrestriction).

TheDellBCM57086SEthernetMezzaninecardistheonlyMezzaninecardrestrictedfromslotC;
allotherexistingMezzaninecardsaresupported.
TheDellBCM57086SEthernetMezzaninecardissupportedinMezzaninecardslotB.
ThislimitationonlyappliestotheM605;thePowerEdgeM600isnotaffected.

EmulexLPe1105M4FCHBA
ThefollowinglistdetailsthefunctionalityoftheEmulexLPe1105M4FCHBA.

Provides4,2,or1GbFCconnectivityamongupto16DellbladeserversandaSANfabric
ComprehensivevirtualizationcapabilitieswithsupportforN_PortIDVirtualization(NPIV)and
VirtualFabric
AdministrationviaHBAnywareforEmulexHBAsanywhereintheSAN
CommondriverwithallgenerationsandtypesofEmulexHBAs
Firmwarereleasesdeployedindependentlyfromthedriver

118|P a g e

BladeServerConfiguration&Troubleshooting

SupportforIndustryStandardFibreChannelSecurityProtocol(FCSP)

QLogicQME2472FCHBA
TheQME2472HBAsisan"EnterpriseClassHBA"specificallydesignedfortheDell4GbpsM1000eBlade
ServerChassisbladesMseriesblades.

Provides4,2,or1GbFCconnectivityamongupto16DellbladeserversandaSANfabric
RobustinteroperabilitysimplifiesdeploymentandupgradesofSANhardwareandsoftware
AdministrationviaSANSurfer
CapabilityofbootingfromSAN

119|P a g e

BladeServerConfiguration&Troubleshooting

QlogicSANBladeQNE247MFibreChannelHBATechnicalSpecifications
HostBusSpecifications
Feature
BusInterface
Memory

Description
X4PCIExpress
1MBSRAMperport
1MBFLASH(SPI)
2KBNVRAM(SPI)
ConformstoPCIExpressBaseSpecificationrev.1.0a
PCIExpressCardElectromechanicalSpecificationrev.1.0a
PCIBusPowerManagementInterfaceSpecificationrevision1.1

Compliance

FibreChannelSpecifications
Feature
Description
DataRate
4/2/1Gbpsautonegotiation(4.25/2.125/1.0625Gbps)
Performance
150,000IOPSperport
Throughput
800Mbpsperport
Topology
PointtoPoint(N_Port)
ArbitratedLoop(NL_Port)
SwitchedFabric(N_Port)
Logins
SupportforF_PortandFL_Portlogin.2,048concurrentloginsand2,048active
exchanges
ClassofService Class2and3
Protocols
FCP(SCSIFCP)
IP(FCIP)
FICON(FCSB2)
FCTAPE(FCP2)
BufferCredits
3creditsavailable(2,112Byteframepayload)
Compliance
SCSI3FibreChannelProtocol(SCSIFCP)
FibreChannelPhysicalandSignalingInterface(FCPH)
FibreChannel2ndGeneration(FCPH2)
ThirdGenerationFibreChannelPhysicalandSignalingInterface(FCPH3)
FibreChannelArbitratedLoop(FCAL2)
FibreChannelFabricLoopAttachmentTechnicalReport(FCFLA)
FibreChannelPrivateLoopDirectAttachTechnicalReport(FCPLDA)
FibreChannelTape(PFTAPE)profile
SCSIFibreChannelProtocol2(FCP2)
SecondGenerationFCGenericServices(FCGS3)
ThirdGenerationFCGenericServices(FCGS3)
FibreChannelFramingandSignaling(FCFS)

120|P a g e

BladeServerConfiguration&Troubleshooting

IOMs
InformationontheIOMsavailableforDell
PowerEdgeM1000ebladeserversfollows.
Dell/EmulexPT1016FibreChannelPassThrough
TheDellPT1016byEmulexisdesignedforusein
situationswhereadirectconnectionbetweeneach
bladeserverandexternalSANswitchesisrequired.
Dell/EmulexPT1016
TheDellPT1016byEmulexprovidesinvestment
protectionfordatacenterswithpreexistingnetworkinginfrastructuresbyprovidingadirectconnection
betweentheserversandtheSAN.ThePTMprovidesupto164GbFibreChannelconnectionswith1:1,
nonswitchedconnectionsbetweentheserverbladeandtheSAN.ThePTMdesignishotpluggableand
providesforhighavailabilityfailoverwhendeployedinpairswithintheserverchassis.

121|P a g e

BladeServerConfiguration&Troubleshooting

Dell/EmulexPT1016FibreChannelPassThrough
Capabilities
Provides4,2,or1Gbconnectivitybetweenupto16DellbladeserversandaSANfabric
Eachportprovidesadedicatedconnectionforguaranteedbandwidthtoeachserverport
With16externalportsinasingleblade,thePTMprovidesthehighestportdensityavailablein
Dellsconnectivityoptions
ProvidesuncompromisingscalabilitytothefullestcapabilityoftheDellchassis
Hotplugcapabilityandredundancyprovidetruehighavailabilityfor24x7datacenteroperation
Investmentprotection:usesexistingSANinfrastructurewithnointeroperabilityissues
KeyFeatures
16portsof4GbFibreChannelconnectivitybetweenserversandSAN
TransparentconnectionbetweenSANandServer
Simplicityofunmanageddesignspeedsdeployment
Simplicityofunmanageddesigneliminatesconfigurationhassles
Eliminatesinteroperabilityissueswithtransparent,passthroughprotocol
Compact,ultradenseformfactor
4,2or1GbFibreChannelconnectivityoneachporttomatchexistingserverandSAN
installation
Hotpluggabledesigneliminatesdowntimeduringserviceactions
Intendedtobeinstalledinpairsforredundancyandhighavailability
Firmwarecanbeupgradedinthefield,eliminatingdowntime(future,notininitialrelease)
Completelydedicatedconnectionforeachserverblade
Upto256Gbfullduplexbandwidth
TechnicalSpecifications
TheDellPT1016byEmulexPassThroughModulehasthefollowingtechnicalspecifications:
ProductSpecificationSummary
Item
Description
Physicalinterface
HotpluggableindustrystandardcopperandopticalSFPtransceivers
atallports
Bladetype
FibreChannelPassThrough
Performance
4GbFibreChannelperport,nonblocking
PortConfiguration
Sixteen4GbFibreChannelinternalconnectionstoServerBlades

Sixteen4GbFibreChannelexternaluplinks
PortType
4GbFibreChannelopticalSFPs,shortwave

122|P a g e

BladeServerConfiguration&Troubleshooting

Brocade4424FibreChannelSwitch
SwitchOverview
TheBrocade4424isa24portFibreChannelmodulewith8externalportsand16internalports
designedforinstallationinabladeserverchassis.Theproductshipswith12portsenabledandis
upgradableto24portsthroughPortsonDemand(POD)licensing.Theseportssupportlinkspeedsupto
4Gbit/sec.

TechnicalSpecifications
ThefollowingtabledescribesthetechnicalspecificationsfortheBrocade4424FibreChannelI/O
modularswitch.

Item
Scalability
Certifiedmaximum
Performance
Fabriclatency
(FabricSwitchModeonly)
Maximumframesize
ClassofService

PortTypes

Fabricservices
(FabricSwitchModeonly)
PowerPC440GPprocessor
SDRAMmemory
123|P a g e

Brocade4424Features
Description
Fullfabricarchitecturewithmaximumof239switches
32switches,7hops
1.063Gbit/seclinespeed,fullduplex
2.125Gbit/seclinespeed,fullduplex
<2.1secwithnocontention
2112bytepayload
Class2
Class3
ClassF(interswitchframes)
FL_port
F_port
E_port
N_port
SimpleNameServer,RegisteredStateChangeNotification
333MHz
256MBforsystemmemoryat64bitswidewith8ECCbits
BladeServerConfiguration&Troubleshooting

Item
Bootflashmemory
Compactflashmemory

Brocade4424Features
Description
4MB
256MB

InfiniBandTechnology
TheInfiniBandstandardisakeyelementinthespeedandefficiencyofDell
bladeserverI/Osystems.
AboutInfiniBand
In1999,twocompetinginput/output(I/O)standardscalledFutureI/O
(developedbyCompaq,IBMandHewlettPackard)andNextGenerationI/O(developedbyIntel,
MicrosoftandSun)mergedintoaunifiedI/OstandardcalledInfiniBand.InfiniBandisanindustry
standardspecificationthatdefinesaninput/outputarchitectureusedtointerconnectservers,
communicationsinfrastructureequipment,storageandembeddedsystems.InfiniBandisatruefabric
architecturethatleveragesswitched,pointtopointchannelswithdatatransfersupto120gigabitsper
second,bothinchassisbackplaneapplicationsaswellasthroughexternalcopperandopticalfibre
connections.
HowInfiniBandWorks
InfiniBandisapervasive,lowlatency,highbandwidthinterconnectthatrequireslowprocessing
overheadandisidealtocarrymultipletraffictypes(clustering,communications,storage,management)
overasingleconnection.Asamatureandfieldproventechnology,InfiniBandisusedinthousandsof
datacenters,highperformancecomputeclustersandembeddedapplicationsthatscalefromtwonodes
uptoasingleclusterthatinterconnectthousandsofnodes.

124|P a g e

BladeServerConfiguration&Troubleshooting


AdvantagesofInfiniBand
PowerfulmulticoreCPUs,servervirtualization,bladearchitectures,increasingstoragedemands,and
theneedtoreducepowerusageareallplacingincreaseddemandsonI/Osolutions.InfiniBandcan
addressthesedemandswhiledeliveringlowlatencyinahighbandwidthinterconnect.Advantagesof
InfiniBandinclude:

SuperiorperformanceInfiniBandistheonlyshippingsolutionthatsupports20Gbhost
connectivityand60Gbswitchtoswitchlinks.InfiniBandalsoprovidesworldclassapplication
latencywithmeasureddelaysof1sendtoend.
HighefficiencyInfiniBandoffloadsalldatatransportmechanismstotheadaptercardsto
relieveCPUssotheycanbefullyusedoncustomerworkloads.Inaddition,InfiniBanduses
RemoteDirectMemoryAccess(RDMA)tooptimizedatatransferandreduceinternalmemory
usagetofurtherenableCPUstofocusonapplicationprocessing.
FabricconsolidationandlowenergyusageWithlessthan5Wper20GbportInfiniBandcan
consolidatenetworking,clustering,andstoragedataoverasinglefabric,significantlylowering
theoverallpower,realestate,andmanagementoverheadrequiredforserversandstorage.
ReliableandstableconnectionsByenablingfullyredundantI/Ofabrics,withautomaticpath
failoverandlinklayermultipathingabilitiestomeetthehighestlevelsofavailability,InfiniBand
isperfectlysuitedtomeetthemissioncriticalneedsoftodaysenterprisedatacenter.

125|P a g e

BladeServerConfiguration&Troubleshooting

DataintegrityMissioncriticalapplicationsrequirethatdatahasnotbeencorruptedasit
travelsacrossthefabric.InfiniBandenablesthehighestlevelsofdataintegritybyperforming
cyclicredundancychecks(CRCs)ateachfabrichopandendtoendacrossthefabrictoensure
thedataiscorrectlytransferred.

TheseadvantagescombinetomakeInfiniBandtheindustrysmostpowerfulinterconnect.
Cabling
InfiniBandusesabase2.5GbpsclockratethatnecessitatesthatInfiniBandportsandcablesarethicker
andbulkierthanUnshieldedTwistedPairorUTP,commonlyfoundinEthernetapplications.Generally,
thefasteralinkisclocked,themorecrosstalkisgenerated.Thisisbecauseoftherapidriseandfallof
thecarriervoltageasbitsaretransmitted.Additionally,becauseofthesignalattenuation,thereceiver
mustbeabletointerpretthesignalcorrectlywhenitisreceived.AthigherInfiniBandspeeds,theeffects
ofcrosstalkanddataskewlimitcablelengthsandconsequentlylimitthenumberofnodesthatcanbe
supportedwithinacluster.Seethetablebelow.
ForInfiniBand4Xinstallations,althoughthecoppercablinglengthsarerelativelylimitedwhen
comparedtoGigabitEthernet,verylargeclusterscanbeconstructedbycarefullyplanningcablingruns
andequipmentplacement.Asanexample,theworldslargestInfiniBandbasedHPCcluster,builtusing
CiscoSFS7000SeriesInfiniBandServerSwitches,consistsofmorethan4,500dualprocessorservers.By
contrast,usingcoppercablingonly,thelargestInfiniBand12XHPCclusterthatcanbebuiltis256nodes
usingstandardsidebysiderackconfiguration.ByarrangingtheracksinaUshape,itispossibleto
builda512nodecluster,althoughthiscomplicatesthecablingandmaintenanceaspectsofthecluster.
LinkSpeeds
ThesevariablesdetermineatwhatspeedtheInfiniBandfabricwillrunanditscapablebandwidth:

LinkWidth:Thenumberofbondedchannels(send/receivepairs)usedforanInfiniBandlink.
DefinedIBlinkwidthsare1X,4Xand12x.
SignalingRate:Thesimplex,raw,bitrateofanInfiniBandchannel.DefinedIBsignalingratesare:
o SDR(SingleDataRate2.5Gbps/channel)
o DDR(DoubleDataRate5Gbps/channel)
o QDR(QuadDataRate10Gbps/channel)
LinkSpeed:ThetotalsimplexcapacityofanInfiniBandlinkcalculatedasthesignalingrateof
eachchannelmultipliedbythenumberofchannelspresentinthelink.Forexample,a4XDDRIB
Linkhasasimplexlinkcapacityof5.0GbpsXfourchannels=20Gbpsineachdirection.

QDRsignalingratestandardhasbeendefined,butsiliconthatisabletodriveasignalatthisrateisnot
yetinproductionbyanyvendor.

126|P a g e

BladeServerConfiguration&Troubleshooting


DefinedIBLinkSpeeds
SDR
1XSDR
4XSDR
12XSDR
DDR
1XDDR
4XDDR
12XDDR
QDR
1XQDR
4XQDR
12XQDR

2.5Gbps
10Gbps
30Gbps
5Gbps
20Gbps
60Gbps
10Gbps
40Gbps
120Gbps

InfiniBandPhysicalLayerCharacteristics
TheInfiniBandphysicallayerspecificationsupportsthreelengthwidths,designated1X,4X,and12X,
overbothcopperandfiberopticmedia,asstatedearlierinthismodule.Thebasedatarate,1Xsingle
datarate(SDR),isclockedat2.5Gbpsandistransmittedovertwopairsofwirestransmitand
receiveandyieldsaneffectivedatarateof2Gbpsfullduplex(2Gbpstransmit,2Gbpsreceive).The25
percentdifferencebetweendatarateandclockrateisdueto8B/10Blineencodingthatdictatesthatfor
every8bitsofdatatransmitted,anadditional2bitsoftransmissionoverheadisincurred.
InfiniBand4Xand12XSDRinterfacesusethesamebaseclockratethatusesmultiplepairs,referredto
aslanes,toincreasebandwidthcapacity.Therefore,anInfiniBand4Xinterfacerealizesadatarateof8
Gbpsusing8pairsofwiresanda12Xinterfacerealizes24Gbpsusing24pairs.Althoughitispossiblefor
InfiniBandswitchestoforwardframesbetweenanInfiniBand1Xand4X,or4Xand12Xlink,thereisa
performancepenaltybecauseofthewaythatInfiniBandtransmitspackets.
InfiniBandswitchesusecutthroughswitchingtoprovidelowlatencytransportbetweendevices,which
doesnotrequiretheentirepackettobebufferedbeforebeingforwarded.ThisenablesInfiniBandto
providelowlatencynetworktransmission,butatthesametimerequiresthattheingressandegresslink
speedsarethesameandthatthepacketisreceivedandtransmittedinthesameformat.
However,whendifferentlinkspeedsaremixedinInfiniBandnetworks,InfiniBandtransmitsportionsof
thepacketinparallel.Therefore,foranInfiniBand1Xlink,apacketistransmittedserially;foran
InfiniBand4Xlink,thepacketisdividedintofourchunksandtransmittedinparallelacrossthefour2.5
Gbpslanes.Fora12Xlink,apacketisdividedinto12chunksandtransmittedinparallelacrosstwelve
2.5Gbpslanes.
ParallelTransmissioninInfiniBandLinks
Becausethepacketistransmittedinparallel,ifapacketisswitchedbetweena12Xanda4Xinterface,a
storeandforwardpenaltyisincurredtochangetransmissionformats.Tomaintaintheperformanceof
127|P a g e

BladeServerConfiguration&Troubleshooting

cutthroughswitching,mostInfiniBandfabricsconsistofasinglelinkspeed.Forexample,alllinksare4X
SDRwithmultipathforwardingusedtoprovideadditionalbandwidthwithintheInfiniBandnetwork.
However,ifanapplicationcantoleratethesmallstoreandforwarddelay,mixing12Xand4Xinthe
sameInfiniBandnetworkmaybeanacceptablesolution.

MellanoxConnectXIBMDIInfiniBandHostChannel
Adapter(HCA)MezzanineCard
TheMellanoxConnectXIBMDIInfiniBandHostChannel
Adapter(HCA)mezzaninecardfortheDellPowerEdgeM
seriesbladeserversdeliverslowlatencyandhighbandwidth
forperformancedrivenserverandstorageclustering
applicationsinEnterpriseDataCenterandHighPerformance
Computingenvironments.Clustereddatabases,parallelized
applicationsandtransactionalservicesapplicationswill
achievesignificantperformanceimprovementsresultingin
reducedcompletiontimeandlowercostperoperation.
ConnectXIBadapterssimplifynetworkdeploymentby
consolidatingclustering,communications,storage,and
managementI/Oandbyprovidingenhancedperformancein
virtualizedserverenvironments.

128|P a g e

BladeServerConfiguration&Troubleshooting

Benefits
ThesearethemainbenefitsoftheMellanoxConnectXDualPortInfiniBandMezzanineCard:

Worldclassclusterperformance
Highperformancenetworkingandstorageaccess
Guaranteedbandwidthandlowlatencyservices
Reliabletransport
Endtoendstorageintegrity
I/Oconsolidation
Virtualizationacceleration
Scalestotensofthousandsofnodes

KeyFeatures
KeyfeaturesoftheMellanoxConnectXDualPortInfiniBandMezzanineCardinclude:

1.2usMPIpinglatency
20GbInfiniBandports
CPUoffloadoftransportoperations
EndtoendQoSandcongestioncontrol
HardwarebasedI/Ovirtualization
TCP/UDP/IPstatelessoffload

AdditionalFeatures
AdditionalfeaturesoftheMellanoxConnectXDualPortInfiniBandMezzanineCardinclude:
WorldClassPerformanceandScalability:Clusteredapplicationsrunningonmultisocketserversusing
multicoreprocessorsbenefitfromthereliabletransportconnectionsandadvancedmulticastsupport
offeredbyConnectXlB.EndtoendQualityofService(QoS)enablespartitioningandguaranteedservice
levelswhilehardwarebasedcongestioncontrolpreventshotspotsfromdegradingtheeffective
throughput.ConnectXlBiscapableofscalingtotensofthousandsofserverandstoragenodes.
HardwareOffloadArchitecture:Clusteredandclient/serverapplicationsachievemaximum
performanceoverConnectXIB,becauseCPUcyclesareavailabletofocusoncriticalapplication
processinginsteadofnetworkingfunctions.Networkprotocolprocessinganddatamovementoverhead
suchasRDMAandSend/ReceivesemanticsarecompletedintheadapterwithoutCPUintervention.
ApplicationsusingTCP/UDP/IPtransportcanachieveindustryleadingthroughputwhenrunover
ConnectXlBanditshardwarebasedstatelessoffloadengines.
I/Ovirtualization:ConnectXlBsupportforhardwarebasedI/OvirtualizationiscomplementarytoIntel
andAMDvirtualizationtechnologies.Virtualmachines(VM)withintheserverareenabledwith
dedicatedI/Oadapterresourcesandguaranteedisolationandprotection.Hypervisoroffloadfeatures
removesoftwarebasedvirtualizationoverheadsandfreeupCPUcyclesenablingnativeoperating
systemperformanceforVMsandhigherserverutilizationbysupportingmoreVMsperphysicalserver.

129|P a g e

BladeServerConfiguration&Troubleshooting

StorageAccelerated:AunifiedInfiniBandclusterforcomputingandstorageachievessignificantcost
performanceadvantagesovermultifabricnetworks.Standardblockandfileaccessprotocolsleveraging
InfiniBandRDMAresultinhighperformancestorageaccess.Datareliabilityisimprovedthroughtheuse
ofTb0compliantDataIntegrityField(DIF).FibreChannel(FC)overInfiniBand(FCoIB)featuresenable
theuseofcosteffectivebridgesforconnectingtoFCSANs.
SoftwareSupport:AllMellanoxadaptercardsarecompatiblewithlegacyTCP/IPandOpenFabricsbased
RDMAprotocolsandsoftware.TheyarealsocompatiblewithInfiniBandandclustermanagement
softwareavailablefromOEMs.Theadaptercardsaresupportedwithmajoroperatingsystem
distributions.
TechnicalSpecifications
ThefollowingtableshowsthetechnicalspecificationsfortheMellanoxConnectXDualPortInfiniBand
MezzanineCard.
PowerEdgeM1000e(InfiniBandInterconnect)Specifications
Description
IBTASpecification1.2compliant
10or20Gbperport
RDMA,Send/Receivesemantics
Hardwarebasedcongestioncontrol
Atomicoperations
16millionI/Ochannels
256to4KbyteMTU
2GBmessages
9virtuallanes:8data+1management
EnhancedInfiniBand
Hardwarebasedreliabletransport
Hardwarebasedreliablemulticast
ScalableReliableConnectedtransport
EnhancedAtomicoperations
ServiceorientedI/O
FinegrainedendtoendQoS
HardwareBasedI/OVirtualization Addresstranslationandprotection
Multiplequeuespervirtualmachine
NativeOSperformance
ComplimentarytoIntelandAMDI/OMMU
AdditionalCPUOffloads
TCP/UDP/IPstatelessoffload
Intelligentinterruptcoalescence
FullsupportforIntelI/OAT
ComplianttoMicrosoftRSSandNetDMA
StorageSupport
T10compliantDataIntegrityFieldsupport
FibreChanneloverInfiniBand(FCoIB)
CPUCompatibility
AMDX86,X86_64
IntelX86,EM64T,IA32,IA64
Feature
InfiniBand

130|P a g e

BladeServerConfiguration&Troubleshooting

PowerEdgeM1000e(InfiniBandInterconnect)Specifications
Feature
Description
PCIExpressInterface
PCIeBase2.0compliant,1.1compatible
2.5GT/slinkratex8(20+20Gbbidrectionalbandwidth)
FitsDellPowerEdgeM1000eserverblades
SupportforMSI/MSIXmechanisms
Connectivity
WorkswithDellPowerEdgeM600andPowerEdgeM605
blades
ManagementandTools
OpenSM
Interoperablewiththirdpartysubnetmanagers
Firmwareanddebugtools(MFT,IBADM)
OperatingSystems/Distributions
NovellSLES,RedHatEnterpriseLinux(RHEL),Fedora,and
otherLinuxdistributions
OpenFabricsEnterpriseDistribution(OFED)
ProtocolSupport
OpenMPI,OSUMVAPICH,HPMPI,IntelMPI,MSMPI,ScaliMPI
IPoIB,SDP,RDS
SRP,iSER,FCoIBandNFSRDMA
uDAPL

InfiniBandIOM:CiscoSFSM7000EIB
Switch
CiscoSFS7000ESeriesInfiniBandSwitches
provideanidealserverinterconnectfor
distributedapplicationenvironments,
suchashighperformancecomputing.The
CiscoSFS7000ESeriesmeetsthe
performancedemandsofdistributed
applicationsbyoffering:

TheunprecedentedbandwidthofInfiniBand
Superiorlatencycharacteristics
Enterpriseclasshighavailabilityandmanagementfeatures

ThelatestevolutionoftheCiscoSFS7000SeriessupportsdualspeedInfiniBand4Xdoubledatarate
(DDR)andsingledatarate(SDR)interfacesthatdeliver20Gbpsand10Gbpsbandwidthrespectively,
perport.ThelowlatencyandhighbandwidthofferedbyCiscoSFS7000DInfiniBandServerSwitches
enableanewclassofdistributedapplicationsandsystemsthatdelivergreaterbusinessagilityand
competitiveadvantage.
Overview
TheCiscoSFSM7000ESwitchdeliversanidealcombinationofpriceandperformancewiththeDell
M1000eBladeplatform.TheswitchprovidesDDRInfiniBandconnectivitytoupto16bladeserver

131|P a g e

BladeServerConfiguration&Troubleshooting

modulesandprovides8DDRInfiniBanduplinkports.EachDDRInfiniBandportcansupport20Gbpsof
fullynonblockingbandwidthtoeachport.Theporttoportlatencythroughtheswitchis140ns.
Whenfullypopulated,theDellM1000ecanhavefourCiscoSFSM7000Eblademodulestodrive1.28
terabitsofnonblockingtrafficforI/Ointensiveapplications.Thebladeswitchisalsoahotpluggable
expansionmodule,allowingontheflycapacityexpansionwithoutapplicationdisruption.TheCiscoSFS
M7000Eisdesignedtobeacompellinghighperformancecomputing(HPC)solutionintheblademarket.
TheCiscoSFSM7000EhasbeencertifiedwiththeCiscoSFS7000SeriesInfiniBandServerSwitchesand
CiscoSFS3000/3500MultifabricServerSwitches.CiscoprovidesacompleteDDRInfiniBandswitching
familytocomplementtheCiscoSFSM7000EswitchtodeliverHPCsolutions.Forexample,byusingthe
CiscoSFS7024DSwitchesinthecore(288port,4xDDRInfiniBandswitches)toaggregatemultipleblade
chassis,acustomercanbuildclusterswithtensofthousandsofprocessorsusingDellM1000blade
systems.
BecausetheCiscoSFSM7000Eisanunmanagedswitch,asubnetmanagerisrequiredfortheoperation
oftheInfiniBandfabric.CiscorecommendsrunningtheCiscoHighPerformanceSubnetManagerona
bladeservermoduleintheInfiniBandfabric.TheCiscoSFSM7000EisalsoqualifiedwithotherSFS
managementapplicationssuchastheFabricAnalysisandCorrelationToolkit.

132|P a g e

BladeServerConfiguration&Troubleshooting

TechnicalSpecifications
ThefollowingtablesshowthetechnicalspecificationsfortheCiscoSFSM7000EInfiniBandDualPort
Switch.
CiscoSFSM7000EInfiniBandDualPortSwitchFeatures
Feature
Description
20Gbps(DDR),4xInfiniBandPorts TheCiscoSFSM7000Eoffers244xDDRIBportsof206bpslinks
forthemostdemandinghighperformancecomputing(HPC)
applications.Theswitchhasfullbisectionalbandwidthof960
Gbps.Eachportisautosensinginbothsingledatarate(SDR)and
doubledatarate(DDR)modes.16portsprovideconnectivityto
servermodulesand8uplinkportsareavailableforexternal
connectivity.
LowLatency
TheCiscoSFSM7000Eisacutthroughswitchwithlessthan140
nanosecondsofporttoportlatency.Theswitchisidealfor
applicationsthathavelargeinterprocesscommunication(IPC)or
simplydependonquickmovementofdataforcompetitive
advantageinthemarket.
PoweredPorts
Allportsarepoweredtoenableconnectivitywithpassiveand
activecoppercablesaswellasopticalcables.
HotPlugSwappable
TheCiscoSFSM7000ESwitchesarehotplugswappableintoanyof
thefourI/OslotsintherearoftheDellM1000Ebladechassis.The
administratorcanreconfigureandmaintainthefabricwithout
poweringdowntheservernodes.
ScaleoutFabricTopologies
Eachserverblademaycontainupto2HCA cards.Whentheblade
chassisisfullypopulatedwiththeCiscoSFSM7000ESwitches,a
customercanbuildmultiple,independentfabricswiththesame
servernodes.Thistypeofconfigurationoffersextremelyhigh
throughput.
UnmanagedSwitch
TheCiscoSFsM7000Eisanunmanagedswitchtoprovideapower
efficient,costeffectivesolutionforHPCcustomers.
CompleteEcosystem
CiscooffersanInfiniBandplatformthathasbeencertifiedand
extensivelytestedtoprovideoptimalproductivity.TheCiscoSFS
M7000EispartofagreaterInfiniBandofferingfromCisco.

CiscoSFSM7000EInfiniBandDualPortSwitchTechnicalSpecifications
Feature
Description
Size(HxWxD)
1.15x12.10x10.75in.
SystemWeight
3.75lb.
Temperature
Operating:40to120F(5to50C)
Nonoperating:40to140F(40to60C)
PowerConsumption
56W
EnvironmentalStandards
ROHS5compliant,WEEEcompliant
MeantimebetweenFailure
>100,000hours
(MTBF)
133|P a g e

BladeServerConfiguration&Troubleshooting

DiagnosticIndicators

DiagnosticLEDs
ThefollowingtableshowsthediagnosticindicatorstotroubleshoottheCiscoSFSM7000EInfiniBand
switch.
LEDType
PortStatusLEDs

Indicator
DiagnosticStatus(DIAG)

Description
BlinkingGreen:TrafficActive
SolidGreen:Logicallinkexists
Off:LinkerrororSubnetManageris
notrunning

SystemStatusLEDs DiagnosticStatus(DIAG)

SolidBlue:Systemisoperating
normally
BlinkingBlue:POSTisinprogress
Amber:Systemisreceivingpowerbut
isnotfunctioningproperly
Off:Systempowerisoff

PowerStatusLEDs

SolidGreen:Powerisappliedtothe
switch
Off:Powerisnotappliedtotheswitch

DiagnosticStatus(DIAG)

PowerOnIndicators
ThefollowingLEDsequenceoccursafterinstallationintothechassisI/Obayandtheswitchisfunctional.
1. SwitchStatusPowerLEDturnsGREENafterafewseconds.CMCmoduleinBladesystemverifies
thattheSwitchiscorrectlyconnectedtoanInfiniBandHCA.
2. PortStatusLEDsturnGREENafterapproximately30seconds.
134|P a g e

BladeServerConfiguration&Troubleshooting

3. SwitchStatusLEDturnsBLUE.

After installation into the chassis I/O bay, if the switch is not operational, the following are true:
1. TheSwitchStatusPowerLEDdoesnotturnGREEN.
2. TheSwitchStatusLEDblinksAMBER.

Summary
FabricAconsistsoftwoembeddedBroadcomEthernetcontrollers,whicharedesignedasa
(CNIC)thatcombinesnetworkfunctionalityintoasinglechip.
o ThebidirectionaldataforasingleBroadcomGbLOMiscarriedoveraPCIExpressx4bus
connectedtotheESB2DSouthbridgeontheM600ortheMCP55ontheM605.
o OptionalLOMfeaturesareactivatedwiththepopulationoftheTOELicenseKey.
OptionalFabricexpansionMezzaninecardsdesignedtosupporthighspeednetwork
communicationtechnologiesinFabricsBandCinclude:
o 10GbEthernet
o FibreChannel
o InfiniBand
ThedataforeachmezzaninefabriciscarriedoveraPCIExpressx8interfaceconnectedtothe
Northbridge.
Mezzaninecardssupportedinclude:
o EmulexLPe1105M4FCHBA
o QLogicQME2472FCHBA
o DellBCM57086SEthernetMezzanine
Mezzaninecards(whichrequireamatchingI/OmoduleinstalledinthecorrespondingI/Obay)
installedintheFabricBorFabricCcardslotrequireamatchingI/Omoduleinstalledinthe
correspondingI/Obaytosupportdataflowforthatfabric.
o ThePowerEdgeM605cannotsupporttheDellBCM57086SEthernetMezzaninecard.
IOMsinDellPowerEdgebladeserversinclude:
o Dell/EmulexPT1016FibreChannelPassThrough,designedforuseinsituationswherea
directconnectionbetweeneachbladeserverandexternalSANswitchesisrequired.
o Brocade442424portFibreChannelswitch
InfiniBandisanindustrystandardspecificationthatdefinesaninput/outputarchitectureused
tointerconnectservers,communicationsinfrastructureequipment,storageandembedded
systems.
o InfiniBandaddressestheneedsoftodayscomplexandpowerfulserversbyproviding:
Superiorperformance
Highefficiency
Fabricconsolidationandlowenergyusage
Reliableandstable
Dataintegrity
135|P a g e

BladeServerConfiguration&Troubleshooting

TheMellanoxConnectXIBMDIInfiniBandHostChannelAdapter(HCA)mezzaninecard
deliverslowlatencyandhighbandwidthforperformancedrivenserverandstorage
clusteringapplications.
CiscoSFS7000ESeriesInfiniBandSwitchesoffer:
TheunprecedentedbandwidthofInfiniBand
Superiorlatencycharacteristics
Enterpriseclasshighavailabilityandmanagementfeatures

136|P a g e

BladeServerConfiguration&Troubleshooting


Module8:Troubleshooting

CommonConfigurationIssuesforBlades

Objectives

Inthissectionyouwilllearnaboutcommonconfigurationissuesthatcanavoidtheneedforparts
replacement.

ReceiveMessagestatingthattheCMCsarenotredundant
Solutions
PlugbothCMCsintomanagementnetworkviaEthernetcable.PrimaryCMCwillhaveabluestatus
indicatorled.
CMCfirmwaredoesnotmatch.MakesurebothCMCsarerunningthesamefirmwareversion.

UpdatetheFirmware
FirmwareUpdatesarecriticalforM1000eandblades.Updateallcomponentstocurrentversionsat
thesametime.Ifthisisnotdone,unexpectederrorsorbehaviormayoccurfromtheCMC,iDRAC,
and/orblade.Planformaintenancewindowstoperformupdates.

137|P a g e

BladeServerConfiguration&Troubleshooting

CMC
iKVM
iDRAC
Bios
M6220firmware
LOMBIOS/firmware
HBAMezzanineBIOS/firmware

FLEXADDRESSINFORMATION:Seehttp://www.delltechcenter.com/page/FlexAddressforinformation
onFlexAddress.

Anexampleofwhyitisimportanttoupdateallfirmware/biostocurrentversionsacrossallcomponents
intheenclosureandbladesasmentionedinitem2.EvenifFlexAddressingisnotenabled,theCMC
requiresallpertinentcomponentfirmwarebeataversionthatsupportsFlexAddressing.Seeitems3
and4.

MessagereceivedintheiDRACsellog
WhenthemessageLinktuningsensorfailedisreceived:
Donotreplacehardware.Updatethefirmwarefortheappropriatedevicelistedintheerrormessage.
ThiscouldbetheiDRAC,LOM,orHBAMezzaninecard.
Exampleoferror:LinkT/FlexAddr:Linktuningsensorfailedtogetlinktuningorflexaddressdatafrom
BMC/iDRAC

FC(QLGCorEMLX)Mezzaninewouldexperience"F1/F2"duringpost.
Donotreplacehardware.UpdatetheQLGCorEMLXmezzanineBIOS.
QLogicQME2472BIOS2.04orlater
EmulexLPe1105M4BIOS3.03a3andfirmware2.72A2orlater

Servernonresponsiveatpowerup
Anewinstallationorconfigurationchange,themostcommonproblemisfabricmismatch.Theblade
status/identificationindicatorwillblinkamberifthereisaninstalledmezzaninecardthatdoesnot
matchtheI/OmoduleinstalledintheM1000eenclosure.
138|P a g e

BladeServerConfiguration&Troubleshooting

Explanationoffabrics:SeeCMCgetdcinfoandgetioinfooutputbelow.Noticethatthefontcolor
matchesthefabrictypes.

FabricAwillalwaysbeEthernetSwitch1andswitch2correspondtotheLOMsineachofthe
blades.NotethattheLOMsarenotlistedundertheserverDaughtercard(DC1type,DC2type)
informationbelow.(thetermdaughtercardisusedinsteadofmezzaninecard)
FabricBcanbeeitherEthernetorfibreswitch3andswitch4correspondtodaughtercard
(DC1type).Notethatbothswitch3andswitch4fabricmustmatch.Exampleofunsupported
configuration:switch3=Ethernetandswitch4=fibre.Alsonotethatthedaughtercardfabric
mustmatchtheswitchfabric.Intheexamplebelow,fabricBisEthernet.
FabricCcanbeeitherEthernetorfibreswitch5andswitch6correspondtothedaughtercard
(DC2type)Notethatbothswitch5andswitch6fabricmustmatch.Thedaughtercard(DC2
type)mustmatchtheswitchfabric.Intheexamplebelow,fabricCisFibre.

Itispossibletoinstalloneswitchforafabricandleavethesecondswitchblank.Thisisavalid
configuration.

Ifaswitchisnotinstalled,nomezzaninecardscanbeinstalledforthatfabric.
Ifaswitchisinstalled,notallserversmusthavethecorrespondingmezzaninecardinstalled.

TroubleshootingResource

DellEnterpriseTechnologyCenterhttp://www.delltechcenter.com
1.UsemenuonleftsideofpagetochooseBlades:

139|P a g e

BladeServerConfiguration&Troubleshooting


2.Atthebottomofthepagechoosefromseveralinformationresources:

Support.dell.com

Firmwarecanmanuallybedownloadedfromthissite.Informationislocatedunderservers.

1.ChooseDriversandDownloadsfromthemainpage:

140|P a g e

BladeServerConfiguration&Troubleshooting

2.Choosemethodforfindinginformationasappropriate.

Ftp.dell.com
FTPsitewhereupdatesareavailable.Thissiteisautomaticallyaccessedbytheservers.

141|P a g e

BladeServerConfiguration&Troubleshooting

Module9:NavigatingDellInformationandTools
DellEnterpriseiscommittedtoofferingcustomersinformationandsupportthroughmanychannels.Our
engineersandsubjectmatterexpertsarededicatedtobringingyouexperienceandguidance,technical
howtotips,tools,bestpracticesandsolutions.
Uponcompletionofthismodule,youwillbeableto:

NavigatethroughmanyofthesupportsitesandtoolsdesignedandofferedbyDellEnterprise.
Recognizethebenefitsofthesetools.
Relatetherelevancyofthisinformationtoyourjob.

DellSystemESupportTool(DSET)
TheDellSystemESupportTool(DSET)providestheabilitytocollecthardware,storageandoperating
systeminformationfromaDellPowerEdgeserver.ThisinformationisconsolidatedintoasingleSystem
ConfigurationReportthatcanbeusedfortroubleshootingorinventorycollectionofasystem.The
browseruserinterfaceprovidesaconvenientmeanstoviewspecificdatathroughhierarchicalmenu
trees.

NavigateandexploretheDellSystemESupportTool(DSET)atthislink:

http://support.dell.com/support/topics/global.aspx/support/en/dell_system_tool?c=us&l=en&s=gen

Support.dell.com
Thiscomprehensivesupportsiteoffersawidevarietyoftechnicalsupportoptions.Youmayresearch
yourissuesusingDellmanualsandforumsaswellascheckonorderstatusanddownloaddriversyou
need.ThesiteofferscomprehensiveinformationonallDellproductsandfeatureslinkstomanytech
support,customercare,andchatoptions.

NavigatetotheDellsSupportpageatthislink:http://support.dell.com/
1. FindtheTroubleshootingandFAQssectionandclickthelink.Followthesestepstousethe
troubleshootingwizard:
a. ChooseSelectaModel.
b. WithSelectyourProductModelhighlighted,selectServers,Storage,
Networking.
c. WithServers,Storage,Networkinghighlighted,selectPowerEdgeServer.
d. WithPowerEdgeServerhighlighted,selectthemodelofyourchoice,then
confirmyourchoice.
142|P a g e

BladeServerConfiguration&Troubleshooting

e. Spend10minutesusingthewizard.
2. Support.dell.comoffersallcustomersoptionsforsigningupandmanagingDellTechnical
Updateinformation.TheoptionsincludeDellTechnicalUpdatesandManagementofYour
Updates.Thesiterequiresthatyousignupinitiallyforanaccountafterwhichupdates
concerningyourequipmentwillautomaticallybeemailedtoyou.
a. Clickthislink,thenreviewthewebpage:
http://support.dell.com/support/topics/global.aspx/support/notifications
/en/index?c=us&l=en&s=gen&~ck=ln&lnki=0
b. Signupforthisservicenoworsavethelinkforfuturereference.

Learndell.com
DellofferscomprehensivetrainingandcertificationprogramsforDellhardwareandpopularsoftware
applications.Trainingcanbecustomizedtomeetyouruniquerequirements.
NavigatetotheLearnDellsiteatthislink:
http://www.learndell.com/Dell/
SpendfiveminutesreviewingthelearningopportunitiesofferedthroughDellLearning.

EnterpriseSupportServices
EachlevelofDellsEnterpriseSupportincorporatesdifferentoptions,allowingDellandyoutocreatethe
rightmatchforyouruniqueneeds.DellalsooffershelptoensureongoingperformanceofyourEMC
technology.
NavigateandexploretheEnterpriseSupportsiteatthislink:
http://www.dell.com/content/topics/global.aspx/services/ent_support/ent_support?c=us&cs=555&l=e
n&s=biz
SpendfiveminutesreviewingthedifferentsupportoptionsofferedbyDellEnterprise.

ThankYou!
ThisconcludestheDellBladeServerConfigurationCourse.Dellinvitesyoutoconsiderthefollowing
relatedcourses:

DellSANDataProtection.
2daycourse
DellSANManagement.
3daycourse
DellOpenManageandSystemManagement.
2daycourse

143|P a g e

BladeServerConfiguration&Troubleshooting

PowerEdgeConfigurationandTroubleshooting.
2daycourse
DellDesktopandLaptopConfigurationandTroubleshooting.
2days

Gotowww.learndell.comtoregister!

144|P a g e

BladeServerConfiguration&Troubleshooting

You might also like