________________________________________________________________________

IBM Flex System

Education

Basic Problem Determination for the IBM
Flex System Enterprise Chassis (8721) &
Options

May 2012
Study guide

GX40650
Release 1.10

This course is owned and published by the IBM MTS
Global Support Skills and Knowledge Enablement
Team and created by Alex Badia.

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Preface
May 2012 2 GX40650r110.pdf

International Business Machines Corporation, 2012
All rights reserved.
IBM MTS Global Support Skills and Knowledge Enablement
IBM Systems, Department EYGA.
Building 203, Post Office Box 12195,
Research Triangle Park, North Carolina 27709-2195
IBM reserves the right to change specifications or other product information without
notice. This publication could include technical inaccuracies or typographical errors.
References herein to IBM products and services do not imply that IBM intends to make
them available in other countries. IBM provides this publication as is, without warranty of
any kind either expressed or impliedincluding the implied warranties of
merchantability or fitness for a particular purpose. Some jurisdictions do not allow
disclaimer of expressed or implied warranties. Therefore, this disclaimer may not apply
to you.
Data on competitive products is obtained from publicly obtained information and is
subject to change without notice. Please contact the manufacturer for the most recent
information.
The following terms are trademarks or registered trademarks of IBM Corporation in the
United States, other countries or both:
Active Memory, Active PCI, AT, BladeCenter, the e-business logo, EasyServ, Enterprise
X-Architecture, EtherJet, HelpCenter, HelpWare, IBM RXE-100 Remote Expansion
Enclosure, IBM XA-32, IBM XA-64, IntelliStation, LANClient Control Manager, Memory
ProteXion, NetBAY3, Netfinity, Netfinity Manager, Predictive Failure Analysis, RXE
Expansion Port, SecureWay, ServeRAID, ServerProven, ServicePac, SMART
Reaction, SMP Expansion Module, SMP Expansion Port, UM Services, Universal
Manageability, Update Connector, Wake on LAN, XceL4 Server Accelerator Cache,
XpandOnDemand scalability.
IBM Corporation Subsidiaries: Lotus, Lotus Notes, Domino, and SmartSuite are
trademarks of Lotus Development Corporation. Tivoli and Planet Tivoli are trademarks
of Tivoli Systems, Inc.
LLC, Adobe, and PostScript are trademarks of Adobe Systems, Inc. Intel Celeron,
LANDesk, MMX, Pentium II, Pentium III, Pentium 4, SpeedStep, and Xeon are
trademarks or registered trademarks of Intel Corporation. Linux is a trademark of Linus
Torvalds. Microsoft Windows and Windows NT are trademarks or registered
trademarks of Microsoft Corporation. Other company, product, and service names may
be trademarks or service marks of others.
For more information, visit: http://www.ibm.com/legal/us/en/copytrade.shtml.

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Preface
May 2012 3 GX40650r110.pdf

Preface
Basic Problem Determination for the IBM Flex System
Enterprise Chassis (8721) & Options
This document may not be copied or sold, either in part or in whole, to non-IBM
personnel.
Please write your name and address below to personalize your copy.
Issued to:
Address:



Current release date: May 2012
Current release level: 1.10
The information contained within this publication is current as of the date of the latest
revision and is subject to change at any time without notice.
Please forward all comments and suggestions regarding the course material format and
content to your local IBM System x Service and Support Education country coordinator
or contact.

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Table of
Contents
May 2012 4 GX40650r110.pdf

Table of Contents
Preface ............................................................................................................................ 3
Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) &
Options........................................................................................................................ 3
Table of Contents ........................................................................................................... 4
Prerequisites ............................................................................................................... 5
Objectives ................................................................................................................... 6
Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721)
& Options ........................................................................................................................ 7
Overview ..................................................................................................................... 7
Flex System Enterprise Chassis & Options ........................................................... 7
IBM Flex System Enterprise Chassis & Options Lightpath ...................................... 8
Chassis Front Panel Card LEDs ........................................................................... 8
Power Supply LEDs ............................................................................................ 11
Compute Node LEDs .......................................................................................... 12
Systems Management .............................................................................................. 17
Service Data Capture Instructions from the CMM CLI ........................................ 17
FSM Basic Chassis Management ....................................................................... 18
Summary ....................................................................................................................... 21

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options
Prerequisites and Objectives
May 2012 5 GX40650r110.pdf

Prerequisites
A curriculum is a combination of specific roadmaps that are attempted in a logical
sequence. Each roadmap is a set of course modules that are also attempted in a logical
sequence. A brand curriculum consists of the roadmap types listed below. Attempting
the courses in each roadmap in the sequence presented affords the best change of
successful completion.
Access the complete IBM Flex System curriculum using the following Global Support Enablement
University (GSEU) link:
https://w3-
connections.ibm.com/wikis/home?lang=en#/wiki/Global%20Support%20Enablement%20University/page/
Flex%20MTS%20RSC%20and%20SSR%20Combined%20Curriculum
Prerequisites: Prerequisite courses provide basic concepts in technologies,
service tasks, and products. All prerequisites should be completed prior to
attempting subsequent roadmaps in the curriculum.
Fundamentals: Fundamental courses describe in general the architectures,
technologies, capabilities, and limitations of the specific product brand.
Fundamental courses should be attempted prior to problem determination and
product specific course modules.
Point of Reference Training: Point of Reference Training (PORT) is the use of
the brand external product documentation to learn about the products as do our
customers. This training is designed to provide service personnel the customer's
point of reference for the products they have installed.
New Technologies: New technologies courses introduce new major concepts
not previously seen within the brand. Over time, new technologies courses are
integrated into the fundamentals courses.
Access and Interface Training: Access and Interface (A and I) training modules
detail the ability to access service data and the use of typical application
interfaces, such as system management, service management, and configuration
management.
Problem Determination Training: Problem Determination (PD) training details
the processes and tasks used to isolate failures and generate a resolution plan
for the products covered within the curriculum.
Product and Options modules: Product and Options course modules provide
specific information for each product covered within the curriculum. These course
modules specifically relate to machine types and option part numbers, and
provide the details of a particular product.
Service Task Training: Service task training describes the processes and
practices used by service personnel to perform problem determination, problem
isolation, and action plans to resolve customer issues. Service task training is

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options
Prerequisites and Objectives
May 2012 6 GX40650r110.pdf

typically hands on classroom or virtual exercises and practice sessions using
brand products.
Compute Environment Training: Compute environment training details the
typical customer use of the products within the curriculum, as well as external
factors that can cause non obvious issues with these products.
Education Assist Elements: Education Assist (EA) elements include installation
and removal videos, simulators, trifold summaries, and self-running product or
application demonstrations. EA elements are designed to be used as daily work
aids, not as stand-alone course modules.
Curriculum Assessments: Assessments for each course module within the
curriculum are provided at the completion of the course module. Curriculum
assessments are taken when a large majority of the curriculum has been
completed.
The prerequisites for this course include the roadmaps and courses listed below:
http://xsnsftp.raleigh.ibm.com/docs/university3/prereq_sum_bycrs.asp?crscode=GX40650
Objectives
Upon completion of this course, you will be able to:
1. Provide an overview of basic problem determination for the IBM Flex System
Enterprise chassis and options.
2. Describe the IBM Flex System and options lightpath.
3. Identify the IBM Flex System locations.
4. Provide systems management information regarding the Flex System Manager
node and chassis management module.

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Overview
May 2012 7 GX40650r110.pdf

Basic Problem Determination for the IBM
Flex System Enterprise Chassis (8721) &
Options
Overview
Flex System Enterprise Chassis & Options
Flex System chassis problem determination methodology is similar to the BladeCenter
environment. Obtain the service data log information from either the chassis
management module (CMM) or Flex System Manager (FSM) node. The most efficient
way to populate the chassis with nodes is from the bottom up. For hardware, ensure
that all connectors are intact and the correct parts are in the proper locations according
to installation and configurations guides.
Lightpath is used to alert the end user of a condition that requires attention. Not all
notifications are fault based. Some instances that are not fault based include, but are
not limited to information notifications, power good, and locator LEDs.
Understand the IBM Flex System Enterprise chassis and option locations. Approach
each problem logically by following the hardware communication path or be sure to
search the appropriate logs to isolate the error.

Figure 1 IBM Flex System Enterprise Chassis


Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Lightpath
May 2012 8 GX40650r110.pdf

IBM Flex System Enterprise Chassis & Options
Lightpath
Chassis Front Panel Card LEDs
There are four LEDs on the front panel, all located at the bottom left corner of the
chassis. The front customer interface panel card has chassis information LEDs and the
fault LED light-up button. When the combination fault LED button is in deferred
maintenance mode, the fault LED of a failed component (e.g., compute node, switch,
power supply, fan, etc.) does not light. Pushing the button illuminates the fault LED of all
failed components, which are visible from the front, or rear of the chassis. Internal fault
LEDs (i.e., board LEDs next to DIMMs) does not light until activated by internal lightpath
switch (todays blue marked button on the system board).

Figure 2 IBM Flex System Enterprise Chassis Information LEDs
There is a personality card in addition to the front panel card. The personality card
provides two EEPROMs that contain chassis manufacturing vital product data (VPD)
and the remaining space is for CMM control and usage. This card is also the control
point for the front and rear service indicators. The state of the front chassis fault switch

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Lightpath
May 2012 9 GX40650r110.pdf

is via the personality card. Two temperature sensors on the personality card provide
inlet temperature information from the switches to the CMMs.
The personality card interfaces via a cable to the front panel card which provides a light
up logo (used to indicate power on), a blue location LED, and a combination fault LED /
switch.
The personality card logic is power by 3.3V received from the CMMs. If there is no CMM
in the system, the personality card forces the fault LED to turn on.

Figure 3 Personality Card

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Lightpath
May 2012 10 GX40650r110.pdf


Figure 4 IBM Flex System Enterprise Chassis Block Diagram







Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Lightpath
May 2012 11 GX40650r110.pdf



Power Supply LEDs
Each power supply has an AC good LED, DC good LED, and a fault LED. The AC/DC
good LEDs illuminate green and the fault LED is amber.


Figure 5 Power Supply







Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Lightpath
May 2012 12 GX40650r110.pdf



Compute Node LEDs
Figure 6 below displays the hard disk drive (HDD) activity and status LEDs as well as
the power, identify, check log, and fault LEDs.



Figure 6 Compute Node Front Bezel Locations

In addition, Figure 7 displays a LED panel located on the top right hand side of the
compute node. This is a lightpath diagnostics panel considered to be located inside the
node because it is not visible when plugged into the chassis. It is necessary to use the
lightpath diagnostics panel when the fault LED is illuminated.


Figure 7 Compute Node Locations Indicating Top LED panel



Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Lightpath
May 2012 13 GX40650r110.pdf



Use the following power on sequence and observe the LEDs to confirm a successful
boot operation. Wait until the LED on the power button is flashing slowly, before
pressing the power button. Button is not responsive until the LED is solid. Flashing LED
indicates node is starting the integrated management module (IMM) and doing low-level
boot and diagnostics. Power button LED is on solid indicates initial IMM boot completed
and the node is continuing to start imbedded OS and management functions. Monitor
Check Log LED and Fault LED for error indications.
Check Log LED on indicates a boot error. In the event this happens the servicer should
first update the firmware. Second, reseat the node. If the first and second steps do not
resolve the boot issue, then replace the system board.
View the lightpath diagnostics panel inside the node when the fault LED is illuminated.
This will indicate the following types of errors!battery, power, DIMM, IO expansion,
memory, CPU, system board, drive, and temperature. Figure 8 displays the lightpath
diagnostics panel.




Figure 8 Lightpath Diagnostics Panel

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Locations
May 2012 14 GX40650r110.pdf

IBM Flex System Enterprise Chassis & Option
Locations
The following figures, 6 and 7, display front and rear chassis locations for the IBM Flex
System. Bay 1 starts at the bottom left of the chassis and ends with bay 14 in the top
right front side corner. At the rear, it displays locations for four switches, two CMMs, six
power supplies, and ten fans.

Figure 9 IBM Flex System Enterprise Chassis Bay Locations

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Locations
May 2012 15 GX40650r110.pdf


Figure 10 IBM Flex System Enterprise Chassis Rear Power Supply, Fan, and Switch Bay
Locations








Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Locations
May 2012 16 GX40650r110.pdf

The IBM Flex System Enterprise chassis has an outer skeleton. Inside there are
compute node bay shelves, midplane, fan distribution cards, personality card, rear LED
card, fan logic module, fillers (node, power, switch, CMM, fan), fans, and power
supplies. Some of these are labeled in Figure 11 below.

Figure 11 IBM Flex System Enterprise Chassis Components

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Systems
Management
May 2012 17 GX40650r110.pdf

Systems Management
The FSM and CMM are used for systems management. Management at the compute
node level is available through the IMM. The IMM logs are passed up to both the CMM
and FSM.
Upon failure, the servicer should access the chassis map when operating with a CMM
and FSM. Depending on the type of error, there will be different views to navigate.
These include, but are not limited to problems view, status view, and CMM log.
A servicer should consider the following when operating with a CMM only. Collect a
CMM or OS log depending on the nature of the issue. A CMM log is necessary for
platform events (memory, memory controllers, clocks, CPU, etc.). If no detail is
available on the event start with an OS log for SAN, fibre, or disk issues. Then check
the chassis switch logs.
Service Data Capture Instructions from the CMM CLI
1. To save the full service data logs type the command: displaysd -save .tgz -i
xxx.xxx.xxx.xxx -T mm[x] Where: = the name of the .tgz file to direct the output
to. xxx.xxx.xxx.xxx = the IP address of the tftp server you are using. X = 1 or 2
depending on the CMM you are working on.

2. To copy the file down to your local system, use the following command prompt
command on your workstation or laptop: tftp -i xxx.xxx.xxx.xxx get .tgz
c:\temp\.tgz Where xxx.xxx.xxx.xxx = the IP address of the TFTP server is the
name of the tgz file you created to hold the service data.

3. When you use the CMM as the TFTP server, you can remove the file from the
CMM storage by entering this command: files -d tftproot/.tgz -T mm[x] Where =
the name of the tgz file you collected from the CMM TFTP server. Where x = 1 or
2 depending on the CMM you are working with.











Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Systems
Management
May 2012 18 GX40650r110.pdf

FSM Basic Chassis Management

The FSM has a chassis manager that allows the user to view chassis lightpath,
compute nodes, switches, fans, and power supplies.

Figure 12 FSM Chassis Manager Screen








Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Systems
Management
May 2012 19 GX40650r110.pdf

There are overlays that include status, access, LEDs, asset tags and energy/thermal.
To display more statuses in a view, the user must select one of the filter settings for
Status View. The user can select different overlays. For example, hardware access
states, hardware LEDs, and component names. Figure 13 displays an overlay.


Figure 13 FSM Overlay Screen
















Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Systems
Management
May 2012 20 GX40650r110.pdf


Active Energy Manager, an advanced manager plug-in, has also been integrated into
the chassis manager. This gives the user a quick sense of where thermal/energy
problems may exist.


Figure 14 Active Energy Manager Screen

Basic Problem Determination for the IBM Flex System Enterprise Chassis (8721) & Options Summary
May 2012 21 GX40650r110.pdf

Summary
This course has enabled you to:
1. Provide an overview of basic problem determination for the IBM Flex System
Enterprise chassis and options.
2. Describe the IBM Flex System and options lightpath.
3. Identify the IBM Flex System locations.
4. Provide systems management information regarding the FSM and CMM.