Professional Documents
Culture Documents
Information
SDU TROUBLESHOOTING
ENGLISH
APR 2004
68P09258A80B
Copyrights
This instruction manual, and the Motorola products described in this instruction manual may be, include or describe copyrighted
Motorola material, such as computer programs stored in semiconductor memories or other media. Laws in the United States and
other countries preserve for Motorola and its licensors certain exclusive rights for copyrighted material, including the exclusive
right to copy, reproduce in any form, distribute and make derivative works of the copyrighted material. Accordingly, any
copyrighted material of Motorola and its licensors contained herein or in the Motorola products described in this instruction manual
may not be copied, reproduced, distributed, merged or modified in any manner without the express written permission of Motorola.
Furthermore, the purchase of Motorola products shall not be deemed to grant either directly or by implication, estoppel, or
otherwise, any license under the copyrights, patents or patent applications of Motorola, as arises by operation of law in the sale of a
product.
Copyrighted Materials
Software and documentation are copyrighted materials. Making unauthorized copies is prohibited by law. No part of the software or
documentation may be reproduced, transmitted, transcribed, stored in a retrieval system, or translated into any language or
computer language, in any form or by any means, without prior written permission of Motorola, Inc.
Trademarks
MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are
the property of their respective owners.
Copyright 2003 Motorola, Inc.
Javat Technology and/or J2MEt: Java and all other Javabased marks are trademarks or registered trademarks of Sun
Microsystems, Inc. in the U.S. and other countries.
UNIXR: UNIX is a registered trademark of The Open Group in the United States and other countries.
REV091302
Table of Contents
SDU Troubleshooting
Software Release 2.16.3.x
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iii
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iv
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vii
General Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ix
Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xi
Chapter 1: Introduction
About this Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-1
SDU Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-2
Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-5
2-1
Chapter 3: Login/Passwords
Log in to SDU Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-1
4-1
5-1
6-1
APR 2004
SDU Troubleshooting
7-1
8-1
8-2
8-3
8-9
9-1
10-1
10-2
10-3
11-1
11-2
11-3
12-1
13-1
14-1
ii
SDU Troubleshooting
15-1
APR 2004
List of Figures
SDU Troubleshooting
Software Release 2.16.3.x
APR 2004
1-2
1-3
1-3
1-4
5-1
Figure 5-2: SDU Power Supply Modules and Circuit Breaker Modules . . . . . .
5-2
SDU Troubleshooting
iii
List of Tables
SDU Troubleshooting
Software Release 2.16.3.x
iv
1-5
7-2
8-4
8-6
8-10
8-11
8-11
8-12
8-13
8-13
8-14
8-15
8-15
8-16
8-16
8-17
8-17
8-18
8-18
SDU Troubleshooting
APR 2004
APR 2004
8-19
8-19
8-20
8-21
8-21
9-2
10-3
10-4
10-4
10-5
10-5
10-6
10-6
10-7
10-7
10-8
10-8
10-9
10-9
10-10
10-10
10-11
10-11
SDU Troubleshooting
List of Tables
vi
continued
10-12
10-12
11-3
11-4
11-4
11-5
11-6
11-6
11-7
11-7
11-8
11-8
11-9
11-10
11-11
11-11
SDU Troubleshooting
APR 2004
Foreword
Scope of manual
This manual is intended for use by cellular telephone system
craftspersons in the day-to-day operation of Motorola cellular system
equipment and ancillary devices.
This manual is not intended to replace the system and equipment
training offered by Motorola, although it can be used to supplement or
enhance the knowledge gained through such training.
Obtaining Manuals
To view, download, or order manuals (original or revised), visit the
Motorola Lifecycles Customer web page at http://services.motorola.com,
or contact your Motorola account representative.
If Motorola changes the content of a manual after the original printing
date, Motorola publishes a new version with the same part number but a
different revision character.
Text conventions
The following special paragraphs are used in this manual to point out
information that must be read. This information may be set-off from the
surrounding text, but is always preceded by a bold title in capital letters.
The four categories of these special paragraphs are:
NOTE
Presents additional, helpful, non-critical information that
you can use.
IMPORTANT
CAUTION
Presents information to identify a situation in which
damage to software, stored data, or equipment could occur,
thus avoiding the damage.
WARNING
Presents information to warn you of a potentially
hazardous situation in which there is a possibility of
personal injury.
APR 2004
SDU Troubleshooting
vii
Foreword continued
+18004335202
+44 (0) 1793565444
+861088417733
+81354633550
+5112124020
SDU Troubleshooting
APR 2004
General Safety
Remember! . . . Safety
depends on you!!
The following general safety precautions must be observed during all
phases of operation, service, and repair of the equipment described in
this manual. Failure to comply with these precautions or with specific
warnings elsewhere in this manual violates safety standards of design,
manufacture, and intended use of the equipment. Motorola, Inc. assumes
no liability for the customers failure to comply with these requirements.
The safety precautions listed below represent warnings of certain dangers
of which we are aware. You, as the user of this product, should follow
these warnings and all other safety precautions necessary for the safe
operation of the equipment in your operating environment.
Ground the instrument
To minimize shock hazard, the equipment chassis and enclosure must be
connected to an electrical ground. If the equipment is supplied with a
three-conductor ac power cable, the power cable must be either plugged
into an approved three-contact electrical outlet or used with a
three-contact to two-contact adapter. The three-contact to two-contact
adapter must have the grounding wire (green) firmly connected to an
electrical ground (safety ground) at the power outlet. The power jack and
mating plug of the power cable must meet International Electrotechnical
Commission (IEC) safety standards.
NOTE
Refer to Grounding Guideline for Cellular Radio
Installations 68P81150E62.
SDU Troubleshooting
ix
SDU Troubleshooting
APR 2004
Revision History
Manual Number
68P09258A80B
Manual Title
SDU Troubleshooting
Software Release 2.16.3.x
Version Information
The following table lists the manual version, date of version, and
remarks on the version.
Version
Level
Date of Issue
Remarks
APR 2004
Added notes stating that logging into SDU boards should be done during
a maintenance window.
APR 2004
SDU Troubleshooting
xi
Chapter 1: Introduction
Table of Contents
APR 2004
1-1
1-1
1-1
SDU Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SDU Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-2
1-2
1-2
Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-5
1-5
SDU Troubleshooting
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
IMPORTANT
NOTE
Additional performance tests may be required to insure
that the SDU still meets ATP specifications after any
device is replaced. Refer to the applicable Field
Replacement Unit (FRU) and ATP manual for this
information.
Associated Manuals
The following manuals and online help systems are referenced in this
manual.
S
S
S
S
S
S
S
S
S
APR 2004
SDU Troubleshooting
1-1
SDU Overview
Overview
The following figures illustrate the SDU frame, card cage, and top I/O
Panel.
Figure 1-1: SDU Frame
75
75
75
75
20
20
20
20
20
20
20
20
20
20
POWER SUPPLY
MODULES (4)
CIRCUIT BREAKER
MODULES (2)
FAN MODULES
FANS MODULES
NOTE:
1. Frame shown with doors removed
2. System configuration determines if upper cage is used.
1-2
SDU Troubleshooting
APR 2004
SDU Overview
continued
APR 2004
BPP
ISB
ISB
SDU Troubleshooting
CCA
1-3
SDU Overview
continued
1-4
SDU Troubleshooting
APR 2004
Acronyms
Acronyms
Table 1-1 defines acronyms used in this manual.
Table 1-1: Acronyms
Item
Description
AN
Access Node
BPP
CBM
CCA
DVM
FRU
GND
Ground
ISB
MBP
MMI
ManMachine Interface
PDU
PSM
RTN
Return
SDU
SPROC
System Processor
STP
T1C
Tier 1 Controller
NOTE
T1C is a different name for the ISB/SPROC.
UTP
APR 2004
SDU Troubleshooting
1-5
Acronyms
continued
Notes
1-6
SDU Troubleshooting
APR 2004
APR 2004
SDU Troubleshooting
2-1
2-1
2-1
2-1
2-1
2-2
2-2
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
Overview
When resetting cards to resolve an alarm, use the hardware reset method
(frontpanel reset switch) as opposed to the software reset method (e.g.
an OMCR reset command).
The BPP cards and CCA cards have hardware reset switches. The reset
switch is located in an unlabeled pinhole in the front panel of the card.
SDU Troubleshooting
2-1
continued
Lock/Unlock
NOTE
This case applies to both an automatic system reset and a
manual reset.
S The OMCR should issue a state change event indicating that the card
has come back into service. You can use an OMCR Display
command to verify this. For example, if you reset a BPP card with
SDF software, the OMCR will indicate that the SDF is in service.
2-2
SDU Troubleshooting
APR 2004
Chapter 3: Login/Passwords
Table of Contents
3
Log in to SDU Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Accessing Devices Using the OMCR . . . . . . . . . . . . . . . . . . . . . . . . .
Retrieving Information from Circuit BTS Configurations . . . . . . . . . .
Accessing the MMI Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Recovery Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
APR 2004
SDU Troubleshooting
3-1
3-1
3-1
3-3
3-4
3-4
3-5
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
Overview
Most of the SDU devices can be accessed remotely (not on site using the
front panel MMI interface) using the OMCR. This action makes it easy
to gain access to the device and collect data. The following procedures
provide the information needed to access the SDU devices using the
OMCR.
Locating SDU IP Addresses
At the OMCR enter the following command:
APR 2004
Description
IP address of the
active SPROC
SDU1 Router 2
IP address of the
active ISB
SDU2 Router 2
IP address of the
standby ISB
BPP<frameid><cageid><slotid> BCP
IP of LOGIN
ADDRESS of SDF
or PCF
BPP<frameid><cageid><slotid> CPE
SDU Troubleshooting
3-1
continued
WARNING
Any of the following actions could prevent future access to
the SDU boards:
S Never use the telnet command to log into any of the
SDU boards.
SPROC
At the OMCR use the following command to access to the SPROC.
3-2
SDU Troubleshooting
APR 2004
continued
The following two items contain directions on how to display items from
a circuit BTS configuration site.
Locating the PSI Address (for Circuit BTS Configurations)
At the OMCR enter the following command:
APR 2004
SDU Troubleshooting
3-3
continued
The MMI console is used to enter commands. Each SDU card has an
MMI console connection.
IMPORTANT
*
3
Configuration
To log in to SDU cards, you will need to obtain a PC with a Serial port,
and a terminal emulation program such as Hyperterminal. A
standalone dumb terminal can also be used.
The settings consist of:
S
S
S
S
Parity: None
Stop bit: 1
Flow control: none
VT100 Emulation
S SPROC
Login: motorola
Password: developer
Exit MMI Password: partymoto
NOTE
Use startMMI to enter MMI mode again.
3-4
SDU Troubleshooting
APR 2004
continued
S BPP
Login: motorola
Password: developer
Secure MMI: altivec
S CCA
Login: motorola
Password: developer
S ISB
Login: motorola
Password: developer
Enter logout to quit.
Recovery Procedure
If the rlogin command for the BPP command does not work, the
following procedure can be performed at the site.
Connect a serial port to the front panel of the BPP board.
At the prompt enter the following:
S lock
S shell Restart
If the lock and restart commands do not work, perform an lock/unlock
command to see if the session can be recovered.
APR 2004
SDU Troubleshooting
3-5
continued
Notes
3-6
SDU Troubleshooting
APR 2004
Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Overview of Black Box Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Running collectbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Running bboxlisten . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Accessing the Collected Black Boxes . . . . . . . . . . . . . . . . . . . . . . . . . .
APR 2004
SDU Troubleshooting
4-1
4-1
4-1
4-1
4-2
4-3
4-4
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
Data Collection
Data Collection
collectbox
The collectbox program is intended to be used on an OMCR to collect
debug information from the SDU. The debug information being gathered
is called a blackbox (airplane lingo) and contains information recorded
on the controller (SPROC).
bboxlisten
The bboxlisten program collects the blackbox logs from the SDU in the
case of an SPROC reset. Bboxlisten also provides some automatic bbox
file maintenance on the OMCR.
Prerequisites
APR 2004
SDU Troubleshooting
4-1
Data Collection
continued
Running collectbox
4
Setting the ne# (network element id or sdu number) or ipaddress is
mandatory. Optional parameters can be entered in any order.
NOTE
If not specified, the ne # is resolved from ip address. If the
ip address is not specified, the OMCR will pull the
address from the MIB based on the ne # specified.
Optional Parameters
Optional parameters:
S [f filter] Grabs the full blackbox using the specified filter, the
blackbox is not reset as part of the collection. The default behavior is
to collect the filtered blackbox from the active side. If neither the f or
p parameters are specified, the full unfiltered blackbox is collected.
Default behavior for the full blackbox is collection from the active
side.
S [p] Grabs the preserved blackboxes from the last reset. The default
behavior is to grab the preserved blackboxes from both the active and
the standby controllers. If neither the f or p parameters are
specified, the full unfiltered blackbox is collected. Default behavior
for the full blackbox is collection from the active side.
4-2
SDU Troubleshooting
APR 2004
Data Collection
continued
Examples
The following examples illustrate the use of collectbox.
S collectbox ne 5
Collects a full blackbox from the active side of SDU5.
S collectbox ne 8 p
Collects the preserved blackboxes from both sides of SDU8.
S collectbox ne 1 f SNMP
Collects a filtered blackbox based on the keyword SNMP from the
active side of SDU1.
Description of Operation
This command does not require the CLI to be up when the command is
executed. The command will attempt to find the SDU in the database
and determine the correct IP address to connect to. The command will
normally ping the platform before attempting to collect the specified
blackbox. The command will display the name and location of the file
collected. Any temporary files created on the platform will be cleaned
up. This program telnets to the platform which grabs console output
temporarily from the front panel serial port.
Running bboxlisten
APR 2004
SDU Troubleshooting
4-3
Data Collection
continued
The black box output is cleaned up on the OMCR and stored in the
/sc/spool/bb directory. The collected black boxes are named based on the
sdu#, time of collection, active/standby, and original file name on
platform.
The following is an example:
/sc/spool/bb/s210041939a0.out
/sc/spool/bb/s210041939s0.out
s2 SDU2
1004 October 4
1939 7:39 PM
a0 preserved file /data/blackbox/0 from
active side
s1 preserved file /data/blackbox/1 from
standby side
sh preserved file
/data/blackbox/hardResetBBox from standby side
af0 preserved file /active/init/hungbbox0
from standby side
4-4
SDU Troubleshooting
APR 2004
5-1
5-1
5-1
5-2
5-2
APR 2004
SDU Troubleshooting
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
Introduction
If the SDU should become completely unresponsive for some reason, the
method of last resort for recovery would be a power cycle of the SDU
cage (which in turn would power cycle every FRU in the SDU cage).
SDU Cage Power Supply
Overview
This section provides a high level overview of the SDUs power supply
and distribution in order to familiarize the user with the location and
purpose of the various power supplies and circuit breakers on the SDU
frame.
There are two power supplies per SDU cage, for a total of 4 per frame.
Each power supply pair (2 vertically stacked on the left and the right)
feeds a group of 7 circuit breakers which in turn feed 7 power zones
(or groups of slots) in a given cage.
There are 7 pushpull switches that correspond to the 7 circuit breakers
(running vertically alongside the power supplies). However, there is a
metal plate over these to prevent them from being manually opened.
These should not be used to perform a cage reset.
In addition to the 7 circuit breakers per cage, there are also 2 circuit
breakers per cage which correspond to the main power supplies to the
cage. These are located near the top of the front side of the SDU cage on
the panel labeled MAIN BREAKER PANEL. There is one circuit
breaker pushpull switch for each power supply and two power supplies
for each cage.
Figure 5-1: SDU Main Breaker Panel (MBP)
75
APR 2004
75
75
SDU Troubleshooting
75
5-1
continued
Figure 5-2: SDU Power Supply Modules and Circuit Breaker Modules
20
20
20
20
20
20
20
20
20
20
To cycle power to a given cage, the two circuit breaker switches for the
two power supplies to that cage must be pulled. The left two switches
control power to the lower cage and the right two switches control power
to the upper cage. Again, both switches must be pulled since the power
supply feeds are redundant.
Once both breaker switches are pulled, wait 1015 seconds and then
push the same two switches back in towards the frame. At this point, the
system should initialize completely and return to operation.
Cage Recovery
Once both circuit breaker switches are pushed back in for the Power
Supply, the SDU Frame will begin to initialize the SPROC and other
Payloads. During this process, the OMCR (NE Manager) gets an Alarm
which indicates the Loss of Communication with SDU Frame. Once
the NE (SPROC) completes the initialization and the ISB learns the
Routes from Access Node, then the SPROC will start sending the
SYNC Required to the OMCR. The OMCR also clears the Loss of
Communication Alarm. Then, the OMCR SYNCs with NE. Once the
NE SYNCs up with OMCR, the SDU Frame will be in the Operational
State.
5-2
SDU Troubleshooting
APR 2004
6-1
6-1
APR 2004
SDU Troubleshooting
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
S Once the SDU boots up after a reset, the LED lights on the front panel
of an SDU frame should appear solid green.
S If the LED light on a board appears red, the board is having an alarm.
S If the LED light on a board appears blinking green, the board is
resetting.
S If the LED light is in a constant blinking green, the board may not be
provisioned at the moment.
Boot progress
To monitor the boot progress of a board in an SDU frame, a console
needs to be connected to the board. Without a console connection, only
the LEDs can provide a limited status. Once the LED light turns into
solid green, the board is finished rebooting.
State Determination
Once the SDU finishes rebooting, the OMCR can be used to check the
state of the SDU. For OMCR Status/Display Commands, please refer
to the System Command Reference.
APR 2004
SDU Troubleshooting
6-1
continued
Notes
6-2
SDU Troubleshooting
APR 2004
7-1
7-1
7-1
7-2
7-3
7-4
7-5
7-6
APR 2004
SDU Troubleshooting
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
Introduction
WARNING
Never reseat or reset the SPROC for any reason, unless
Motorola CNRC has given the instruction to do so. If this
is the case, first lock the SPROC from the OMCR.
Symptom
If the SPROC (either active or standby) fails to join in the cluster and is
unable to boot up, use the following information to troubleshoot the
problem.
Possible Causes
Possible causes for this symptom include one of the following:
S LAN failure
S While the OMCR is configuring the ISBs, the SPROCs unmanaged
timeout timer expires.
Recovery Action
Reset the SPROC, and verify that the ISBs are in a stable state.
APR 2004
SDU Troubleshooting
7-1
continued
Symptom
The SPROC initialization does not complete if the SPROC console
displays node not in database and then resets.
Possible Causes
Possible causes for this symptom include one of the following:
S The configuration data file (baseNECDF .xml) for the SPROC either
contains errors or is corrupted.
For example, the configuration data may include an incorrect cage or
slot number.
Action
If the SPROC is not correctly seated, correctly seat the
SPROC.
NOTE
Refer to the CDMA2000 1X RAN Hardware Installation for
details.
2
NOTE
Refer to the CDMA2000 1X Cellular System Administration
Guide for additional details.
7-2
2a
2b
SDU Troubleshooting
APR 2004
continued
Symptom
If the SPROC console displays going unmanaged and then resets,
this indicates that the SPROC lost management control and is
performing a reset to clear the underlying problem.
Possible Causes
This symptom may be due to one of the following causes:
S With a 2N redundant setup, if both the ISBs reset and initialize again,
one of the SPROCs will display going unmanaged and reset as
a result of a split brain resolution process. There is a possibility that
the node which was a manager (the active one) may also reset.
APR 2004
SDU Troubleshooting
7-3
continued
Symptom
NOTE
A hung thread is like a hanging software task, only it is a
thread instead of a process.
If the hung thread problem occurs on the SDU, the SPROC console
displays something like the following output:
[D2D37D8] PANIC: Starting Emergency Exit
Handlers
[D2D37D8] PANIC: Called Default Emergency Exit
Handler
[D2D37D8] Writing Emergency Exit Trace
Information for 1 threads
[D2D37D8] goahead/hap Error: HUNG THREADS
DETECTED: srk tid = 14000011 (OS id = c43fa70)
[D2D37D8] Default EEH: PANIC for srk tid =
14000011 (OS id = c43fa70).
[D2D37D8] Finished Default Emergency Exit
Handler
[D2D37D8] PANIC: Finished Emergency Exit
Handlers
[D2D37D8] PANIC: Closing SRK
[D2E3D50] Closing Extensions
[D2E3D50] RIM Extension Shutting Down.
NEI : NEIX: Possible GA hung thread. Stack trace
in black box.Rebooting in 30 secs...
7-4
SDU Troubleshooting
APR 2004
continued
Recovery Action
A full SDU reset from the OMCR may help to clear out the hung thread
problem, but use caution.
WARNING
A full SDU reset will cause a site outage. If possible,
perform the reset during an appropriate maintenance
window.
Symptom
If the SPROC software becomes corrupted, the SPROC will reboot.
Possible Causes
See the Symptom description.
Recovery Action
No recovery action is necessary.
The SDU automatically resolves the problem using one of the following
methods:
S For a single SPROC node setup, the SPROC will reboot ten times,
then it will switch the partition and attempt to initialize using the other
partition.
APR 2004
SDU Troubleshooting
7-5
continued
Symptom
If the active LAN fails, the active SPROC will transition to the OOS
state, and the redundant SPROC will take over.
The SPROC console displays output similar to the following:
NEIX: active interface x with an ip address
x.x.x.x and subnet x failed
Possible Causes
The ISB connected to the active LAN is not functioning properly.
Recovery Action
None. This is a normal behavior.
7-6
SDU Troubleshooting
APR 2004
APR 2004
ISB/SPROC Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Overview of ISB/SPROC Troubleshooting . . . . . . . . . . . . . . . . . . . . . .
8-1
8-1
8-2
8-2
8-2
8-3
8-3
8-3
8-4
8-5
8-5
8-6
8-7
8-9
8-9
8-11
8-11
8-12
8-13
8-15
8-20
SDU Troubleshooting
8-7
8-7
8-8
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
ISB/SPROC Troubleshooting
Overview of ISB/SPROC
Troubleshooting
APR 2004
SDU Troubleshooting
8-1
8-2
SDU Troubleshooting
APR 2004
Overview
Symptom
The ISB is unable to download the software load from the SPROC.
Possible Causes
The possible cause may be one of the following. The ISB:
APR 2004
SDU Troubleshooting
8-3
continued
Symptom
The SPROC is unable to get IP service from the ISB.
Possible Causes
Ensure that the debug ethernet port of the SPROC isnt connected to an
outside network. If, when the SPROC boots up, the console display
indicates it is loading up on the debug interface, the debug port is
connected to the network.
Recovery Action
Perform the recovery actions in Table 8-1.
Table 8-1: SPROC Unable to get IP address from ISB
Step
Action
Ensure that the front panel Ethernet Jack does not have a
cable connected to it. The front panel Ethernet Jack is the
debug port.
8-4
SDU Troubleshooting
APR 2004
continued
Symptom
The symptoms include at least one of the following:
Symptom
The ISB reset causes payloads (BPPs, CCAs) to reset.
Possible Causes
If the ISB failover didnt occur quickly enough, the payloads will also
reset.
Recovery Action
None. The automatic ISB reset should correct the problem.
APR 2004
SDU Troubleshooting
8-5
continued
Symptom
The ISB fails to establish the gigabit link.
Possible Causes
Possible causes include:
S Cable failure
S Span I/O failure
S The SPROC database was not updated.
Recovery Action
Check/replace cables and/or span I/O (swap out); check/update the
SPROC database for gigabit links.
Table 8-2: ISB Fails to establish Gigabit Link
Step
1
Action
At the OMCR, verify the settings for the
MLS_SDU_CON links, the Gigabit connections from the
AN to the SDU.
If necessary, correct or change those settings.
NOTE
If you change these settings, additional steps from the
OMCR will be necessary. For example, if you modify a
configuration from the OMCR, you need to regenerate the
data files (the Generate Data File command) and reactive the
devices.
For more details, and the complete procedural context, refer
to the System Command Reference and the Cellular System
Administration Guide.
8-6
SDU Troubleshooting
APR 2004
continued
Symptom
During the SPROC failover, the ISB automatically reset.
Possible Cause
The route updates were not fast enough.
Recovery Action
None. The automatic reset should correct the problem.
Pulling out the standby ISB
and the standby SPROC
causes the payloads to reset
Symptom
The Payloads reset after a Dual standby pull.
Possible Causes
The route updates were not fast enough.
Recovery Action
None. The automatic reset should correct the problem.
The Active ISB reset causes
the SPROC (active and
standby) to reset
Symptom
The active ISB reset causes both SPROCs to reset.
Possible Causes
APR 2004
SDU Troubleshooting
8-7
continued
Symptom
When the standby ISB reset, the SPROC also reset.
Possible Causes
The route updates were not fast enough.
Recovery Action
None. The automatic reset should correct the problem.
8-8
SDU Troubleshooting
APR 2004
ISB/SPROC Alarms
NOTE
In addition to maintaining troubleshooting logs, you may
also want to notify CNRC of problems if they frequently
reappear.
NOTE
For a more detailed description of each alarm, refer to
System Output Messages. Also, if you are not sure when
you need to resolve the problem, refer to the alarm
documentation (in the System Output Messages) for the
severity level.
APR 2004
SDU Troubleshooting
8-9
continued
OMCR command
display mmsdfralinksdu#cbsc#mmsdfralink#
mmsdfralinkconf
display mmpcfralinksdu#cbsc#mmpcfralink# status
display mmpcfralinksdu#cbsc#mmpcfralink#
mmpcfralinkconf
status sdulinksdulink#
display
sdupdsnlinksdu#sdupdsncluster#sdupdsnlink#
pdsnlinkconf
NOTE
For information on editing, deleting and reprovisioning devices, refer to the Cellular
System Administration Guide.
8-10
SDU Troubleshooting
APR 2004
continued
Action
Check the physical link connection at the SDU and the farend connection of the physical link at the
AN IP Switch frame (Catalyst MLS chassis). Determine whether it is a physical link connectivity
error or a bad cable.
Refer to the CDMA2000 1X RAN Hardware Installation manual for details on physical cabling and
connectivity between the SDU and AN.
If it is not a physical link problem, replace the ISB. Refer to the CDMA2000 1X AN & SDU FRU for
details.
Action
Add additional processing capacity to the site or reduce the traffic loading.
APR 2004
SDU Troubleshooting
8-11
continued
Action
The system automatically attempts to reset the card. The card will initialize if the automatic reset is
successful.
If the card initializes, no further action is necessary.
If the card fails to initialize, continue at the next step.
NOTE
If the card initializes properly, there should be a state change event indicating that the board has
returned to the inservice state.
2
If the reset fails, lock and replace the card using the procedures contained in the CDMA2000 1X AN &
SDU FRU manual.
8-12
SDU Troubleshooting
APR 2004
continued
Action
If there are (in addition to this alarm) alarms on the farend ISB, or if other BPPs and CCAs are also
reporting similar alarms, the problem is most likely with the ISB. In that case, the ISB problems
should be addressed first. Otherwise the problem is probably local to the BPP, and the BPP should be
reset at a convenient time.
If the problem continues, replace the BPP. Refer to the CDMA2000 1X AN & SDU FRU.
IMPORTANT
Action
Verify that the device specified in the alarm output is present and operational, using the applicable
OMCR CLI commands.
If the device is not present or operational, replace it with an operational device. Refer to the
CDMA2000 1X AN & SDU FRU manual.
APR 2004
SDU Troubleshooting
8-13
continued
Action
Once the device is unlocked using the unlock command, the alarm will clear.
Refer to the Cellular System Administration Guide for details on the unlock command.
8-14
SDU Troubleshooting
APR 2004
continued
Action
The system disables the affected MMSDFRALINK or MMPCFRALINK. To clear the alarm, it is
necessary to delete and reprovision the link from the OMCR.
NOTE
For details, refer to the Cellular System Administration Guide for topics that deal with the SDULINK
software object and its subelements (the MMSDFRALINK and MMPCFRALINK).
Action
The failed SPROC becomes the standby SPROC. If the problem is not resolved in the reset, the
standby SPROC continues to fail. In this case, replace the card.
Refer to the CDMA2000 1X AN & SDU FRU manual for details.
APR 2004
SDU Troubleshooting
8-15
continued
Action
The system disables the affected SDUSPROC. To clear the alarm, it is necessary to delete the
SDUSPROC from the OMCR and reprovision in an attempt to bring it back into service.
NOTE
For details, refer to the Cellular System Administration Guide and System Command Reference.
Action
The system disables the affected device shown in the alarm. To clear the alarm, it is necessary to delete
the affected device from the OMCR and reprovision in an attempt to bring it back into service.
For details, refer to the Cellular System Administration Guide and System Command Reference.
NOTE
For details, refer to the Cellular System Administration Guide and System Command Reference.
8-16
SDU Troubleshooting
APR 2004
continued
Action
NOTE
For details, refer to the Cellular System Administration Guide and System Command Reference.
Action
Wait for the alarm to clear. The card resets in an attempt to recover. The alarm clears when the failed
SPROC recovers.
NOTE
If the standby SPROC fails to initialize, the active SPROC is not affected.
If the active SPROC has this alarm, a failover occurs.
2
The failed SPROC becomes the standby SPROC. If the problem is not resolved in the reset, the
standby SPROC continues to fail. In this case, replace the card.
Refer to the CDMA2000 1X AN & SDU FRU manual for details.
APR 2004
SDU Troubleshooting
8-17
continued
Action
The system disables the affected MMSDFRALINK, or MMPCFRALINK. To clear the alarm, it is
necessary to delete the affected link type from the OMCR and reprovision in an attempt to bring it
back into service.
For details, refer to the Cellular System Administration Guide and System Command Reference.
NOTE
For details, refer to the Cellular System Administration Guide and System Command Reference.
2826750 IP Failure
An internal software failure occurred in the protocol stack on the
SPROC in the onboard communications protocol software.
Perform the procedure in Table 8-15 to resolve this alarm.
2826760 Multicast Failure
An application on the SPROC was unable to join a multicast group.
Perform the procedure in Table 8-15 to resolve this alarm.
2826770 Capacity Greatly Reduced
Call capacity has been greatly reduced
Perform the following procedure to resolve this alarm.
8-18
Action
Check the status and alarms for all SDF boards, and correct problems as indicated for those alarms.
Resolving the SDF problems should restore the capacity.
SDU Troubleshooting
APR 2004
continued
Action
To clear the alarm, it is necessary to lock, delete, and reprovision the BPP board to attempt bringing
it back into service. Refer to the Cellular System Administration Guide and System Command
Reference for details.
Action
The SDU is disabled.
To clear the alarm, regenerate the objects for download. This can be accomplished with the Generate
SDU Data File CLI command at the OMCR. Refer to Table 10-1, step 2.
APR 2004
SDU Troubleshooting
8-19
continued
Action
No action necessary. The system will take the link out of service. The MM will try to reestablish the
link. When the link is reestablished, the alarm will clear.
NOTE
The following link types exist between the SDU and the MM: MMSDFRALINK (for SDU SDF
resource allocation) and MMPCFRALINK (for SDU PCF resource allocation). For details on
configuring and displaying the status of these links, refer to the Cellular System Administration Guide
and/or the System Command Reference.
2
If other links are failing locally, a problem may exist with the network connection or the ISB. In this
case, check the ISB for alarms and check the farend MM for failures.
8-20
SDU Troubleshooting
APR 2004
continued
Action
No action is necessary. The system will take the link out of service. The MM will try to reestablish
the link. When the link is reestablished, the alarm will clear.
NOTE
The following link types exist between the SDU and the MM: MMSDFRALINK (for SDU SDF
resource allocation) and MMPCFRALINK (for SDU PCF resource allocation). For details on
configuring and displaying the status of these links, refer to the Cellular System Administration Guide
and/or the System Command Reference.
2
Check the farend MM for lock or failure status. Refer to the Cellular System Administration Guide
and System Command Reference for details on displaying the status of the MM.
Action
Wait for the alarm to clear as the system automatically attempts to reestablish the SDUPDSNLINK.
The alarm will clear when the SDU reestablishes the SDUPDSNLINK, the call processing link
between the SDU and PDSN.
From the OMCR, you can verify the status of the SDUPDSNLINK.
NOTE
The SDUPDSNLINK is the call processing link between the SDU and the PDSN. Until this link is
reestablished, the SDU cannot assign new calls to the PDSN. For more details on SDUPDSNLINK,
refer to the Cellular System Administration Guide and System Command Reference.
2
APR 2004
SDU Troubleshooting
8-21
continued
Notes
8-22
SDU Troubleshooting
APR 2004
9-1
9-1
9-1
9-1
9-2
APR 2004
SDU Troubleshooting
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
Introduction
Symptom
Any command from the OMCR to the SDU fails.
Possible Cause
The SDU is not in a state where it can synchronize with OMCR (or the
SDU is not connected to the OMCR).
Recovery Action
To verify that the SDU and OMCR are not in sync, type
snmpConnStateShow at the SPROC command line.
IMPORTANT
At the OMCR, initiate the audit command to the SPROC. Refer to the
System Command Reference for details.
SDU fails to send sanity
(heartbeat) message to OMCR
Symptom
The SDU fails to send the sanity (heartbeat) message to the OMCR
every two minutes.
NOTE
The OMCR monitors heartbeat messages to verify that it
is still in contact with the device.
Possible Cause
Connectivity does not exist between the SDU and the OMCR.
Recovery Action
If the OMCR is not receiving heartbeat messages from the SDU,
perform the following procedure.
APR 2004
SDU Troubleshooting
9-1
continued
Action
Log in to the SPROC and enter:
snmpConnStateShow
The command output will indicate whether the SPROC is
communicating with the OMCR.
* IMPORTANT
Logging into the SPROC should be performed during a
scheduled maintenance window to avoid system disruption.
2
Symptom
The OMCR cannot reset the SDU or the particular SDU card.
Possible Cause
The OMCR command does not reach the SDU.
This could be due to a transient error or a connectivity problem.
Recovery Action
Retry the OMCR command.
If it still doesnt work, determine whether there are any connectivity
problems (are there alarms for contact being lost with SDU).
Troubleshoot those issues based on the applicable alarm procedure.
9-2
SDU Troubleshooting
APR 2004
BPP Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Overview of BPP Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10-1
10-1
10-2
10-2
10-3
10-3
10-3
10-4
10
APR 2004
SDU Troubleshooting
Table of Contents
continued
Notes
10
SDU Troubleshooting
APR 2004
BPP Troubleshooting
Overview of BPP
Troubleshooting
10
APR 2004
SDU Troubleshooting
10-1
10
10-2
SDU Troubleshooting
APR 2004
Overview
NOTE
For a more detailed description of each alarm, refer to
System Output Messages. Also, if you are not sure when
you need to resolve the problem, refer to the alarm
documentation (in the System Output Messages) for the
severity level.
Action
SDU Troubleshooting
10
10-3
continued
Action
The system automatically attempts to reset the card. The card will initialize if the automatic reset is
successful.
If the card initializes, no further action is necessary.
If the card fails to initialize, continue at the next step.
NOTE
If the card initializes properly, there should be a state change event indicating that the board has
returned to the inservice state. Also, there should be an attribute value change indicating that the
SDU capacity (either SDF or PCF) has increased by one BPPs worth of capacity.
2
Reset the hardware using the reset switch on the front panel of the specific BPP card.
If the hardware reset fails, lock and replace the card using the procedures contained in the CDMA2000
1X AN & SDU FRU manual.
Action
Verify whether the software load is valid and complete. To do so, check that the current software load
is actually the one intended to be running, and that all of the pertinent software files are present. Refer
to the SDU Software Installation documentation (in the Release Binder) for details.
If the software load is corrupt, replace it with a valid load and codeload the card with the new
software. Refer to the SDU Software Installation documentation (in the Release Binder) for details.
10
10-4
SDU Troubleshooting
APR 2004
continued
Action
The system automatically attempts to reset the card. The card will initialize if the automatic reset is
successful.
If the card initializes, no further action is necessary.
If the card fails to initialize, continue at the next step.
NOTE
If the card initializes properly, there should be a state change event indicating that the board has
returned to the inservice state. Also, there should be an attribute value change indicating that the
SDU capacity (either SDF or PCF) has increased by one BPPs worth of capacity.
2
Reset the hardware using the reset switch on the front panel of the specific BPP card.
If the hardware reset fails, lock and replace the card using the procedures contained in the CDMA2000
1X AN & SDU FRU manual.
Action
NOTE
As this alarm has minimal impact, the reset can be delayed almost indefinitely.
1
10
APR 2004
SDU Troubleshooting
10-5
continued
Action
The system automatically attempts to reset the card. The card will initialize if the automatic reset is
successful.
If the card initializes, no further action is necessary.
If the card fails to initialize, continue at the next step.
NOTE
If the card initializes properly, there should be a state change event indicating that the board has
returned to the inservice state. Also, there should be an attribute value change indicating that the
SDU capacity (either SDF or PCF) has increased by one BPPs worth of capacity.
2
If the reset fails, reset the card using the reset switch on the front panel of the specific BPP card.
If the hardware reset fails, lock and replace the card using the procedures contained in the CDMA2000
1X AN & SDU FRU manual.
Action
The system automatically resets the card. No action is necessary, unless the card fails to initialize. If
the card fails to initialize, attempt a hardware reset using the reset switch on the frontpanel of the
BPP card.
If the hardware reset fails, lock and replace the card. Refer to the CDMA2000 1X AN & SDU FRU for
details.
10
10-6
SDU Troubleshooting
APR 2004
continued
Action
If the additional text in the alarm output indicates that the board should be replaced and returned,
replace the card and return it to Motorola. To replace the card, refer to the CDMA2000 1X AN & SDU
FRU manual.
If the additional text does not indicate that the board should be replaced and returned, reset the card.
If the card fails to reinitialize properly after a reset, or if the alarm continues to occur, the card should
be replaced.
Action
The system automatically attempts to reset the card. The card will initialize if the automatic reset is
successful.
If the card initializes, no further action is necessary.
If the card fails to initialize, continue at the next step.
NOTE
If the card initializes properly, there should be a state change event indicating that the board has
returned to the inservice state. Also, there should be an attribute value change indicating that the
SDU capacity (either SDF or PCF) has increased by one BPPs worth of capacity.
2
If the card fails to initialize, reset the hardware using the reset switch on the front panel of the specific
BPP card.
If the reset fails, or if the problem recurs several times, lock and replace the card using the procedures
contained in the CDMA2000 1X AN & SDU FRU manual.
10
APR 2004
SDU Troubleshooting
10-7
continued
Action
Reset the card at the earliest opportunity.
NOTE
If the card initializes properly, there should be a state change event indicating that the board has
returned to the inservice state. Also, there should be an attribute value change indicating that the
SDU capacity (either SDF or PCF) has increased by one BPPs worth of capacity.
2
If the reset fails, lock and replace the card using the procedures contained in the CDMA2000 1X AN &
SDU FRU manual.
Action
The system automatically resets the card. The card should be replaced if the automatic reset does not
result in a complete recovery.
To replace the card, refer to the CDMA 2000 1X AN & SDU FRU manual.
NOTE
If the card initializes properly, there should be a state change event indicating that the board has
returned to the inservice state. Also, there should be an attribute value change indicating that the
SDU capacity (either SDF or PCF) has increased by one BPPs worth of capacity.
10
10-8
SDU Troubleshooting
APR 2004
continued
Action
Action
If there are any ISBtype alarms occurring concurrently with this particular alarm, address the ISB
alarms first. Resolving the ISB problems should also resolve the BPP problems triggering this alarm.
If there are no ISB alarms, but other cards are also reporting link alarms, the ISB is probably the
source of the problem. Reset the ISB at a convenient time. When the ISB recovers, the BPP alarm
should clear.
If the ISB and other cards appear to be fine, or if resolving the ISB problems does not clear the alarm,
reset the BPP at a convenient time. If the problem remains, replace the BPP.
10
APR 2004
SDU Troubleshooting
10-9
continued
Action
Action
Determine whether the problem is a general SDU system problem or whether it is specific to just one
card.
The trouble is local to the one card if all of the following cases are true:
S
S
S
S
If the problem is specific to the one card, reset the card. If the problem persists, replace the card.
If the problem is not specific to the one card, it is necessary to assess and correct the general
environmental conditions (e.g. fan, airflow) surrounding the SDU frame. When the environmental
problem has been corrected, the alarm will clear.
The environmental conditions must meet the specifications outlined for the SDU frame in the
CDMA2000 1X RAN Hardware Installation manual.
If a card reports high temperature for an extended time and later exhibits unusual behavior, the card
may have been damaged and should be replaced.
10
10-10
SDU Troubleshooting
APR 2004
continued
Action
Follow the procedure in Table 10-15.
Action
If the alarm indicates Waiting for Personality, assign either a PCF or an SDF personality to the
BPP card. Refer to the Cellular System Administration Guide for details on assigning PCF or SDF
functionality to a BPP card.
Otherwise, skip to the next step.
10
APR 2004
SDU Troubleshooting
10-11
continued
Action
The system automatically resets the card. The alarm will clear after the card successfully initializes.
Verify that the software load is valid and complete. To do so, check that the current software load is
actually the one intended to be running, and that all of the pertinent software files are present. If the
software load is corrupt, replace it with a valid load and codeload the card with the new software.
NOTE
Refer to the SDU Software Installation documentation (in the Release Binder) for details.
Action
Physically check the SDU card (BPP or CCA) specified in the alarm output. Determine whether it is
properly connected into the SDU backplane.
Refer to the CDMA2000 1X RAN Hardware Installation manual for details on card placement and
connections.
If it is not a card connectivity problem, then it is necessary to replace the FRU identified in the alarm.
Refer to the CDMA2000 1X AN & SDU FRU manual for details.
10
10-12
SDU Troubleshooting
APR 2004
11
APR 2004
CCA Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Overview of CCA Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11-1
11-1
11-2
11-2
11-3
11-3
11-3
11-4
11-7
11-11
SDU Troubleshooting
11
Table of Contents
continued
Notes
SDU Troubleshooting
APR 2004
CCA Troubleshooting
11
Overview of CCA
Troubleshooting
APR 2004
SDU Troubleshooting
11-1
11
11-2
SDU Troubleshooting
APR 2004
11
Overview
NOTE
For a more detailed description of each alarm, refer to
System Output Messages. Also, if you are not sure when
you need to resolve the problem, refer to the alarm
documentation (in the System Output Messages) for the
severity level.
Troubleshooting CCA
Processing Faults
Action
If this alarm occurs, contact CNRC.
APR 2004
SDU Troubleshooting
11-3
11
continued
Action
If this alarm occurs, contact CNRC.
Troubleshooting CCA
Equipment Faults
Action
Verify that maintenance was being performed on the alarmed device. If not, the device requires
replacement. Refer to the CDMA2000 1X AN & SDU FRU manual.
If maintenance was being performed, either a fan tray or power supply module was removed from the
system. The specified device should be placed back into the system.
NOTE
When a device is removed from the system wait 10 seconds before replacement to allow for software
adjustments needed for EID reread (and other internal adjustments) upon reinsertion.
11-4
SDU Troubleshooting
APR 2004
continued
11
S Equipment malfunction
S Multiplexer problem
S I/O Device failure
Device List: CCA, ISB, SDUFANTRAY, SDUPSMDCDC.
Perform the following procedure to resolve this alarm.
Table 11-4: SDU Equipment Alarm 2823690:
Equipment Fault, Hardware Fault Troubleshooting Procedure
Step
Action
Replace the FRU specified in the alarm output. Refer to the CDMA2000 1X AN & SDU FRU manual
for details.
If this alarm is a fan tray alarm, the fan tray alarm indicates a problem with the fan tray hardware
has occurred. When this alarm condition is detected on any fan all other fan trays in the system will
boost until the alarm condition clears.
Causes/debugging:
S Something is lodged in the fan, the voltage has dropped, or the motor is bad, causing the rotation of
the fan blades to stop or slow down.
S Fan alarms are produced by faulty fan units which should be replaced.
S The automatic boost of the fans may place the rotation speed within the alarm threshold which in
turn will clear the alarm. The fans will then unboost and the alarm will occur again creating a cycle
or bounce condition on the alarm and fan boost.
3
If this alarm is a power supply controller board alarm, this indicates a hardware failure and could be
caused by an overstress condition. Replace the hardware since it is not functional.
APR 2004
SDU Troubleshooting
11-5
11
continued
Action
Replace the FRU specified in the alarm output. Refer to the CDMA2000 1X AN & SDU FRU manual
for details.
S
S
S
S
Action
Check the input power to the frame, and the circuit breaker to the PSM. Trace the fuse to make sure
that it is connected.
11-6
SDU Troubleshooting
APR 2004
continued
11
Troubleshooting CCA
Environmental Faults
Action
Verify that the input voltage is within the specified range for the SDU. (Refer to the CDMA2000 1X
SDU ATP manual for details on the acceptable input voltage range for the SDU).
Carefully check possible causes for this alarm. This alarm is indeterminate and could be caused by
power supply hardware, the AC filter, etc.
If the power supply voltage alarm condition also exists, this may indicate that the power supply
modules have been operating on batteries and the batteries are near being fully discharged.
If the power supply voltage alarm condition does not exist, this alarm indicates an unknown hardware
failure, so replace the power supply module.
Action
Evaluate and correct the power situation by verifying and ensuring that the input voltage is within the
specified range for the SDU.
Refer to the CDMA2000 1X SDU ATP manual for details on the acceptable input voltage range for the
SDU.
APR 2004
SDU Troubleshooting
11-7
11
continued
Action
Evaluate and correct the environmental condition cited by the alarm.
NOTE
Cabinet sensor 1 alarm occurs due to the cabinet / cage door being open.
Cabinet sensors 2 5 have not yet been defined.
Action
Evaluate and correct the situation being alarmed by the customer defined sensors.
NOTE
Since the sensors are customer defined, the troubleshooting action will depend upon the specific
implementation. To troubleshoot this problem it is necessary to know how the sensors were wired for
the particular installation.
11-8
SDU Troubleshooting
APR 2004
continued
11
Action
Correct this problem by cooling the SDU environment or by removing dust from the fan input.
Check the following:
S
S
S
S
NOTE
When working with hardware follow all safety precautions specified in the CDMA2000 1X RAN Data
Hardware Installation and CDMA2000 1X AN & SDU FRU manuals.
APR 2004
SDU Troubleshooting
11-9
11
continued
Action
Evaluate and correct the environmental condition cited by the alarm:
S If this alarm is coupled with fan failure alarms, correcting the fan problem should resolve this issue.
S If this alarm is occurring without any fan failure indications, evaluate and correct the temperature
conditions which may be causing the power supplies to overheat.
2
S
S
S
S
11-10
SDU Troubleshooting
APR 2004
continued
11
Troubleshooting CCA
Communication Faults
Action
Reseat the PSM. This alarm will occur if the PSM is either missing or not seated correctly.
NOTE
Refer to the appropriate documentation for installing/seating the PSM.
2
If you reseated the PSM and the alarm still occurs, contact CNRC.
Action
Physically check the SDU card (BPP or CCA) specified in the alarm output. Determine whether it is
properly connected into the SDU backplane.
Refer to the CDMA2000 1X RAN Hardware Installation manual for details on card placement and
connections.
If it is not a card connectivity problem, then it is necessary to replace the FRU identified in the alarm.
Refer to the CDMA2000 1X AN & SDU FRU manual for details.
APR 2004
SDU Troubleshooting
11-11
11
continued
Notes
11-12
SDU Troubleshooting
APR 2004
APR 2004
SDU Troubleshooting
12-1
12-1
12-1
12
Table of Contents
continued
Notes
12
SDU Troubleshooting
APR 2004
Overview
This section provides a listing of OMCR alarms that apply for multiple
network elements but also apply to the SDU.
Refer to the System Output Messages manual for the alarm description
and operator action.
OMCR Alarms
S
S
S
S
APR 2004
SDU Troubleshooting
12-1
12
continued
Notes
12
12-2
SDU Troubleshooting
APR 2004
13
System Initialization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SPROC Boot Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
OMCR Commissioning Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . .
APR 2004
SDU Troubleshooting
13-1
13-1
13-1
13-2
Table of Contents
continued
Notes
13
SDU Troubleshooting
APR 2004
Introduction
13
SDU Troubleshooting
13-1
continued
13
S Verify that you can ping the SPROC external IP address from the
OMCR.
IMPORTANT
13-2
SDU Troubleshooting
APR 2004
APR 2004
SDU Troubleshooting
14-1
14-1
14-1
14-1
14-1
14-1
14-1
14-2
14-2
14
Table of Contents
continued
Notes
14
SDU Troubleshooting
APR 2004
Introduction
14
The XC PKTIF must receive the cBTS Proxy Request in order to set up
the call on that end. If this doesnt happen, a CFC will occur.
APR 2004
SDU Troubleshooting
14-1
continued
14
The A interfaces related to SDU are A8/A9 (between SDU SDF and
SDU PCF) and A10/A11 (between the SDU PCF and the PDSN).
The following steps must occur or else CFCs will occur and packet data
calls will not be established. In such a case, refer to the troubleshooting
procedure for the applicable CFC.
S
S
S
S
14-2
SDU Troubleshooting
APR 2004
15-1
15-1
15-2
15-5
15
APR 2004
SDU Troubleshooting
Table of Contents
continued
Notes
15
SDU Troubleshooting
APR 2004
Overview
Introduction
This chapter contains CFC troubleshooting procedures for SDU.
NOTE
When displaying status data from the OMCR, it may be
helpful to log the data to a file, especially if you plan on
contacting CNRC.
APR 2004
SDU Troubleshooting
15-1
To view errors and events logged by the BPP, use the spel <option>
command at the BPP command line.
The command format is spel <option> where <option> may be:
S
S
S
S
S
S
S
S
S
S
S
S
15-2
SDU Troubleshooting
APR 2004
S The following commands check for forward and reverse frame rates
(they also display other Mux settings such as Radio Configuration):
for 1X: display servoptcbsc#servopt# so1xcrc
for IS95A/B:
display servoptcbsc#servopt# so95abcrc
Corresponding edit commands are available to correct the settings if
necessary.
IMPORTANT
SdfCpePrintStore
Use the SdfCpePrintStore command to determine whether calls are being
allocated across all CPEs (Channel Processing Elements) on the SDF
card.
This command will also identify the IP addresses of the individual
CPEs.
At the SPROC, enter: SdfCpePrintStore
IMPORTANT
APR 2004
SDU Troubleshooting
15-3
For example:
[SDUSPROC2]> SdfCpePrintStore
***** Printing Cpe Store for SDF ******
//
SDF Cpe Index = 0
status = INS
ipAddress
= a01013d
capacity
= 3200
availableBwu = 3200
usedBwu
= 0
//
//
SDF Cpe Index = 1
status = INS
ipAddress
= a01013e
capacity
= 3200
availableBwu = 3200
usedBwu
= 0
//
15-4
SDU Troubleshooting
APR 2004
Description
Layer 2 Failure Reported by SDU DSP
Layer 2 Failure Reported by SDU L2
RF Layer 2 Failure
Troubleshooting
CFC=4 RF Loss
This CFC is generated when the SDF CPE detects an RF loss condition.
RF Loss is defined by L3CC on the SDF receiving a Traffic Channel
State Change indication for Invalid Speech (erasure) frames from a valid
speech state. This will start the RF Loss Timer, if valid speech frames
are not received before the timer expires, the call will be torn down for
RF Loss.
Causes:
Detailed Cause
0x111A
Description
RF Loss
Troubleshooting:
RF Optimization may be necessary if RF Losses occur often.
SDU Troubleshooting
15-5
Causes:
Detailed Cause
0x111B
0x2005
Description
No TCH Preamble Detected
MccCpT1 Expired
Troubleshooting:
CFC 05 indicates either an RF environment problem or a possible
internal BTS problem. Use the following options to narrow down the
problem.
Problem is due to the RF environment:
S Compare with an SMAP log to get the full picture of what is sent and
received on both ends.
Description
No STRAU Synch
Troubleshooting:
15-6
SDU Troubleshooting
APR 2004
Description
CP Timeout Awaiting MS Acquisition
Troubleshooting:
S If CFC 05 occurs, then CFC 07 will also occur. If this is the case,
instead troubleshoot CFC 05.
S If CFC 05 did not occur, and CFC 07 did occur, then there is probably
a disconnect:
Collect a sniffer log to determine what bearer frames are present.
Collect a sniffer log at multiple points to determine the source of
disconnect; from the Catalyst to the SDU ISB, and the SDU ISB to
the SDU BPP.
Description
SDF Call Setup Timeout
No Vocoder Connect Received at XC
No connect order received
SDF Assignment Timeout. No SDF Resource
Update is received at the SPROC before
Troubleshooting:
S Verify that the SDF Call Setup Timer is shorter than the Vocoder
Connect Timer at the XC before troubleshooting this failure, as this
can be a misleading failure cause.
S Dump some SPEL events on the BPP to see if any errors are being
reported by the SDF.
IMPORTANT
SDU Troubleshooting
15-7
Causes:
Detailed Cause
0x1123
Description
No service option ack received from MS
Troubleshooting:
S Compare with SMAP data to see what was actually sent and received
by the BTS.
S Check the value of the XcdrCkt. ServType in the Servopt database for
the specified service option.
S Check the value of the TchCkt. Serv Type in the Servopt database for
the specified service option.
S Check the value of the Setup Action Time, SDF SN Setup A Timer.
S Determine if the SDU (SDF L3CC) is sending a Service Option
Response Order/Service Connect message. You can use SMAP to
monitor these messages.
S Dump some SPEL events on the BPP to determine if there are any
errors being reported by the BPP:
Check to see if Timer 23 (Sdf Service Connect Complete Timer) is
expiring. This cant be done on a system under call load.
Check that L2 received the Service Option Response Order/Service
Connect message. This cant be done on a system under load.
Check that the BTS received the Scap Service Configuration
Change Request message.
IMPORTANT
15-8
SDU Troubleshooting
APR 2004
Description
Not enough status information received from Mobile
Troubleshooting:
S Also determine whether the BTS received the Status Response using
SMAP.
S Dump some SPEL events on the BPP to determine if there are any
errors being reported by BPP.
IMPORTANT
S Collect a BTS TCH Audit Log to determine if the BTS is getting any
erased frames in the Status Response message.
APR 2004
SDU Troubleshooting
15-9
Description
No mutual service option/configuration
Troubleshooting
S From the Mobile DM tool and SMAP, determine the response of the
Mobile to the Service Request/Response message and whether the
response is received by the BTS.
The Mobile can respond with a Service Response/Request or a MS
Reject order or send no response at all.
S From the Mobile DM tool and SMAP, determine the response of the
Mobile to the Service Connect /Option Response Order message and
whether the BTS receives the response.
S Using SMAP, check to see if the SDU (SDF L3CC) sends out a
Service Connect Message.
S In the servopt database, check the IS95 A/B and 1x Forward and
Reverse frame rates for the service option for which negotiation is
taking place.
15-10
SDU Troubleshooting
APR 2004
Description
No response from SDURA
Troubleshooting:
Collect a sniffer log to verify that the MM is sending resource requests
over SCTP.
Description
No SDF Resources available
Troubleshooting:
S Check the OMCR event logs to verify that the SDFRG (SDF
Resource Group) is not reporting capacity changes.
S Check the resource allocation at the SPROC, is it close to the limit for
the SDFs?
APR 2004
SDU Troubleshooting
15-11
Description
MM RM Link Failure
MM RM Session ID Change
MM RM Deregistration
MM RM ReRegistration Failure
SDU CP Internal Failure
SDU CPE Failure
SDU BPP Failure
PSISIG Failure
PSICE Failure
PSI SDU Failure
Backhaul Bearer Loss
Network Bearer Loss
Troubleshooting:
Description
No PSILINK in MM DB
No SDFRA in MM DB
Troubleshooting:
15-12
SDU Troubleshooting
APR 2004
Description
No PSILINK in MM DB
No SDFRA in MM DB
Troubleshooting:
Description
PCF Pkt Oriented Setup Timer Expired
Troubleshooting:
APR 2004
SDU Troubleshooting
15-13
Description
Protocol error between MS and BS
TA8 Inactivity Timer Expired
Troubleshooting:
Description
MM SDFRA link is OOS
SDU is OOS (no MMSDU connection)
Troubleshooting:
S Verify you can ping the SPROC from the MM. Refer to your IP
planning documentation for the applicable IP addresses.
15-14
SDU Troubleshooting
APR 2004
Description
cBTS Proxy Resource Request Timeout
cBTS Proxy Resource Configure Timeout
cBTS Proxy Resource Mismatch
Troubleshooting:
Description
PDSN Resources Not Available
Troubleshooting:
Description
No PCF Resources available
Troubleshooting:
APR 2004
SDU Troubleshooting
15-15
Description
PDSN Reg Denied Reason Unspecified
PDSN Reg Denied Administratively Prohibited
PDSN Reg Denied Insufficient Resources
PDSN Reg Denied MS Failed Authentication
PDSN Reg Denied Identification Mismatch
PDSN Reg Denied Poorly formed request
PDSN Reg Denied Unknown PDSN Address
Troubleshooting:
15-16
SDU Troubleshooting
APR 2004
*68P09258A80B*
68P09258A80B
Technical
Information
SDU TROUBLESHOOTING
ENGLISH
APR 2004
68P09258A80B
SDU TROUBLESHOOTING
SOFTWARE RELEASE 2.16.3.X
SDU
CDMA2000 1X
APR 2004
ENGLISH
68P09258A80B
APC:
of
Print Vendor:
Date
PAPER:
Body: 70 lb.
Inside Cover: 65 lb. Cougar
Tabs: 110 lb. Index
Binder Cover: Standard TED
cover 10 pt. Carolina
Single Sided
5 Cuts
Clear Mylar
Pantone 2706C
Black Ink
Single Sided
7 Cuts
Clear Mylar
White
Black Ink
FINISHING:
3Ring Binder
SlantD
3Hole Punched
(5/16in. dia.)
Shrink Wrap Body
7X9
8.5x11
11x17
NONSTANDARD SPECIFICATIONS
Tape Bound
Corner Stitch
SPECIAL INSTRUCTIONS