ATN 910&910I&910B&950B V200R003C10 Emergency Maintenance 02 (CLI)

ATN 910&910I&910B&950B Multi-Service Access
Equipment
V200R003C10
Emergency Maintenance
Issue 02
Date 2014-04-30
HUAWEI TECHNOLOGIES CO., LTD.

Copyright © Huawei Technologies Co., Ltd. 2014. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior written
consent of Huawei Technologies Co., Ltd.
Trademarks and Permissions
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and the
customer. All or part of the products, services and features described in this document may not be within the
purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,
and recommendations in this document are provided "AS IS" without warranties, guarantees or representations
of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
Huawei Technologies Co., Ltd.

Address: Huawei Industrial Base
Bantian, Longgang
Shenzhen 518129
People's Republic of China
Website: http://www.huawei.com
Email: support@huawei.com
Issue 02 (2014-04-30) Huawei Proprietary and Confidential i

Copyright © Huawei Technologies Co., Ltd.
Equipment
Emergency Maintenance About This Document
About This Document
Purpose
This document describes how to troubleshoot emergent faults on ATN in aspects of basic
concepts, operation process, and operation guidelines. The last chapter in this manual also
provides a table for recording emergent troubleshooting operations and other related information
for reference in the future.
Related Version
The following table lists the product version related to this document.
Product Name Version
l ATN 910 V200R003C10

l ATN 910I
l ATN 910B
l ATN 950B
Intended Audience
This document is intended for:
l System maintenance engineers
Symbol Conventions
Symbol Description
Indicates an imminently hazardous situation which, if not

avoided, will result in death or serious injury.
Issue 02 (2014-04-30) Huawei Proprietary and Confidential ii

Equipment
Symbol Description
Indicates a potentially hazardous situation which, if not

avoided, could result in death or serious injury.

avoided, may result in minor or moderate injury.

avoided, could result in equipment damage, data loss,
performance deterioration, or unanticipated results.
NOTICE is used to address practices not related to personal
injury.
Calls attention to important information, best practices and

tips.
NOTE is used to address information not related to personal
injury, equipment damage, and environment deterioration.
Command Conventions
Convention Description
Boldface The keywords of a command line are in boldface.
Italic Command arguments are in italics.
[] Items (keywords or arguments) in brackets [ ] are optional.
{ x | y | ... } Optional items are grouped in braces and separated by

vertical bars. One item is selected.
[ x | y | ... ] Optional items are grouped in brackets and separated by

vertical bars. One item is selected or no item is selected.
{ x | y | ... }* Optional items are grouped in braces and separated by

vertical bars. A minimum of one item or a maximum of all
items can be selected.
[ x | y | ... ]* Optional items are grouped in brackets and separated by

vertical bars. Several items or no item can be selected.
Issue 02 (2014-04-30) Huawei Proprietary and Confidential iii

Equipment
GUI Conventions
Convention Description
Boldface Buttons, menus, parameters, tabs, window, and dialog titles

are in boldface. For example, click OK.
> Multi-level menus are in boldface and separated by the ">"

signs. For example, choose File > Create > Folder.
Change History
Updates between document issues are cumulative. Therefore, the latest document issue contains
all updates made in previous issues.
Changes in Issue 02 (2014-04-30)

This document has the following updates:
Known bugs are fixed.
Changes in Issue 01 (2014-01-31)

This document is the first release of the V200R003C10 version.
Issue 02 (2014-04-30) Huawei Proprietary and Confidential iv

Equipment
Emergency Maintenance Contents
Contents
About This Document.....................................................................................................................ii

1 Overview of Emergency Maintenance......................................................................................1
1.1 Definition of Emergency Maintenance...........................................................................................................................2
1.2 Definition of Emergencies..............................................................................................................................................2
1.3 Source of Emergency Maintenance................................................................................................................................2
1.4 Principle of Emergency Maintenance.............................................................................................................................3
1.5 Precautions for Emergency Maintenance.......................................................................................................................3
1.6 Technical Support...........................................................................................................................................................4
2 Flow of Emergency Maintenance...............................................................................................5

2.1 Emergency Maintenance Flow Chart.............................................................................................................................6
2.2 Flow of Emergency Maintenance...................................................................................................................................6
2.2.1 Notifying Huawei of the Emergency...........................................................................................................................6
2.2.2 Collecting Fault Information.......................................................................................................................................7
2.2.3 Handling the Emergency.............................................................................................................................................7
2.2.4 Seeking Help................................................................................................................................................................9
2.2.5 Checking the Handling Result.....................................................................................................................................9
2.2.6 Recording Emergency Maintenance Information........................................................................................................9
3 Emergency Maintenance Guidelines.......................................................................................10

3.1 Guide to Collecting Fault Information.........................................................................................................................11
3.1.1 Collecting Basic Fault Information...........................................................................................................................11
3.1.2 Collecting Device Fault Information.........................................................................................................................11
3.2 Guide to Handling Emergencies...................................................................................................................................12
3.2.1 Failed to Log in to the System Through the Serial Interface.....................................................................................12
3.2.2 Failed to Start the System..........................................................................................................................................15
3.2.3 Status of Hardware Components Is Abnormal..........................................................................................................17
3.2.4 Interface Status Is Abnormal.....................................................................................................................................20
3.3 Guide to Restarting a Device........................................................................................................................................23
3.3.1 Preparations for the Restart.......................................................................................................................................23
3.3.2 Guide to Restarting the ATN.....................................................................................................................................24
3.3.3 Verification After the Restart....................................................................................................................................25
3.3.4 Solution to the Restart Failure...................................................................................................................................26
Issue 02 (2014-04-30) Huawei Proprietary and Confidential v

Equipment
Emergency Maintenance Contents
3.4 Built-in System Software Is Incorrect or Does Not Exist............................................................................................26

3.4.1 Loading the System Software Package in BIOS Mode.............................................................................................26
4 Emergency Maintenance Record Table...................................................................................30

4.1 Emergency Maintenance Notice...................................................................................................................................31
4.2 Emergency Handling Record........................................................................................................................................31
Issue 02 (2014-04-30) Huawei Proprietary and Confidential vi

Equipment
Emergency Maintenance 1 Overview of Emergency Maintenance
1 Overview of Emergency Maintenance
About This Chapter
This chapter describes the definition of emergencies, and the definition, sources, handling
guidelines, and precautions of emergencies maintenance.
1.1 Definition of Emergency Maintenance

This section describes the definition and functions of emergency maintenance.
1.2 Definition of Emergencies

This section describes the definition and classification of emergencies.
1.3 Source of Emergency Maintenance

This section describes in what situations emergency maintenance is to be applied.
1.4 Principle of Emergency Maintenance

This section describes the principles of emergency maintenance.
1.5 Precautions for Emergency Maintenance

This section describes the precautions for emergency maintenance.
1.6 Technical Support

This section describes how to seek Huawei technical support.
Issue 02 (2014-04-30) Huawei Proprietary and Confidential 1

Equipment
1.1 Definition of Emergency Maintenance

This section describes the definition and functions of emergency maintenance.
Emergency maintenance is a rectification method to handle emergencies, such as power-down

or service interruption. With emergency maintenance, you can rapidly rectify the fault and
restore the normal operation of the system and device, and thus minimize the loss.
Emergency maintenance also guides maintenance personnel to take preventive measures to

protect the system from a great amount of service traffic.
This document describes how to apply emergency maintenance to the ATN.
1.2 Definition of Emergencies

This section describes the definition and classification of emergencies.
An emergency is an unexpected fault on a device or a network, which happens suddenly, has

extensive involvement, and has serious consequences on the network security and quality of
service (QoS).
Emergencies that may occur on the ATN can be classified into:
l Abnormal system - all services being interrupted

l Abnormal main processing unit (MPU) - all services being interrupted
l Abnormal service board - partial services being interrupted
l Abnormal link - services on the interface being interrupted
An emergency is an extreme situation, which may be foretold by abnormality alarms and logs.
You can confirm whether an emergency occurs by checking either alarms and logs or a
customer's complaint.
NOTE
This section describes the roadmap for handling emergencies. The roadmap for locating common faults,
refer to the troubleshooting.
1.3 Source of Emergency Maintenance

This section describes in what situations emergency maintenance is to be applied.
An emergency may be caused by a software or hardware failure, an incorrect setting, improper

maintenance operation, maintenance misoperations, a line failure, or a natural disaster.
Emergency maintenance is expected in either of the following situations:
l Customer' complaint
A customer's complaint is the main reason for the application of emergency maintenance.
When a fault reported by a customer or the Customer Service Center conforms to the
conditions in section "1.2 Definition of Emergencies", emergency maintenance needs to
be applied.
l Alarm messages

Equipment
When the alarm messages output by the Network Management System (NMS) or displayed
on the terminal initiate a large scale of service interruption, emergency maintenance needs
to be applied.
l Natural disaster
When a natural disaster such as the earthquake, fire, or flood happens, it is required to
temporarily power off devices and apply emergency maintenance.
1.4 Principle of Emergency Maintenance

This section describes the principles of emergency maintenance.
Emergencies easily cause network access failures of numerous users, device breakdown, and
service interruption, resulting in serious consequences. To improve the efficiency in handling
emergencies and minimize the losses, you must comply with the following guidelines when
maintaining devices:
l To keep the stable running of a device and minimize the occurrence probability of
emergencies, refer to the Routine Maintenance.
l The core function of emergency maintenance is to recover system operation and service
provision as soon as possible. To efficiently handle emergencies, you need to set up schemes
to handle various emergencies according to the emergency maintenance manual. Managers
and maintenance personnel must be well-trained and familiar with these schemes.
l The maintenance personnel must attend the emergency maintenance training to learn
methods of identifying and handling emergencies.
l When an emergency occurs, keep calm and check whether the hardware and route of the
ATN are normal. Then, check whether the emergency is caused by the ATN. If so, handle
the emergency according to the predetermined emergency handling schemes or the
processing procedures in this manual.
l The CF card contains important data. When an emergency occurs, do not format the CF
card before consulting Huawei engineers.
l Contact the Customer Service Center or the local office of Huawei early for technical
support during troubleshooting.
l Once the emergency is solved, collect related alarm information and send the handling
report, device alarm files, and log files to Huawei for analysis. This can help Huawei to
improve the after-sales service.
1.5 Precautions for Emergency Maintenance

This section describes the precautions for emergency maintenance.
To ensure the security of the device and safety of the operators, comply with the following
guidelines.
Static Electricity
Wear an ESD wrist strap before operating a board or the backplane, and comply with the
following rules:

Equipment
l For precautions and procedures of board replacement, see "Replacing Boards" in Parts
Replacement.
l Place a board in an ESD bag before installing it.
l Place a removed board in an ESD bag.
Laser/LED
When you maintain a device with an optical module or optical interface, comply with the
following rules:
l When installing and maintaining the optical fiber, do not look into the optical fiber without
eye protection.
l When replacing the pluggable optical module, do not look into the connector of the optical
fiber without eye protection.
l Only personnel who attend the mandatory training can operate the optical module and
optical fiber on the ATN.
NOTE
When you install and maintain the optical fiber, keep the connector of the optical fiber clean, unfolded,
and straight.
1.6 Technical Support

This section describes how to seek Huawei technical support.
During the equipment operation and maintenance, if a fault occurs and is difficult to locate or
rectify, or if you cannot rectify the fault by referring to the after-sales customer documentation,
contact Huawei for assistance (Huawei engineers will provide guidance remotely or on site on
troubleshooting).
Call local Huawei branches or representative offices or contact Huawei Customer Service
Center.
l Call local Huawei branches or representative offices or contact Huawei Customer Service
Center.
l Contact Huawei Customer Service Center: support@huawei.com.
Maintenance personnel need maintain a detailed record of the emergency maintenance

procedures, notify Huawei of the type of the board to be replaced, and apply for a backup board
according to the warranty articles. This can help Huawei technical support personnel to rectify
the fault sooner. You can fax an Emergency Maintenance Notice to Huawei. For detailed
information about the format, refer to Appendix A "4 Emergency Maintenance Record
Table."

Equipment
Emergency Maintenance 2 Flow of Emergency Maintenance
2 Flow of Emergency Maintenance
About This Chapter
This chapter describes the flow of emergency maintenance.
2.1 Emergency Maintenance Flow Chart

This section describes the flow chart of emergency maintenance.
2.2 Flow of Emergency Maintenance

This section describes the flow of emergency maintenance.

Equipment
2.1 Emergency Maintenance Flow Chart

This section describes the flow chart of emergency maintenance.
NOTE
l During troubleshooting, maintain detailed records of operation procedures and results, which can
provide reference for Huawei technical support personnel and thus handle the emergency sooner.
l When a fault persists, contact Huawei Customer Service Center. For contact information, see section
"1.6 Technical Support."
The main purpose of emergency maintenance is to recover a system operation and service
provision as soon as possible. Figure 2-1 shows the flow chart of emergency maintenance.
Figure 2-1 Emergency maintenance flow chart

Start
Notify Huawei of the

Emergency
Collect fault
information
Locate the fault
Rectify the fault
No
Service recovery ? Obtain help
Yes
Check the handling
result
Record information
about emergency
maintenance
End
2.2 Flow of Emergency Maintenance

This section describes the flow of emergency maintenance.
2.2.1 Notifying Huawei of the Emergency

Equipment
When an emergency occurs, immediately contact Huawei for technical support.
NOTE
Even if you can independently complete emergency maintenance with the guidance of this manual, notify
Huawei of the emergency. Then, Huawei technical personnel maintain records of the fault to improve after-
sales services.
2.2.2 Collecting Fault Information

When an emergency occurs, collect and back up fault information in real time for reference.
Then, provide the collected information to Huawei engineers for fault location and rectification.
For details about fault information collection, see the chapter "3.1 Guide to Collecting Fault
Information".
Recording Basic Fault Information

Collect the following basic information:
l Time when the fault occurs

l Detailed description of the fault
l Software version of the ATN
l Measures have been taken and the results
l Severity level of the problem and expected time of system recovery
Backing Up Fault Information

Back up the following information:
l Indicator status of the boards, power modules, and fans

l Alarm information
l Log information
l Configuration information
l Debugging information (if debugging is enabled)
2.2.3 Handling the Emergency

During emergency maintenance, follow the flow chart as shown in Figure 2-2 to verify the type
and range of the fault according to the method as shown in Table 2-1.

Equipment
Figure 2-2 Flow chart of verifying the fault type
Start
Can log in No
through the console
Interface?
Yes
System starts No
normally?
Yes
Board status is No
normal?
Yes
Interface status No
is normal?
Yes
A service fault occurs A device fault occurs
Table 2-1 Methods of verifying the fault type
Item Method
Login through Connect the serial interface of a PC or terminal to the console interface of
the console the ATN and set relevant parameters on the terminal. For details, refer to
interface the ATN Multi-service Access Equipment Configuration Guide - Basic
Configurations.
Check that a terminal display is provided.
System startup Check whether the system can be started normally and the command
prompt such as <HUAWEI> is displayed.
Board status Run the display device command on the terminal to check whether the
status of all boards is Normal. In the case of a local fault, check the status
of the service board connected to the customer who reports the fault.
Interface status Run the display interface command on the terminal to check whether the
status of the interface connected to the customer who reports the fault is
Up and whether the number of packets received on the interface increases.

Equipment
After the fault type is verified, apply emergency maintenance according to the description in the
Chapter "3.2 Guide to Handling Emergencies".
l 3.2.1 Failed to Log in to the System Through the Serial Interface

l 3.2.2 Failed to Start the System
l 3.2.3 Status of Hardware Components Is Abnormal
l 3.2.4 Interface Status Is Abnormal
2.2.4 Seeking Help

Seek Huawei technical support according to the contact information given in section "1.6
Technical Support."
2.2.5 Checking the Handling Result

After services resume, check the device status, board indicators, and alarms to confirm that the
system runs normally. Make a dialing test to prove that services are normal. For detailed
operations, refer to the Routine Maintenance.
You are recommended to arrange technical personnel to monitor the system running during the
service peak time so that further problems can be handled immediately.
2.2.6 Recording Emergency Maintenance Information

Record emergency maintenance information, including the emergency maintenance time,
version, fault symptom, handling procedures, and result, for a future query. For the format of
the maintenance record information table, refer to Appendix A "4 Emergency Maintenance
Record Table."

Equipment
Emergency Maintenance 3 Emergency Maintenance Guidelines
3 Emergency Maintenance Guidelines
About This Chapter
This chapter guides you how to perform emergency maintenance.
3.1 Guide to Collecting Fault Information

This section guides you how to collect and back up fault information as the reference for
troubleshooting.
3.2 Guide to Handling Emergencies

This section guides you how to handle emergencies.
3.3 Guide to Restarting a Device

This section guides you how to restart a device when a fault on the device causes services to be
interrupted and the device fails to restart automatically.
3.4 Built-in System Software Is Incorrect or Does Not Exist

Incorrect or no system software was delivered along with the device. You need to load the system
software through the BIOS.

Equipment
3.1 Guide to Collecting Fault Information

This section guides you how to collect and back up fault information as the reference for
troubleshooting.
When an emergency occurs, you need collect and back up fault information on time for reference.
When seeking Huawei technical support, you need provide the collected information to Huawei
engineers for fault location and rectification.
When a fault occurs, collect the following information:
l Basic fault information
l Device fault information
3.1.1 Collecting Basic Fault Information

This section describes what items need to be collected and how to collect basic fault information.
Table 3-1 lists the basic fault information to be collected when a fault occurs.
Table 3-1 Basic fault information table
No. Item Collection Method
1 Occurrence time Record the time when the fault occurs. The value should
be accurate to a minute.
2 Fault symptom Collect the fault symptom and maintain detailed records.
3 Fault severity level Record the fault severity level according to the range and
the severity of the fault.
4 Software version Collect information about the software version on console

through the display version command when you can log
in to the device through Telnet or the serial interface.
5 Networking Draw a networking diagram showing upstream and

information downstream devices and the peer interfaces.
6 Taken measures Record the measures that have been taken and the results.
NOTE
When collecting fault information through command lines, you can copy the information displayed on the
console, such as the serial interface or the Telnet terminal, and then attach it to a txt. file for a record.
3.1.2 Collecting Device Fault Information

This section describes what items need to be collected and how to collect the device fault
information.
When a fault occurs and you can log in to the device through Telnet or the serial interface, collect
the following information.

Equipment
Table 3-2 Fault information table
1 Device information Run the display device command to collect the device
information.
2 Temperature Run the display temperature command to collect the

temperature information.
3 CPU usage Run the display cpu-usage command to collect the CPU
usage information.
4 Routing table Run the display ip routing-table command to collect the

information routing table information.
5 Log information Run the display logbuffer command to collect the log
information.
6 Alarm information Run the display trapbuffer command to collect the alarm
information.
7 Configuration Run the display current-configuration command to

information collect the configuration information.
8 Diagnostic Run the display diagnostic-information file-name

information command to collect the diagnostic information.
NOTE
By default, running the display diagnostic-information file-
name command saves the device's diagnostic information to the
file with a specified filename in Cfcard:/.
9 Interface information Run the display interface command to collect the

interface information.
10 Network connectivity Run the ping command to collect information about the
information network connectivity and record the results.
NOTE
When a device runs normally, you are recommended to back up the historical traps and logs in the CF card
through the Trivial File Transfer Protocol (TFTP) or File Transfer Protocol (FTP).
3.2 Guide to Handling Emergencies

This section guides you how to handle emergencies.
3.2.1 Failed to Log in to the System Through the Serial Interface

Equipment
Fault Symptom
After the serial interface of a PC or a terminal is connected to the console interface of the
ATN and the relevant parameters are set, nothing is displayed on the terminal.
Fault Information Collection

If you are unable to log in to the system through the serial interface, collect the following
information besides the generic information described in the Chapter "3.1 Guide to Collecting
Fault Information" for future reference.
1 Communications Check whether the communication parameters set for the

parameters of the software such as Windows-based HyperTerminal for the
serial interface communication between serial interfaces are consistent
with the communication parameters set on the console
port of the ATN, such as the bard rate, data bit, parity
check or not, stop bit, and flow control or not.
2 Indicator status Check the status of the following indicators:

l STAT and ACT indicators of the main board
l PWR indicators of the PIU
l FUN indicators of the fans

Equipment
Processing Procedure
Figure 3-1 Flowchart of solving the problem that users cannot log in to a system through the
serial interface
Start
Does No Repair the Yes

Is the fault
the power module power supply
rectified?
work normally? system
Yes No
No Yes
Is the cable in Replace the Is the fault
good condition? cable rectified?
Yes No
Are No
Is the fault Yes
parameters for the Modify the
serial interface parameters rectified?
correct?
Yes No
Does No Exchange or
Is the fault Yes
the MPU/SRU work replace the
rectified?
normally? MPU/SRU
Yes No
Is the fault Yes

Reset the system
rectified?
Seek technical
End
support
NOTICE
All the following steps are performed only when the customer's services are already interrupted,
and therefore have no adverse effect on services. If the customer's services are not interrupted,
collect fault information and provide it to Huawei engineers for further processing.

Equipment
Procedure
Step 1 Check and repair the power supply system.
When you find that the indicators of all the boards are off and all the fans fail to work (which
can be identified by fan's rotating), or the indicator of the power module is abnormal, the power
supply system of the device is possibly faulty and need repairing. The power supply system
consists of the following:
l Power supply system of the equipment room, chassis, or cabinet

l Power module
l Power supply system of the backplane
You can solve the problem according to the steps:
1. Check whether the power module is switched on. When there are multiple power modules,
ensure that at least one works normally.
2. When none of the preceding problems is found, but the power supply system fails to work,
seek Huawei technical support according to 1.6 Technical Support.
Step 2 Check and replace the cable.

Check whether the cable is in good condition. You can replace the cable with a new one to check
that you can normally log in.
Step 3 Check parameters of the serial interface.
Check whether parameters set for the serial interface are identical with those for the console
interface on the ATN. If they are not identical, modify the parameters of the serial interface.
Step 4 Exchange and replace the MPU.

If the serial interface, cable, power supply system are normal, the problem may be caused by
the fault on the MPU.
Step 5 Exchange and replace the system control board.

If the serial interface, cable, power supply system are normal, the problem may be caused by
the fault on the system control board. When two system control boards are installed in the system,
remove the master system control board and insert the terminal of the cable into the console port
of the slave system control board. When there is only one system control board, you can replace
it with a spare one.
Step 6 Reset the system.

If the fault persists after the preceding steps are performed, you can reset the system by switching
off the power module and then switching it on three minutes later to solve the problem. For
details, see 3.3 Guide to Restarting a Device.
Step 7 Seek Huawei technical support.

If the fault persists after the preceding steps are performed, you can seek Huawei technical
support according to "1.6 Technical Support."
----End
3.2.2 Failed to Start the System

Equipment
Fault Symptom
The system is failed to be started and the prompt may be displayed on the terminal as follows:
l "XXXXX selftest.........FAIL!", which indicates that the self-test of a certain module fails.
l The system remains in the phase of file decompression for a long time.
l The system is repeatedly restarted.

If you are unable to start the system, collect the following information besides the generic
information described in the chapter "3.1 Guide to Collecting Fault Information" for future
reference.

1 System startup Copy system startup information on the console such as

information the serial interface or Telnet terminal to a .txt file for
storage.
2 Name of the startup Check the name of the startup file through the Basic Input/
file Output System (BIOS) menu.
Figure 3-2 Flow chart of solving the problem that the system cannot be started
Start
Is the No Repair the Yes

Is the fault
power module work power supply
rectified?
normally? system
No
Yes
Is the Yes Debug and Yes

Is the fault
module self-check replace the
rectified?
failed? MPU/SRU
No
No
Is the file Yes Re-upload the Yes

Is the fault
decompression startup file
rectified?
failed? through BIOS
No No
Make the
Is the Yes startup files of
the master and Is the fault Yes
system repeatedly
slave system rectified?
restarted?
control board No
identical
No
Seek technical
End
support

Equipment
NOTICE
do not perform the following steps. Instead, collect fault information and feed it back to Huawei
engineers for further processing.
Procedure
Step 1 Check and repair the power supply system.
When you find that the indicators of all the boards are off and all the fans fail to work (which
can be identified by fan's rotating), or the indicator of the power module is abnormal, the power
supply system of the device is possibly faulty and need repairing. The power supply system
consists of the following:

l Power module
Step 2 Upload the startup file through BIOS again.

When the system remains in the phase of file decompression or is repeatedly restarted, the startup
file is possibly incorrect or damaged. You can try to upload the startup file through BIOS.
It is complicated to upload the startup file through BIOS. Contact Huawei technical support
personnel and perform the uploading under their guidance. For detailed operation procedures,
refer to Appendix B "3.4 Built-in System Software Is Incorrect or Does Not Exist."

Seek Huawei technical support according to "1.6 Technical Support."
----End
3.2.3 Status of Hardware Components Is Abnormal
Fault Symptom
Hardware components refer to hardware modules including board modules, power supply
modules, and fan modules. The abnormality of hardware component status includes (one or
multiple items):

Equipment
l When you run the display device command in any view to view information about a
hardware component where services are interrupted, the hardware component status is
Abnormal.
l When you run the display device command in any view to view information about a
hardware component where services are interrupted, the hardware component status is
Unregistered.
l The RUN or STATUS indicator of a hardware component blinks or is off, or the ALM
indicator of the hardware component is on.

When you find that the status of the hardware component is abnormal, collect the following
information besides the generic information described in the Chapter "3.1 Guide to Collecting
Fault Information" for future reference.

1 Indicator status of a Check that the status of the indicator on a hardware

hardware component component is red or green.
2 Detailed information Run the display device slot-id command in any view to
about a hardware view detailed information about the specified hardware
component component.
3 Status of a PIC Run the display device pic-status command in any view
channel to collect information about the status of a PIC channel.
Figure 3-3 Flow chart of solving the problem that the hardware component status is abnormal
Start
Reset the component
Yes Is the fault

rectified?
No
Replace the component
No Cut over the services

Is the fault
on the board and seek
rectified?
technical support
Yes
End

Equipment
NOTICE
Procedure
Step 1 Reset the hardware component.
The processing of the hardware component abnormality is complex.

l Power module
If the abnormality occurs on the fan modules, directly replace the fan modules.
If a board is abnormal and the situation is urgent, reset and replace the board. Relevant cause
location will be performed by Huawei technical support personnel.
You can reset a board by using the reset slot slot-id command in the user view, pressing the
RESET button on the panel, or pulling out and inserting the board.
NOTE
You are recommended not to pull out and insert in the board for resetting. This can avoid damages on the
board.
Step 2 Replace the hardware component.

When the problem cannot be solved by resetting the hardware component, you can try to replace
the hardware component with a spare one.
Step 3 Cut over services on the board and seek Huawei technical support.
When the preceding methods cannot solve the problem, you can cut over services on the faulty
board to a normal board or a board in an vacant slot. For operation details, contact Huawei
technical support personnel or comply with your cutover scheme.
In addition, report the fault information to the local Huawei office for technical support.
----End

Equipment
3.2.4 Interface Status Is Abnormal
Fault Symptom
The abnormality of the interface status includes:
l When the display interface [ interface-type interface-number ] command is run in any
view to check an interface where services are interrupted, the interface status is Down.
view to check an interface where services are interrupted, the number of packets transmitted
on the interface remains unchanged.
view to check the interface where services are interrupted, a large number of CRC packets
are received.
l The indicator status of an interface is abnormal. For example, the LINK indicator of the
interface is off.

When you find the interface status is abnormal, collect the following information besides the
generic information described in the chapter "3.1 Guide to Collecting Fault Information" for
future reference.
1 Indicator status of an Check whether the indicator (LINK/ACT) on a faulty

interface interface is off, on, or blinking. If the indicator is on, the
link is Up. If the indicator is blinking, the data is being
received or transmitted. If the indicator is off, the link is
Down.
2 Detailed information Run the display interface [ interface-type interface-

about an interface number ] command in any view to collect detailed
information about an interface.
3 Brief IP-related Run the display ip interface brief [ interface-type

information about an interface-number ] command in any view to collect brief
interface IP-related information about an interface.
4 Brief information Run the display interface brief command in any view to
about all interfaces collect brief information about all interfaces.

Equipment
Figure 3-4 Flow chart of solving the problem that the interface status is abnormal
Start
Status
of interface Proceed to the
Yes
indicator flow for handling
normal? service faults
No
Interface No Yes
status Is manually Shut up the
shut down? interface
is Up?
No
Yes Yes
Fault
Detect the link End
rectified?
Packets are No Perform Cut over

transeived a local No
theservices
normally? loopback test on the
board and
Yes
Check and modify the seek board
Reset the No Is the status Yes configuration of the and seek
interface normal? data link layer or the support
upper layer protocol
Fault No
rectified?
Yes
End
NOTICE

Equipment
NOTE
Usually, the interface fault is caused by the problem of the cable or optical module.
l When the cable is broken or the optical module is damaged, the interface fails to go Up.
l When the cable or optical module on an interface has been being used for many years, the signal
attenuation may be severe. In this case, although the interface is Up, a large number of packets are
discarded.
Replace the cable or optical module on the faulty interface. If the problem persists, perform the following
operations.
Procedure
Step 1 Start the interface manually.
Run the display this in the interface view to check the configuration files of the faulty interface.
If you find that an interface is shut down through the shutdown command, run the undo
shutdown command in the interface view to start it.
Step 2 Check the link and rectify the link fault.

Before checking a link, check whether the LINK indicator of the interface is on.
If so, it indicates that the physical link is Up and you can detect the link as follows:
1. Run the display this interface command in the interface view to check whether the
interface parameters at both ends of the link are identical, such as the duplex mode and rate.
2. In the case of optical interfaces, use the optical power meter to check whether the receiving
and sending optical signals at both ends are normal. If it is not convenient to use the optical
power meter, you can use the optical power detecting function in the optical module: run
display this interface in the error interface and compare the parameters in the display
information with the optical module parameters, and check whether the power of the send
or receive optical signals are in normal range. If optical interfaces only send or receive
optical signals, the optical module is possibly faulty or the optical fiber does not match the
optical module. Then, you can try to replace the optical module or the optical fiber.
3. If the interface is an electric interface, observe the pinouts in the RJ-45 connectors on both
ends of the case to check whether the cable should be straight-through or crossover cable
for the specified interface.
DANGER
When you check the receiving and sending of optical signals, do not look into the optical fiber
without eye protection. You must use the optical power meter to measure the optical power.
When the LINK indicator of the interface is off, you can check the link as follows:
1. Perform a physical loopback test on the device. That is, connect the faulty interface to an
interface is in the normal state through an optical fiber or cable in good condition. Pay
attention to the two interfaces' type should match with each other.
2. If the LINK indicator is on, it indicates that the interface is normally. In this situation, you
need to check whether the optical fiber or cable is damaged and the trunk link runs normally.
Usually, you need to check the optical fibers, cables, and trunk links at the neighboring
sites.

Equipment
3. If the LINK indicator is off, it indicates that the interface hardware is faulty. When a
pluggable optical module is used, you can replace the optical module; otherwise, cut over
services on the faulty interface to other interfaces in the normal state.
Step 3 Check and modify the data link layer or upper layer protocol.
If the interface still fails to send and receive packets in the local loopback test, check the data
link layer or upper layer protocol. For example, check the Point-to-Point Protocol (PPP) protocol
at both ends is identical and the routing protocol runs normally.
Step 4 Reset the interface.

If the fault persists after the preceding steps are performed, you can reset the interface to solve
the problem.
Run the shutdown command to disable the interface, and then run the undo shutdown command
to enable the interface to reset the interface.

Seek Huawei technical support according to 1.6 Technical Support.
----End
3.3 Guide to Restarting a Device

This section guides you how to restart a device when a fault on the device causes services to be
interrupted and the device fails to restart automatically.
NOTICE
l Restart the ATN with caution. If it is required to restart the ATN, go over the principles and
precautions described in Chapter 1, or perform the restart operation under the guidance of
Huawei technical personnel.
l Before the ATN is successfully restarted, all services on the ATN are interrupted unless the
dual-system hot backup networking is adopted.
When a critical fault occurs on the ATN during the equipment running, the ATN is automatically
restarted. After the restart, the ATN runs normally. The ATN needs to be restarted manually
only in emergency or exception, for example, services are interrupted because of the fault
occurred on the ATN, and the ATN fails to automatically restart or recover by using other
methods.
3.3.1 Preparations for the Restart

This section describes the preparations for the restart of the ATN.
Before restarting the ATN, confirm whether configuration files of the ATN need to be backed
up. Configuration files should be backed up and executed automatically after the restart. In this
case, services can be automatically resumed.

Equipment
3.3.2 Guide to Restarting the ATN

This section describes procedures and precautions of restarting the ATN.
You can restart the ATN in any of the following methods:

l Running the reboot command
l Running the schedule reboot command
l Operating through the NMS
NOTE
You are recommended not to restart the ATN remotely. Otherwise, once the restart operation fails, services
may be interrupted for a long period.
Checking Whether Configuration Files Are Correct

Run the display startup command to check whether configuration files are correct. For example:
<HUAWEI> display startup
MainBoard:
Configured startup system software: cfcard:/V200R003C10.cc
Startup system software: cfcard:/V200R003C10.cc
Next startup system software: cfcard:/V200R003C10.cc
Startup saved-configuration file: cfcard:/vrpcfg.cfg
Next startup saved-configuration file: cfcard:/vrpcfg.cfg
Startup paf file: cfcard:/paf-V200R003C10.txt
Next startup paf file: cfcard:/paf-V200R003C10.txt
Startup license file: cfcard:/license-V200R003C10.txt
Next startup license file: cfcard:/license-V200R003C10.txt
Startup patch package: cfcard:/patch.bat
Next startup patch package: cfcard:/patch.bat
Running the reboot Command

Before manually restarting the ATN, run the reboot command on the configuration terminal.
Enter the reboot command in the user view and press Y after the display to restart the ATN.
The operation example is as follows:
<HUAWEI> reboot
mpu 2:
Paf: V200R003C10
License: V200R003C10
Next startup saved-configuration file: cfcard:/vrpcfg.zip
Info: The system is now comparing the configuration, please wait.

Info: If want to reboot with saving diagnostic information, input 'N' and then e
xecute 'reboot save diagnostic-information'.
System will reboot! Continue?[Y/N]:
NOTE
The reboot command output varies with system versions. Take the command output of the current system
version as the standard.
Running the schedule reboot Command

The schedule reboot command is run in the user view and has two modes, namely, schedule
reboot delay and schedule reboot at.

Equipment
l The schedule reboot delay command is used to enable the scheduled reboot function and
set the wait delay for the ATN.
The wait delay set for the scheduled restart of the ATN can be expressed in two formats:
"hour: minute" and "absolute minutes". The total minutes cannot be greater than 30 x 24 x
60 minutes.
l The schedule reboot at command is used to enable the scheduled restart function and set
the specific restart date and time for the ATN. Note that the specified date cannot be 30
days later than the current date.
If the schedule reboot at command sets a specific date (yyyy/mm/dd) and the date is a
future date, the ATN is restarted at the set time and the error is within 1 minute. If no specific
date is set, the following situations occur:
– If the set time is later than the current time, the ATN is restarted at this time that day.
– If the set time is earlier than the current time, the ATN is restarted at this time the next
day.
After the schedule reboot delay or schedule reboot at command is run, the system prompts
you to confirm the restart. Enter Y or y, and the configuration takes effect. If the related
configuration exists, the latest configuration overrides the previous one.
NOTE
If you adjust the system time through the clock command after running the schedule reboot delay or
schedule reboot at command, the parameter set through the schedule reboot delay or schedule reboot
at command becomes invalid.
You can run the undo schedule reboot command to remove the parameter set through the
schedule reboot delay or schedule reboot at command.
You can run the display schedule reboot command to view the parameter set through the
schedule reboot delay or schedule reboot at command.
Operating Through the NMS

The operation for restarting the ATN differs for different type of NMS. For detail, refer to the
NMS Online Help.
3.3.3 Verification After the Restart

This section describes how to verify the device after the restart.
NOTICE
After the ATN is restarted, check that the configuration data is recovered correctly and
completely. The loss of configuration data will result in the service interruption and you are
therefore required to manually add the configuration data and save it.
Verifying the Restart Information

If you log in to the ATN through the console interface and restart the ATN by running the
reboot command, the display on the configuration terminal after the restart is as follows:

Equipment
The preceding display shows that the device is restarted successfully. You can press Enter and
enter the user view.
Checking Whether the Software Version Is Correct

Run the display version command to check whether the software version is correct. For example:
<HUAWEI> display version
Huawei Versatile Routing Platform Software
VRP (R) software, Version VRP 5.150 (ATN
V200R003C10)
Copyright (C) 2000-2010 Huawei Technologies Co., Ltd
HUAWEI ATN910 uptime is 0 day, 17 hours, 40 minutes
......
The preceding command output shows the Versatile Routing Platform (VRP) version, host
version, and patch version. You can check whether the version number before and after the
restart is identical.
Checking Whether Configuration Files Are Correct

Run the display startup command to check whether configuration files are correct. For example:
<HUAWEI> display startup
MainBoard:
Configed startup system software: cfcard:/V200R003C10.cc
Startup system software: cfcard:/V200R003C10.cc
Startup saved-configuration file: cfcard:/vrpcfg.cfg
Next startup saved-configuration file: cfcard:/vrpcfg.cfg
Startup paf file: cfcard:/paf-V200R003C10.txt
Next startup paf file: cfcard:/paf-V200R003C10.txt
Startup license file: cfcard:/license-
V200R003C10.txt
Next startup license file: cfcard:/license-
V200R003C10.txt
Startup patch package: cfcard:/patch.bat
Next startup patch package: cfcard:/
patch.bat
3.3.4 Solution to the Restart Failure

This section describes how to handle the failure in the restart of the ATN.
If the system fails to be upgraded, to ensure that the ATN operates normally, you need to roll
back the version. For the rollback procedures, see the Upgrade Guide.
If any problem arises during the restart of the ATN, contact the local Huawei technical support
personnel.
3.4 Built-in System Software Is Incorrect or Does Not Exist

Incorrect or no system software was delivered along with the device. You need to load the system
software through the BIOS.
3.4.1 Loading the System Software Package in BIOS Mode

This section describes how to load the ATN system software package in BIOS mode to upgrade
the system if the package cannot be loaded in common ways.

Equipment
Context
When you load the system software package in BIOS mode, only FTP is allowed and the
operation terminal (a PC) must be connected to the device through a serial port, network port,
or ATN. The PC and device communicate with each other using hyper terminal.
NOTE
In this section, the active and standby system control boards refer to the ones that are working in the system
before the software package is loaded. After the system software package is loaded, the active/standby
status of the system control boards will change.
For ATN, the system software package must be separately loaded to the active and standby system control
boards. First load the software package to the active system control board and in the meanwhile remove
the standby system control board. Then load the software package to the standby system control board in
the same way as you load it to the active system control board.
Procedure
Step 1 Connect the system control board's console port to the PC's COM port and configure the hyper
terminal.
The FTP server and operation terminal can be the same PC.
The following uses an example of configuring the hyper terminal on Windows XP to illustrate
how to configure hyper terminal.
1. Run Windows XP and choose Start > Accessories > Communications > Hyper
Terminal. In the window that is displayed, enter a name in the Name field.
Figure 3-5 Hyper terminal operation GUI 1
2. Click OK. The following window is displayed. In the window, select a COM port.

Equipment
3. Click OK. The following window is displayed. In the window, set B to 38400 and retain
the default settings for other parameters.
Click OK. Configuring the hyper terminal is now complete.
Step 2 Run the FTP server on the PC and create an FTP user.
NOTE
The FTP setting display depends on the FTP software.
Set the FTP server parameters, including the home directory, user name, and password.

Equipment
Step 3 Run the reboot command to restart ATN. The device startup information is displayed in the
hyper terminal window. The device startup information includes [LAN2]TCP Server Recv
Task Begin! and Boot pkg check begin.... If [DMM] Beat Timer Proc Begin! is displayed,
the system control board has successfully entered the BIOS state.
NOTE
If the system control board fails to enter the BIOS state, contact Huawei technical support engineers.
Step 4 When [DMM] Beat Timer Proc Begin! is displayed, press Enter and enter devs to check
directories. Ensure that there is a cfcard: directory.
drv name 0 /null 1 /tyCo/0 1 /tyCo/1 5 bootHost: 8 /vio 3 cfcard: 3 ofs1 3 ofs2 3
mfs value = 25 = 0x19
Step 5 Run the ifShow "qefcc2" command to obtain the NE IP address.

qefcc (unit number 2): Flags: (0x8063) UP BROADCAST MULTICAST ARP RUNNING Type:
ETHERNET_CSMACD Internet address: 129.10.6.32 Broadcast address: 129.9.255.255
Netmask 0xffff0000 Subnetmask 0xffff0000 Ethernet address is 28:6e:d4:cb:a7:97
Metric is 0 Maximum Transfer Unit size is 1500 0 octets received 202 octets sent 0
packets received 5 packets sent 0 non-unicast packets received 2 non-unicast
packets sent 0 unicast packets received 3 unicast packets sent 0 input discards 0
input unknown protocols 0 input errors 0 output errors 0 collisions; 0 dropped
value = 1 = 0x1
In the preceding information, "Internet address: 129.10.6.32" refers to the IP address of the ATN
NE. The ATN NE's IP address and PC's IP address must be in the same network segment. If
they are in different network segments, a login to the ATN NE will fail.
Step 6 On Windows XP, choose Start > Run, enter the FTP address (for example, ftp://129.10.6.32),
and click OK to log in to the FTP server.
Step 7 Copy the bcf.txt, configuration file, and system software that are stored on the PC to the
cfcard: directory by FTP. After that, shut down FTP.
Step 8 Run the reboot command to restart the device.
If the following message is displayed, the system has successfully started up.
Recover configuration...OK!
Press ENTER to get started.
<HUAWEI>
For a device with a single system control board , loading the software package is completed after
the startup. Then, go to next step.
For a device with two system control boards, remove the active system control board and insert
the standby system control board. Repeat the preceding steps to load the software package to
the standby system control board.
Step 9 Verify the upgrade.

<HUAWEI> check startup crc
Warning: This operation will take several minutes! Continue?[Y/N]:y
Check startup software CRC
................
Info: ok
If "ok" is displayed, the upgrade is successful. If other information is displayed, contact Huawei
technical support engineers.
----End

Equipment
Emergency Maintenance 4 Emergency Maintenance Record Table
4 Emergency Maintenance Record Table
About This Chapter
This chapter describes how to fill up the emergency maintenance record table.
4.1 Emergency Maintenance Notice
4.2 Emergency Handling Record

Equipment
4.1 Emergency Maintenance Notice

If your maintenance personnel cannot remove a fault through emergency maintenance, please
call or fax Huawei technical support personnel for help. The maintenance personnel need
maintain a detailed record of the emergency maintenance procedures, notify Huawei of the type
of the board to be replaced, and apply for a spare one according to the warranty articles. This
help Huawei technical support personnel rectify the fault sooner.
You can fax an emergency maintenance notice to Huawei. The format of the notice is shown in
Table 4-1:
Table 4-1 Emergency maintenance notice
* Filled by the customer
Telecom Device Capacity

office model
Customer Phone Version

number
Complaint Required In warranty Yes ¨ No

time response
time
Fault description and handling procedure (in detail):

Approved by: Signature:
* Filled by Huawei engineers
Handling ¨ Guide in call ¨ Remote maintenance ¨ On-site support

method
Result (attachment):
Handled by: Date:
Known anomalies:
4.2 Emergency Handling Record

Emergency handling record
Site:__________________Date: MM/DD/YY_________________.
Emergency Emergency
occured at handled at

Equipment
Person on Emergency
duty handled by
Fault source Customer Basic information:

complaint
Routine
maintenance
Alarms
Other sources
Fault symptom:
Solution and result:


ATN 910&910I&910B&950B V200R003C10 Emergency Maintenance 02 (CLI)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ATN 910&910I&910B&950B V200R003C10 Emergency Maintenance 02 (CLI)

Uploaded by

Copyright:

Available Formats

ATN 910&910I&910B&950B Multi-Service Access

HUAWEI TECHNOLOGIES CO., LTD.

Trademarks and Permissions

Huawei Technologies Co., Ltd.

Issue 02 (2014-04-30) Huawei Proprietary and Confidential i

About This Document

Product Name Version

l ATN 910 V200R003C10

l System maintenance engineers

Indicates an imminently hazardous situation which, if not

Issue 02 (2014-04-30) Huawei Proprietary and Confidential ii

Indicates a potentially hazardous situation which, if not

Indicates a potentially hazardous situation which, if not

Indicates a potentially hazardous situation which, if not

Calls attention to important information, best practices and

Boldface The keywords of a command line are in boldface.

Italic Command arguments are in italics.

[] Items (keywords or arguments) in brackets [ ] are optional.

{ x | y | ... } Optional items are grouped in braces and separated by

[ x | y | ... ] Optional items are grouped in brackets and separated by

{ x | y | ... }* Optional items are grouped in braces and separated by

[ x | y | ... ]* Optional items are grouped in brackets and separated by

Issue 02 (2014-04-30) Huawei Proprietary and Confidential iii

Boldface Buttons, menus, parameters, tabs, window, and dialog titles

> Multi-level menus are in boldface and separated by the ">"

Changes in Issue 02 (2014-04-30)

Known bugs are fixed.

Changes in Issue 01 (2014-01-31)

Issue 02 (2014-04-30) Huawei Proprietary and Confidential iv

About This Document.....................................................................................................................ii

2 Flow of Emergency Maintenance...............................................................................................5

3 Emergency Maintenance Guidelines.......................................................................................10

Issue 02 (2014-04-30) Huawei Proprietary and Confidential v

3.4 Built-in System Software Is Incorrect or Does Not Exist............................................................................................26

4 Emergency Maintenance Record Table...................................................................................30

Issue 02 (2014-04-30) Huawei Proprietary and Confidential vi

1 Overview of Emergency Maintenance

About This Chapter

1.1 Definition of Emergency Maintenance

1.2 Definition of Emergencies

1.3 Source of Emergency Maintenance

1.4 Principle of Emergency Maintenance

1.5 Precautions for Emergency Maintenance

1.6 Technical Support

Issue 02 (2014-04-30) Huawei Proprietary and Confidential 1

1.1 Definition of Emergency Maintenance

Emergency maintenance is a rectification method to handle emergencies, such as power-down

Emergency maintenance also guides maintenance personnel to take preventive measures to

This document describes how to apply emergency maintenance to the ATN.

1.2 Definition of Emergencies

An emergency is an unexpected fault on a device or a network, which happens suddenly, has

Emergencies that may occur on the ATN can be classified into:

l Abnormal system - all services being interrupted

1.3 Source of Emergency Maintenance

An emergency may be caused by a software or hardware failure, an incorrect setting, improper

Issue 02 (2014-04-30) Huawei Proprietary and Confidential 2

1.4 Principle of Emergency Maintenance

1.5 Precautions for Emergency Maintenance

Issue 02 (2014-04-30) Huawei Proprietary and Confidential 3

1.6 Technical Support

Maintenance personnel need maintain a detailed record of the emergency maintenance

Issue 02 (2014-04-30) Huawei Proprietary and Confidential 4

2 Flow of Emergency Maintenance