0513 - 3 Workstation Network Maintenance

NOKIA NMS/2000
FOR MANAGING CELLULAR NETWORKS
WORKSTATION NETWORK MAINTENANCE
System Administrator's Guide
Document number/Issue
Copyright Nokia Telecommunications Oy
NTC TAN 0513/3 en
The information in this document is subject to change without notice and describes only the product dened in the introduction of this documentation. This document is intended for the use of Nokia Telecommunications' customers only for the purposes of the agreement under which the document is submitted, and no part of it may be reproduced or transmitted in any form or means without the prior written permission of Nokia Telecommunications. The document has been prepared to be used by professional and properly trained personnel, and the customer assumes full responsibility when using it. Nokia Telecommunications welcomes customer comments as part of the process of continuous development and improvement of the documentation. The information or statements given in this document concerning the suitability, capacity, or performance of the mentioned hardware or software products cannot be considered binding but shall be defined in the agreement made between Nokia Telecommunications and the customer. However, Nokia Telecommunications has made all reasonable efforts to ensure that the instructions contained in the document are adequate and free of material errors and omissions. Nokia Telecommunications will, if necessary, explain issues which may not be covered by the document. Nokia Telecommunications' liability for any errors in the document is limited to the documentary correction of errors. Nokia Telecommunications WILL NOT BE RESPONSIBLE IN ANY EVENT FOR ERRORS IN THIS DOCUMENT OR FOR ANY DAMAGES, INCIDENTAL OR CONSEQUENTIAL (INCLUDING MONETARY LOSSES), that might arise from the use of this document or the information in it. This document and the product it describes are considered protected by copyright according to the applicable laws. NOKIA logo is a registered trademark of Nokia Corporation. Other product names mentioned in this document may be trademarks of their respective companies, and they are mentioned for identication purposes only. Copyright Nokia Telecommunications Oy 1997. All rights reserved.
No. of pages 101/MiG
Edited by M. Ganszauge 22 Jan 1998
Author M. Ganszauge 22 Jan 1998
Approved by J. Karppelin 22 Jan 1998
Previous issue (2) approved 15 May 1997
NTC TAN 0513/3 en
TABLE OF CONTENTS
1 WHAT IS NEW. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1 1.2 Changes from T8 to T9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Changes from T9 to T10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
ABOUT THIS MANUAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1 2.2 2.3 2.4 What you need to know first . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 How to use this manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Where to find more . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Typographic conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4.1 Text styles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4.2 Command line conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Concepts and terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.5.1 Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.5
QUICK REFERENCE TO ADMINISTRATIVE TASKS . . . . . . . . . 17

3.1 Maintenance tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.1.1 Periodic tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.1.2 Occasional tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
FAULT MANAGEMENT TASKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.1 4.2 4.3 Modifying alarm lifting rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Adding and deleting alarm pipes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Checking that network monitoring works . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.3.1 Verifying that the monitored network element is online . . . . . . . 26 4.3.2 Verifying that the NMS is showing the true state of the network 27 Blocking alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.4.1 Blocking alarms in a specific network element . . . . . . . . . . . . . . 28 4.4.2 Blocking BTS alarms in the BSC . . . . . . . . . . . . . . . . . . . . . . . . 28 4.4.3 Disabling and enabling network alarms and measurements in the workstations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4
5 6
PERFORMANCE MANAGEMENT TASKS . . . . . . . . . . . . . . . . . . . 30

5.1 Monitoring the network performance data. . . . . . . . . . . . . . . . . . . . . . . . . 30
TAKING BACKUPS OF THE NOKIA NMS . . . . . . . . . . . . . . . . . . . 32

6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 About backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taking a full backup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Backing up the critical files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Restoring a full backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Restoring a critical files backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taking a backup of the files in the Front End . . . . . . . . . . . . . . . . . . . . . . Restoring a Front End backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taking a backup of the router configuration . . . . . . . . . . . . . . . . . . . . . . . 33 33 39 39 41 41 42 43
NTC TAN 0513/3 en
UNIX ADMINISTRATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
7.1 7.2 7.3 7.4 7.5 System shutdown and startup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Shutting down and starting up the system . . . . . . . . . . . . . . . . . . . . . . . . . 47 Rebooting the NMS servers and workstations . . . . . . . . . . . . . . . . . . . . . . 49 Shutting down and starting up the Nokia NMS . . . . . . . . . . . . . . . . . . . . . 50 Administering MC/ServiceGuard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 7.5.1 Halting the MC/ServiceGuard packages . . . . . . . . . . . . . . . . . . . 51 7.5.2 Starting the MC/ServiceGuard packages . . . . . . . . . . . . . . . . . . . 52 7.5.3 Halting an MC/ServiceGuard node . . . . . . . . . . . . . . . . . . . . . . . 53 7.5.4 Starting an MC/ServiceGuard node . . . . . . . . . . . . . . . . . . . . . . . 54 7.5.5 Halting the MC/ServiceGuard cluster . . . . . . . . . . . . . . . . . . . . . 55 7.5.6 Starting the MC/ServiceGuard cluster . . . . . . . . . . . . . . . . . . . . . 56 7.5.7 Shutting down and restarting a server after halting a node or the whole cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 7.5.8 Enabling MC/ServiceGuard node switching . . . . . . . . . . . . . . . . 57 7.5.9 Mounting and unmounting file systems within a MC/ServiceGuard cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 7.5.9.1 Mounting the disks in clustering mode . . . . . . . . . . . . . 59 7.5.9.2 Mounting the disks in non-clustering mode . . . . . . . . . 59 7.5.10 Verifying the functionality of the MC/ServiceGuard cluster. . . . 60 Managing the Nokia NMS processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 7.6.1 Configuring Process Supervision and Recovery Service. . . . . . . 61 7.6.2 Terminating and starting the Process Supervision and Recovery application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 7.6.3 Terminating and starting a process under supervision. . . . . . . . . 62 Resource Supervision Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.7.1 Configuring disk supervision . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.7.2 Monitoring the disk space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 7.7.3 Monitoring log files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 7.7.3.1 Log les monitored by the system . . . . . . . . . . . . . . . . . 66 7.7.3.2 Log les to be monitored by the system administrator . 67 7.7.4 Removing core image files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Generic Supervision Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 7.8.1 Supervising the X.25 adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 7.8.2 Supervising mirrored disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 7.8.3 Supervising the database index tablespaces. . . . . . . . . . . . . . . . . 70 7.8.4 Supervising the flow of BSC traffic measurements. . . . . . . . . . . 70 7.8.5 Supervising the workstation message service . . . . . . . . . . . . . . . 70 7.8.6 Supervising the OSI connections . . . . . . . . . . . . . . . . . . . . . . . . . 71 Modifying the Alarm Flow Supervision parameters . . . . . . . . . . . . . . . . . 72 Checking the processes manually . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Monitoring the wcltoday.log File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Checking revisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.12.1 Creating revision files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.12.2 Comparing revisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Managing crontab files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Tasks on the Nokia NMS servers with no MC/ServiceGuard software. . . 81 7.14.1 Shutting down and starting up the Nokia NMS servers. . . . . . . . 81
7.6
7.7
7.8
7.9 7.10 7.11 7.12
7.13 7.14
NTC TAN 0513/3 en
7.14.2 7.14.3 7.14.4
Starting up and shutting down the Nokia NMS . . . . . . . . . . . . . . 81 Closing and opening the X.25 connections . . . . . . . . . . . . . . . . . 84 Starting up and shutting down the Nokia NMS database . . . . . . 85
SYSTEM MODIFICATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
8.1 8.2 8.3 8.4 8.5 8.6 8.7 Setting up a printer in Alarm Viewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing customised sounds for Alarm Viewer . . . . . . . . . . . . . . . . . . . . Changing the IP address of a server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring the TFTP server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the NIS configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exporting directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Restricting access to certain services on the local host . . . . . . . . . . . . . . . 87 88 89 91 92 94 96
INDEX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
NTC TAN 0513/3 en
What is new
WHAT IS NEW
This chapter outlines the most significant changes that have occurred in the Nokia NMS software, related HP-UX operating system software and documentation since release T8. If you are an experienced user of the Nokia NMS, we strongly advise you to read this section to get an overview of what system-level changes have been implemented in Nokia NMS release T9 and T10. New features in applications and new applications are described in the respective manuals.
1.1
Changes from T8 to T9
This section describes the changes that have occurred in the Nokia NMS workstation maintenance procedures and in the documentation between releases T8 and T9 of the Nokia NMS. Supervision Framework Nokia NMS now includes a feature called Supervision Framework which monitors the X.25 adapters, BSC traffic measurements, database index tablespaces, mirrored disks, WS NW message service and OSI connections. Formore information see 7.8, Generic Supervision Service. Oracle database The Oracle database version used in T9 was 7.2.3. Also, three new scripts were created to verify the structure of the database.
1.2
Changes from T9 to T10

The following changes were made between release T9 and T10: HP-UX operating system The HP-UX operating system version has been upgraded from 10.01 to 10.20. For more information on what is new for HP-UX 10.20, see HP-UX 10.20 Operating System Help in the HP Help Manager. Oracle database The Oracle database version used in T10 is 7.3.2. As a result, the NMS database file structure has changed from previous releases. For more information, see Database Maintenance, TAN0514. NMS documentation NMS documentation is now available online. Information on basic tasks and user interfaces previously found in the Users Guides can now be found in the respective application help. If you have the optional Online Library installed, all documentation is available to you via the NMS Online Help. For more
NTC TAN 0513/3 en
What is new
information, see Section 2.2, Where to find information on system management in System Management Basic Operating Principles and Procedures, TAN 0732. New alarm system The Nokia NMS has a new alarm system that uses pipes and supports alarm collection from third-party network elements. For more information, see System Management Basic Operating Principles and Procedures, TAN 0732, Fault Management Basic Operating Principles and Procedures, TAN 0704 and Chapter 4, Fault Management tasks in this manual.
NTC TAN 0513/3 en
About this manual
ABOUT THIS MANUAL

The set of System Administrator's Guides for the Nokia NMS is intended for the system administrators of the Nokia NMS. In general, system administrators are the most privileged users in the network and their responsibility is to set up and maintain the environment for normal users. The Nokia NMS system administrator usually also acts as the system administrator of the UNIX servers (the root user). This manual explains the tasks the system administrator needs to carry out to ensure that the Nokia NMS remains functional at all times. The descriptions in this manual are release-specific, so we urge you not to use any previous issues of this document with Nokia NMS release T10. The supported releases of network element software are S6 for BSCs and M7B for MSCs unless otherwise stated. This document also deals with the HP 9000 servers and workstations and HP UX version 10.20, unless otherwise stated. The manual describes the Standard Server Configuration for Workstations which comprises one Communications Server, one Database Server, one Standby Server and one or more Application Servers. If your server configuration is different, you may have to carry out the task on different servers from the ones mentioned in the procedure descriptions. For more information on Nokia NMS server configurations, see the Technical Reference Guide, TAN 0453. More specifically, this document covers the following topics: Chapter 3, Quick reference to administrative tasks introduces the basic concepts used in maintaining Nokia NMS workstation network. The chapter also explains the difference between periodic and occasional tasks. Chapter 4, Fault Management tasks explains the basic procedures related to monitoring and investigating alarms from the network elements, and modifying the alarm system operation. Chapter 6, Taking backups of the Nokia NMS provides instructions for taking and restoring a full backup of the Nokia NMS workstations, and taking and restoring the most critical files in the system. Chapter 7, UNIX administration describes the tasks related to the built-in supervision and recovery functionality of the Nokia NMS and outlines the most common tasks related to regular system administration of the WS network. Chapter 8, System modifications describes the most common tasks related to the reconfiguration of the WS network.
Note: The chapters include administration procedures for servers both with and without the HP MC/ServiceGuard software. If the procedures are different
Document number/Issue Copyright Nokia Telecommunications Oy
NTC TAN 0513/3 en
About this manual
depending on whether you have or do not have MC/ServiceGuard, the procedure headings clearly show if you should carry out the procedure or not. Chapter 7, UNIX administration also includes a special section 7.14, Tasks on the Nokia NMS servers with no MC/ServiceGuard software.
2.1
What you need to know first

This document assumes that you are familiar with the following areas: The concepts of the Nokia NMS and GSM/PCS/DCS networks in general A text processing utility, such as vi. This is required for you to be able to edit certain configuration files. The system administration tasks of the UNIX operating system. An overview of the responsibilities of the system administrator can be found in the HP-UX System Administration Concepts. This manuald complements information contained in the following Nokia NMS/2000 documents, and you should also familiarise yourself with the information contained in these documents. Database Maintenance, TAN 0514 Technical Reference Guide, TAN 0453 Expanding and Reconfiguring Networks, TAN 0432
2.2
How to use this manual

This manual can serve both as a guide to the system maintenance concepts and as a quick task reference. To conveniently locate the information you need, follow these guidelines: What You Need What What to Look for in the Manual Examples 6.1, About backups
Information About... about the basic concepts, descriptions and features available. Descriptions of individual tasks and related windows and dialogs. ...ing
4.4.2, Blocking BTS alarms in the BSC
NTC TAN 0513/3 en
About this manual
What You Need What Quick hands-on task reference.
What to Look for in the Manual How to... or To...
Examples To take a backup of the critical files
Table 1. How to Use This Manual Note that the fastest way to get started is to have a look at the task reference in Chapter 3, Quick reference to administrative tasks.
2.3
Where to find more

When you perform the tasks described in this document, you may need to refer to other documentation for further information. The document list below contains manuals that you will find useful as well as a brief description of their contents. NMS manuals For information on system management concepts, principles and system management applications for the Nokia NMS refer to System Management Basic Operating Principles and Procedures, TAN 0732. For information on integrating the Nokia BSS with the Nokia NMS, installing and configuring hardware, creating X.25 connections and configuring I/O systems, please refer to DCN Integration, TAN 0839. For more information on NMS database functions, database recovery, database backups and other database administration tasks that the system administrator should perform, see Database Maintenance, TAN 0514. For information on hardware, NMS file and directory structure, NMS applications and processes, configuration files and optional features, see Technical Reference Guide, TAN 0453. For further information on other NMS manuals, see NMS/2000 Description of Documentation, TAN 442.
Oracle manuals For conceptual information on the information contained in other Oracle manuals, refer to Oracle7 Server Concepts Manual. For more information on managing the an Oracle 7 server, refer to the Oracle7 Server Administrators Guide. For more information on the messages generated by the Oracle 7 server, the SQL processor, PL/SQL Server Manager, Export and Import utilities refer to Oracle7 Server Messages.
10
NTC TAN 0513/3 en
About this manual
For more details on how to use Oracle 7 server utilities for data transfer, maintenance and database administration, see the Oracle7 Server Utilities Users Guide For more information on using SQL*Net V2.1, see Understanding SQL*Net
Hewlett-Packard documents For more information on the available on HP-UX manuals refer to HP-UX 10.0 Documentation Map and the HP-UX manual page manuals (5). For further information on NFS administration, please refer to the Installing and Administering NFS Services manual. For further information on managing MC/ServiceGuard, please refer to the Managing MC/ServiceGuard manual. For further information on HP-UX system administration tasks and solving HP-UX-related problems, please refer to the HP-UX System Administration Tasks manual.
2.4
Typographic conventions
Several types of typographic conventions are used in this manual to describe different actions and restrictions. The tables below present the conventions used in this manual.
2.4.1
Text styles
The following table presents the typefaces and their indications. Text Style Initial Upper-Case Lettering Explanation Application names Hardware components Names of windows and dialogs Emphasis State, status or mode Referenced documents Referenced chapters within a document All components on a window or dialog in the user interface, for example, "Choose Exit from the File menu".
Italicised text Initial Upper-Case Letters in Italics Initial Upper-Case Lettering in Bold
11
NTC TAN 0513/3 en
About this manual
Text Style
The Courier font
Explanation User names File and directory names Names of database tables Counters in database tables Parameters System output
UPPER-CASE LETTERING
<bracketed text>
Keys on the keyboard. The text within the brackets presents information about a parameter. Replace the whole expression with the correct value. Further information about command-line
Shading
Table 2. Text Styles in This Document
2.4.2
Command line conventions
Command line prompts The following UNIX command line prompts are used to indicate which user should enter the command(s). Note that you should not include the prompt in the command. Prompt
# username>% %
As Which User As the root user As the user indicated by username. As any user.
Table 3. Command Line Prompts Environment variables This document assumes that the following environment variables have the values indicated below: Variable
$OMCCONFPATH $OMCROOT
Value
/global/NokiaOMC/conf/global /usr/local/NokiaOMC/<omc_build>
Table 4. Environment Variables
12
NTC TAN 0513/3 en
About this manual
Line breaks For layout purposes, long command lines may sometimes be split onto two or more separate lines. This is indicated as follows: With UNIX commands and UNIX file descriptions, a backslash ( \ ) at the end of a printed line is used to indicate that the command continues on the following line. For example:
# cat /etc/cmcluster/pkgcomm/control.sh.log | grep \ "crontab" | more
With MML commands, everything between the initial Z and the semicolon ( ; ) constitute a single line. For example:
ZWTP:OMU:AS7_U,<PIU-index>,<track>::X25,<PCM-type >,<PCM-number>,TSL,0;
Optional parameters With all types of commands, square brackets ( [ ] ) are used to indicate that the portion of the command enclosed in brackets is optional or not always required. For example:
ZUSU:<unit>[,<index>];
The above example indicates that if you are dealing with several units of the same type (for example, the BDCUs), it is necessary to also specify the index. With single units (for example, the OMU), the index parameter is not necessary. Conditional parameters With all types of commands, the braces and the vertical bar ( { | } ) are used to indicate that the portion of the command enclosed in brackets is conditional. For example:
# cmmodpkg { -e | -d } <package>
The above example indicates that you must select between the -e and -d options when you enter the command. Text variables The following mark-up for string variables is used throughout this document to represent certain system concepts. When you encounter one of these variables, replace the string within the brackets (including the brackets) with the actual text relevant in each context.
13
NTC TAN 0513/3 en
About this manual
Variable
<omc_build>
Explanation This variable refers to the name of the releasespecific directory of the Nokia NMS. For the actual name, refer to Chapter Release Information in the NMS Customer Release Notes, which can be found in the Nokia NMS release binders. This variable refers to the name of the Communications Server. This variable refers to the name of the Database Server. This variable refers to the name of the Standby Server. This variable refers to the name of the Application Server N. Note that there is often more than one Application Server, so the commands presented in these contexts must be executed on all Application Servers. This variable refers to the domain name of Network Information Service. This variable refers to the name of a server.
<Communications> <Database> <Standby> <App_X>
<domain_name> <server>
Table 5. Text variables
2.5
Concepts and terminology

This subsection explains the concepts, terms and abbreviations used in this document. Term Application Server Explanation A server which has the Nokia NMS software installed locally. This server runs the graphical NMS applications. A combined Communications and Database Server which runs the Nokia NMS database and also functions as a communications gateway between the workstation network and other network elements. A server that functions as a communications gateway between the workstation network and other network elements. A server that runs the Nokia NMS database.
Comm&DB Server
Communications Server
Database Server
14
NTC TAN 0513/3 en
About this manual
Term HP 9000 server
Explanation Abbreviation used to refer to a HP 9000 G, H and K-Class servers. The HP 9000 servers used with the Nokia NMS have the following roles: Communications Server Database Server Standby Server Depending on the configuration, a single HP 9000 server may perform only a part of one of these functions or even several of these functions. Abbreviation used to refer to a HP 9000 Series 700 workstation. The HP 9000 Series 700 workstations used with the Nokia NMS have the following role: Application Server Network File System. The NFS allows file systems to be accessed from different servers. A server which can mount a file system from another server. A server whose disks can be mounted from other servers. Network Information Service. The NIS provides generic database access facilities for the distribution of information, such as password and group files, within a network. A server which receives NIS information from one of the NIS servers. A server which contains the main NIS maps and distributes information about users and groups to the NIS network. A server that takes over the tasks of the NIS master if the NIS master itself is not functional. Describes the type of network connection, hard disk configuration and the number and type of HP 9000 servers. For more information on each of these configurations, see the Technical Reference Guide. A server which monitors the Communications and Database Servers or Comm&DB Server and replaces it in case of server failure. Standby Server has its own connections to the network elements. A graphical terminal which can display graphical applications that run on an Application Server.
HP 9000 workstation
NFS NFS client NFS server NIS
NIS client NIS master
NIS slave Server configuration
Standby Server
X terminal
15
NTC TAN 0513/3 en
About this manual
Term X.25
Explanation An interface between the DTE and the DCE in a packet switched network using X.21 at the physical layer.
Table 6. Terminology used in this document
2.5.1
Abbreviations
The following table presents the abbreviations and acronyms which appear in this document: Abbreviation AS BSC BTS CDS CS DAT DB DS FE HP IP IS MML NFS NIS PID RNW SAM SS SW CDE WS Explanation Server role: Application Server Base Station Controller Base Transceiver Station Server role: Comm&DB Server Server role: Communications Server Digital Audio Tape Database Server role: Database Server Front End Hewlett-Packard Internet Protocol Intermediate System Man-Machine Language Network File System Network Information Service Process ID Radio Network System Administrator Manager Server role: Standby Server Software Common Desktop Environment Workstation
Table 7. Abbreviations used in this document
16
NTC TAN 0513/3 en
Quick reference to administrative tasks
QUICK REFERENCE TO ADMINISTRATIVE TASKS

Workstation network maintenance is the general concept used to refer to all the administrative tasks you have to perform to ensure that the Nokia NMS remains functional at all times. It includes both preventive and corrective procedures which together minimise the possibility of data loss. The aim is to guarantee that the NMS workstations have correct and up-to-date information about the cellular network, and that the information can be exploited to the fullest. The following section, Maintenance tasks, gives you a brief overview of the tasks involved in keeping the Nokia NMS functional and efficient. We suggest that you familiarise yourself with this section to gain a general idea of the tasks associated with workstation network maintenance. For detailed information about the actual tasks, refer to the appropriate chapters later in this manual. They describe each maintenance area in more detail and provide you with step-by-step instructions on how to use the Nokia NMS maintenance applications.
3.1
Maintenance tasks
This section describes some of the procedures involved in keeping the Nokia NMS system functional and efficient. The maintenance procedures can be divided into two categories: Periodic tasks: tasks that have to be carried out at regular intervals. Occasional tasks: tasks that have to be carried out when needed.
We advise you to review the tables below to obtain an overview of workstation network maintenance procedures.
3.1.1
Periodic tasks
This section lists the tasks that have to be regularly carried out by the system administrator or automatically by the system. What to do? Check the reasons for automatic process restarts Check the status of the MC/ ServiceGuard cluster Free disk space taken by log files When to do it? Daily Where to find instructions? 7.10, Checking the processes manually 7.5, Administering MC/ ServiceGuard
Daily or if there are problems
Weekly.
7.7.3, Monitoring log files
17
NTC TAN 0513/3 en
What to do? Take a backup of the critical files
When to do it? The frequency depends on the usage
Where to find instructions? 6.3, Backing up the critical files 6.2, Taking a full backup
Take a full backup Every time you make of the file systems changes to the Nokia NMS system Table 8. Periodic tasks
3.1.2
Occasional tasks
This section lists the tasks that have to be carried out by the system administrator when needed. The frequency of the tasks to be carried out below may vary greatly. What to do? Monitor the available disk space Block and filter alarms Change the IP address Change the NFS configuration Change the NIS configuration Check revisions Compare revisions Configure disk supervision When to do it? If there are problems If a network element is excessively generating alarms. If you shift the system is to another address range. If you add or remove hosts. If you add or remove hosts to/from the NIS domain. When needed When needed Default settings are set during the commissioning. Reconfigure when needed. When needed When adding network elements that use new alarm interfaces. When needed Where to find instructions? 7.7.2, Monitoring the disk space 4.4, Blocking alarms
8.3, Changing the IP address of a server 8.6, Exporting directories 8.5, Changing the NIS configuration 7.12, Checking revisions 7.12.2, Comparing revisions 7.7.1, Configuring disk supervision
Configure alarm lifting rules Configure alarm pipes
4.1, Modifying alarm lifting rules 4.2, Adding and deleting alarm pipes
Configure a printer for Alarm Viewer
8.1, Setting up a printer in Alarm Viewer
18
NTC TAN 0513/3 en
What to do? Configure the TFTP server Configure automatic process supervision
When to do it? If TFTP server needed in file transfer Default settings are set during the commissioning. Reconfigure when needed. When needed When shutting down servers or taking a full backup When needed
Where to find instructions? 8.4, Configuring the TFTP server 7.6.1, Configuring Process Supervision and Recovery Service
Create revision files Halt an MC/ ServiceGuard package, node or cluster Installing new sounds for Alarm Viewer Restrict access to workstations Restore a critical files backup Restore a file system backup Shut down and start up HP 9000 workstations Shut down and start up the HP 9000 servers Shut down the NMS
7.12.1, Creating revision files 7.5, Administering MC/ ServiceGuard
8.2, Installing customised sounds for Alarm Viewer 8.7, Restricting access to certain services on the local host 6.5, Restoring a critical files backup 6.4, Restoring a full backup
If additional security required. In case of an emergency or for test purposes. In case of an emergency or for test purposes. When needed
7.2, Shutting down and starting up the system 7.2, Shutting down and starting up the system 7.4, Shutting down and starting up the Nokia NMS 7.4, Shutting down and starting up the Nokia NMS 4.3.1, Verifying that the monitored network element is online 5.1, Monitoring the network performance data
When needed
Only if instructed by NMS documentation to do so. When needed. When needed
Start up the NMS Verify that a network element monitored is online Verify that measurement information is correct
When needed
19
NTC TAN 0513/3 en
What to do? Verify that measurements are coming Verify that the NMS reflects the true state of the network
When to do it? When needed
Where to find instructions? 5.1, Monitoring the network performance data 4.3.2, Verifying that the NMS is showing the true state of the network
When needed
Table 9. Occasional tasks Each subsection describes an individual maintenance area and gives you references to the more detailed sections in this manual.
20
NTC TAN 0513/3 en
Fault Management tasks
FAULT MANAGEMENT TASKS

This chapter describes Fault Management tasks related to the NMS and the network elements controlled by it. The following, topics are covered: Modifying alarm lifting rules Adding and deleting alarm pipes Checking that network monitoring works Blocking alarms
4.1
Modifying alarm lifting rules

The alarm lifting rules are defined in the configuration file arcobjmodelmx.cf in $OMCCONFPATH on the Communications Server. The lifting rules have the following syntax:
# ( LEGAL_ALARM_LIFTS "" # ( <source object class> # )
"<target object class>" )
( LEGAL_ALARM_LIFTS "" ( 24 "4" ) # TRX alarms are lifted to BTS ( 10 "108" )# alarms from FU are lifted to HLR )
Example 1. Lifting rules in the arcobjmodelmx.cf file To change alarm lifting rules Caution: Usually there is no need to modify the default alarm lifting rules. If, however, you wish to change the rules, we recommend that you contact your local Nokia representative first. 1. 2. Log into the Communications Server as the omc user. Change to the $OMCCONFPATH directory and make a backup copy of the arcobjmodelmx.cf configuration file:
omc>% cp arcobjmodelmx.cf arcobjmodelmx.cf.bak
3.
Open the arcobjmodelmx.cf configuration file in that directory. Use a text editor to change the contents of the file.
21
NTC TAN 0513/3 en
4.
In the arcobjmodelmx.cf configuration file, look for the section titled LEGAL_ALARM_LIFTS. As you can see in example above there is one line for each alarm lifting rule in the configuration file. Change the lifting rules as follows: If the source object class is already included in the file and has a lifting rule assigned to it, and you want to change those objects target object class, you need to change the target object class entry in the respective line. For example, if there is a lifting rule for all Channel (5) alarms to be lifted to the Transceiver (24), and you want to lift the Channel alarms to the BTS (4), you need to change the line
( 5 "24" )
5.
to
( 5 "4" )
If the source object class is not included in the file, you need to add a new row to the LEGAL_ALARM_LIFTS section, where you define the source object class and the target object class as shown in example on the previous page.
Note that the target object class must be a superior within the distinguished name of the source object class. Note also that the alarm lifting rules are read hierarchically and use the first match. 6. 7. Save the changes and close the text editor. Find out the process IDs of the arcpcsmx processes and kill the arcpcsmx processes. Give the following commands:
omc>% mx | grep arcpcsmx omc>% kill -INT <pid> <pid> ...
Process Supervision and Recovery Service (wpmanamx) will restart the alarm processor processes automatically.
22
NTC TAN 0513/3 en
4.2
Adding and deleting alarm pipes

Alarm pipes are defined in the arcpipmx.cf configuration file on the server which runs the alarm pipe processes. An example of the arcpipmx.cf file is given below:
# # Alarm pipe for WS alarms. # (ALARM_PIPE "0 <Comm>" (COLLECTOR "arccolmx \ -forward -pipeId 0 \ -interface WS \ -writeBufferId ARCCOL.0") (PROCESSOR "arcpcsmx \ -pipeId 0 \ -readBufferId ARCCOL.0 \ -writeBufferId ARCPCS.0") (FORWARDER "arcfrwmx \ -pipeId 0 \ -readBufferId ARCPCS.0") ) # # Alarm pipe for Q3 alarms. # (ALARM_PIPE "1 <Comm>" (COLLECTOR "arcq3cmx \ -forward \ -pipeId 1 \ -writeBufferId ARCCOL.1") (PROCESSOR "arcpcsmx \ -pipeId 1 \ -readBufferId ARCCOL.1 \ -writeBufferId ARCPCS.1") (FORWARDER "arcfrwmx \ -pipeId 1 -readBufferId ARCPCS.1") ) <other alarm pipes...>
Example 2. Example of the configuration file for alarm pipes. Note: The internal alarm pipes with pipe identifier 0 should not be modified in any way.
23
NTC TAN 0513/3 en
Because of MC/ServiceGuard, the arcpipmx.cf configuration file is stored in three places in the NMS. The following table tells you where the files are located: Every time the arcpipmx.cf file is changed, the changes have to be made to all the three files. Directory
$OMCCONFPATH $OMCCONFPATH/cs $OMCCONFPATH/cds
Server Name Communications Server Role-specific directory (Communications Server) Role-specific directory (Comm&DB Server)
Table 10. Alarm pipe locations in the NMS When a network element that uses a different type of alarm interface from the existing one(s) is added under the supervision of the NMS, you need to install a new alarm pipe to ensure that the alarms generated in the network element are forwarded to the right target. To add new alarm pipes Warning: You should only add a new alarm pipe if there does not exist a pipe for the particular alarm interface. The Nokia NMS supports alarm collection from several types of third-party network elements. For more information on these, refer to Fault Management Basic Operating Principles and Procedures, TAN 0704. We also recommend that you contact your local Nokia representative before you start modifying the alarm pipe configuration. 1. Create the buffer for the alarm pipe. Each alarm pipe has to have buffers between its processes.
omc>% tubeezmx create <buffername> 65536 omc sysop 700
2.
Change to the $OMCCONFPATH directory and make a backup copy of the arcpipmx.cf configuration file:
omc>% cp arcpipmx.cf arcpipmx.cf.bak
24
NTC TAN 0513/3 en
3.
Update the contents of the $OMCCONFPATH/arcpipmx.cf configuration file. Add the following section to the file:
(ALARM_PIPE "<no.> <Comm>" (COLLECTOR "<collector_process> \ -forward -pipeId <no.> \ -interface WS \ -writeBufferId ARCCOL.<no.>") (PROCESSOR "arcpcsmx \ -pipeId <no.> \ -readBufferId ARCCOL.<no.> \ -writeBufferId ARCPCS.<no.>") (FORWARDER "arcfrwmx \ -pipeId <no.> \ -readBufferId ARCPCS.<no.>") )
<no.> <collector_process>
ID for the alarm pipe Name of the alarm collector process, which can be:
arcq3cmx arccolmx
Remember to take backup copies and update the files $OMCCONFPATH/ cs/arcpipmx.cf and $OMCCONFPATH/cds/arcpipmx.cf, too. 4. To start the processes of the new alarm pipe, execute arcstamx on all workstations where the new alarm pipe should run.
To delete an alarm pipe Warning: You should only delete an alarm pipe if there are no more network elements that use the particular alarm interface. The Nokia NMS supports alarm collection from several types of third-party network elements. For more information on these, refer to Fault Management Basic Operating Principles and Procedures, TAN 0704. We also recommend that you contact your local Nokia representative before you start modifying the alarm pipe configuration. To remove an alarm pipe, carry out the following steps on the Communications Server:
25
NTC TAN 0513/3 en
1.
Log in as the omc user and then find out which processes belong to the alarm pipe in question by entering the following command:
omc>% ps -fp 99999 ; ps -ef | grep pipeId <no.>
This command lists the processes which belong to the alarm pipe in question. The process ID is the number given in the PID column. 2. Kill the processes by entering
omc>% kill -TERM <pid> <pid> ...
3.
Delete the buffers that belong to the alarm pipe you want to remove by entering the following commands for each buffer:
omc>% tubeezmx delete ARCCOL.<no.> omc>% tubeezmx delete ARCPCS.<no.>
4.
Change to the $OMCCONFPATH directory and make a backup copy of the arcpipmx.cf configuration file:
omc>% cp arcpipmx.cf arcpipmx.cf.bak
5.
Open the file $OMCCONFPATH/arcpipmx.cf in a file editor and delete the section that defines the alarm pipe in question. For more information on how the entries in this configuration file are structured, see the examples on the previous pages. Remember to take backup copies and remove the section in question also in the files $OMCCONFPATH/cs/arcpipmx.cf and $OMCCONFPATH/ cds/arcpipmx.cf.
4.3
Checking that network monitoring works

From time to time, you should make sure that the alarm status in the NMS and a network element is up-to-date. The alarm status in the NMS and network element may be out of sync if, for example, the O&M connection between the NMS and network element has been down, or if you have made any modifications to the NMS or network elements.
4.3.1
Verifying that the monitored network element is online
To verify that you are receiving alarms from the network elements to the NMS, open the Alarm Viewer application. You should see new alarms or warnings appear in the Alarm Viewer every now and then. You can check the time when the alarm has been generated to verify that there has not been a backlog of alarms which have been buffered somewhere for a long time. If your network comprises only few network elements and practically nothing seems to be happening in your network (you see no alarms or warnings being generated in your network), you may want to generate an alarm yourself in a network element to make sure that everything is functioning properly.
26
NTC TAN 0513/3 en
Generating alarms is also a way to verify the status of a particular network element. To generate an alarm, you can change the Spare Message Bus to TEST state for a short while, which causes an Incorrect Working State alarm. Open an MML session to the network element and first find out the index of the Spare Message Bus by entering
ZUSI:MB;
To change the Spare Message Bus to TEST state and back to SPARE state, give the following commands:
ZUSC:MB,<index>:TE; ZUSC:MB,<index>:SP; <index>
The index of the Spare Message Bus
Some network elements also support Alarm Flow Supervision, which automatically generates an alarm if a network element is not online. For more information, see 7.9, Modifying the Alarm Flow Supervision parameters in this manual and System Management Basic Operating Principles and Procedures, TAN 0732.
4.3.2
Verifying that the NMS is showing the true state of the network
You can verify that the NMS is showing the true state of the network either directly from the Top-level User Interface or by opening an MML session. To check the current alarm status of one network element as seen by the NMS 1. 2. 3. Open the Alarm History application by selecting Alarm History Search Criteria from the pop-up menu. Select both acknowledged and non-acknowledged alarms from the Alarm Ack State pane. Click the Other Criteria button, select the Order by Time option and click the Search button
To check the alarm status in the network element you can open an MML session to the network element. Display active alarms with the MML command as follows:
ZAHO; (use ZEOL; if you intend to check BTS alarms)
27
NTC TAN 0513/3 en
The differences between the alarms in the NMS and network elements could also be caused by alarm reclassification in the NMS, or alarm blocking in the network element. If there is a severe mismatch between the NMS and network element alarms, you should correct the situation with the Alarm Upload application.
4.4
Blocking alarms
If you are carrying out some maintenance work in a network element, it is advisable to block all the alarms coming from that network element in order to prevent excessive number of alarms from arriving to the database. The alarm block can be set using an MML session to the network element.
4.4.1
Blocking alarms in a specific network element
The following table presents the MML commands which can be used to block a the alarms in a network element. MML Command
ZABB:<alarm number>:,; ZABU:<alarm number>:,; ZABO;
Description Blocks the alarms of a specific number Unblocks the alarms of a specific number Lists all blocked alarms in a network element.
Table 11. Blocking Alarms in a Network Element
4.4.2
Blocking BTS alarms in the BSC
It is possible to block the alarms coming from the BTSs in the parent BSC. This will prevent the alarms from arriving to the NMS. The BTS alarm block can be set using an MML session to the parent BSC. The following table presents the MML commands which can be used to block a the BTS alarms in a BSC. MML Command
ZEOB:<alarm number>; ZEOU:<alarm number>; ZEOE;
Description Blocks the BTS alarms of a specific number in the BSC Unblocks the BTS alarms of a specific number in the BSC Lists the blocked BTS alarms in the BSC
Table 12. Blocking BTS Alarms in a BSC
28
NTC TAN 0513/3 en
4.4.3
Disabling and enabling network alarms and measurements in the workstations
Network measurements and alarms should always be disabled before the database is shut down. When the alarms and measurements are disabled, the elements start buffering events to avoid data loss. The alarms and measurements must also be enabled after the database startup when buffering is no longer necessary and forwarding the alarms and measurements to the database can be resumed. To disable measurements and alarms 1. 2. Log into the Communications Server as the omc user. Enter the following command to find out the process ID of the edmmanmx process:
omc>% mx | grep edmman
3.
Enter the following command to disable network alarms and measurements:

omc>% kill -USR1 <edmmanmx_PID> <edmmanmx_PID>
Process ID of the edmmanmx process
When edmmanmx process receives the USR1 signal, it ceases to forward network alarms and measurements to the database. However, the process must stay up to be able to resume event forwarding. To enable measurements and alarms After the database startup the alarms and measurements have to enabled. 1. 2. Log into the Communications Server as the omc user. Enter the following command to find out the process ID of the edmmanmx process:
omc>% mx | grep edmman
3.
Enter the following command to enable network alarms and measurements:

omc>% kill -USR1 <edmmanmx_PID> <edmmanmx_PID>
Process ID of the edmmanmx process
When edmmanmx process receives the USR1 signal, it resumes forwarding network alarms and measurements to the database. For eddmanmx to be able to resume event forwarding, it must not be killed.
29
NTC TAN 0513/3 en
Performance Management tasks
PERFORMANCE MANAGEMENT TASKS

This chapter describes performance management tasks related to the NMS and the network elements controlled by it. The following, topics are covered: Monitoring the network performance data
5.1
Monitoring the network performance data

If the O&M connection between the NMS and a network element has been down or if you have made some modifications in the NMS or network elements, you should check that network performance data is available in the NMS. To verify that measurements from the BSC are transferred to the NMS 1. 2. 3. Select Performance Mgmt DB Contents from the BSC pop-up menu. Click the Select All button to select all the available measurement types Click the From button in the Time pane and set the date to current date and the time to the beginning of the day, for example. Keeping the period relatively short speeds up the verification process. However, you should not set the time to shorter than your measurement interval or you will not be able to see any measurements coming in. Click File Find. If you do not see any measurements coming, you should also check the measurement schedule, output interval and the state of the measurement. Some observation reports are generated only when the requirements for the observation are met. Generic Supervision Service also monitors the transfer of measurements from BSCs. For more information, refer to 7.8.4, Supervising the flow of BSC traffic measurements in this manual. To verify that BSC measurements are not buffered in the BSC 1. 2. Click with the mouse button on the right on top of the BSC and select MML Session Give the following command:
ZIFO:OMU,MEASUR:1&&300;
4. 5.
30
NTC TAN 0513/3 en
Performance Management tasks
You will see similar output to the one below:

298 TRANSFERRED 18 08:01:09 299 TRANSFERRED 18 08:16:07 300 TRANSFERRED 18 08:31:09 OK/-OK/-OK/-- - - - - - 1998-01-18 08:01:00 1998-01-18 08:15:59 1998-01-18 08:30:59 1998-011998-011998-01-
Example 3. Output from ZIFO:OMU,MEASUR:1&&300;command If measurements are not buffered, the buffer is empty and there are no files in FULL state. To verify that measurement information is correct in the NMS 1. Click with the mouse button on the right on top of the network element you want to check and select Performance Mgmt Administration from the pop-up menu. Select a measurement by clicking it. Select Action Interrogate. The application informs you, if there are any discrepancies between the network element and NMS concerning this measurement or observation type. Repeat steps 2 and 3 for all measurement types on the list.
2. 3.
4.
31
NTC TAN 0513/3 en
Taking backups of the Nokia NMS
TAKING BACKUPS OF THE NOKIA NMS

This chapter describes the procedures associated with taking and restoring backups of the files in the Nokia NMS system. This procedure is divided into the following parts: About backups Taking a full backup Backing up the critical files Restoring a full backup Restoring a critical files backup
For more information on backup principles, refer to System Management Basic Operating Principles and Procedures, TAN 0732. This chapter also contains instructions on how to take backups of the files in the Nokia NMS Front End.
32
NTC TAN 0513/3 en
6.1
About backups
It is highly recommended that you take regular backups of the system to avoid data loss in case of a system failure. The backup frequency depends on the system usage. The following procedures explain both the full backup and the backup of the most critical files in the system. The subsections give instructions on how to take the backups both with and without the MC/ServiceGuard software.
6.2
Taking a full backup

A full backup saves a copy of all the file systems in the workstation network onto DATs, from where it can be restored in case something goes wrong. To take a full backup, all applications which are currently running on the server, need to be stopped. In the Nokia NMS environment, this means shutting down the database and all the Nokia NMS processes. Only after this has been done can the files be copied onto the backup medium. The backup procedure presented here takes separate copies of the root file system and of the disks mounted under the /d directory hierarchy (/d is not a mount point!). Please note that one DAT may not be enough to store all the files in the system. To take a full backup with MC/ServiceGuard in use 1. On the Communications Server, halt the MC/ServiceGuard packages:
# cmhaltpkg -v db # cmhaltpkg -v comm
Halting the packages may take several minutes. Halting the comm package on the Communications Server also halts the omcdb package on the Database Server and the spare package on the Standby Server. 2. Insert a DAT into the tape drive on the Communications Server and copy the root file system onto the tape. If you have a compressing DAT drive, give the following commands on the Communications Server:
# cd / # tar cvf /dev/dat ./
If you do not have a compressing DAT drive, you can speed up the backup process considerably by compressing the files while moving them onto the DAT:
# tar cvf - ./ | compress > /dev/dat
33
NTC TAN 0513/3 en
3.
To access the other disks, you have to remount them. Give the following commands:
# /etc/cmcluster/sgsdskmx.sh -e mount db # /etc/cmcluster/sgsdskmx.sh -e mount comm
4.
Copy all the files from /d onto the tape, or copy different file systems onto different tapes: If you have a compressing DAT drive, give the following commands on the Communications Server to copy all the files onto the same DAT:
# cd / # tar cvf /dev/dat ./d
# tar cvf - ./d | compress > /dev/dat
Note: When the tape becomes full, the system asks you to insert another tape. As an alternative, you can copy the file systems mounted under /d/ global and /d/dbXX each onto separate tapes. This may help you to keep track what files are stored where and make restoring the files easier. For example, to copy the files under /d/global onto a separate DAT, insert a new tape into the DAT drive on the Communications Server and do the following:
# cd /d # tar cvf /dev/dat ./global
Remove the tape when all files from /d/global have been copied and insert another tape for another file system. 5. Unmount the disks before starting up the Nokia NMS and Nokia NMS database:
# /etc/cmcluster/sgsdskmx.sh -e umount db # /etc/cmcluster/sgsdskmx.sh -e umount comm
6.
Enable package switching for the packages:

# cmmodpkg -e db comm spare omcdb
You can view the status of the MC/ServiceGuard cluster by giving the command cmviewcl and verify that the packages are starting as they
34
NTC TAN 0513/3 en
should. The startup usually takes a while, so you may have to repeat the command a few times. If the MC/ServiceGuard packages do not start up after you have enabled the package switching attributes, you have to restart the packages manually by giving the following command:
# cmrunpkg -v -n <server> <package>
You have to specify the server if the primary node for the package is different from the server on which you are giving the command. To take a full backup with MC/ServiceGuard not in use Note: You should follow the instructions in this section if your system does not have the MC/ServiceGuard software installed. We assume that you have a Comm&DB Server only. 1. 2. Log into the Comm&DB Server as the root user. Close the connections to the network elements: If you use X.25 connections, give the following commands on the Comm&DB Server:
# x25stop -d /dev/x25_0 # exit
Wait a while for the processes to flush the channels. Note: If you have several X.25 cards, you must repeat the x25stop command for each X.25 card, replacing the device specification as necessary, for example:
# x25stop -d /dev/x25_1
If you use LAN connections kill the OTS processes. Give the following commands:
# ps -ef | grep ots
Make a note of the Process IDs (PIDs) of the otsamd and otslogd processes. Kill the processes:
# kill -TERM <otsamd_pid> <otslogd_pid>
35
NTC TAN 0513/3 en
<otsamd_pid> <otslogd_pid>
The PID of the otsamd process. The PID of the otslogd process.
3.
Shut down the Nokia NMS background processes on all servers:

% su - omc omc>% mx | grep wpmana
Make a note of the PID of the wpmanamx process and kill the process:
omc>% kill -TERM <wpmanamx_pid> <wpmanamx_pid>
The PID of the wpmanamx process
4.
Stop the cron daemon by giving the following command on the Comm&DB Server as the root user:
# /sbin/init.d/cron stop
5.
Shut down the database:

# su - oracle oracle>% svrmgrl SVRMGR> connect internal SVRMGR> shutdown immediate
If shutting down the database seems to take an excessive amount of time, the RDBMS may be performing a rollback or another time-consuming operation. You can view the activities the RDBMS carries out by monitoring the alert_omc.log file. Enter the following command:
oracle>% tail -f $ORACLE_HOME/rdbms/log/alert_omc.log
If the RDBMS is carrying out a time-consuming task, simply wait for it to complete. When the database has been shut down using the immediate option, start it up and shut it down again:
SVRMGR> startup open SVRMGR> shutdown normal SVRMGR> exit
You should receive a notification that the database has been shut down. Then exit the oracle prompt by entering:
oracle>% exit
6.
Unmount the file systems in /d/global and /d/dbXX:

# cd / # umount /d/global /d/db01 /d/db02 [ /d/dbXX ]
36
NTC TAN 0513/3 en
7.
Copy all the files on the root file system onto a DAT. If you have a compressing DAT drive, enter the following commands on the Communications Server as the root user:
# tar cvf /dev/dat ./
If you do not have a compressing DAT drive, you can speed up the backup process by compressing the database while moving it onto the DAT. Enter the following commands on the Comm&DB Server as the root user:
# tar cvf - ./ | compress > /dev/dat
The workstation backup may take several hours. You will be prompted for another DAT if the one in the DAT drive becomes full. 8. Mount the file systems back under /d/global and /d/dbXX:
# mount -ae
9.
Copy all the files from /d onto the tape, or copy different file systems onto different tapes. If you have a compressing DAT drive, give the following commands on the Communications Server to copy all the files onto the same DAT:
# cd / # tar cvf /dev/dat ./d
# tar cvf - ./d | compress > /dev/dat
Note: When the tape becomes full, the system asks you to insert another tape. As an alternative, you can copy the file systems mounted under /d/ global and /d/dbXX each onto separate tapes. This may help you to keep track what files are stored where and make restoring the files easier. For example, to copy the files under /d/global onto a separate DAT, insert a new tape into the DAT drive on the Communications Server and do the following:
# cd /d # # tar cvf /dev/dat ./global
Remove the tape when all files from /d/global have been copied and insert another tape for another file systems.
37
NTC TAN 0513/3 en
10.
If you use X.25 connections proceed to step 11. If you use LAN connections, take the following steps to restore the normal mode of operation: Reboot the server:
# shutdown -r -y 0
Note: The backup procedure with LAN connections ends here. When the server restarts it restores the network element connections and brings the Nokia NMS back online. 11. When the backup is finished, start the cron daemon by entering the following command as the root user:
# /sbin/init.d/cron start
12.
Start up the database:

# su - oracle oracle>% svrmgrl SVRMGR> connect internal SVRMGR> startup open
You should receive a notification that the database has been started up. Then exit the oracle prompt by entering:
oracle>% exit
13.
Open the X.25 connections by entering the following commands on the Comm&DB Server:
# x25init -c /etc/x25/x25config_0 # exit
Note: If you have several X.25 cards, you must repeat the x25init command for each X.25 card, replacing the device specification as necessary, for example:
# x25init -c /dev/x25_1
14.
Restart the Nokia NMS processes on all servers:

# su - omc omc>% omkickmx -v
38
NTC TAN 0513/3 en
6.3
Backing up the critical files

The most critical files in the workstation configuration are the following: Data files in /usr/oracle/dbs Database archive log files in /usr/oracle/dbs/arc Nokia NMS configuration files in /usr/local/NokiaOMC/conf Network views in /usr/local/NokiaOMC/views Customisation files in /usr/local/NokiaOMC/custom Home directories of the users in /home
The NMS software automatically copies these directories from the Communications Server to the Standby Server every night. We advise you to check daily if the automatic backup operation has been successful. This can be done by verifying the time stamps of the files on the backup disk. The time stamps should show the time when the backup was last taken. The backup procedures are run by the cron daemon for omc, root and oracle users. If any problems occur, check the crontab entries of each the users with the crontab l command. For more information, see the 7.13, Managing crontab files. The database files can be backed up taking advantage of the features offered by the HP MirrorDisk/UX software. For information on how to take an offline backup of the database files, please refer to Database Maintenance, TAN 0514. To take a backup of the critical files 1. 2. 3. Insert the backup tape into the DAT drive on the Communications Server. Log into the Communications Server as the root user. Take a backup of the critical files in the directories by giving following command:
# tar cvfh /dev/dat /usr/local/NokiaOMC/conf \ /usr/local/NokiaOMC/views \ /usr/local/NokiaOMC/custom \ /home
6.4
Restoring a full backup

This subsection gives you instructions on how to restore a full file system backup. We assume that you have taken backups as described in 6.2, Taking a full backup.
39
NTC TAN 0513/3 en
Note: These instructions assume that the backup is restored to a system which has suffered a disk failure or has in some other way been disabled. This means that the Nokia NMS is not running and that the Nokia NMS database is down. However, we assume that the HP-UX operating system is functional. We also assume that the disk and hardware configuration are the same as when the backup was taken. To restore a full backup To restore a full backup of all the files in the workstation network of the Nokia NMS you should have plenty of free space on one of the disks. The backup is restored to a temporary directory and the necessary files are copied from there to their correct locations. We recommend that you use the backup disks, for example. 1. Insert the backup DAT into the DAT drive. If you did not compress the data files, give the following commands as the root user to restore the backup:
# cd <disk_with_free_space> # tar xvfp /dev/dat
If you have compressed the data files when taking the backup, you should now uncompress and restore them with the following command:
# uncompress < /dev/dat | tar xvfp -
Note: If you want to restore a specific file, give the following commands:
# cd / # tar xvfp /dev/dat/ <file>
<file>
File or directory name with full relative path specification
Note: If you get an error message saying that the files you are trying to restore are busy, you have to rename the files on the hard disk with the mv command. Repeat the above step for all directories where you have data files.
40
NTC TAN 0513/3 en
2.
Move or copy the required files to their correct locations. Remember to use the -p flag with the cp command to preserve permission and ownership settings. If you overwrite any operating system files we recommend that you reboot the servers. For instructions, refer to Chapter 7, UNIX administration.
3.
6.5
Restoring a critical files backup

This subsection gives instructions on how to restore a critical files backup. The backup is assumed to have been taken as described in 7.8.2, Supervising mirrored disks. To restore a critical files backup 1. 2. 3. Insert the backup tape of the critical files into the DAT drive on the Communications Server. Log into the Communications Server as the root user. Restore the critical files by giving the following command:
# cd / # tar xvfp /dev/dat
Note: If you only want to restore specific directories or files, enter the following commands:
# cd / # tar xvfp /dev/dat/ <file>
<file>
File or directory name with full relative path specification
6.6
Taking a backup of the files in the Front End

The Nokia NMS Front End contains duplicated hard disks that hold the system data. Even with this precaution, it is advisable to take regular backups of the system files in the Front End onto a diskette or one of the hard disks in the Front End. You can take a backup of the system files by: Disabling file updates from the PIUs Creating a backup build Enabling file updates from the PIUs
For exact instructions, see the tasks below.
41
NTC TAN 0513/3 en
To take a backup of the current build onto a hard disk 1. Disable file updates to the disk from the units:
ZDUP:OMU:DISK; ZDUP:FCMU:DISK; ZDUP:PFMU:DISK;
2.
Take a backup copy of the build:

ZWQS:ALL:NAME=<backup_name>,DIRE=<backup_dir>,:; <backup_name> <backup_dir>
Any string up to eight characters. Any string up to eight characters.
3.
If you wish to update the LFILES directory of the backup to the level of the active build, give the following command:
ZWQS:DATA;
4.
Enable file updates to the disk:

ZDUR:OMU:DISK; ZDUR:FCMU:DISK; ZDUR:PFMU:DISK;
To take a backup of LFILES onto a floppy disk 1. Give the following commands:
ZIWY:S:SYSTEM=<FE_NAME>,UNIT=OMU,PATH=-LFILES/, DRIVE=WDU-S; ZIWY:D:SYSTEM=<FE_NAME>,UNIT=OMU,DRIVE=FDU-N0,PATH=/; ZIBC;
6.7
Restoring a Front End backup

You can either activate the backup build on the other hard disk or restore the build from a diskette. To restore a Front End backup from a diskette 1. Give the following commands:
ZIWY:S:SYSTEM=<FE_NAME>,UNIT=OMU,PATH=/,DRIVE=FDU-N0; ZIWY:D:SYSTEM=<FE_NAME>,UNIT=OMU,DRIVE=WDU-S, PATH=-LFILES/; ZIBC;
2.
Copy the files from the SYSTEM drive onto the BACKUP drive:
ZIWY:S:SYSTEM=<FE_NAME>,UNIT=OMU,DRIVE=WDU-S, PATH=-LFILES/; ZIWY:D:SYSTEM=<FE_NAME>,UNIT=OMU,
42
NTC TAN 0513/3 en
DRIVE=WDU-B,PATH=-LFILES/; ZIBC;
To activate the Front End backup 1. If you have to take the backup into use, give the following command:
ZWSD:STAT=FB;
2.
Restart the Front End:

ZUSS:SYM:C=DSK,F=TOT;
6.8
Taking a backup of the router configuration

In case the router configuration is accidentally lost, it is good to have a backup of the router configuration. The task sequence below shows how to take backup copies of running-configuration or startup-configuration files with the TFTP server (tftpd). To take a backup of the router configuration 1. 2. Log into the Communications Server as the root user. Create the configuration file into the tftp user's home directory. The file name must be exactly the same as the one you are going to use when you make copy of router's configuration file, for example, router-confg. If the TFTP service is not available, please refer to 8.4, Configuring the TFTP server.
# touch /usr/tftpdir/router_conf/router-confg
3.
Change the file owner to tftp:guest and give the user read and write permissions.
# chown tftp:guest /usr/tftpdir/router_conf/\ router-confg # chmod 600 /usr/tftpdir/router_conf/router-confg
4.
Copy the configuration file to the TFTP server

router# copy running-config startup-config remote host []? <IP_address> Name of configuration file to write [router-confg]? /router_conf/router-confg
Note: Ensure that the full path name the client is requesting from the server exists within the tftp users home directory.
43
NTC TAN 0513/3 en
Write file /router_conf/router-confg on host 111.222.122.92? [confirm] Building configuration... Writing /router_conf/router-confg !! [OK] router# exit
5.
Copy the configuration file from the TFTP server. When you have copied the configuration file from the TFTP server to the startup-configuration, the router has to be restarted to activate the changes.
router# copy tftp startup-config Address of remote host [255.255.255.255]? <IP_address> Name of configuration file [router-confg]? /router_conf/router-confg Configure using router-confg from 111.222.122.92? [confirm] Loading router-confg from 111.222.122.92 (via Ethernet0): ! [OK - 1172/32723 bytes] [OK]
6.
Activate the new start-up configuration by restarting the router:

router# reload Proceed with reload? [confirm]
44
NTC TAN 0513/3 en
UNIX administration
UNIX ADMINISTRATION
This section describes the most common system administration tasks related to the HP-UX operating system and the Nokia NMS. This chapter cover the following topics: Note: For information on tasks to be carried out on servers without the MC/ ServiceGuard software can be found in Section 7.14, Tasks on the Nokia NMS servers with no MC/ServiceGuard software. System shutdown and startup Shutting down and starting up the system Rebooting the NMS servers and workstations Shutting down and starting up the Nokia NMS Administering MC/ServiceGuard Managing the Nokia NMS processes Resource Supervision Service Generic Supervision Service Checking the processes manually Monitoring the wcltoday.log File Checking revisions Managing crontab files Tasks on the Nokia NMS servers with no MC/ServiceGuard software
7.1
System shutdown and startup

Some operating system processes may reduce the available system resources over time. Therefore it may be necessary to reboot the servers and workstations occasionally. This chapter gives instructions on how to halt or restart the system using the shutdown command and describes what happens when the server is brought to a halt or when it is rebooted. The instructions given in this chapter apply to all server configurations.
45
NTC TAN 0513/3 en
UNIX administration
About the shutdown command Using the shutdown command is the correct way to reboot or halt the system. When you enter the shutdown command to halt or reboot the servers, the shutdown program shuts the system down in an orderly and cautious manner. There are a few things that should be borne in mind when you use the shutdown command: If you enter shutdown without specifying any command line arguments, the system is brought down to single-user mode. This mode is often used for system maintenance tasks. In single-user mode you can either enter / sbin/reboot to reboot the system or /sbin/reboot -h to bring the system to a halt. Note that you should not run shutdown -r to reboot the system when you are already in single-user mode but use the reboot command as described above. If you specify -h, the system is halted after the system is brought to single-user mode. If you specify -r, the system is rebooted immediately after it has been brought to single-user mode.
Note: You must not enter the /sbin/reboot command directly in multi-user mode to reboot or halt the system. In the example sequence below, we assume that the current runlevel is 3, which is the default runlevel used by the Nokia NMS servers. The shutdown program goes through the following steps: 1. The PATH environment variable is reset. By default it is set to /usr/ bin:/usr/sbin:/sbin. The value of the variable is modified later when the process kill scripts are run. The IFS (Internal Field Separator) environment variable is set to tab, new line and space. The Bourne shell and other programs uses this variable to split the command line arguments passed to it into distinct arguments where any of the characters are found. The shutdown program checks to see if the current user is authorised to execute the shutdown command. The current working directory is set to the root directory (/). This is done to ensure that file systems can be unmounted. All superblocks on the mounted file systems are updated with the sync command to ensure the integrity of the file systems.
2.
3. 4. 5.
46
NTC TAN 0513/3 en
UNIX administration
6. 7. 8.
The real user ID is set to that of the superuser. The shutdown program sends a broadcast message to all users logged on the system telling them to log out. The program executes the /sbin/rc script to bring down any subsystems. The steps are roughly the following: The PATH environment variable is set to /sbin. The script determines the current runlevel The script executes the process kill scripts in /sbin/ rc<runlevel>.d directory. By running these scripts with the stop parameter all processes are killed in an orderly fashion. When the server is being halted or rebooted, the scripts for the intermediate lower runlevels and the new (target) runlevel will be executed. In other words, with the current runlevel being 3, the scripts for runlevels 2, 1 and 0 are run. In the Nokia NMS environment the following processes (among others) are stopped at this stage if they are running: MC/ServiceGuard cluster CDE login process Nokia NMS processes Oracle SQL*Net processes NMS database processes NMS time synchronisation processes NFS server subsystem System message logging daemon X.25 connections NIS subsystem other processes/subsystems...
9.
The system is halted or rebooted with /sbin/reboot depending on whether -h or -r was specified when the shutdown command was given.
7.2
Shutting down and starting up the system

This section describes how to shut down and start up the NMS servers (Communications Server or Comm&DB Server, Database Server and Standby Server), as well as the NMS workstations (Application Servers).
47
NTC TAN 0513/3 en
UNIX administration
To shut down the NMS servers 1. 2. Log into all servers from the server console as the root user. Shut down the Communications Server:
# shutdown -h -y 0
3. 4.
Repeat the above command on the Standby Server and then on the Database Server. When the servers have been halted and the system informs you that you can switch off the server, turn off the power in the following order. Switch off the power on the servers. Switch off the power on all external disks, screens and other external devices.
To start up the NMS servers 1. 2. 3. 4. Power on all the external devices (disks, screens, and other devices). Wait until the external devices have warmed up. Power on the Communications Server, the Standby Server and the Database Server. Verify that all NMS processes are running. For instructions refer to 7.10, Checking the processes manually. If you have MC/ServiceGuard in use, verify that the MC/ServiceGuard cluster starts up successfully:
# cmviewcl
The table below shows how the MC/ServiceGuard packages have been defined to run on the different servers. The table sums up the supported Nokia NMS server configurations and their MC/ServiceGuard packages (primary nodes). Server Configuration CDS + SS (1 + 1) CS + DB (2 + 0) CS + DB + SS (2 + 1) CDS
db + comm
CS comm comm
DB db + omcdb db + omcdb
SS
spare
spare
Table 13. MC/ServiceGuard packages on the Nokia NMS servers To shut down an NMS workstation 1. Log into the Application Server from the console as the root user.
48
NTC TAN 0513/3 en
UNIX administration
2.
Shut down the Application Server:

# shutdown -h -y 0
3. 4.
Repeat steps 1 and 2 on all other Application Servers that need to be shut down. When the workstation has been halted and the system informs you that you can switch off the workstation, turn off the power in the following order. Switch off the power on the workstation. Switch off the power on all external disks, screens and other external devices.
To start up an NMS workstation 1. 2. 3. Power on all the external devices (disks, screens, and other devices). Wait until the external devices have warmed up. Power on the workstation. Verify that all NMS processes are running. For instructions refer to 7.10, Checking the processes manually.
7.3
Rebooting the NMS servers and workstations

This section describes how to reboot the NMS servers (the Communications Server or Comm&DB Server, Database Server and Standby Server) and workstations (Application Servers). To reboot the NMS servers 1. 2. Log into all servers from the server console as the root user. Reboot the Communications Server:
# shutdown -r -y 0
3. 4. 5.
Repeat the above command on the Standby Server and then on the Database Server. Verify that all NMS processes are running. For instructions refer to 7.10, Checking the processes manually. If you have MC/ServiceGuard in use, you should verify the status of the MC/serviceGuard cluster.
# cmviewcl
When the servers are up and running, you should also reboot the Application Server(s). For more information, see below.
49
NTC TAN 0513/3 en
UNIX administration
To reboot the NMS workstations 1. 2. Log into the Application Server from the console as the root user. Reboot the Application Server:
# shutdown -r -y 0
3. 4.
Repeat Steps 1 and 2 on all other Application Servers that need to be rebooted. Verify that all NMS processes are running. For instructions refer to 7.10, Checking the processes manually.
7.4
Shutting down and starting up the Nokia NMS

As mentioned in Chapter 7, UNIX administration, the wpmanamx process takes care of process supervision and recovery. With Nokia NMS release T10, the wpmanamx process is normally supervised by MC/ServiceGuard, which is why you have to shut down the Nokia NMS by halting the comm package. To terminate all the Nokia NMS processes Warning: Terminating all NMS processes will temporarily make the NMS unusable. Do not terminate the processes unless a maintenance manual or some other NMS documentation specifically instructs you to do so. Also note that if you manually terminate all the processes under supervision, they also have to be restarted manually. See Subsect1 7.6.3, Terminating and starting a process under supervision for details. 1. Enter the following command as the root user on the node running the comm package:
# cmhaltpkg -v comm
The above command will cause Process Supervision and Recovery to be halted as well as the Nokia NMS processes. 2. If you need to access the disks included in the comm MC/ServiceGuard package, you have to remount the disks with the following command:
# /etc/cmcluster/sgsdskmx.sh -e mount comm
To start up the Nokia NMS MC/ServiceGuard supervises the wpmanamx process which is responsible for the monitoring of the other NMS processes. The instructions given in this section presuppose that you have stopped the Nokia NMS processes as described above.
50
NTC TAN 0513/3 en
UNIX administration
1.
If you have mounted the disks belonging to the comm package while the MC/ServiceGuard daemon has been down, you have to unmount the disks before starting up the comm package and the Nokia NMS processes. To unmount the disks give the following command:
# /etc/cmcluster/sgsdskmx.sh -e umount comm
2.
To start all the Nokia NMS processes, give the following command on the Communications Server:
# cmrunpkg comm
This starts the comm MC/ServiceGuard package and the Nokia NMS processes.
7.5
Administering MC/ServiceGuard
This chapter describes the basic management procedures for MC/ServiceGuard in the Nokia NMS environment. The basic concepts are presented in System Management Basic Operating Principles and Procedures, TAN 0732. For more information on MC/ ServiceGuard, see the HP-UX man pages on the cmhaltpkg, cmrunpkg, cmmodpkg, cmhaltcl and cmruncl commands, or refer to the HP manual Managing MC/ServiceGuard. MC/ServiceGuard affects many of the routine maintenance tasks within the Nokia NMS workstation network. If you intend to shut down the database or kill the Nokia NMS processes, you have to do it by closing down the MC/ ServiceGuard services in one of the ways presented below.
7.5.1
Halting the MC/ServiceGuard packages
Many routine system administration tasks require that you halt the MC/ ServiceGuard packages. If you shut down the Nokia NMS database, for example, without first stopping MC/ServiceGuard, the MC/ServiceGuard daemon will try to restart the database as soon as it finds out that the database is down. Therefore the procedure for shutting down the Nokia NMS processes or Nokia NMS database must include the command for halting an MC/ServiceGuard package or the whole cluster. The general syntax for halting an MC/ServiceGuard package is given below:
# cmhaltpkg [ -n <node_name> ] [ -v ] <package> <node_name> <package>
The name of the server The name of the package (comm, db, omcdb or spare)
51
NTC TAN 0513/3 en
UNIX administration
If you do not specify the node, the package will be halted regardless of where it is currently running. If the node is specified, the package will only be halted if it is running on the specified node. Halting the MC/ServiceGuard package also makes the disks belonging to the package inaccessible. After halting the MC/ServiceGuard package you can remount the disks if you need to access them. To halt the MC/ServiceGuard packages 1. Log into the Communications Server as the root user and give the following commands:
# cmhaltpkg -v db comm
2.
After halting the packages you have two options: You can proceed to halting the nodes or the whole cluster. For more information, refer to 7.5.3, Halting an MC/ServiceGuard node or 7.5.5, Halting the MC/ServiceGuard cluster. You can continue working on the node. To access the disks owned by the MC/ServiceGuard cluster they have to be remounted. Give the following commands:
# /etc/cmcluster/sgsdskmx.sh -e mount db # /etc/cmcluster/sgsdskmx.sh -e mount comm
For more information on mounting and unmounting disks on the Nokia NMS servers running MC/ServiceGuard, refer to 7.5.9, Mounting and unmounting file systems within a MC/ServiceGuard cluster.
7.5.2
Starting the MC/ServiceGuard packages
The general syntax for starting an MC/ServiceGuard package is given below:

# cmrunpkg [ -n <node_name> ] [ -v ] <package>
The cmrunpkg command starts the specified package within a cluster. If you do not specify the node with the -n option, the package starts running on the server where you give the command. Sometimes it may be necessary to specify the node, if one of the servers is down and the package has to be moved from one server to another, for example. If you have stopped an MC/ServiceGuard package and remounted the disks belonging to the package, restarting a MC/ ServiceGuard package requires that you first unmount the disks and then restart the MC/ServiceGuard package. The MC/ServiceGuard packages can also be started by enabling the node switching attributes on the packages. For more information on how to enable the node switching attributes, refer to 7.5.8, Enabling MC/ServiceGuard node switching.
52
NTC TAN 0513/3 en
UNIX administration
To start the MC/ServiceGuard packages 1. Log into the Communications Server as the root user. If you have not mounted the disks while the packages have been down, proceed to Step 3. 2. Unmount the disks before restarting the packages:
# /etc/cmcluster/sgsdskmx.sh -e umount db # /etc/cmcluster/sgsdskmx.sh -e umount comm
3.
Restart the packages by enabling the node switching attributes on the packages:
# cmmodpkg -e db comm
The command starts the packages on their primary nodes. 4. Verify that the packages start up normally. Give the following command:
# cmviewcl
The startup takes some time. You may have to repeat the command a few times to see that the packages start up normally. 5. If the MC/ServiceGuard packages do not start up after you have enabled the package switching attributes, you have to restart the packages manually by giving the following command:
# cmrunpkg -v [ -n <server> ] <package>
You must specify the server if the primary node for the package is different from the server on which you are giving the command.
7.5.3
Halting an MC/ServiceGuard node
Instead of halting an MC/ServiceGuard package, you can halt or restart an MC/ ServiceGuard node. You can halt the MC/ServiceGuard daemon on a specific node by giving the following command:
# cmhaltnode [ -f ] [ -v ]
The cmhaltnode command causes the node to stop its MC/ServiceGuard daemon, optionally halting all packages in the process if you specify the -f flag. If you have not halted the packages at an earlier stage on the Communications or the Database Server, halting the node with the -f flag will cause MC/ ServiceGuard to switch the package(s) from the current node to the adoptive node. To halt an MC/ServiceGuard node 1. Log into the server as the root user
53
NTC TAN 0513/3 en
UNIX administration
2.
Halt the node by giving the following command:

# cmhaltnode -v
Note: If you halt one of the primary nodes (CS, DS or CDS) and force the shut down of the packages with the -f flag, MC/ServiceGuard moves the MC/ ServiceGuard package(s) to the adoptive node after halting the primary node. 3. After halting the node you have two options: You can proceed to halting another node, the whole cluster or shutting down the server. For more information on halting the cluster, refer to 7.5.5, Halting the MC/ServiceGuard cluster and 7.5.7, Shutting down and restarting a server after halting a node or the whole cluster. You can continue working on the node. To access the disks owned by the MC/ServiceGuard cluster they have to be remounted. Give the following command:
# /etc/cmcluster/sgsdskmx.sh -e mount <package>
For more information on mounting and unmounting disks on the Nokia NMS servers running MC/ServiceGuard, refer to 7.5.9, Mounting and unmounting file systems within a MC/ServiceGuard cluster.
7.5.4
Starting an MC/ServiceGuard node
The general syntax for starting an MC/ServiceGuard node is given below:

# cmrunnode [ -v ] [ <node> ]
The cmrunnode command causes the node to join itself to the MC/ ServiceGuard cluster. To start an MC/ServiceGuard node 1. Log into the server in the cluster as the root user. If you have not mounted the disks while the node has been down, proceed to Step 3. 2. Unmount the disks before restarting the node:
# /etc/cmcluster/sgsdskmx.sh -e umount <package>
54
NTC TAN 0513/3 en
UNIX administration
Repeat the command for all the packages whose disks you have mounted while the node has been down. 3. Start the node by giving the following command:
# cmrunnode -v
If you have halted the package(s) for which the current server is as a primary node, proceed to Step 4. If there are packages that are currently running on an adoptive node, they have to be manually stopped on the adoptive node and moved back to the primary node. For instructions on how to halt and start packages, see 7.5.1, Halting the MC/ServiceGuard packages and 7.5.2, Starting the MC/ ServiceGuard packages. 4. Restart the package(s) by enabling the node switching attributes on the packages:
# cmmodpkg -e <packages>
5.
Verify that the node and the packages start up normally. Give the following command:
# cmviewcl
The startup takes some time. You may have to repeat the cmviewcl command a few times to see that the packages start up normally. 6. If the MC/ServiceGuard packages do not start up after you have enabled the package switching attributes, you have to restart the packages manually by giving the following command:
# cmrunpkg -v [ -n <server> ] <package>
You must specify the server if the primary node for the package is different from the server on which you are giving the command.
7.5.5
Halting the MC/ServiceGuard cluster
The general syntax for halting the MC/ServiceGuard cluster is given below:
# cmhaltcl [-f] [-v]
The cmhaltcl command causes all nodes configured in the MC/ServiceGuard cluster to stop their MC/ServiceGuard daemons, optionally halting all packages in the process if you specify the -f flag. The command halts all the MC/ ServiceGuard daemons running on the current system and shuts down the Nokia NMS database and the Nokia NMS processes. To halt the MC/ServiceGuard cluster 1. Log into the Communications Server as the root user.
55
NTC TAN 0513/3 en
UNIX administration
2.
Halt the MC/ServiceGuard cluster by giving the following command:

# cmhaltcl -v
The command halts the cluster if the packages have been halted at an earlier stage. If you have not halted the packages before giving the command, you have to specify the -f flag. 3. After halting the cluster there are two options: You can proceed to shutting down or restarting the servers. For instructions, refer to 7.5.7, Shutting down and restarting a server after halting a node or the whole cluster. You can continue working on the node. To access the disks owned by the MC/ServiceGuard cluster they have to be disowned and remounted. Give the following commands:
# /etc/cmcluster/sgsdskmx.sh disown <package> # /etc/cmcluster/sgsdskmx.sh -a mount <package>
7.5.6
Starting the MC/ServiceGuard cluster
The general syntax for starting up the MC/ServiceGuard cluster is given below:
# cmruncl [ -v ]
The cmruncl command causes all nodes in a cluster to start their MC/ ServiceGuard daemons. On the Nokia NMS servers, the cmruncl command mounts the required file systems and starts the Nokia NMS database as well as the Nokia NMS processes. However, if the disks have been disowned while the cluster has been halted, they have to be included in the cluster. For instructions on how to carry out these operations, see 7.5.9, Mounting and unmounting file systems within a MC/ServiceGuard cluster and 7.5.8, Enabling MC/ ServiceGuard node switching. To start the MC/ServiceGuard cluster 1. Log into the Communications Server as the root user. If you have not mounted the disks while the cluster has been down, proceed to Step 4. 2. Unmount the disks before restarting the cluster:
# /etc/cmcluster/sgsdskmx.sh -a umount <package>
Repeat the command for all the packages whose disks you have mounted. 3. Include the disowned disks back in the MC/ServiceGuard cluster:
# /etc/cmcluster/sgsdskmx own <package>
56
NTC TAN 0513/3 en
UNIX administration
Repeat the command for all the packages whose disks you have mounted. 4. Start the cluster by giving the following command:
# cmruncl -v
5.
Verify that the cluster and the packages start up normally. Give the following command:
# cmviewcl
The cluster startup takes some time. You may have to repeat the command a few times to see that the packages start up normally.
7.5.7
Shutting down and restarting a server after halting a node or the whole cluster
After halting an MC/ServiceGuard node or the cluster you can shut down or reboot the Nokia NMS servers. To shut down a server 1. 2. Log into the server as the root user. Bring the server to a halt. Give the following command:
# shutdown -h -y 0
To reboot a server 1. 2. Log into the server as the root user. Reboot the server. Give the following command:
# shutdown -r -y 0
For more information on shutting down and starting up the Nokia NMS servers refer to 7.1, System shutdown and startup and 7.3, Rebooting the NMS servers and workstations.
7.5.8
Enabling MC/ServiceGuard node switching
For MC/ServiceGuard to be able to switch a package onto a different node, you have to enable the switching attributes for the package. You also have to enable node switching after restarting the MC/ServiceGuard cluster for the software to work correctly. You have to do this every time if you have halted a package and restarted it again. The general syntax for enabling the node switching attributes on a package is as follows:
# cmmodpkg { -e | -d } [ -n <node_name> ] [ -v ] <package>
57
NTC TAN 0513/3 en
UNIX administration
The cmmodpkg command above enables the ability of an MC/ServiceGuard package to switch to an adoptive node in case of node failure and starts the package on the primary node if the package has been halted. You can also use the command to prevent a package from running on a certain node by using the -d flag. To enable MC/ServiceGuard package switching attributes 1. Log into the Communications Server as the root user. Enable the node switching attributes on the packages by giving the following command:
# cmmodpkg -e db comm
The command also starts the packages on their primary nodes if they have been halted at an earlier stage.
7.5.9
Mounting and unmounting file systems within a MC/ ServiceGuard cluster
Nokia NMS release T10 includes the /etc/cmcluster/sgsdskmx.sh script for manipulating the access to volume groups and file systems configured in a MC/ServiceGuard package or cluster. You can use the /etc/cmcluster/ sgsdskmx.sh script to mount and unmount file systems, if you need to access the hard disks after halting a MC/ServiceGuard package or cluster. Depending on which commands you use to halt the MC/ServiceGuard packages you have to specify the -a or -e flag after the /etc/cmcluster/ sgsdskmx.sh command. Below is the command line syntax of the script:
# /etc/cmcluster/sgsdskmx.sh { -e | -a } <cmd> <package> <cmd> <package>
Command (mount, umount, own, disown) The name of the package (comm, db, omcdb or spare)
The table below explains the command line arguments of the /etc/ cmcluster/sgsdskmx.sh script: Option
-a
Meaning The flag to be used if the disks are not owned by the MC/ ServiceGuard cluster, non-clustering mode (if you have halted the whole MC/ServiceGuard cluster with the cmhaltcl command, for example).
58
NTC TAN 0513/3 en
UNIX administration
Option
-e
Meaning The flag to be used if the disks are currently owned by the MC/ServiceGuard cluster, clustering mode (if you have halted an MC/ServiceGuard package with cmhaltpkg command, for example).
Table 14. Flags to Be Used with the /etc/cmcluster/sgsdskmx.sh Script Command

mount umount own disown
Meaning Mount the disks configured in the package Unmount the disks configured in the package. Change the volume group to clustering mode. Change the volume group to non-clustering mode.
Table 15. Commands Understood by the sgsdskmx.sh Script.
7.5.9.1
Mounting the disks in clustering mode
The following two task sequences below show you how to mount and unmount the disks belonging to the MC/ServiceGuard packages. You should use this procedure when you have halted a package or node. To mount the disks after halting a package or node 1. 2. Log into the Communications Server as the root user. Mount the disks belonging to the halted package:
# /etc/cmcluster/sgsdskmx.sh -e mount <package>
Repeat the command for all the packages whose disks you are going to mount. The disks are still owned by the cluster after completing the steps above, but they can be accessed normally. To unmount the disks owned by the MC/ServiceGuard cluster 1. 2. Log into the Communications Server as the root user. Mount the disks belonging to the halted package:
Repeat the command for all the packages whose disks you have mounted.
7.5.9.2
Mounting the disks in non-clustering mode
The following two task sequences below show you how to mount and unmount the disks belonging to the MC/ServiceGuard packages. You should use this procedure when you have halted the whole MC/ServiceGuard cluster.
59
NTC TAN 0513/3 en
UNIX administration
To mount the disks after halting the cluster 1. 2. Log into the Communications Server as the root user. Change the disks to non-clustering mode:
# /etc/cmcluster/sgsdskmx.sh disown <package>
Repeat the command for all the packages whose disks you are going to mount. 3. Mount the disks belonging to the halted package:
# /etc/cmcluster/sgsdskmx.sh -a mount <package>
Repeat the command for all the packages whose disks you are going to mount. The disks are not owned by the cluster after completing the steps above. To unmount the disks not owned by the MC/ServiceGuard cluster 1. 2. Log into the Communications Server as the root user. Mount the disks belonging to the halted package:
Repeat the command for all the packages whose disks you have mounted. 3. Change the disks back to clustering mode:
# /etc/cmcluster/sgsdskmx.sh own <package>
Repeat the command for all the packages whose disks you have mounted.
7.5.10
Verifying the functionality of the MC/ServiceGuard cluster
You can view the status of the MC/ServiceGuard cluster by giving the command cmviewcl and verify that the packages are running as they should. For more information on which packages should be running on the servers, refer to System Management Basic Operating Principles and Procedures, TAN 0732. To check the status of the MC/ServiceGuard cluster 1. 2. Log into the Communications Server as the root user Give the following command:
# cmviewcl
The output should show that the required MC/ServiceGuard packages are running. The example below is taken from the Standard Server Configuration for Workstations (2+1).
60
NTC TAN 0513/3 en
UNIX administration
CLUSTER <name> NODE <hostname> PACKAGE comm NODE <hostname> PACKAGE omcdb db NODE <hostname> PACKAGE spare
STATUS up STATUS up STATUS up STATUS up STATUS up up STATUS up STATUS up STATE running STATE running STATE running STATE running running STATE running STATE running PKG_SWITCH enabled NODE <hostname> PKG_SWITCH enabled enabled NODE <hostname> <hostname> PKG_SWITCH enabled NODE <hostname>
Example 4. MC/ServiceGuard Status Report
7.6
Managing the Nokia NMS processes

The functionality of the Nokia NMS system is based on a number of processes which run on the servers. Some processes are transparent to the user in the sense that they are started, run and terminated in the background by the Nokia NMS system. Other processes are started by the user, for example, by selecting Utils Fault Mgmt Alarm Monitor... on the Top-level User Interface. For more information on process supervision in the Nokia NMS refer to System Management Basic Operating Principles and Procedures, TAN 0732.
7.6.1
Configuring Process Supervision and Recovery Service
The Process Supervision and Recovery application only monitors the processes which have been defined to be under its supervision. These definitions are stored in the wpmanamx.cf configuration file, located in the $OMCCONFPATH/ <server> directory. Note: A default configuration file is supplied during commissioning, and normally the system administrator has no need to edit the file. See Technical Reference Guide, TAN 0453 for the syntax of this file.
61
NTC TAN 0513/3 en
UNIX administration
7.6.2
Terminating and starting the Process Supervision and Recovery application
The Process Supervision and Recovery application can be terminated and started manually without disturbing other processes. If you terminate the Process Supervision and Recovery application manually, you must also restart it manually. Note: You should only follow the instructions in this section if your system does not have the MC/ServiceGuard software installed. If your system has MC/ ServiceGuard you cannot terminate the Process Supervision & Recovery application alone. To terminate the Process Supervision and Recovery application To terminate only the Process Supervision and Recovery application, take the following steps: 1. Give the following command to determine the process ID of the wpmanamx process:
omc>% mx | grep wpmana
2.
Send an INT signal to the wpmanamx process:

omc>% kill -INT <PID of wpmanamx>
Entering the above command will terminate the Process Supervision and Recovery application without disturbing the processes under its supervision. To start the Process Supervision and Recovery application If Process Supervision and Recovery was terminated manually, it can be restarted by entering:
omc>% omkickmx -v -p wpmanamx
If Process Supervision and Recovery was terminated in an abnormal way, the cron daemon will start it after the time period specified in the crontab file. The Process Supervision and Recovery application will not be started automatically if one is running already or if it has been terminated manually.
7.6.3
Terminating and starting a process under supervision
Individual processes on the server can also be terminated and restarted manually. Note that if you terminate a NMS process, the functions it normally performs are
62
NTC TAN 0513/3 en
UNIX administration
temporarily disabled. Thus, it is advisable to terminate processes only when you are specifically instructed to do so by a NMS maintenance manual or other NMS documentation. Note: If you manually terminate a process under supervision, it has to be manually restarted. See To restart a terminated process below for details. Manually terminating and restarting a process under supervision requires you to determine the command line arguments the process needs to start again. You can do this by checking the reference file of the process. See below for detailed instructions. When you terminate a process manually, Process Supervision and Recovery will terminate all processes which depend on this process. These "child" processes will automatically be restarted as soon as the "parent" process is running again. To terminate a process under supervision Before terminating a NMS process, you must check the reference file of the process to be able to restart it. This has to be done before terminating the process, as the reference file will be removed when the process is terminated. The reference file specifies the path, the process file name and the command line arguments required to restart the process. To find out this information, take the following steps: 1. 2. Log in as the omc user. Change the working directory to $OMCCONFPATH/<server>. This directory contains the wpmlibmx.cf configuration file, which specifies the directory where the reference files are stored. Check the wpmlibmx.cf configuration file for th.e directory specification. The directory entry should be found on the last line of the file. It may look something like the following:
(directory "/etc/daemons/refs")
3.
This example would mean that the reference files are stored in the /etc/
daemons/refs directory.
4.
Change to the directory specified in the wpmlibmx.cf configuration file and enter ls. You will see the reference files for all the processes currently running. The names of the reference files are of the form <name of the
server>.<process name>.<process ID>
5.
Now you are ready to terminate the process. You can do this by sending the process the TERM signal as the omc user:
63
NTC TAN 0513/3 en
UNIX administration
omc>% kill -TERM <pid>
<pid>
Process ID
The reference file of the process will be removed and the process will be terminated. Note: If you wish to change the arguments of a process which is running in a server, the TERM signal has to be used. If you terminate the process by using some other signal, the process will typically be recovered using the original arguments. To restart a terminated process To be able to restart a NMS process correctly, you must use the appropriate command line arguments. Note that all processes do not require additional arguments. 1. To restart a NMS process, enter the following command as the omc user:
omc>% omkickmx -v -p <process>
7.7
Resource Supervision Service

All the tasks related to hard disk supervision are presented in this section
7.7.1
Configuring disk supervision
Disk supervision parameters are stored in the spymanmx.cf configuration file. The configuration file is stored in the $OMCCONFPATH/<server> directory. For more information on the configuration file, see Technical Reference Guide, TAN 0453. The spymanmx.cf configuration file defines which file systems are monitored by Resource Supervision, together with the alarm points and alarm limits for each alarm class. The working principle of these parameters is basically the same as in database supervision: Field Name
<alarm class>_ENABLE <alarm class>_POINT <alarm class>_LIMIT
Description Specifies whether a certain alarm class is enabled for an individual file system The usage percentage when an alarm is generated The usage percentage when the alarm is cancelled 64
NTC TAN 0513/3 en
UNIX administration
Table 16. Disk supervision parameters in spymanmx.cf The alarms associated with disk supervision are as follows: Alarm No. 9314 9312 9304 9205 9104 9009 Alarm Class Warning Warning Warning Minor Major Critical
Table 17. Disk supervision alarms The system administrator can use the spymanmx.cf configuration file to specify the file systems that needs to be monitored by Resource Supervision. To modify the parameters to suit the needs of your system, use a text editor to edit the $OMCCONFPATH/<server>/spymanmx.cf configuration file. See the Technical Reference Guide for the syntax of this file.
7.7.2
Monitoring the disk space
You should continuously monitor the available disk space. To achieve the optimum performance, the usage of database disks should never exceed 80% of the total capacity. You can monitor the disk usage with Disk Supervision which will send an internal alarm if the specified alarm point is exceeded. Refer to Subsect1 7.7.1, Configuring disk supervision for instructions. If there is less disk space than recommended, try to clean up the disk by looking for unnecessary files, core files or temporary files which could be deleted. These tasks are explained below.
7.7.3
Monitoring log files
The HP-UX operating system writes information into certain log files which will grow unrestricted from a server restart to the next restart. Normally, the workstations may run without restarts for lengthy periods. Therefore it is necessary to truncate and possibly archive log files. The system keeps track of certain log files, and you do not have to pay any special attention to them. Some log files, however, are not monitored by the system, and it is up to you to ensure that they have not grown excessively. The following subsections cover both log types in more detail.
65
NTC TAN 0513/3 en
UNIX administration
7.7.3.1
Log files monitored by the system
The files listed in this section are monitored by the system, which ensures that they do not unnecessarily occupy hard disk space. The wcltoday.log file The NMS uses the UNIX system log to record system management events (command log). This log is located in the /var/adm directory and its name is wcltoday.log. The log files of the last two weeks (14 days) are stored in the same directory and renamed to wclYYMMDD.log. YYMMDD is the date for the log records. The system stores information on some events into this log. These events are, for example: Startup and termination of processes (wpmanamx) Startup and termination of supervision programs (sfwmgrmx) Startup and termination of backup (dbbackmx) MML Session starts as well as commands (wsesammx) Object creations (nedmanmx) Radio Network events (cevmgrmx)
The renaming is carried out every night. Other log files The following log files are also monitored by the system: Information on the SQL*Net TCP/IP Network Servers operations, such as orasrv restarts and connection requests from outside, is stored into the / usr/oracle/tcp/log/orasrv.log file. This file is purged every night. alert file /usr/oracle/rdbms/log/ alert_<ORACLE_SID>.log stores information on all Oracle error situations as well as database closings and openings. It is moved to /usr/ oracle/rdbms/log/alert_<ORACLE_SID>.old every 14 days. The Oracle Listener log file /usr/oracle/tcp/log/tnslsnr.log The $OMCLOGDIR/werlog<number>.log files on each server store information about actions of the Nokia NMS processes. Their sizes are monitored by the system, and extensive growth is not possible. The Oracle RDBMS
Temporary files are often created in the /tmp and /usr/tmp directories. There is an NMS feature which automatically removes the old temporary files (which are older than two days) from these directories. We advise you not to store any permanent data into these two directories.
66
NTC TAN 0513/3 en
UNIX administration
7.7.3.2
Log files to be monitored by the system administrator
The following files should be checked at least every month to ensure that they have not grown excessively: File Name
/var/adm/syslog/ syslog.log
Description This log file is managed by the syslogd process and it stores information according to the settings in the /etc/ syslog.conf file. The ypserv daemon writes its messages, log diagnostics and error messages into this log file. This is only done on the NIS server. You can use the ypwhich command to find out the master NIS server on the WS network. The ypxfr process copies NIS maps from a NIS server to the local host. It writes all its messages to this log file on the NIS server. This log file stores all reboots and halts of the system as well as the user ID of the user who gave the command. All actions by System Administration Manager (SAM) are logged into this log file. The wtmp log holds information on all logins and logouts. The log is used by commands such as last, who, write and login. This log file stores information on remote and console logins. MC/ServiceGuard package log file on workstations equipped with the MC/ ServiceGuard software.
/usr/etc/yp/ypserv.log
/usr/etc/yp/ypxfr.log
/var/adm/shutdownlog
/var/sam/log/samlog
/etc/wtmp
/var/adm/sulog /etc/cmcluster/pkg/ <package>/control.sh.log
Table 18. Files to Be Monitored by the System Administrator Note: If any one of the above listed files grows excessively, its contents should be examined for any potential configuration problems. To clear a log file If a periodic check reveals that one of the above log files is taking up too much disk space and you want to clear the log file, take the following steps.
67
NTC TAN 0513/3 en
UNIX administration
1. 2.
Log in as the root user. Rename the existing log file. For example:
# mv <log_file> <log_file>.old <log_file>
Name of the log file
3.
Create a new file for logging purposes by entering:

# touch <log_file>
For <log_file> substitute the original name of the log file you renamed in step 1. 4. Ensure that the file permissions are correct by entering ls -l <log_file> for both the file you renamed in step 1, and the empty file you created in step 2. If the ownership and permissions are not correct, use chown to change the ownership and then chmod to modify the permissions of the new file. For further study, you can move the <log_file>.old onto a tape or onto another medium. For example:
# tar cvf /dev/dat <log_file>.old
5.
6.
Remove the old version of the file.

# rm <log_file>.old
7.
If the file is managed by syslogd, inform syslogd of the file change by entering:
# kill -HUP `cat /etc/syslog.pid`
7.7.4
Removing core image files
In certain error situations, such as memory violations and illegal instructions in the process code, the system creates a core image of the memory area of the illbehaved process and dumps it in its working directory. The resulting image files are titled core, and they can be quite large. Although these files contain information about the error and can be used to trace the cause of the interrupt, they can often be disregarded and removed. To remove core image files 1. Give the following command as the root user:
# cd /; find . -name core -type f -print \ -exec rm -i {} \;
You will be asked to verify the deletion of each file if the find command finds any core files.
68
NTC TAN 0513/3 en
UNIX administration
7.8
Generic Supervision Service

The parts supervised by Generic Supervision Service (Supervision Framework) are monitored by separate programs, which can also be executed from the command line for testing purposes.
7.8.1
Supervising the X.25 adapters
The sfwx25mx.perl program monitors the X.25 adapters installed on the local NMS workstation. Generic Supervision Service controls the execution of the program according to the parameters set up in the sfwmgrmx.cf configuration file. The program executes the x25ifstate command which is used for probing the X.25 interface cards installed on the NMS workstations. If the x25ifstate detects any anomalies in the X.25 interfaces, the sfwmgrmx.cf program passes an error message on to Generic Supervision Service, which informs the user of the possible error by generating an internal alarm and displaying the error message. You can also run the program from the UNIX command line. Below you see an example of the output when run from the UNIX command line.
(version "1.0" (object "/etc/x25/x25config_0" (info "ok") (status "0")))
Example 5. Sample output from the sfwx25mx.perl script. The program does not require any command-line parameters.
7.8.2
Supervising mirrored disks
The sfwmirmx.perl program monitors all the mirrored logical volumes on the Nokia NMS workstations. The program informs Generic Supervision Service if mirroring has not been activated for a particular logical volume, if a mirror disk fails or if the status of a mirrored disk cannot be determined. The program uses the lvdisplay command to report the status of the logical volumes. If you run the program from the UNIX command line (no parameters required), the program prints out its messages to the standard output. Below you see an example of the output when run from the UNIX command line.
(version "1.0" (object "/dev/vg00/lvol1" (info "ok") (status "0")) (object "/dev/vg01/lvol5" (info "Mirroring is not active on logical volume /dev/vg01/lvol5") (status "1")) (object "/dev/vgdb01/lvol1" (info "ok") (status "0")) (object "/dev/vgdb02/lvol1" (info "ok") (status "0")) (object "/dev/vgglobal/lvol1" (info "ok") (status "0"))
Example 6. Sample output from the sfwmirmx.perl script
69
NTC TAN 0513/3 en
UNIX administration
7.8.3
Supervising the database index tablespaces
The sfwdbimx.perl program monitors the index space allocation of the Nokia NMS database tables. The program supervises that the amount of allocated tablespaces does not exceed a given percentage and that the size of NEXT_EXTENTS to be allocated does not exceed a given percentage of free tablespace. The program cannot be run from the UNIX command line. For more information on other database supervision-related topics, refer to Database Maintenance, TAN 0514.
7.8.4
Supervising the flow of BSC traffic measurements
The sfwmeamx.perl program monitors that all the BSCs that have traffic measurements activated transfer the measurement data to the Nokia NMS. The program looks for measurement data in the Nokia NMS database and checks the stop times of the measurements. If the measurements seem to be late, the program informs Generic Supervision Service, which in turn will generate an internal alarm for the user. If you run the program from the UNIX command line, the program prints out its messages to the standard output. You should use the -f option if you run the program from the command line on another server than the Database Server. Below you see an example of the output.
(version "1.0" (object "BSC" (intid "102645") (info "Traffic measurement data is at least 60 minutes late.") (status "1")) (object "BSC" (intid "114105") (info "Traffic measurement data is at least 60 minutes late.") (status "1")) (object "BSC" (intid "114106") (info "Traffic measurement data is at least 60 minutes late.") (status "1")) (object "BSC" (intid "130388") (status "0"))
Example 7. Sample output from the sfwmeamx.perl script
7.8.5
Supervising the workstation message service
The wnesfwmx program monitors that all locally configured Workstation Network Communication Service Manager applications work properly. The program supervises only the local NMS connections not the gateway connections.
70
NTC TAN 0513/3 en
UNIX administration
If you run the program from the UNIX command line, the program prints out its messages to the standard output. Below you see an example of the output when run from the UNIX command line.
# # amazon Mon Feb 3 14:16:05 1997 # (version "1.0" (object "wnemgrmx connection from amazon to goljat" (info "") (status "1") ) (object "wnemgrmx connection from amazon to sarah" (info "") (status "0") ) ) # end of output
Example 8. Sample output from the wnesfwmx program. In the example above, the workstation message service connection to goljat has failed.
7.8.6
Supervising the OSI connections
OSI Connections Supervision Service (sfwosimx) supervises the OSI connections from the Nokia NMS WSs to an IS. With older configurations equipped with the Nokia NMS FE, OSI Connections Supervision Service monitors the CONS X.25 connections only. With newer Nokia NMS systems, OSI Connections Supervision Service monitors the CLNS LAN connections. When supervising the connections the Generic Supervision Service application (sfwmgrmx) executes the sfwosimx application at given intervals. The sfwosimx application, in turn, uses the sfwosimx.perl script whose execution of the script is controlled by command line arguments.
71
NTC TAN 0513/3 en
UNIX administration
If you run the sfwosimx.perl script from the UNIX command line, the program prints out its messages to the standard output. You have to be logged on as the root user to run the script. Below you see an example of the output when run from the UNIX command line.
# # amazon Mon Feb 3 14:17:05 1997 # (version "1.0" (object "primary X.25" (info "ok") (status "0") ) (object "secondary X.25" (info "Secondary route to front end failed") (status "2") ) ) # end of output
Example 9. Sample output from the sfwosimx.perl script
7.9
Modifying the Alarm Flow Supervision parameters

Alarm Flow Supervision monitors the transfer of alarms from network elements to the NMS and generates an internal alarm if the alarm flow is delayed too much or interrupted. Alarm Flow Supervision can be used with BSCs running version S6 of the BSC software and MSCs running version M8 of the MSC software. You can edit the arcsupmx.cf file to change the supervision parameters to better meet the supervision needs of your network. Below you see an example of the contents of the configuration file:
(objectClass "3" # BSC (maxAllowedAlarmDelay "20") # 20 minutes ) (alarmCheckInterval "120") # 120 seconds
Example 10. Contents of the arcsupmx.cf file If the default delay of 20 minutes is not appropriate for the particular type of network element, you can set the delay longer or shorter. The check interval defines how often network elements are scanned and whether an alarm should be generated. The value is given in seconds and can be changed if the default value is not appropriate. There may also be need to add individual network elements under supervision with a maximum alarm delay of their own. If you know that a certain network element should normally send an alarm at certain intervals, it would make sense
72
NTC TAN 0513/3 en
UNIX administration
to create a separate entry for the particular network element. Create an entry for the network element as in the example below:
(intId "<NE_internal_ID>") (maxAllowedAlarmDelay "<minutes>")
Example 11. Entry for an individual network element You should list all individual elements before the maxAllowedAlarmDelay parameter, because there can be only one common value for all individual network elements.
7.10 Checking the processes manually

The Process Supervision and Recovery application monitors the Nokia NMS processes which have been defined under its supervision. If Process Supervision and Recovery detects that a process has been terminated in an abnormal way, it attempts to restart the process. For more information about the process supervision, see 7.6, Managing the Nokia NMS processes above. The NMS Process Supervision and Recovery application normally takes care of most tasks associated with process supervision. However, there are a number of tasks that the system administrator has to carry out to guarantee the full functionality of the Nokia NMS. If you want to verify that Process Supervision and Recovery is working normally and that all the processes are running properly, there are several ways of monitoring the processes manually. See the following subsections for details. To check that all processes are running The processes which should be running in a server are listed in the file $OMCCONFPATH/<hostname>/wpmanamx.cf. To check which processes are running, take the following steps: 1. Give the following command:
omc>% mx
2.
If one or several of the processes listed in the wpmanamx.cf file are not running, start them:
omc>% omkickmx -v -p <process> <process>
name of the process that you want to start
The following commands can be used to verify the functionality of the Nokia NMS processes:
73
NTC TAN 0513/3 en
UNIX administration
Command
mx mxo
Description Lists the active NMS processes. This command is useful in every server of the configuration. Lists the Oracle processes. This command is useful on the Database Server. You should see the entries for the following processes:
ora_pmon_omc ora_lgwr_omc ora_arch_omc ora_dbwr_omc ora_ckpt_omc ora_smon_omc
x25stat -v
Displays the status of the X.25 connections. This command is useful on the Communications and Standby Servers.
Table 19. Process verification commands. To check automatic process restarts Automatic process restarts and recoveries by Process Supervision are logged into the wcltoday.log file on each server. 1. You can check the contents of the log file for any restart entries on the server by entering the following command:
omc>% grep WPMANA /var/adm/wcltoday.log
If the command displays restart entries, you should make sure that all the Nokia NMS processes (ending in -mx) in the /etc/daemons/refs directory (or in another directory specified in the $OMCCONFPATH/<server>/wpmlibmx.cf file) have their reference files in the directory. You should also check the werlog error log file of the problematic process. They can provide valuable information about the problem situation. The error log files are located in the $OMCLOGDIR directory on each server. For more information error logs, see Technical Reference Guide, TAN 0453.
7.11 Monitoring the wcltoday.log File

You should check the /var/adm/wcltoday.log file daily. The file should not contain any serious error messages. The Process Supervision and Recovery (wpmanamx), Database Connection Supervision (dbkeepmx) and Generic Supervision Framework (sfwmgrmx) applications add entries to the log, which provides valuable information for troubleshooting cases. The file also includes information about any network changes. The Radio Parameter Event Manager (cevmgrmx) application adds an entry for every object creation event on the
74
NTC TAN 0513/3 en
UNIX administration
network to this file. This file usually contains log entries for the Nokia NMS processes, such as database backup (dbbackmx), database cleanup (do-delete), resource supervision (spymanmx, spysizmx, spyanamx), database functions (dbrmuimx), radio network management (rnwmgrmx), BTS hardware management (hwnhndmx), MML sessions (wsesammx) and MML command execution (exemmlmx).
If you find any log entries caused by errors, you should investigate them very carefully.
7.12 Checking revisions

The Nokia NMS has a feature which allows you to check workstation software revisions and control changes in the software. With this tool, you can view the existing versions of the software and check what has been changed and what are the present revisions. It is also possible to compare the present situations at two different sites, for example. The revision control tool consists of two parts. The first part, the torquemx.sh shell script, can be used to create revision files which can be analysed later. The actual analysis is carried out using the torchemx.sh script. It allows you to compare different revision situations and to track the Change Note status, for example. Details about the usage of the revision tracking tools are given in the following subsections.
7.12.1
Creating revision files
As mentioned above, the torchemx.sh script is used to create revision files, which can later be used to analyse changes to the system software. The tool can be used in two ways: To create the original revision file To create revision snapshot files
75
NTC TAN 0513/3 en
UNIX administration
The original revision file can only be created once, and only by the root user. The file is automatically named as original_revisions.txt. As the name suggests, it can be used to compare software revisions with the original configuration. The revision snapshot files can be created later, and compared with the original revision file or other snapshot files. Revision snapshot files can be named revisions_sitename.date, for example. The revision files are saved in the $OMCROOT/lib/build_log directory. Note: You can enter torquemx.sh -help to display a help screen. To create the original revision file To create the original revision file original_revisions.txt, take the following steps. Note that you have to be logged in as the root user. 1. Change the working directory as follows:
# exec /bin/csh # cd $OMCROOT/lib/torche
2.
Give the following command to create the original revision file:

# ./torquemx.sh
Creating the revision file may take a while. Note: If you interrupt torchemx.sh by pressing CTRL + C, all the temporary files in the $OMCROOT/lib/build_log directory are deleted. Some files, however, are stored into the /tmp directory and must be deleted manually at certain time intervals. The output file is automatically named original_revisions.txt and stored in the $OMCROOT/lib/build_log directory. If a file of this name already exists, an error message will be displayed. This file reflects the software revision situation, and it includes the name, version, checksum and date for the revisions. For more information on how to compare revisions, see Section 7.8.2, Comparing Revisions.
76
NTC TAN 0513/3 en
UNIX administration
To create revision snapshot files When you give a file name as a parameter to the torquemx.sh script, it will create a revision snapshot file. This file presents the software revision situation at the time of the script execution. To create a revision snapshot file, give the following commands as the omc user: 1. Change the working directory as follows:
omc>% cd $OMCROOT/lib/torche
2.
Give the following command to create the original revision file:

omc>% ./torquemx.sh <file name>
The torquemx.sh tool will asks you to confirm the file name. If you do not accept this file name, the process will be cancelled. Creating the revision file may take a while. The file is saved in the $OMCROOT/lib/build_log directory. You can create as many snapshots as you wish. A serial number is added to the file name so that no existing files can be overwritten.
7.12.2
Comparing revisions
The torchemx.sh tool allows you to compare software revisions. There are three ways in which the tool can be used: The current software revisions can be compared with the original ones The current software revisions can be compared with the revisions in a given revision snapshot file The software revisions in a revision snapshot file can be compared with the revisions in another snapshot file
Note that the torchemx.sh tool cannot be used to create revision files. For information on how to create revision files, see Section 7.8.1, Creating Revision Files. To compare revisions The software revisions can be compared using the torchemx.sh tool. There are a number of command line parameters which can be used to specify the type of action you want. To compare software revisions, take the following steps: 1. Change the working directory as follows:
% cd $OMCROOT/lib/torche
77
NTC TAN 0513/3 en
UNIX administration
2.
Specify the type of comparison you want to carry out. See the Options and the Examples subsections for details. For example:
% ./torchemx.sh -ch > revision_report.january1997
A revision report is saved to the given file. For information on how to interpret the report, see Section 7.8.2.1, Contents of the Revision Report. Note: You can enter torchemx.sh -help to view a help screen. Options of the torchemx.sh script If you do not specify any file names, the current revisions in the software will be compared with the original revisions which were created using the torquemx.sh tool. You can also choose whether you want to all files or only the changed revisions. If the software has been changed or if some files have been edited, information on the revision date is given. Since the report can be quite lengthy, it is a good idea to redirect the output of the torchemx.sh script to a printer or a file using the standard UNIX commands. The following command line parameters can be given to specify the type of comparison: Command Line Parameter(s) -all Description Current revisions of all files and binaries are compared with the original revisions. If a file has been edited but the revision has not changed, the date is shown. As above, with the exception that only the files and binaries whose revisions have changed or which have been edited are shown. The given revision file is compared with the original revisions. The -all and -ch options can be used to specify which files are shown. Two revision files are compared with each other. You can use this option to compare the SW revision at two different sites, for example, or check which Change Notes have been installed and which have not. The -all and -ch options are applicable as above
-ch
<filename>
<filename1> <filename2>
Table 20. Command Line Parameters for torchemx.sh Usage examples of the torchemx.sh script The following examples present some uses of the torchemx.sh script:
78
NTC TAN 0513/3 en
UNIX administration
Comparing the current software revisions with the original revisions

torchemx.sh -all
Comparing a revision snapshot file with the original revisions so that only the changed and edited files are displayed:
torchemx.sh -ch <filename>
Comparing <filename1> with <filename2>, showing only the changed and edited files:
torchemx.sh -ch <filename1> <filename2>
Contents of the revision report The following table shows how the changes in the software versions are reflected in the revision report. Status Revision of a file has not been changed A file did not exist in the previous revision file Report The existing revision is printed The present revision is displayed together with a message which indicates that the file did not exist in the previous revision file The old and the new revisions are shown The message revision unsolved is displayed
The revision of a file has changed The revision cannot be found
Table 21. Contents of the Revision Report
7.13 Managing crontab files

The user crontabs are different for each user and the tasks defined in the crontabs are dependent on the servers role. To modify the omc, root or oracle user's crontab file, you have to be logged on as the user whose crontab file you intend to modify. The user must also be able to write into the working directory. With MC/ServiceGuard, the user crontabs are located in the role-specific directories in $OMCCONFPATH/<role>. If you need to modify the user crontab files, you have to edit the crontabs in the role-specific directories.
<role>
The name of the role-specific directory under $OMCCONFPATH: aps, cs, ds, cds or sps.
With MC/ServiceGuard, the execution of the user cron jobs is controlled by the MC/ServiceGuard control scripts in the /etc/cmcluster/<package>
79
NTC TAN 0513/3 en
UNIX administration
directory. The control scripts always reactivate the user crontab files from the role-specific directories when the MC/ServiceGuard packages switch nodes, or if the packages or the whole MC/ServiceGuard cluster are halted and then restarted. Even if the global disk is visible on all servers and you can edit the crontabs on one server, you have to activate the crontabs on the server that corresponds to its role. If you are on the Communications Server, for example, you can activate the crontabs in the $OMCCONFPATH/cs directory. An example of a crontab entry is given below:
18 0 * * * csh -c 'nice $OMCROOT/bin/runmemx' > /dev/null 2>&1
Example 12. Example of a Crontab Entry To modify a user's crontab file 1. 2. Log into the Communications Server as the user whose crontab file you want to modify. Change to a role-specific directory:
% cd $OMCCONFPATH/<role>
3. 4.
Edit the users crontab file crontab.<user>.cf in the directory with a text editor. Repeat Steps 2 and 3 with the other server roles in your server configuration. You may have to edit as many as four crontab files for one user. Change to the role-specific directory of the current server.
% $OMCCONFPATH/<role>
5.
6.
Remove the active crontab and install the new crontab:

% crontab -r % crontab crontab.<user>.cf
7.
Verify that the changes you made are correct:

% crontab -l | more
8.
Activate the rest of the modified crontabs on the other servers in your configuration by logging into them and repeating steps 5 - 7 on them.
80
NTC TAN 0513/3 en
UNIX administration
7.14 Tasks on the Nokia NMS servers with no MC/ ServiceGuard software
This section describes tasks to be carried out on servers without the HP MC/ ServiceGuard software. Note: You should only follow the instructions in this section if your system does not have the MC/ServiceGuard software installed. We also assume that your server configuration includes the Comm&DB Server only.
7.14.1
Shutting down and starting up the Nokia NMS servers
The shutdown and startup procedures are the same for servers with and without the MC/ServiceGuard software. For instructions on how to start up the Nokia NMS servers, refer to Section 7.2, Shutting down and starting up the system above.
7.14.2
Starting up and shutting down the Nokia NMS
To shut down the Nokia NMS Note: If you use a LAN for connecting to network elements and shut down the Nokia NMS, you have to reboot the Comm&DB Server to restore Nokia NMS. For information on restarting servers, refer to 7.3, Rebooting the NMS servers and workstations. The following steps should be carried out on each server which runs the Nokia NMS processes. 1. 2. Log into the server as the root user. Close the connections to the network elements: If you use X.25 connections, give the following commands on the Comm&DB Server:
# x25stop -d /dev/x25_0 # exit
Wait a while for the processes to flush the channels.
81
NTC TAN 0513/3 en
UNIX administration
Note: If you have several X.25 cards, you must repeat the x25stop command for each X.25 card, replacing the device specification as necessary, for example:
# x25stop -d /dev/x25_1
If you use LAN connections kill the OTS processes. Give the following commands:
# ps -ef | grep ots
Make a note of the Process IDs (PIDs) of the otsamd and otslogd processes. Kill the processes:
# kill -TERM <otsamd_pid> <otslogd_pid> <otsamd_pid> <otslogd_pid>
The PID of the otsamd process. The PID of the otslogd process.
3.
Change to be the omc user and find out the process ID of the wpmanamx (Process Supervision and Recovery) process:
# su - omc omc>% mx | grep wpmana
The process ID is indicated by the parameter PID in the command output. Note the process ID for use in the following step. 4. Enter the following command to terminate the wpmanamx process:
omc>% kill -TERM <wpmanamx_pid> <wpmanamx _pid >
The PID of the wpmanamx process
When the wpmanamx process receives the TERM signal, it begins terminating other NMS processes. Wait a while for the processes to terminate. 5. Enter the following command to check whether some processes are still running:
omc>% mx
If you see entries for NMS processes, terminate them manually by entering:
omc>% kill <pid>
82
NTC TAN 0513/3 en
UNIX administration
<pid>
The PIDs of the processes still running
When the mx command no longer reports running processes, the processes have terminated. 6. Check that none of the processes left stale reference files when they were terminated by entering:
omc>% cd /etc/daemons/refs omc>% ls
If the directory listing displays reference files, remove them by entering:

omc>% rm *
Repeat the above steps on all servers which run NMS processes. To start up the Nokia NMS The Nokia NMS processes can be started using the omkickmx command. This program is responsible for providing the correct environment and related settings for the user and background processes and for starting the background processes. The configuration is based on the workstation supervision configuration generated during the commissioning of the Nokia NMS. 1. The omkickmx program activates all the processes configured for the NMS to operate. To start all NMS processes, take the following step. As the omc user, give the following command:
omc>% omkickmx -v
Note: You should not normally run this program from the command line, because it causes the processes currently running to be restarted and the program itself does not check for duplicate startups. 2. All the Nokia NMS processes are started. If the X.25 connection has been down, you should start it after starting up the Nokia NMS processes. As the root user, open the X.25 connections:
omc>% su - root # x25init -c /etc/x25/x25config_0
83
NTC TAN 0513/3 en
UNIX administration
Note: If you have several X.25 cards, you must repeat the x25init command for each X.25 card, replacing the device specification as necessary, for example:
# x25init -c /etc/x25/x25config_1
7.14.3
Closing and opening the X.25 connections
You have to shut down and start up the X.25 connections before you wish to shut down the Nokia NMS and after the Nokia NMS has started up. The order of these tasks is important because by closing the X.25 connections before shutting down the Nokia NMS ensures that no events are lost while the Nokia NMS is off-line. This is also why the X.25 connections have to be restored only after the Nokia NMS has been started up. To close the X.25 connections The following steps should be carried out on each server which has X.25 cards installed. 1. 2. Log into the server as the root user. Enter the following command to close the X.25 link:
# x25stop -d /dev/x25_<no.> <no.>
Number of the X.25 card, normally beginning from 0.
Note: If the server has several X.25 adapters installed, you should repeat the above command for each X.25 adapter on the server, changing the card number as necessary. 3. Repeat the above steps on each server equipped with X.25 adapters.
To open the X.25 connections The following steps should be carried out in each server which has X.25 cards installed. 1. 2. Log into the server as the root user. Enter the following command to open the X.25 link:
# x25init -c /etc/x25/x25config_<no.>
84
NTC TAN 0513/3 en
UNIX administration
<no.>
Number of the X.25 card, normally beginning from 0.
Note that if the server has several X.25 cards installed, you should repeat the above command for each X.25 card in the server, changing the card number as necessary. Repeat the above steps in each server which has X.25 cards installed.
7.14.4
Starting up and shutting down the Nokia NMS database
There are several cases in which the database has to be shut down: Certain system maintenance and upgrade tasks which make changes in the system configuration Offline backup The servers need to be shut down or rebooted
Generally speaking, the database should be started back up as soon as possible to enable it to start storing network events again. To shut down the database Note that when you shut down the database, all connections to the database will be lost, and network events will not be stored into the database. Because of this, the X.25 links to the network elements should be closed and the Nokia NMS processes should be shut down in order to enable the network elements to start buffering events. Warning: You should shut down the database only when a task described in a manual or other NMS documentation instructs you to do so. Shutting down the database on other occasions will cause data loss. Take the following steps to shut down the database: 1. 2. Log into the Database Server as the oracle user. Enter the following commands to shut down the database:
# su - oracle oracle>% svrmgrl SVRMGR> connect internal SVRMGR> shutdown immediate
If shutting down the database seems to take an excessive amount of time, the RDBMS may be performing a rollback or another time-consuming
85
NTC TAN 0513/3 en
UNIX administration
operation. You can view the activities the RDBMS carries out by monitoring the alert_omc.log file. Enter the following command:
oracle>% tail -f $ORACLE_HOME/rdbms/log/\ alert_omc.log
If the RDBMS is carrying out a time-consuming task, simply wait for it to complete When the database has been shut down using the immediate option, start it up and shut it down again:
SVRMGR> startup open SVRMGR> shutdown normal SVRMGR> exit
You should receive a notification that the database has been shut down. Then exit the oracle prompt by entering:
oracle>% exit
The immediate option terminates the current database sessions directly, without waiting for each database user to log out. After the database has been shut down using this method, it is started up again. Then, shutting it down normally ensures that all data is written from the redo log files into the database to avoid data loss. To start up the database To start up the database, take the following steps: 1. 2. Log into the Database Server as the oracle user. Enter the following commands to start up the database:
oracle>% svrmgrl SVRMGR> connect internal SVRMGR> startup open
You should receive a notification that the database has been started up. Then exit the oracle prompt by entering:
oracle>% exit
86
NTC TAN 0513/3 en
System modifications
SYSTEM MODIFICATIONS
This chapter describes various changes that may have to made to the workstation network configuration parameters. For a more comprehensive review of network-related topics, refer to DCN Management, TAN 0377. This chapter covers the following sections: Setting up a printer in Alarm Viewer Installing customised sounds for Alarm Viewer Changing the IP address of a server Restricting access to certain services on the local host Changing the NIS configuration Exporting directories
8.1
Setting up a printer in Alarm Viewer

The Alarm Viewer application has a feature for printing alarms to a local printer. This requires a printer that is able to print messages sent in ASCII format. Normal line printers are most suitable for this purpose. To set up a printer for Alarm Viewer 1. 2. 3. 4. 5. 6. Log into the Application Server as the root user. Start the HP System Administration Manager (SAM). In the SAM Areas window, select Printers and Plotters. In the Printers and Plotters window, select LP Spooler. In the Printers and Plotters LP Spooler window, select Printer and Plotters. In the spooler window, select Actions Add Local Printer/Plotter Add Parallel Printer/Plotter or Add Serial Printer/Plotter depending on the port you are going to use. Select the appropriate interface card and click OK. If there are several possible interfaces, you also have to specify the port number. Enter alarm as the printer name and set the model to dumb. If the print spooler was not running before you added the printer, SAM prompts you to start the spooler. Click Yes. Print out a test print from Alarm Viewer to make sure that the installation was successful. For instructions on how to print from Alarm Viewer, see the Alarm Viewer help.
7. 8. 9. 10.
87
NTC TAN 0513/3 en
8.2
Installing customised sounds for Alarm Viewer

With Alarm Viewer you can use customised sounds to inform the users about incoming alarms. To use audio requires a workstation or X station with Audio hardware. Audio hardware is built into all Series 700 workstations except the 720, 730, and 750, and the 705 does not include audio software. To use audio on an X station, you need either an HP ENVIZEX or ENTRIA X station that includes an audio accessory kit. To install customised sounds for Alarm Viewer 1. 2. Log into the Application Server as the user who is going to use the customised sounds. Open an xterm session and ensure that the NCSWAVE and AUDIO environment variables have been set (the AUDIO variable should not have the value none either).
% env | grep NCSWAV % env | grep AUDIO
If the variables have not been set, insert the following two lines into the users .cshrc file:
setenv NCSWAVE <path_to_sound_files> setenv AUDIO <audio_server> <path_to _sound_files>
The path to the directory where the sound files are located, ~/sounds, for example. The name of the workstation used for playing back the sounds.
<audio_server>
Note: You have to restart the whole X session to activate the changes made to the .cshrc file. 3. 4. Start Alarm Viewer from the Top-level User Interface and select Configuration Change Configuration... In the Current Alarm and Regions Settings window, select Change Sound... You see a list of available sound files. Change the sound for the type of alarm or warning by clicking the desired audio file button.
If you have problems with making the audio work, see Solving NMS/2000 Problems, TAN 0470.
88
NTC TAN 0513/3 en
8.3
Changing the IP address of a server

If you change the IP address of a NMS workstation, you must edit several files on the servers. You also have to update the NIS maps when you change the IP addresses of the WSs or the MC/ServiceGuard packages. The following table sums up the changes to be made on different occasions.
IP Address to be changed Edit hosts Edit netconf Edit cmclconf.ascii Edit /etc/cmcluster/ Edit /etc/cmcluster/ pkgcomm/control.sh pkgdb/control.sh Run cmapplyconf Update NIS
CS DS SS AS comm pkg db pkg

Table 22. IP Address Changes to Be Made on Different Servers To change the IP address of a server Note: This procedure cannot be used for changing the hostname of a server. To modify the network configuration files, take the following steps: 1. 2. Log in to the NIS master server as the root user. Edit the /etc/hosts file and replace the IP address entries for all the WSs whose IP address changes. The following table is an example of the contents of the file.
# IP address 127.0.0.1 111.222.111.001 111.222.250.001 111.222.111.005 111.222.111.002 111.222.250.002 111.222.111.003 111.222.111.004 hostname localhost <Comm> <Comm>2 <DB> <Standby> <Standby>2 <App_X> <XTerm_X> comment # loopback # Communications Server # Redundant LAN on # the Communications Server # Database Server # Standby Server # Redundant LAN on # the Standby Server # Application Server # X-Terminal
Example 13. Contents of the /etc/hosts File
89
NTC TAN 0513/3 en
3. 4.
Log into the server, whose IP address is to be changed, as the root user. Edit the /etc/rc.config.d/netconf file and replace the IP address entry for the workstation. The following file listing is an example of the contents of the file.
<...> HOSTNAME="tarzan" OPERATING_SYSTEM=HP-UX LOOPBACK_ADDRESS=127.0.0.1 <...> INTERFACE_NAME[<IF_number>]=lan<IF_number> IP_ADDRESS[<IF_number>]="<IP_address>" SUBNET_MASK[<IF_number>]="<subnet_mask>" BROADCAST_ADDRESS[<IF_number>]="<broadcast_address>" LANCONFIG_ARGS[<IF_number>]="ether ieee" <...>
Example 14. Sample of the /etc/rc.config.d/netconf File

<IF_number> <IP_address> <subnet_mask> <broadcast_address>
The number of the LAN interface The IP address to be assigned to the LAN interface The bit mask for the network Broadcast address for the LAN interface
5.
Halt the MC/ServiceGuard cluster:

# cmhaltcl -f -v
Wait a while for the cluster to halt. 6. On all servers, edit the /etc/cmcluster/cmclconf.ascii, /etc/ cmcluster/pkgcomm/control.sh, and /etc/cmcluster/pkgdb/ control.sh files. Replace the old IP addresses with the new ones. To update the NIS maps, run ypmake on the NIS master server as the root user:
# /var/yp/ypmake
7.
8.
Mount the cluster lock disk for updating the configuration files:
# cd /etc/cmcluster # ./sgsdskmx.sh disown db # ./sgsdskmx.sh -a mount db
9.
Update the MC/ServiceGuard binary files on the Communications Server:

# cmapplyconf -C /etc/cmcluster/cmclconf.ascii \ -P /etc/cmcluster/pkgcomm/pkgcommconf.ascii \ -P /etc/cmcluster/pkgdb/pkgdbconf.ascii \
90
NTC TAN 0513/3 en
-P /etc/cmcluster/pkgsps/pkgspareconf.ascii \ -P /etc/cmcluster/pkgomcdb/pkgomcdbconf.ascii
10.
Copy the updated configuration files to all other servers in the MC/ ServiceGuard configuration:
# rcp -r /etc/cmcluster root@<server_name>:/etc
11.
Unmount the cluster lock disk:

# cd /etc/cmcluster # ./sgsdskmx.sh -a umount db
12.
Reboot the servers (one right after another) by repeating the following command on all servers in the MC/ServiceGuard configuration:
# shutdown -r -y 0
8.4
Configuring the TFTP server

Configuring TFTP (Trivial File Transfer Protocol) server (tftpd) on your system allows you to make files available to remote clients that support TFTP. This protocol is needed when taking a backup of the router configuration, for example. Caution: Implementations of TFTP present a security problem since no user authorisation mechanism exists within the protocol. The combination of the home directory and file access permissions enforces strict security on what remote systems can read and write to your system via TFTP. Furthermore, implementing these features through the /etc/passwd entry forces the system administrator to make a conscious decision about what files are accessible with TFTP. This prevents the administrator from accidentally configuring the TFTP server to be insecure. To configure the TFTP server 1. 2. Log into the Communications Server as the root user. Add the TFTP protocol entry to the /etc/services file:
ftp 69/udp # ARPA Trivial file transfer protocol
3.
Add the following entry to the /etc/inetd.conf file:

tftp dgram udp wait root /usr/lbin/tftpd tftpd
4.
Reconfigure /etc/inetd:
# /usr/sbin/inetd -c
91
NTC TAN 0513/3 en
5.
Add the user tftp to the /etc/passwd file:

tftp:*:510:301:tftp directory: \ /usr/tftpdir: /bin/false
6.
Add the group guest to the /etc/group file:

guest:*:301:tftp
7.
Create the home directory for the user tftp. Create the directory and the file owned by the user tftp, and ensure that the directory and the file gives the user tftp read, write, and execute permissions. For example:
# mkdir -p /usr/tftpdir/router_conf # chown -R tftp:guest /usr/tftpdir/router_conf # chmod -R 700 /usr/tftpdir/router_conf # touch /usr/tftpdir/router_conf/router-confg # chown tftp:guest /usr/tftpdir/router_conf/\ router-confg # chmod 700 /usr/tftpdir/router_conf/router-confg
8.
Place files that you want to make available via TFTP in the home directory of the user tftp. If you want to give remote systems permission to retrieve a file via TFTP, the file must not only exist in the home directory of the user tftp, but also be readable by the user tftp. Verify your tftpd installation. Create a file and use the tftp program to perform a file transfer: Place the file in the home directory, /usr/ tftpdir. Ensure that the file is readable by the user tftp. For example:
# cd /tmp # echo "Hello, I feel good!" > /usr/tftpdir/testfile # chown tftp /usr/tftpdir/testfile # chmod 400 /usr/tftpdir/testfile # tftp localhost tftp> get testfile tftp> quit
9.
Compare the files:

# diff /tmp/testfile /usr/tftpdir/testfile diff should not report any differences.
8.5
Changing the NIS configuration

In the Standard Server Configuration for Workstations, the Communications Server is usually configured as the NIS Master, the Standby Server as an NIS client, and one of the Application Servers must be configured as an NIS slave. If workstations or servers in other subnetworks connect to the Nokia NMS, each of these subnetworks also needs a separate NIS server.
92
NTC TAN 0513/3 en
To change the NIS configuration 1. 2. Log into the NIS master as the root user. Find out the name of your NIS domain and write it down. Give the following command:
# domainname
3.
Stop the NIS services on all servers. Give the following commands on the NIS servers as the root user:
# /sbin/init.d/nis.server stop # /sbin/init.d/nis.client stop
Give the following commands on the NIS clients as the root user:
# /sbin/init.d/nis.client stop
4.
Remove the old NIS data from the NIS servers. Delete the directory that contains the NIS maps.
# rm -rf /var/yp/<domain_name> <domain_name>
The name of the NIS domain
5.
Disable the automatic NIS startup at reboot on all servers. Edit the /etc/ rc.config.d/namesvrs file as below:
NIS_MASTER_SERVER=0 NIS_SLAVE_SERVER=0 NIS_CLIENT=0
6. 7.
Reboot the servers. For instructions on how to reboot the servers refer to 7.3, Rebooting the NMS servers and workstations. Define the NIS server configuration as desired by editing the /etc/ rc.config.d/namesvrs file. Bear in mind what was said about the number of masters, slaves and clients above. On a NIS master server edit the file as follows:
On a NIS slave server edit the file as follows:

On a NIS client edit the file as follows: 93
NTC TAN 0513/3 en
8.
Re-initialise the NIS. Give the following commands on the NIS master as the root user:
# domainname <domain_name> # ypinit -m
Give the following commands on the NIS slaves as the root user:
# domainname <domain_name> # ypinit -s <NIS_master> <NIS_master>
The name of the NIS master server
9.
Restart the NIS processes. Give the following commands first on the NIS master and then on the NIS slaves as the root user:
# # # # /sbin/init.d/nis.server /sbin/init.d/nis.server /sbin/init.d/nis.client /sbin/init.d/nis.client stop start stop start
Give the following commands on the NIS clients as the root user:
# domainname <domain_name> # /sbin/init.d/nis.client stop # /sbin/init.d/nis.client start
8.6
Exporting directories
By default, the NFS mount request server (mountd) does not allow anyone to mount directories from the server. To permit one or more hosts to NFS-mount a directory or a file system, it must be exported. If you want to change the NFS configuration of your workstation network by adding or removing hosts, you have to specify the directories to be exported in the /etc/exports file and list the servers that are allowed to mount the exported directories. To export a directory 1. 2. Log into the Communications Server as the root user. Edit the /etc/exports file and add any hosts to the list or remove the ones that should not be able to mount the directory in question.
<dir> -anon=65534,access=<Standby>:<App_X>,root=<Standby>
94
NTC TAN 0513/3 en
<dir>
The name of the directory (or file system) made available via NFS.
3.
Run /usr/sbin/exportfs to make changes valid:

# /usr/sbin/exportfs -a
4.
Edit the /etc/fstab file on the remote systems. If you added the server to the list of permitted hosts, insert the required parameters to mount the file systems at system startup. If you removed any hosts from the list of permitted hosts, delete the mount requests for the directories from the / etc/fstab file. An example of the /etc/fstab file on a remote server is given below.
# directory mnt pnt type options bu fsck # --------------------------------------------------------------/dev/vg00/lvol1 / hfs defaults 0 1 /dev/vg00/lvol1 /var vxfs delaylog 0 2 fbar:/d/global /m/global nfs rw,retry=100,bg,suid 0
Example 15. Sample of the /etc/fstab File To unexport a directory It may be necessary on some occasions to remove a directory from the list of exported directories. To unexport a directory, take the following steps: 1. Log into the NFS server as the root user and check which NFS clients have mounted the directory you want to unexport:
# /usr/sbin/showmount -a
2.
On every NFS client that has the directory mounted, enter the following command to see the process IDs and user names that are using the mounted directory:
# /usr/sbin/fuser -u <server_name>:<dir>
3.
Instruct any users to change to another directory and kill any processes using the directory:
# /usr/sbin/fuser -ck <local_mount_point> <dir> <server_name> <local_mount_point>
The full name and path of the directory (or file system) made available via NFS. The name of the server containing the exported file system. The name of the directory acting as a mount point for the exported directory on a local workstation.
95
NTC TAN 0513/3 en
4.
On every NFS client that has the directory mounted, edit the /etc/fstab file and comment out or remove the line containing the mount parameters for the unexported directory. On every NFS client that has the directory mounted, give the following command to unmount the directory:
# /usr/sbin/umount <servername>:<dir>
5.
6. 7.
On the NFS server, edit the /etc/exports file comment out or remove the line that contains the parameters for the directory you want to unexport. On the NFS server, enter the following command to unexport the directory:
# /usr/sbin/exportfs -u <dir>
8.7
Restricting access to certain services on the local host

On some occasions, it may be necessary to restrict remote access to certain internet services on the local host. You can restrict the access to the local host by editing the /var/adm/ inetd.sec file. You can define rules that control the access of particular hosts or networks to the internet services on the local host. The internet super-server (inetd) reads this file at startup and when a remote host connects to the local host requesting a certain service, inetd checks the address of the host against the rules defined in the /var/adm/inetd.sec and allows or denies access to the service requested. The /var/adm/inetd.sec file syntax is as follows
<service_name> { allow | deny } [ <IP_address> ] [ <name> ] <service_name> <IP_address> <name>
The name of the services defined in the /etc/ services, or with RPC services, /etc/rpc file. The IP address of a host or network The name of a host or network.
96
NTC TAN 0513/3 en
You can use a text editor or SAM to edit the /var/adm/inetd.sec file. See below for a sample of the file.
# allow rlogin from these two hosts login allow 192.168.1.3 192.168.1.4 # deny ftp access from the network 191.215.0.0 ftp deny 191.215.* s # deny all access to the shell service (remsh) shell deny * # deny telnet from these hosts telnet deny tarzan jane boy
Example 16. Sample of the /var/adm/inetd.sec File The services not listed in the /var/adm/inetd.sec fall back to the default security settings offered by the service itself.
97
NTC TAN 0513/3 en
INDEX
A
access inetd.sec 96 restricting 96 adding alarm pipes 23 alarm lifting modifying rules 21 alarm pipes 23 adding 24 deleting 25 Alarm Viewer customising sounds 88 setting up printer 87 alarms blocking 28 disabling 29 enabling 29 Application Server 14 starting 56 clustering mode 59 Comm&DB Server 14 Communications Server 14 comparing revisions 77 configuring customised sounds for Alarm Viewer 88 disk supervision 64 IP addresses 89 NFS 94 NIS 92 printer for Alarm Viewer 87 TFTP 91 core dumps removing files 68 creating revision snapshots 77 critical files backing up 39 restoring backup 41 crontabs examples 80 managing 79
B
backups 32 critical files 39 Front End 41 restoring a Front End backup 42 restoring a full backup 39 restoring critical files 41 routers 43 taking a full backup 33 blocking alarms 28
D
database shutting down 85 starting up 85 Database Server 14 deleting alarm pipes 23 core image files 68 disabling network alarms and measurements 29
C
Changes from T8 to T9 6 checking cluster 60 network element status 26 NMS status 27 process restarts 74 processes manually 73 closing X.25 connections 84 cluster halting 55
E
edmmanmx disabling network alarms and measurements 29 enabling network alarms and measurements 29 enabling network alarms and measurements 29 node switching 57
98
NTC TAN 0513/3 en
F
Fault Management 21 Front End backing up 41 restoring backups 42 full backup 33 restoring 39
G
Generic Supervision Service 69
H
halting clusters 55 nodes 53 packages 52 hard disks monitoring free space 65 HP 9000 server 15 HP 9000 workstation 15
cmrunpkg 52 cmviewcl 60 mounting disks 58 non-clustering mode 59 sgsdskmx.sh 58 MC/ServiceGuard Packages on the Nokia NMS Servers 48 measurements checking the BSC 30 checking the NMS 31 disabling 29 enabling 29 modifying Alarm Flow Supervision 72 crontab files 79 monitoring disk space 65 log files 65 PM data 30 wcltoday.log 74 mounting disks in cluster 58 disks in clustering mode 59 disks in non-clustering mode 59
I
IP addresses changing 89
N
network moitoring alarm status 27 network monitoring checking elements 26 verifying functionality 26 NFS 15 exporting directories 94 NFS client 15 NFS server 15 NIS 15, 92 reconfiguring 92 NIS client 15 NIS master 15 NIS slave 15 NMS database shutting down 85 starting up 85 NMS processes wpmanamx 73 node cmhaltnode 53 halting 53 node switching 57 restarting server 57 starting 54
L
log files monitored by system 66 monitored by system administrator 67 monitoring 65 Oracle Listener 66 Oracle RDBMS 66 other 66 wcltoday.log 66
M
MC/ServiceGuard 51, 52, 53, 54, 55, 57, 60 checking status 60 clustering mode 59 cmhaltcl 55 cmhaltnode 53 cmhaltpkg 51 cmmodpkg 57 cmruncl 56 cmrunnode 54
99
NTC TAN 0513/3 en
node switching enabling 57 Nokia NMS shutting down 50 starting up 50 non-clustering mode 59
router taking a backup of the configuration 43
S
server configuration 15 shutdown Nokia NMS 50 Nokia NMS without MC/ServiceGuard 81 without MC/ServiceGuard 81 shutting down NMS database 85 Nokia NMS 50 Standby Server 15 starting clusters 56 nodes 54 packages 53 processes under supervision 62 starting up NMS database 85 Nokia NMS 50 startup Nokia NMS 50 Supervision Framework 6, 69 BSC traffic measurements 70 database index tablespaces 70 mirrored disks 69 OSI connections 71 sfwdbimx.perl 70 sfwmeamx.perl 70 sfwmirmx.perl 69 sfwosimx 71 sfwosimx.perl 72 sfwx25mx.perl 69 wnesfwmx 70 Workstation Message Service 70 X.25 adapters 69
O
Occasional tasks 18 opening X.25 connections 84 OSI Connections Supervision Service 71
P
packages cmhaltpkg 51 halting 52 starting 53 Performance Management 30 Periodic tasks 17 Process Supervision and Recovery Service 61, 73 terminating 62 processes checking manually 73 recovering 61 restarting 64 supervision 61 terminating supervised processes 62 verification commands 74 wpmanamx 62
R
recovering processes 61 reports revision 79 Resource Supervision Service 64 alarms 65 configuring 64 parameters 65 spymanmx.cf 64 restoring full backup 39 revision report 79 revisions checking 75 comparing 77 creating with torquemx.sh 75
T
terminating Process Supervision and Recovery Service 62 processes under supervision 62 To change the NIS configuration 93 To Change the Workstation IP Address 89 To Check That All Processes Are Running 73 To Check the Automatic Process Restarts 74 To check the status of the MC/ServiceGuard cluster 60 To Clear a Log File 67
100
NTC TAN 0513/3 en
To close the X.25 connections 84 To compare revisions 77 To create revision snapshot files 77 To create the original revision file 76 To enable MC/ServiceGuard package switching attributes 58 To export a directory 94 To halt an MC/ServiceGuard node 53 To halt the MC/ServiceGuard cluster 55 To halt the MC/ServiceGuard packages 52 To modify a user's crontab file 80 To mount the disks after halting a package or node 59 To mount the disks after halting the cluster 60 To open the X.25 connections 84 To Remove Core Image Files 68 To restart the HP 9000 workstations 50 To restore a critical files backup 41 To shut down an HP 9000 workstation 48 To shut down the database 85 To shut down the Nokia NMS 81 To start an MC/ServiceGuard node 54 To start the MC/ServiceGuard cluster 56 To start the MC/ServiceGuard packages 53 To Start the Process Supervision and Recovery Application 62 To start up the database 86 To start up the HP 9000 servers 48 To start up the HP 9000 workstation 49 To start up the Nokia NMS 50, 83 To take a backup of the critical files 39 To take a backup of the new build 42 To Terminate a Process Under Supervision 63 To terminate all the Nokia NMS processes 50 To terminate the Process Supervision and Recovery application 62 To unexport a directory 95 To unmount the disks not owned by the MC/ ServiceGuard cluster 60 To unmount the disks owned by the MC/ ServiceGuard cluster 59 To verify that BSC measurements are not buffered in the BSC 30 To verify that measurement information is correct in the NMS 31 To verify that measurements from the BSC are transferred to the NMS 30 torchemx.sh 75, 77 examples 78 torquemx.sh 77 truncating log file 67
W
wcltoday.log 66, 74 monitoring 74 workstation network maintenance 17 Workstation Network Modifications 87
X
X terminal 15 X.25 16 X.25 connections closing 84 opening 84
101
NTC TAN 0513/3 en
102
NTC

0513 - 3 Workstation Network Maintenance

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

0513 - 3 Workstation Network Maintenance

Uploaded by

Copyright:

Available Formats

NOKIA NMS/2000

FOR MANAGING CELLULAR NETWORKS

WORKSTATION NETWORK MAINTENANCE

System Administrator's Guide

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

No. of pages 101/MiG

Edited by M. Ganszauge 22 Jan 1998

Author M. Ganszauge 22 Jan 1998

Approved by J. Karppelin 22 Jan 1998

Previous issue (2) approved 15 May 1997

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

ABOUT THIS MANUAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

QUICK REFERENCE TO ADMINISTRATIVE TASKS . . . . . . . . . 17

FAULT MANAGEMENT TASKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

PERFORMANCE MANAGEMENT TASKS . . . . . . . . . . . . . . . . . . . 30

TAKING BACKUPS OF THE NOKIA NMS . . . . . . . . . . . . . . . . . . . 32

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

7.9 7.10 7.11 7.12

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

7.14.2 7.14.3 7.14.4

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

Changes from T9 to T10

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

About this manual

ABOUT THIS MANUAL

NTC TAN 0513/3 en

About this manual

What you need to know first

How to use this manual

4.4.2, Blocking BTS alarms in the BSC

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

About this manual

What You Need What Quick hands-on task reference.

What to Look for in the Manual How to... or To...

Examples To take a backup of the critical files

Where to find more

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

About this manual

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

About this manual

Table 2. Text Styles in This Document

Command line conventions

Table 4. Environment Variables

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

About this manual

Copyright Nokia Telecommunications Oy

NTC TAN 0513/3 en

About this manual

<Communications> <Database> <Standby> <App_X>

Table 5. Text variables

Concepts and terminology