Module

6
Monitoring Utilities
Objectives
At the end of this module, you will be able to:
n List the utilities for monitoring a server
n Use the System Monitoring Interface to run SQL statements to
obtain information about the server

Monitoring Utilities 2018 08-98

1998 Informix Software, Inc.

6-189

Monitoring Utilities
There are several utilities that you can use to monitor a server:
n System Monitoring Interface (SMI)
n onstat
n oncheck

Monitoring Utilities 2018 08-98 190

There are several utilities to monitor server activity.

n The System Monitoring Interface (SMI)
n The onstat utility
n The oncheck utility
Many of the options are used for debugging by Informix technical support only. The
options that can be used for monitoring and tuning purposes are detailed in the
following chapters.

Monitoring Utilities 6-190

SMI
The System Monitoring Interface (SMI) is a read-only access
method to administrative information related to a Dynamic
Server system that:
n Provides SQL access to shared memory structures
n Provides profile information for specific
Shared
user sessions
Memory
n Allows a Dynamic Server administrator to
SYSMASTER
easily automate the system monitoring
Database
process.
SQL

Monitoring Utilities 2018 08-98 191

The SMI provides you with point-in-time information about the contents of the
Dynamic Servers shared memory data structures. The SMI is implemented via the
sysmaster database. There is one sysmaster database for each Dynamic Server
instance. The sysmaster database contains its own system catalog tables and a set
of virtual tables that serve as pointers to shared memory data. The SMI is useful
because it simplifies the task of programming monitoring functions for repetitive
use.
The sysmaster database is documented in the Informix Dynamic Server
Administrators Guide. The sysmaster database schema is also provided and
commented in the $INFORMIXDIR/etc/sysmaster.sql file.

Monitoring Utilities 6-191

The sysmaster Database

Data Dictionary
for sysmaster

Shared
Memory

The sysmaster database is created automatically the first

time a server initializes.
n The database contains data dictionary information that
points to shared memory structures.
n Some sysmaster tables (used by ON-Archive) are real
tables.
Monitoring Utilities 2018 08-98 192

The sysmaster database holds the tables used by SMI to retrieve diagnostic
information. There is one sysmaster database created for each server instance. The
database is created automatically the first time the server is initialized.
Most of the SMI tables do not hold any data. Instead, the data dictionary structures
for that table point to structures in shared memory.
There are some tables included in the sysmaster database that are used by the ONArchive utility. These tables actually hold the data in the table and are not pointing
to shared memory structures.
All users have permission to query the supported tables in sysmaster.

Monitoring Utilities 6-192

How SMI Works

SELECT *
from
syslocks

Shared
Memory

The server recognizes

syslocks as an SMI
table, and reads from
shared memory
instead of disk/buffer
pool.

Monitoring Utilities 2018 08-98 193

When a SELECT statement is executed on a regular table, the server reads the data
dictionary information for the table to find the partition number and other
information about the table. Then it will access the data from disk, if it is not in the
buffer pool.
When a SELECT statement is executed on an SMI table, the server still reads the
data dictionary information for the table listed in the SELECT statement. The SMI
tables have a special partition number (the dbspace number within the partition
number is 0). When the server detects the special partition number, it knows to read
a specific set of data in shared memory to satisfy the query.
Because the SELECT statement is accessing real-time data in shared memory, the
data between one SMI table and another may not be synchronized.

Monitoring Utilities 6-193

SMI Restrictions
n

You cannot lock SMI tables or use isolation levels

n INSERT, UPDATE, and DELETE statements are not
allowed
n dbschema and dbexport cannot be used on the sysmaster
database
n SELECT rowid will return unpredictable results

Monitoring Utilities 2018 08-98 194

There are a few restrictions in using SMI tables. They are listed below:
n You cannot lock the non-permanent SMI tables or use isolation levels. Since
these tables are nothing but shared memory structures, the traditional SQL
locking mechanisms are ineffective (and not desired).
n INSERT, UPDATE, and DELETE statements are not allowed against nonpermanent SMI tables.
n The dbschema and dbexport utilities cannot be used for the sysmaster
database. You will receive the following error:
Database has pseudo tables - cant build schema
n The use of rowid in a SELECT statement is not relevant and will return
inconsistent results.

Monitoring Utilities 6-194

The Sysmaster Database

n
n
n
n
n
n
n
n
n
n
n
n
n

sysdatabases - Databases in the server

systabnames - Tables within a database
syslogs - Logical log information
sysdbspaces - Dbspace information
syschunks - Chunk information
syslocks - Lock information
sysvpprof - VP information
syssessions - Session information
syssesprof - Session level profile information
sysextents - Extent information
syschkio - I/O statistics by chunk
sysptprof -Tblspace profile information
sysprofile - System profile information
Monitoring Utilities 2018 08-98 195

The sysmaster database consists of over 50 tables. Of these tables, only some
tables and some views are supported and documented by Informix. For your
protection, only the supported tables and views should be used in any programs as
the unsupported tables may change between releases. The supported tables and
views are:
n The sysdatabases table lists databases, owner and characteristics of the
database.
n The systabnames table contains the names of all tables in the server. To
retrieve all tables in a database, run:
SELECT tabname FROM systabnames WHERE dbsname = db_name

n The syslogs view contains information about the logical logs. You can use

n
n
n
n
n

syslogs to determine if the logs need to be backed up. If size = used, then the
log is full.
The sysdbspaces view contains information about dbspaces.
The syschunks view contains the chunks in the server. The nfree column
shows the number of pages in the chunk that are free.
The syslocks view lists all active locks.
The sysvpprof view contains all the active virtual processors.
The syssessions view lists information about each session.
Monitoring Utilities 6-195

n The syssesprof view contains more information about each session.

n The sysextents view lists extents allocated in the server.
n The syschkio view contains I/O statistics by chunk.

Continued

n The sysptprof view lists information about the tblspaces at any one point in

time. Only tables currently being used are listed in this view. Once the last user
closes the table, the tblspace structure in shared memory is freed, and
subsequently any profile statistics are lost.
n The sysprofile view lists certain events in the server such as disk reads, disk
writes, roll backs, checkpoints, etc. Each row contains one profiled event and
its value. Some of the values of the name column are shown below:
dskreads
bufreads
dskwrites
bufwrites
isamtot
isopens
isstarts
isreads
iswrites
isrewrites
isdeletes
iscommits
isrollbacks
ovtbls
ovlock
ovuser
ovtrans
compress
fgwrites
lruwrites
chunkwrites

latchreqs
latchwts
buffwts
lockreqs
lockwts
ckptwts
deadlks
lktouts
numckpts
plgpagewrites
plgwrites
llgrecs
llgpagewrites
llgwrites
pagreads
pagwrites
flushes
dpra
rapgs_used
seqscans
btradata

Example
You can use the syschunks table (it is actually a view) to find out the percent used
space in all of the chunks in your server:
Select chknum,(100 * (chksize - nfree)/chksize)
from sysmaster:syschunks
This information can also be derived from the following onstat command:
onstat -d

Monitoring Utilities 6-196

SMI Tables or Views (cont.)

n
n
n
n
n

sysadtinfo - auditing configuration information

sysaudit - auditing event masks
sysconfig - Configuration parameter values
sysdri - Data-replication information
sysseswts - Users wait time on each of several objects

Monitoring Utilities 2018 08-98 197

n The sysadtinfo table contains information about the auditing configuration for

the server. You must be user informix to retrieve information from this table.
n The sysaudit table contains the hexadecimal representation of each defined
audit mask. To list, modify, or add an audit mask you must use the onaudit
utility. You must be user informix to retrieve information from the sysaudit
table.
n The sysconfig table describes the effective, original, and default values of the
configuration parameters.
n The sysdri table provides information on the data-replication status of the
database server.

Monitoring Utilities 6-197

n The sysseswts table provides information on the amount of time users wait for

the following database objects:

Continued

unspecified
buffer
lock
asynchronous I/O
mt yield 0
mt yield n
mt yield
checkpoint
log I/O
log copy
condition

lock mutex
lockfree mutex
lockfree mutex
deadlock mutex
lrus mutex
tblsp mutex
log mutex
ckpt mutex
mutex
mt ready
mt yield x
running

Example
You can join the syslocks and syssessions tables (they are actually views) to list the
current locks, who owns them, and if any session is waiting for a lock:
SELECT username, sid, waiter, dbsname, tabname,
rowidlk, keynum, type
FROM sysmaster:syslocks, sysmaster:syssessions
WHERE syssessions.sid = syslocks.owner
Output from the above statement is similar to the output from the
onstat -u and onstat -k commands.

Monitoring Utilities 6-198

The onstat Utility

Onstat:
n Lists what is in the server shared memory structures at the
instant the command is run
n No disk I/O is performed
n No locking is performed, therefore, it does not impact the
performance of the server

Monitoring Utilities 2018 08-98 199

The onstat utility will read the server shared memory structures and report the
contents of shared memory at the instant that it is run. This means the contents of
shared memory may be changing as they are being printed (as no memory locking
is done by onstat).
The onstat utility prints out the contents of the various internal tables (or data
structures) maintained in shared memory. It does not print out the contents of the
buffer pool. Since these tables keep track of all activity in the server, this tool gives
a good picture of what is going on in the system at the time it is run.
Generally, onstat does no disk I/O; it reads from shared memory alone (there are a
few options that will read from disk files). Because it places no locks on shared
memory resources, it does not impact the performance of the server.

Monitoring Utilities 6-199

Onstat Options
n
n
n
n
n

Interactive option:
onstat -i
Multithreaded options:
onstat -g sub_options
List all options:
onstat -Show status of instance:
onstat Repeat/refresh:
onstat -r <value>
Monitoring Utilities 2018 08-98 200

There are many onstat options available. Many of the options are used for
debugging by Informix technical support only. The options that can be used for
monitoring and tuning purposes are detailed in the following chapters.

Interactive Option
To bypass the overhead of having to connect to shared memory every time an onstat
command is run, you can connect to shared memory once using onstat -i. This
option displays a prompt from which you can enter any onstat option (omit the
hyphen (-) in front of the option). For example:
onstat -i
> u
You can also use the repeat option in interactive mode; for example:
> r 5
> u
This example will run the u option every five seconds. To discontinue the repeated
option, press Control-c.
To exit from the interactive option, press Control-c.

Monitoring Utilities 6-200

Multithreaded Options

Continued

Most of the onstat options display activities in the servers multithreaded subsystem.
These three letter options must be preceded by -g (unless being run with the
interactive option). For example:
onstat -g ath

Monitoring Utilities 6-201

Onstat Header Output

Version X.XX.XXX -- On-Line -- Up 18:35:19 -- 8872 Kbytes
Mode

Uptime

shared memory size

n On-Line
n Off-Line
n Quiescent
n Shutdown
n Recovery
n (CKPT REQ) checkpoint requested, but not started
n (CKPT) checkpoint in progress

Monitoring Utilities 2018 08-98 202

The first line in any onstat output is the header. It contains information about the
current state of the server.

Monitoring Utilities 6-202

Onstat Syntax and Options

Continued

usage: onstat [-abcdghklmpstuzBDFRX][-i] [-r seconds] [-o file] [infile]

-a
Print all info
-b
Print buffers
-c
Print configuration file
-d
Print dbspaces and chunks
-g
MT sub-command (default: all)
-i
Interactive mode
-h
Print buffer hash chain info
-k
Print locks
-l
Print logging
-m
Print message log
-p
Print profile
-s
Print latches
-t
Print tblspaces
-u
Print user threads
-z
Zero profile counts
-B
Print all buffers
-C
Print btree cleaner requests
-D
Print dbspaces and detailed chunk stats
-F
Print page flushers
-R
Print LRU queues
-x
Print transactions
-X
Print entire list of sharers and waiters for buffers
-r
Repeat options every n seconds (default: 5)
-o
Put shared memory into specified file (default:
onstat.out)

Monitoring Utilities 6-203

Onstat Options
onstat -g option

Continued

Multithreaded Options
ath
wai
act
rea
sle
spi
sch
lmx
wmx
con
stk <tid>
glo
mem <pool name|session id>
seg
rbm
nbm
afr <pool name|session id>
ffr <pool name|session id>
ufr <pool name|session id>
iov
iof
ioq
iob
ppf [<partition number> | 0],
tpf [<tid> | 0],
ntu
ntt
ntm
ntd

Print all threads

Print waiting threads
Print active threads
Print ready threads
Print all sleeping threads
Print spin locks with long spins
Print VP scheduler statistics
Print all locked mutexes
Print all mutexes with waiters
Print conditions with waiters
Dump the stack of a specified thread
Print MT global information
Print pool statistics
Print memory segment statistics
Print block map for resident segment
Print block map for non-resident segments
Print allocated pool fragments
Print free pool fragments
Print pool usage breakdown
Print disk IO statistics by vp
Print disk IO statistics by chunk/file
Print disk IO statistics by queue
Print big buffer usage by IO VP class
Print partition profiles
Print thread profiles
Print net user thread profile information
Print net user thread access times
Print net message information
Print net dispatch information

Monitoring Utilities 6-204

Onstat Options (cont.)

Multithreaded Options
Continued

nss <session id>

nsc <client id>
nsd
ppf
sts
dic
qst
wst
prc
dsc
ses <session id>
sql <session id>
dri

Print net shared memory status

Print net shared memory status
Print net shared memory data
Print fragment read/write calls
Print max and current stack sizes
Print dictionary cache information
Print queue statistics
Print thread wait statistics
Print procedure cache information
Print data distribution cache information
Print session information
Print sql information
Print data replication information

Monitoring Utilities 6-205

The oncheck Utility

The oncheck utility is used to:
n Locate index and data page corruption.
n Repair index page corruption.
n Examine data structures on disk.
n Print reports on the various data structures.
n Some oncheck options will place a shared lock on the table
it is processing. This will prevent other users from updating
the table.

Monitoring Utilities 2018 08-98 206

The oncheck utility is used to repair index and data page corruption on disk. It can
also be used to examine other data structures on disk in the server. It also has the
functionality to allow you to print reports containing information on the various
data structures on disk and their usage.
The oncheck utility can only work on the same computer as the server it is checking;
it does not support network or distributed operations.
Some oncheck commands (-pt or -pT) place a shared lock on the table it is
processing. This will prevent other users from updating the table. For very large
tables, examining every data and index page could take a long time.

The -c and -p Options

The syntax and options for the oncheck utility are shown on the following pages.
Notice that there are two primary options, -c and -p, and that these primary options
have their own sub-options. The -c options indicate that certain things should be
checked. The -p options indicate that certain things should be checked and printed
to the screen.

Monitoring Utilities 6-206

Oncheck
Syntax and Options
Continued

Usage: oncheck [-clist] [-plist] [-qny]

[{ database[:[owner.]table]
| TBLSpace number | Chunk number }
{ rowid | page number } ]
-c
Check
r
Reserved pages
e
Extents
c
Database catalogs [database]
i
Table indexes database[:[owner.]table]
I
Table indexes and rowids in index
database[:[owner.]table]
R
Table index check against rows (must be accompanied by
i or I)
d
TBLSpace data rows including bitmaps
database[:[owner.]table]
D
TBLSpace data rows including bitmaps, remainder pages
and blobs database[:[owner.]table]

-p
r
e
c
k
K
l
L
d
D
t

Print
Reserved pages (-cr)
Extents report (-ce)
Catalog report (-cc) [database]
Keys in index (-ci) database[:[owner.]table]
Keys and rowids in index (-pI) database[:[owner.]table]
Leaf node keys only (-ci) database[:[owner.]table]
Leaf node keys and rowids (-cI) database[:[owner.]table]
TBLSpace data rows (-cd) database[:[owner.]table]
[rowid]
TBLSpace data rows including bitmaps, remainder pages
and blobs (-cD) database[:[owner.]table] [page number]
TBLSpace report database[:[owner.]table]

Monitoring Utilities 6-207

Oncheck Options
T

Continued
p
P
B
-q
-n
-y

TBLSpace disk utilization report

database[:[owner.]table]p dump page for the given [table

and rowid | TBLSpace and page number]
Dump page for the given [table[,fragdbs] and rowid |
TBLSpace and page number]
Dump page for the given chunk number and page number
[chunk num and page number]
BLOBSpace utilization for given table(s)
database[:[owner.]table]
Quiet mode - print only error messages
Answer NO to all questions
Answer YES to all questions

Monitoring Utilities 6-208

Lab Exercise
Lab
Exercise

This exercise requires the demonstration database. After this exercise, drop the
demonstration database using DB-Access. You will create it again in a later
module.
Create the demonstration database by entering the following at the command line
(verify that your INFORMIXSERVER environment variable is set first):
dbaccessdemo7 stores# -log
where # represents your student number.
Answer N to abort when you are prompted by the script.
Using sysmaster database tables and views, execute SQL commands to obtain the
following information:
1. The databases in the current system.
2. A list of all chunks that are on-line.
3. The table names in the demo database.
4. The number of extents allocated in each of the tables in the demo database.
5. The percentage of each of the logs that have been filled.
6. The number of locks held by each session (list the user name and the process id
of the client).
7. You may use the function dbinfo(sessionid) from within an SQL statement
to return your sessionid. Use this function to retrieve information from the
sysmaster database regarding your session.
Execute dbaccess and connect to your database. In a separate window, use onstat to
find out the following information:
8. The Informix session id and the UNIX process id (PID) of your dbaccess
session.
9. The status of your dbspaces and chunks.
10. The status of your logical logs.
Use oncheck to obtain the following information:
11. A disk utilization report of the items table in your database.
12. Any indexes and rowids in your database that are inconsistent.

Monitoring Utilities 6-209

Solutions

Solution
The SQL statements are:
Solution

1.

SELECT
FROM

name database
sysmaster:sysdatabases;

2.

SELECT

c.chknum chunk_number,
d.name dbspace,
On-Line status
sysmaster:syschunks c,
sysmaster:sysdbspaces d
c.dbsnum = d.dbsnum
c.is_offline = 0;

FROM
WHERE
AND
3.

SELECT t.tabname
FROM
sysmaster:systabnames t
WHERE t.dbsname = demo_database;

4.

SELECT tabname,
count(*) num_extents
FROM
sysmaster:sysextents
WHERE dbsname = demo_database
GROUP BY tabname
ORDER BY tabname;

5.

SELECT uniqid,
ROUND ((used/size) * 100, 2) percent_used
FROM
sysmaster:syslogs;

6.

SELECT ss.username || - || ss.sid user_session,

sp.locksheld,
ss.pid
FROM
sysmaster:syssesprof sp,
sysmaster:syssessions ss
WHERE sp.sid = ss.sid;

Monitoring Utilities 6-212

Solution
7.

SELECT *
FROM
sysmaster:syssessions ss,
sysmaster:syssesprof sp
WHERE sp.sid = dbinfo ("sessionid")
AND
sp.sid = ss.sid;

8.

onstat -g ses

9.

onstat -d

Solution

10. onstat -l
11. oncheck -pT demo_database:items
12. oncheck

-cI demo_database

Monitoring Utilities 6-213