Professional Documents
Culture Documents
Todays Enterprise
Most of the large organizations have heterogeneous
systems on multiple platforms from multi various vendors.
Due to the advancements in networking, the systems in an
organization are made to discuss with other systems that
reside outside the organization too.
In such a highly complex environment the data has to flow
between different platforms both inside and outside
organizations.
Enterprise Application Integrator (EAI) is a technology
that aims at making the applications talk to each other in
such complex environments.
Traditional Approaches
Remote Procedure Call
Distributed Transaction Processing
RPC
Remote Procedure Call (RPC), the clients can
access the data in a remote machine giving all
the parameters.
But, there is no guarantee that the server
would send the results as the errors in the
network are not accountable.
DTP
DTP followed a method of wrapping the flow of queries
and results between the client and server as single unit of
work.
The popular databases like IBMs DB2, MS SQL, Oracle,
etc. follow DTP. They will take care of the network
failures.
But the complexities involved in developing such
applications are tremendous and they are not supported on
wide range of hardware.
For example, IBMs CICS is available largely on
Mainframes, Tuxedo is available only on Unix systems,
and MTS is available only on Windows. Though CICS is
available on Unix, Windows, and OS/2, it is popular only
on mainframes.
EAI - Advantage
The above systems have their own disadvantages and
advantages.
But the main feature is they are all synchronous, meaning
the server indefinitely waits for a clients request. Both
client and server should be executing together.
EAI - Advantage
EAI - Advantage
MQSeries is the answer!
Rather than communicating directly with the application,
they store the requests in a queue as messages.
They are then forwarded once the receiving application is
ready to process the data.
MQSeries - Middleware
MQSeries is a message queuing system that provides
assured, asynchronous, once-only delivery of data across a
broad range of hardware and software platforms.
MQSeries enables applications / programs to communicate
with each other across a network of dissimilar platforms
using an application program interface called Message
Queue Interface (MQI).
Message queuing is a method of program-to-program
communication. Programs within an application
communicate by writing and retrieving application-specific
data (messages) to/from queues, without having a private,
dedicated, logical connection to link them.
Message Queuing
MQSeries is a middleware that understands message
queuing applications. The fundamental concepts in
MQSeries are Messages, Queues and Queue Managers.
Messaging means that programs communicate with each
other by sending data in messages and not by calling each
other directly.
Queuing means that programs send their messages to a
queue from which they are forwarded to the other
application. Programs communicating through queues need
not be executed concurrently.
Queue managers are the owners of the queues.
What is MQSeries ?
MQ Series is an Enterprise Application Integrator.
It is used in a client/server or distributed environment.
It is a middleware that enables communication between
different applications residing in different platforms.
It provides API called Message Queue Interface (MQI)
that is consistent across all platforms supported by
MQSeries.
Applications use MQI to get messages and put messages
into the queue
Characteristics of MQSeries
Program to Program
Queue
Program to Program
A program can send messages to another program residing
in another computer through MQSeries.
MQSeries will be installed in the computers whose
applications have to communicate with each other.
For example, computer A is an Windows NT and computer
B is a Unix system, then computer A shall have MQSeries
for Windows NT installed, and the computer B shall have
MQSeries for Unix installed.
The applications can communicate with each other with
the help of their respective MQSeries. The queue
managers act as the middle-man sending and receiving
messages between applications.
Asynchronous Model
Queue 1
Queue 2
Asynchronous Model
There are two types of communication between tow
application:
1. Synchronous type in which the program sends a message
to another program and waits for the reply.
2. Asynchronous type, in which an application sends a
message to another without waiting for the reply.
MQSeries adopts the asynchronous model and enables
communication between any number of applications.
Program A can continue to put messages on Queue 1 and is
not blocked by having to wait for a reply to each message.
It can continue to put messages on queue 1 even if Program
B fails.
Time Independent
B
Queue
B
Queue
Not available
B
Queue
A
B
Not available
Queue
A
Not available
Not available
Time Independent
It is time independent in such a way that there is no requirement for
two programs to communicate with each other at the same time.
Even if the applications are not active, the messages are stored safely
in the queue, and once the program becomes active the messages are
supplied to it.
As depicted in the above diagram, Program A can put messages on the
queue and Program B will get them whenever it is ready.
If Program B is busy or is not active, the messages get stored in the
queue until program B is ready to get them. Program A need not be
active till the messages are delivered to the queue.
Even if both the programs are not active, the messages pertaining to
both will wait in the queue.
Architecture of MQSeries
Architecture of MQSeries
MQSeries on
Mainframe
MQSeries on
Windows NT
Application 1
Application 3
MQI
Queue Manager 1
MQI
Application 2
MQI
MM
CC
AA
TCP/IP, SNA
MM
CC
AA
Queue Manager 2
MQI
Application 4
MQSeries Objects
MQSeries Objects
The queue manager component of MQSeries manages
certain resources called MQSeries objects.
It also identifies itself as another object. Those MQSeries
objects are:
Queue managers
Queues
Namelists (Only on OS/390 Version 5.1 products)
Process definitions
Channels
Distribution Lists
MQSeries Objects
Application
Queue Manager
Distribution List
Queues
Name Lists
Process Definitions
Channels
Messages
MQSeries message consists of two parts:
1. Data that is sent from one program to another, and
2. The message descriptor or message header.
Message Descriptor
Message ID
Control Info
Application Data
Name
Account Name
Amount Requested
Message Descriptor
Message
(MQMD)
Descriptor
Details
Version
Message ID / Correlation
ID
Persistent
persistent
Non-
Message Descriptor
Priority
Lifetime of a message
(Expiry)
Message Descriptor
Return address
Format
Message Descriptor
Report options
Feedback
(COA,COD)
Backout Counter
Segmenting
/
grouping information
Queue
Queue
Queue Manager
Queue Manager
Queue manager is an important component of MQSeries
that owns and manages queues and other objects.
A queue manager defines the properties / attributes of
different objects.
Message Queue Interface (MQI) calls can be used to
control these objects when used in an application.
configurations
for
Queues
Queues
An MQSeries queue is a named object on which
applications can put messages, and from which
applications can get messages.
Messages are stored on a queue. Applications access a
queue by using the Message Queue Interface (MQI) calls.
MQI
The MQI comprises the following:
Calls through which programs can access the queue
manager and its facilities.
Structures that programs use to pass data to and from
the queue manager.
Elementary data types for passing data to and from the
queue manager.
MQI
Before a message can be put on a queue, the queue must
have already been created.
A queue is owned by a queue manager, and one queue
manager can own any number of queues.
However, each queue must have a name that is unique
within that queue manager.
Queues are managed physically by their queue managers
but this is transparent to an application program.
Queues
Types of Queue
Type of Queue
Description
Local Queue
Remote Queue
Transmission Queue
Initiation Queue
Dynamic Queue
Alias Queue
Reply-to-Queue
Model Queue
Repository queue
Local Queue
Remote Queue
Remote queue identifies a queue belonging to another
queue manager.
Actually, it is the local definition of a remote queue,
meaning, the messages for an application belonging to
another queue manager are put in the remote queue only.
The definition specifies the name of the remote queue
manager where the queue exists as well as the name of the
local queue in the remote queue manager.
The information you specify when you define a remote
queue object enables the queue manager to find the remote
queue manager, so that any messages destined for the
remote queue go to the correct queue manager.
Alias Queue
An alias queue appears to be a queue, but it is actually a
different name given to another queue.
This means that more than one program can work with the
same queue, accessing it using different names and
different attributes.
Cluster Queue
A cluster queue is a local queue defined by the cluster
queue manager, to be known to all the queue managers of
that cluster.
It is defined in such a way that the queue managers in the
cluster can put messages to a cluster queue without
needing a corresponding remote queue or channel
definitions.
One cluster queue can be advertised in more than one
cluster also.
Transmission Queue
A transmission queue is a local queue, which holds
messages destined for a remote queue.
Every remote queue should be associated with a
transmission queue. Transmission queues are used
internally by the queue managers and are transparent to the
applications.
They are referenced in the remote queue definitions.
Typically, a queue manager can have only one
transmission queue definition, to send messages to any
number of queue managers.
Initiation Queue
An initiation queue is a local queue on which the queue
manager puts a message for the purpose of automatically
starting an application when certain conditions / triggers
are met on a local queue.
Other Queues
System command queue
The system command queue is a queue to which suitably
authorized applications can send MQSeries commands.
System default queue
When a queue is created (other than a dynamic queue),
MQSeries uses the queue definitions stored in the system
default queues.
Channel queue
Channel queues
management.
are
used
for
distributed
queue
Channels
Channels
Two queue managers send messages between each other
via channels.
A channel is the logical communication path between
queue managers.
They shield applications from the underlying
communications protocols.
The queue managers may exist on the same, or different
platforms.
For queue managers to communicate with one another, one
channel object has to be defined at the queue manager that
is to send messages, and another, complementary one, at
the queue manager that is to receive them.
Type of Channels
1. Message Channel
2. MQI Channel
Message Channel
Sender
Receiver
Server
Requester
Cluster sender
Cluster receiver
Compatible Channels
A message channel is defined using one of these types
defined at one end, and a compatible type at the other end.
Possible combinations are:
Sender-receiver
Requester-server
Requester-sender (callback)
Server-receiver
Cluster sender-cluster receiver
MQI Channels
A Message Queue Interface (MQI) channel connects an
MQSeries client to a queue manager in its server machine.
Clients dont have a queue manager of their own.
An MQI channel is bi-directional. The following diagram
shows the context of both the channel definitions.
A channel can use the following transport protocols: SNA
LU 6.2, TCP/IP, NetBIOS, SPX and DEC..
Net. Each platform supports different protocol.
MQI Channel
MQ
Client
MQI Channels
MQ
Client
Queue
Manager
Message Channels
Queue
Manager
Process Definition
Process Definition
A process definition object defines an application to an
MQSeries queue manager.
Typically, in MQSeries, an application puts or gets
messages from one or more queues and processes them.
A process definition object is used for defining
applications to be started by a trigger.
It is used to describe the attributes of an application that is
invoked when an MQSeries trigger event occurs.
The definition includes the application ID, the application
type, and application specific data.
For example, it contains the name of the executable to be
invoked.
Distribution Lists
Distribution List
A distribution list enables an application to send a message
to several destinations with a single MQPUT call.
The list of destinations is supplied by the application.
If more than one of these destinations uses the same
transmission queue, only one copy of the message data is
kept on the transmission queue, and sent down the channel.
It helps to avoid invoking more MQPUT calls.
Name List
Name List
A name list is an MQSeries object that contains a list of other
MQSeries objects.
It is available only in S/390 platform and not on Windows NT.
Typically, name lists are used by applications such as trigger monitors,
to identify a group of queues.
The advantage of using a name list is that it is maintained
independently of applications; that is, it can be updated without
stopping any of the applications that use it.
Also, if one application fails, the name list is not affected and other
applications can continue using it.
Name lists contain the name of one or more cluster. It is created when
a queue manager or its resources need to be available in more than one
cluster.
A cluster is group of queue managers that are logically associated.
MQ Clients
MQ Clients
An MQSeries client is a part of an MQSeries product that
can be installed on a machine without installing the full
queue manager.
It accepts Message Queue Interface (MQI) calls from
application programs, and passes MQI requests to an
MQSeries server that is residing on some other machine.
The MQSeries server is a full queue manager, which can
accept MQI calls directly from application programs that
are running on the server machine; also, it can accept MQI
requests from MQSeries clients.
Advantages of MQ Clients
When there is no full MQSeries implementation for the
machine (for example, because it is a DOS platform).
When the client machine is too small, or of insufficient
processing power, to run a full queue manager with good
performance.
When the application program on the client processor need
to get connected to multiple queue managers on different
server processors.
When systems administration effort has to be reduced.
MQ Clients
When an application program in the client issues an MQI
call, the client formats the parameter values of the call into
an MQI request, and sends the request to the server.
The server receives the request, performs the action
specified in the request, and sends a response back to the
client.
The response is used by the client to generate information
that is returned to the application program using the
normal MQI return mechanism.
MQSeries clients allows a program to be connected to
more than one queue manager at a time, that are residing
either in the same machine or on different machines.
channel
is
MQI is Bi-directional
One MQI channel can be used to send requests in one
direction, and responses in the opposite direction.
With message channels, data can be passed in one
direction only. If two-way communication is required
between two queue managers (for example, in the situation
where reply messages are to be sent to the same queue
manager that handled an initial request message), then two
message channels are required, one to handle messages
traveling in one direction, and another for messages
traveling in the other direction.
Communication Synchronous
When an MQI request is transmitted from a client to a
server, the MQSeries client product must wait for a
response from the server before it can send the next MQI
request.
With message channels, the message traffic on the channel
is time-independent.
Multiple messages can be sent from one queue manager to
the other, without the sending queue manager having to
wait for any replies from the receiving queue manager.
Client Communication
MQ Client
1
TCP/IP
LQueue1
MQ Client
2
LQueue2
Net
BIOS
MQ Server
Queue
Manager 1
LQueue1
LQueue2
LQueue3
SNA LU 6.2
MQ Client
3
LQueue3
Queue
Manager 2
Client Communication
The transmission protocol that is to be used on an MQI
channel is specified as part of the channel definition.
The MQSeries program is unaware of the particular
protocol that is being used on the channel.
If a client has to be connected with more than one
MQServer, then MQI channels have to be defined for each
connection.
Each MQI channel can use different protocols, for
example, an application program could connect to one
queue manager using TCP on one channel, and to another
queue manager using NetBIOS on a different Channel.
A program can define different channel types according to
the requirement, but the same channel name should be
used at both ends of the channel.
Clusters
Clusters
A Cluster is a collection of queue managers that may be on different
platforms but serves a common application. MQSeries supports the
concept of clusters to simplify the setup and other operations.
A queue manager can belong to none, one, or several clusters based on
the requirement.
The queue managers in a cluster can exist on the same or different
machines.
Queue managers in a cluster can make their queues available to every
other queue manager in the cluster.
Clusters
A queue manager can send a message to any other queue manager in the
same cluster without the need for explicit channel definitions, remotequeue definitions, or transmission queues for each destination.
Every queue manager in a cluster has a single transmission queue that
transmits messages to any other queue manager in the cluster.
Each queue manager needs to define only one cluster-receiver channel
and one cluster-sender channel.
A queue manager in the cluster contains the repository of information
about the other queue managers such as the queues, channels, etc. and it
is called Repository queue manager.
Other queue managers in the cluster are called Cluster queue manager.
Clusters
All the cluster queue managers get the information about the
queues from the repository queue manager.
Once the cluster queue manager gets the information about
the queues of the other participating queue managers, it
retains them in for future reference. So, all of them will
have their own partial repositories.
If the same queue has been defined on more than one queue
manager in the cluster, the messages can be routed in the
most efficient manner enabling workload balancing and
failure.
Cluster
QM1
Cluster Queue Manager
CLUSDR
TO QM2
CLUSSDRA
TO QM1
QM2
Repository Queue Manager
CLUSRCVR
TO QM2
Repository
Queue
CLUSRCVR
TO QM1
QM3
Cluster Queue Manager
CLUSDR
TO QM2
CLUSRCVR
TO QM3
Clusters
The cluster shown in the diagram has three queue
managers, QM1, QM2, and QM3.
QM2 is configured as the repository queue manager and it holds
the details about the queues that are created in the other queue
managers.
QM1 and QM3 are the cluster queue managers. Each queue
manager will have just local queue definitions.
Clusters
Each cluster queue managers will have a cluster sender
channel (CLUSSDR) and a cluster receiver channel.
All cluster queue managers need a channel initiator to
monitor the system-defined initiation queue
SYSTEM.CHANNEL.INITQ. This is the initiation queue
for all transmission queue including the cluster
transmission queue.
Also, channel listener has to be run on all the queue
managers to listen to the incoming requests and connect to
the appropriate receiver channel.
Advantages of Clustering
Reduced system administration - Establishing a network
of queue managers in a cluster involves fewer definitions
than a non-clustered MQSeries network.
Also changes to the definitions can be made easily.
Increased availability and workload balancing - In
clusters, more than one queue manager can have a queue in
the same name, which means, a message can be sent to
more than one queue manager at one time.
Triggering
Triggering
Messages can be made to automatically start or stop an
application as soon as they arrive in a local queue, which is
called triggering.
Triggers can be enabled in the local queue in which the
name of the process definition is also mentioned. Process
definition is an object that contains information about the
application (name of the executable) to be invoked.
As soon as a message arrives at the trigger enabled local
queue (application queue), it is informed to the queue
manager through the trigger events.
The queue manager extracts the application to be triggered
from the process definition and creates the trigger message
and puts it in the initiation queue.
Triggering
Initiation queue is continuously polled by another
application called trigger monitor which reads the trigger
message and invokes the application associated with it.
The MQSeries objects that are involved in triggering are:
application queue,
process definition,
initiation queue,
trigger events,
trigger-monitors and
trigger message
Triggering Process
Queue Manager
Trigger Event
Trigger
Message
Application A
Application Queue
Process
Definition
Start Application
Application B
Initiation Queue
Trigger
Monitor
Application Queue
An application queue is a local queue, for which the
triggering is enabled. Three conditions can be set for
triggering:
First (triggers the application when the first message
arrives),
Every (triggers the application for very message) and
Depth (triggers the application when nth message
arrives where n is the depth of the queue)
Process Definition
An application queue can have a process definition object
associated with it that holds details of the application that
will get triggered.
It has the information about the executable (for example,
notepad.exe) to be invoked when the trigger event occurs.
The name of the process definition is specified in the
application queue.
Trigger Event
A trigger event is an event that causes a trigger message to
be generated by the queue manager.
This happens usually when a message arrives in an
application queue, but it can also occur at other times.
MQSeries has a range of options to control the conditions
that cause a trigger event.
Trigger Message
The queue manager creates a trigger message when it
recognizes a trigger event.
It copies information about the application to be started
into the trigger message.
This information comes from the application queue and the
process definition object associated with the application
queue.
Trigger messages have a fixed format
Initiation Queue
An initiation queue is a local queue on which the queue
manager puts trigger messages.
A queue manager can own more than one initiation queue,
and each one is associated with one or more application
queues.
Trigger Monitor
A trigger monitor is a continuously running program that
serves one or more initiation queues.
When a trigger message arrives on an initiation queue, the
trigger monitor retrieves the message, and based on the
information on the trigger message it issues a command to
start the application associated with it.
MQ Application Programming
Application Programming
MQSeries has a common application programming
interface called the message queue interface (MQI).
It helps to write applications to communicate with other
applications on same or different platforms through
MQSeries.
The calls made by the applications and the messages they
exchange are common.
This makes it much easier to write and maintain
applications than using traditional methods.
It also facilitates the migration of message queuing
applications from one platform to another.
MQI
The MQI comprises the following:
Calls through which programs can access the queue
manager and its facilities.
Structures that programs use to pass data to and from the
queue manager.
Elementary data types for passing data to and from the
queue manager.
Queue manager
Queues
Attributes
Transactions
MQCONN
MQCONNX
MQDISC
MQI Queues
MQOPEN
MQPUT
MQPUT1
MQGET
MQCLOSE
MQINQ
MQSET
MQCMIT
Notify syncpoint for all the PUTs and GETs since last
syncpoint
MQBACK
MQ System Administration
MQSeries Administration
MQSeries administration involves creation and
manipulation of different MQSeries objects viz.
While creating the MQSeries objects, the administrator
should follow certain rules like, naming the objects.
The names of the objects should be unique within the
network of interconnected queue managers.
A queue and a process definition can have the same name,
but two queues or two process definitions cannot have the
same name.
Administration Commands
MQSeries provides three command sets to handle the
objects and to perform administration tasks:
Control commands
MQSC commands
PCF commands
Advantages of MQSeries
Advantages
MQSeries applications have the following advantages:
Unix
TCP/IP
C
B
TCP/IP
S/390
SNA LU6.2
With MQSeries
Win NT
M
Q
I
Queue
Manager
Unix
M
C
A
M
C
A
Queue
Manager
M
Q
I
Parallel Processing
Application A
MQI Pseudonym
MQ_Reserve
Flight
MQ_Rent Car
MQ_Reserve
Hotel
Flight
Queue
Car Queue
Confirm
R
E
P
Confirm L
Y
Hotel Queue
Confirm
Queue
Manager
(MQ Server)
Queue
MQ Client
Travel Agent 1
MQ Client
Travel Agent 2
MQ Client
Travel Agent 3
Travel
Database
DB1
DB2
Read
Send
Receive
Write
Syncpoint
Syncpoint
DB1
Read
Put
Syncpoint
Unit of Work 2
Unit of Work 1
Get
Write
Syncpoint
DB2
Unit of Work 3
Unit of Works
Unit of Work 1 Data read from DB1 and put on a queue
for transmission
Unit of Work 2 Data transmitted from one queue to
another queue
Unit of Work 3 Data written on the destination, database
DB2 from the queue.
Unit of Work 1
The first application reads from a database DB1 and puts
the message on a queue and then issues a syncpoint to
commit the read operation.
The message contains the data needed to update the second
database DB2 on the other system.
This unit of work 1 is committed when, this message is
given to the transmission queue for transmission.
Unit of Work 2
In the second unit of work, the MCA of the first system
gets a message from the transmission queue and sends it to
the receiving MCA.
The receiving MCA then puts the message on the
destination queue.
When this unit of work 2 is committed , the message is
made available for the second application.
Unit of Work 3
In the third unit of work, the second application gets the
message from the destination queue updates the database
DB2 using the message received.
Security
MQSeries ensures complete security and data integrity in its
messaging process.
It provides the following features for implementing security:
A queue manager can be authorized to check whether a user is
authorized to enter commands used to manage the queue manager.
A queue manager can check whether a user or an application is
authorized to access an MQSeries object.
A message can be encrypted before it is sent by an MCA to
another MCA.
A message can carry fields like user ID and message context
information to authorize a specific user to access some resources
(both MQSeries and non-MQSeries) on the destination system.
Security
Commands
Programs
User
Queue
Local Q Manager
MCA
MCA
MQMD
User IDMessage
Context
Queue
Remote Q Manager
DB