Salvisberg AQ Design For Best Performance 02

AQ Design for Best Performance
Philipp Salvisberg
Technology Manager
Application Performance Mgmt
philipp.salvisberg@trivadis.com
DOAG Conference
Nürnberg, 1st December 2008
Basel · Baden · Berne · Lausanne · Zurich · Düsseldorf · Frankfurt/M. · Freiburg i. Br. · Hamburg · Munich · Stuttgart · Vienna
Who am I?
Technology Manager APM and partner at Trivadis AG

philipp.salvisberg@trivadis.com
www.trivadis.com
Head of the
Main focus on database centric development with Oracle DB

Application Performance Management (APM)
Application Development
Business Intelligence
20 years experience in using Oracle products
AQ Design for Best Performance 2 © 2008

Application Performance Management Context
Business Logic
Presentation
Fat Client Services
Business Logic
Presentation
Integration
Rich Client (ESB, EIB Databases
Data Access)
Business Logic
Thin Client Presentation Files
Frontend Middleware Resources

Agenda
Introduction
Queue Types
Case Study
Benchmarks
Comprehensive
Textmarkierung
Daten sind
Schrift Verdana
consultancy
Grösse 16 is
Recommendations
immer
Farbe weiss im Spiel.
the through ball
to your Conclusion
performance.
Bildmarkierung

What is Advanced Queuing?
Simply put, it‘s enqueue…
… and dequeue

What is Advanced Queuing good for?
Decoupling systems, components through asynchronous processing
Designing workflows as a sequence of services where each service

passes data as messages via dedicated queues to subsequent services
Concurrent execution of subsequent services to increase overall

throughput
Exchange messages between other queues

Content based
Local / remote
Oracle AQ, IBM WebSphere MQ, TIBCO Rendezvous
Visualize process state and increase stability

Using Transactions (local or XA)
Defined restart procedure after a system crash

Where is Advanced Queuing used?
Examples:
Applications based on an ESB (SOA, BPEL, …)
Data Replication with Oracle Streams
Oracle Scheduler
Avaloq Banking System
Trivadis Output Management Service (TOMAS)

Agenda
Introduction
Queue Types
Case Study
Benchmarks
Comprehensive
Textmarkierung
Daten sind
Schrift Verdana
understanding
Grösse 16 is
Recommendations
immer
the through ball
to your Conclusion
performance.
Bildmarkierung

Single Consumer Queue (Point-to-Point)
App A App B
Request Queue
enqueue dequeue
Producer Consumer
dequeue enqueue
Consumer Producer
Response Queue

Message States*
Ready
Expired Waiting
Processed
*= additional sub-states exist, e.g. for ready: “in memory”, “deferred”, “spilled”, “deferred spilled”
Single Consumer Queue – Table Structure
Column Type PK Index 1 Index 2 Comment
Q_NAME VARCHAR2(30) 1
MSGID RAW(16) 1 Generated GUID
CORRID VARCHAR2(128) Correlation ID
PRIORITY NUMBER 3 Message Priority
STATE NUMBER 2
DELAY TIMESTAMP(6)
EXPIRATION NUMBER
TIME_MANAGER_INFO TIMESTAMP(6) 1
LOCAL_ORDER_NO NUMBER 7
CHAIN_NO NUMBER 6
CSCN NUMBER
DSCN NUMBER
ENQ_TIME TIMESTAMP(6) 4
ENQ_UID VARCHAR2(30)
ENQ_TID VARCHAR2(30)
DEQ_TIME TIMESTAMP(6)
DEQ_UID VARCHAR2(30)
DEQ_TID VARCHAR2(30)
RETRY_COUNT NUMBER
EXCEPTION_QSCHEMA VARCHAR2(30)
EXCEPTION_QUEUE VARCHAR2(30)
STEP_NO NUMBER 5
RECIPIENT_KEY NUMBER
DEQUEUE_MSGID RAW(16)
SENDER_NAME VARCHAR2(30)
SENDER_ADDRESS VARCHAR2(1024)
SENDER_PROTOCOL NUMBER
USER_DATA SYS.AQ$_JMS_MESSAGE Payload
USER_PROP SYS.ANYDATA

Multi Consumer Queue (Publish-Subscribe)
App A App B
Producer Consumer
dequeue
enqueue B B B
C C C
App C
dequeue Consumer

Multi Consumer Queue – Table Structure
AQ$_SAMPLE_MC_QT_T SAMPLE_MC_QT AQ$_SAMPLE_MC_QT_I AQ$_SAMPLE_MC_QT_S
*PK NEXT_DATE Q_NAME *pfK SUBSCRIBER# *PK SUBSCRIBER_ ID

*PK TXN_ID
0..* 1 *PK MSGID *PK NAME * QUEUE_NAME
*pfK MSGID CORRID *PK QUEUE # NAME
ACTION PRIORITY *PK MSG_PRIORITY ADDRESS
STATE *PK MSG_ENQ_TIME PROTOCOL
DELA Y 1 0..* *PK MSG_STEP_NO 0..* 1 SUBSCRIBER_TYPE
EXPIRATION *PK MSG_CHAIN_NO RULE_NAME
TIME_MANAGER_INFO *PK MSG_LOCAL_ORDER_NO TRANS_NAME
LOCAL_ORDER_ NO *pfK MSGID RULESET_NAME
CHAIN_ NO HINT NEGATIVE_RULESET_NAME
CSCN SPARE
DSCN
ENQ_T IME
ENQ_ UID
ENQ_T ID
DEQ_T IME
AQ$_SAMPLE_MC_QT_H
DEQ_ UID
DEQ_T ID *pfK MSGID
RETRY_COUNT *pfK SUBSCRIBER#
EXCEPTION_QSCHEMA *PK NAME
EXCEPTION_QUEUE *PK ADDRESS#
STEP_NO DEQUEUE_TIME
RECIPIENT_KEY 1 0..* TRANSACTION_ID 0..* 1
DEQUEUE_MSGID DEQUEUE_ USER
SENDER_NAME PROPAGATED_MSGID
SENDER_ADDRESS RETRY_COUNT
SENDER_PROT OCOL HINT
USER_DATA SPARE
USER_PROP
AQ$_SAMPLE_MC_QT_G
*pfK MSGID
*pfK SUBSCRIBER#
*PK NAME
1 0..* 0..* 1
*PK ADDRESS#
SIG N
DBS_ SIGN

Buffered Queues
Officially since 10gR2 in Memory

(SGA)
Stores Queues in Memory
SGA
stream_pool_size
“faster”
Same API with new

Delivery-Mode „Buffered“ for
enqueue and dequeue

Buffered Queues – Table Structure
AQ$_SAMPLE_MC_QT_P SYS.QT<...>_ BUFFER AQ$_SAMPLE_MC_QT_S
Q_NAME ADDR *PK SUBSCRIBER_ ID

*PK MSGID INDX * QUEUE_NAME
CORRID INST _ID NAME
PRIORITY OBJNO ADDRESS
ST ATE *pfK QUEUE_ ID PROTOCOL
DELA Y FK MSGID SUBSCRIBER_TYPE
EXPIRATION CORRID 0..* 1 RULE_NAME
TIME_MANAGER_INFO SEQUENCE_NUM T RANS_NAME
LOCAL_ORDER_ NO *pfK MSG_NUM RULESET_NAME
CHAIN_ NO ST ATE NEGAT IVE_RULESET_NAME
CSCN PRIORITY
DSCN EXPIRAT ION
ENQ_T IME ENQ_T IME
ENQ_ UID ENQ_ UID
ENQ_T ID 1 0..* ENQ_USER_NAME
DEQ_T IME RETRY_COUNT
DEQ_ UID SENDER_NAME
DEQ_T ID SENDER_ADDRESS AQ$_SAMPLE_MC_QT_D
RETRY_COUNT SENDER_PROTOCOL
EXCEPTION_QSCHEMA DEQUEUE_MSGID *PK QID
EXCEPTION_QUEUE SRCSEQUENCE_NUM *PK MSGNUM
ST EP_NO *pfK SUBSCRIBER_ ID *PK SUB
RECIPIENT_KEY EXCEPTIONQ_SCHEMA MSGID
DEQUEUE_MSGID EXCEPTIONQ_NAME SEQNUM
0..1 1
SENDER_NAME DBMS_AQ_BQVIEW.GET_ADT _PAYLOAD RSUBS
SENDER_ADDRESS
SENDER_PROT OCOL
USER_DATA
USER_PROP

Buffered Queues – Downside
Cannot participate in transactions

Autonomous transactions only!
Possible Data loss after instance shutdown

In-memory storage only!
Data consistency becomes a responsibility of the application
ORA-25293: Lob attributes must be Null for buffered operations

JMS payloads cannot be used for large texts and binary data
No restriction for large object type like SYS.ANYDATA
ORA-25307: Enqueue rate too high, flow control enabled

Producer is blocked when more than 5000 messages are not consumed
Prevents application from flooding the shared pool
To increase the flow control limit see Metalink Note 551516.1

Main Types of Advanced Queues - Recap
Persistent, Single Consumer Queue Persistent, Multi Consumer Queue
Queue tables stored on disk Queue tables stored on disk
Participating or autonomous transactions Participating or autonomous transactions
Persistent
LOBs supported LOBs supported

Propagation support for destination only Full propagation support
Buffered, Single Consumer Queue Buffered, Multi Consumer Queue

Primarily in-memory-queue (SGA) Primarily in-memory-queue (SGA)
Flow control limits queue length Flow control limits queue length
Buffered
Autonomous transactions only Autonomous transactions only

Data loss on instance shutdown Data loss on instance shutdown
LOBs not supported LOBs not supported
Propagation support for destination only Full propagation support except over
but not over messaging gateway messaging gateway
Single Consumer Multi Consumer

Agenda
Introduction
Queue Types
Case Study
Benchmarks
Comprehensive
Textmarkierung
Daten sind
Schrift Verdana
understanding
Grösse 16 is
Recommendations
immer
the through ball
to your Conclusion
performance.
Bildmarkierung

Project Context – High Volume Printing
KPIs Today:
B/W Printing
500 - 1000 pages per minute
Full-Color Printing
Up to 300 page per minute
“Tuning”
Double-sided printing on A3
Get 2 A4 pages by cutting
Quadruple Throughput
~ 4000 pages per minute
~ 67 pages per second

Trivadis BC Architecture Blueprint

TOMAS – Transactional Behavior
1st Transaction
2nd Transaction
3rd Transaction (per Document)
4th Transaction
5th
6th Transaction

TOMAS – Number of Messages
Example:
5 Mio Docs
Control Flow
500 Input Files
2’000 Enq.
2’000 Deq.
Document Flow
10’000 Docs/File
5 Mio Enq.
5 Mio Deq.

Agenda
Introduction
Queue Types
Case Study
Benchmarks
It's about sindtime for
Textmarkierung
Daten
Schrift Verdana
some
Grösse 16 results
Recommendations
immer
Conclusion
Bildmarkierung

msg/sec
Measuring What Counts?
Commit Interval
Operation
1000
Enq, Deq
Volume
16000, 4800
Consumer
Single, Multi
Message Type
Persistent, Buffered
Recipients
1
Block Size Metrics
8K, 2K Response Time,
Processing
DB Time,
AS, DB
Logical I/O
Other Parameter
“fixed values” Queue Usage
Shared, Private
DOP
1, 2, 4, 8, 16 Bulk?
No, Yes (10)
Payload Size
100 (~330) Payload Type
aq$_jms_message

Test Environment
Application Server Database Server
Intel XEON E5345 IBM PowerPC_POWER5
2 Quad-Cores CPU 4 Dual-Core CPU

at 2.33 GHz at 1.5 GHz
16 GB RAM 27 GB RAM
SLES9, 64-bit AIX 5.3 64-bit
JDK 1.5.0_14 Oracle RDBMS 10.2.0.3

Single Consumer Enqueue – Overview
4'000 Blocksize - Bulk? - Queue Usage
MESSAGE_TYPE Persistent
3'500
VOLUME 16000
CONSUMER Single
OPERATION Enqueue
3'000 PAYLOAD_SIZE 100
MESSAGE_PROCESSING AS
2'500 2 - No - Private
2 - No - Shared
msg/sec
2 - Yes - Private
2'000
2 - Yes - Shared
8 - No - Private
1'500 8 - No - Shared
8 - Yes - Private
8 - Yes - Shared
1'000
500
0
1 2 4 8 16
DOP
Single Consumer Enqueue – No Bulk
12'000 160'000
VOLUME 16000
CONSUMER Single
140'000 OPERATION Enqueue
120'000 BLOCKSIZE 8
BULK_PROCESSING No
8'000 QUEUE_USAGE Private
100'000
Logical I/O
msg/sec
6'000 80'000 msg/sec (Response Time)

msg/sec (DB Time)
LIO
60'000
4'000
3'425
2'892 40'000
2'000
1'632 20'000
845
411
0 0
1 2 4 8 16
DOP
Single Consumer Enqueue – Bulk 10 Msgs
3'500 180'000
VOLUME 16000
160'000
CONSUMER Single
PAYLOAD_SIZE 100
140'000 MESSAGE_PROCESSING AS
2'591 BLOCKSIZE 8
2'500 BULK_PROCESSING Yes
120'000 QUEUE_USAGE Private
2'000
100'000
Logical I/O
1'864
msg/sec
msg/sec (Response Time)

80'000 msg/sec (DB Time)
1'500
1'374 LIO
60'000
1'000
779 40'000
500 547
20'000
0 0
1 2 4 8 16
DOP
Single Consumer Dequeue – Overview
3'000 Blocksize - Bulk? - Queue Usage
VOLUME 16000
2'500
CONSUMER Single
OPERATION Dequeue
PAYLOAD_SIZE 100
2'000
2 - No - Private
2 - No - Shared
msg/sec
2 - Yes - Private
1'500
2 - Yes - Shared
8 - No - Private
8 - No - Shared
1'000
8 - Yes - Private
8 - Yes - Shared
500
0
1 2 4 8 16
DOP
Single Cons. Dequeue – Private vs. Shared Queue
3'000 4'000'000 MESSAGE_TYPE Persistent
VOLUME 16000
2'814
CONSUMER Single
3'500'000 OPERATION Dequeue
3'000'000 BLOCKSIZE 8
BULK_PROCESSING No
2'000
1'926
2'500'000
Logical I/O
msg/sec
Private - msg/sec (Response Time)

1'500 2'000'000
Shared - msg/sec (Response Time)
Private - LIO
1'122 1'500'000 Shared - LIO
1'000
1'000'000
661
500
339 500'000
167'112
0 0
1 2 4 8 16
DOP
Single Cons. Enqueue – Private vs. Shared Queue
4'000 200'000 MESSAGE_TYPE Persistent
VOLUME 16000
180'000 CONSUMER Single
3'425
PAYLOAD_SIZE 100
160'000 MESSAGE_PROCESSING AS
3'000 BLOCKSIZE 8
2'892 143'345 BULK_PROCESSING No
140'000
2'500
120'000
Logical I/O
msg/sec
Private - msg/sec (Response Time)

2'000 100'000
Shared - msg/sec (Response Time)
1'632 Private - LIO

80'000
1'500 Shared - LIO
60'000
1'000
845 40'000
500
411 20'000
0 0
1 2 4 8 16
DOP
Summary AQ Throughput (Single Consumer)
5 Mio Enqueues
3'349 3'425
Shared Queue, 3500
3349 Mgs/sec, 1493 sec 3000 2'833

Private Queue,
Messages/Second
2500
3425 Mgs/sec, 1460 sec
2000
5 Mio Dequeues 1500 1'239
Shared Queue,
1239 Mgs/sec, 4035 sec 1000
Private Queue, 500

3425 Mgs/sec, 1765 sec
0
Messaging Overhead Enqueue, shared Queue

30 to 54 Minutes Enqueue, private Queue
Dequeue, shared Queue
Dequeue, private Queue

Agenda
Introduction
Queue Types
Case Study
Benchmarks
Comprehensive
Textmarkierung
Daten sind
Schrift Verdana
consultancy
Grösse 16 is
Recommendations
immer
the through ball
to your Conclusion
performance.
Bildmarkierung

Reduce Number of Messages
The easiest way to reduce the messaging overhead is to reduce

the number of messages
In this case a message could contain an array of document IDs

instead a reference to a single document
An array size of e.g. 10 could reduce the messaging overhead by

factor 10
Changing the size of an array payload is a matter of configuration

Parallelize to Increase Message Throughput
Parallelize enqueue operations
Parallelize dequeue operations
Be aware that enqueue operations scale two or three times better

than dequeue operations (shared queues)
Consider different degrees of parallelism for enqueue and

dequeue operations

Reduce Concurrent Dequeue Operation (1)
Enqueueing Application expects to write to a single queue
Dequeueing Application expects to read from a single queue
A transparent solution is wanted
Implements required JMS

Interfaces
AQProxy Supports various
Distribution Strategies, e.g.
Random
Sequential
Reverse
PrioritySequential
Maps logical queue names to
physical queues, e.g.
„Logical“ Queue (Multiplexer) queue[1-4] to queue1, …
queue[2 4 8] to queue2, …

Reduce Concurrent Dequeue Operation (2)
AQProxy is just one option for a “logical” queue
Using Multi Consumer instead of Single Consumer Queues is

another option (assigning consumers to dequeue processes)
The downside of using “logical” queues

The processing sequence might be affected, which e.g. requires new
concepts for high priority messages
More database objects to be managed
Partitioning is not an option

It’s not supported by AQ
Finding an appropriate key for partition elimination would have been a
precondition anyway

Java Related Performance Recommendations
Minimize creation of QueueSession, TopicSession

Costly because of SQL statements executed for queue initialization
Need to be recreated whenever a JDBC connection is closed (e.g.
through transaction managers)
JMS wrapper may have significant performance implications when

queue related objects are recreated automatically:
E.g. Spring Framework’s JMS template
Use local instead of distributed transactions whenever possible
For better control use

Plain JMS
Plain JDBC via PL/SQL interface
Plain JDBC via new 11g JDBC AQ classes, see
http://download.oracle.com/docs/cd/B28359_01/java.111/b31224/stream
saq.htm#BABBIAAC

General Performance Recommendations
Create a dedicated queue table for each queue
Use NEXT_MESSAGE whenever possible
Schedule queue maintenance tasks

Alter table … coalesce
Alter table … shrink space
See Metalink Notes 271855.1, 284692.1, 421474.1 for details
Check out further documentations

Performance Tuning Advanced Queuing Database and Applications
Metalink Note 102926.1 contains an overview of AQ performance related
themes and links to related Oracle documentation chapters
Generic AQ information
Metalink Note 267933.1 lists AQ related Metalink documents

Agenda
Introduction
Queue Types
Case Study
Benchmarks
Comprehensive
Textmarkierung
Daten sind
Schrift Verdana
consultancy
Grösse 16 is
Recommendations
immer
the through ball
to your Conclusion
performance.
Bildmarkierung

Conclusion
Consider using Buffered Queues when the reliability and transaction

support of persistent tables is not required
Single and Multi Consumer Queues have similar throughput, but may
require different configuration (e.g. block size)
Increase the throughput by increasing the degree of parallelism,

however, enqueue scales better than dequeue
Increase the scalability by assigning queues to enqueue and dequeue

processes or consumers to dequeue processes (private queues)
Make the granularity of a message a part of your design considerations,

that’s the key to improve overall messaging performance

Thank you!
?
Basel · Baden · Berne · Lausanne · Zurich · Düsseldorf · Frankfurt/M. · Freiburg i. Br. · Hamburg · Munich · Stuttgart · Vienna

Salvisberg AQ Design For Best Performance 02

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Salvisberg AQ Design For Best Performance 02

Uploaded by

Copyright:

Available Formats

AQ Design for Best Performance

Technology Manager APM and partner at Trivadis AG

Main focus on database centric development with Oracle DB

20 years experience in using Oracle products

AQ Design for Best Performance 2 © 2008

Fat Client Services

Thin Client Presentation Files

Frontend Middleware Resources

AQ Design for Best Performance 3 © 2008

AQ Design for Best Performance 4 © 2008

Simply put, it‘s enqueue…

AQ Design for Best Performance 5 © 2008

Decoupling systems, components through asynchronous processing

Designing workflows as a sequence of services where each service

Concurrent execution of subsequent services to increase overall

Exchange messages between other queues

Visualize process state and increase stability

AQ Design for Best Performance 6 © 2008

Applications based on an ESB (SOA, BPEL, …)

Data Replication with Oracle Streams

Avaloq Banking System

Trivadis Output Management Service (TOMAS)

AQ Design for Best Performance 7 © 2008

AQ Design for Best Performance 8 © 2008

AQ Design for Best Performance 9 © 2008

AQ Design for Best Performance 11 © 2008

AQ Design for Best Performance 12 © 2008

*PK NEXT_DATE Q_NAME *pfK SUBSCRIBER# *PK SUBSCRIBER_ ID

AQ Design for Best Performance 13 © 2008

Officially since 10gR2 in Memory

Same API with new

AQ Design for Best Performance 14 © 2008

Q_NAME ADDR *PK SUBSCRIBER_ ID

AQ Design for Best Performance 15 © 2008

Cannot participate in transactions

Possible Data loss after instance shutdown

Data consistency becomes a responsibility of the application

ORA-25293: Lob attributes must be Null for buffered operations

ORA-25307: Enqueue rate too high, flow control enabled

AQ Design for Best Performance 16 © 2008

LOBs supported LOBs supported

Buffered, Single Consumer Queue Buffered, Multi Consumer Queue

Autonomous transactions only Autonomous transactions only

Single Consumer Multi Consumer

AQ Design for Best Performance 17 © 2008

AQ Design for Best Performance 18 © 2008

AQ Design for Best Performance 19 © 2008

AQ Design for Best Performance 20 © 2008

AQ Design for Best Performance 21 © 2008

AQ Design for Best Performance 22 © 2008

AQ Design for Best Performance 23 © 2008

AQ Design for Best Performance 24 © 2008

Application Server Database Server

Intel XEON E5345 IBM PowerPC_POWER5

2 Quad-Cores CPU 4 Dual-Core CPU

SLES9, 64-bit AIX 5.3 64-bit

JDK 1.5.0_14 Oracle RDBMS 10.2.0.3

AQ Design for Best Performance 25 © 2008

6'000 80'000 msg/sec (Response Time)

msg/sec (Response Time)

Private - msg/sec (Response Time)

Private - msg/sec (Response Time)

PK NEXT_DATE Q_NAME pfK SUBSCRIBER# *PK SUBSCRIBER_ ID