<Insert Picture Here>

Real Application Testing for Oracle Database 9i / 10g / 11gR1/R2

Jan van Tiggelen
Principal Sales Consultant Oracle EMEA
Outline

Oracle’s Testing Lifecycle for Database and Applications

SQL Performance Analyzer (SPA): Overview

Database Replay (DBReplay): Overview

New in 11g Release 2 and Best Practices

Real-world Customer Case Studies

Conclusion

2
Oracle Application Quality Management
High quality testing for all tiers of application stack

• Testing Application Changes

• Application Testing Suite
• Ensure quality and performance with
end-to-end application testing

• Testing Infrastructure Changes

• Real Application Testing
• Designed and optimized for testing
database tier infrastructure changes

• Test Data Management

• Data Masking Pack
• Achieve security & compliance objectives by
obfuscating sensitive data in test environments

3
 Oracle’s Testing Lifecycle
Production Test Production Test

Capture
Load Testing Functional
Testing

Application
Real Application
Application

Testing

Comprehensive
Application

testing Replay

Change

Mask Clone

Clone Data Masking

Staging
Application Logic Database Logic

4
 Real Application Testing Features
SPA and Database Replay

End-to-end testing with real workloads

Create Test
System
Capture
Workload

Replay
Workload Deploy Replay
Clients

• SQL Performance Analyzer • Database Replay

• SQL unit testing for response time • Load and performance testing for throughput

• Identify and tune regressed SQL • Remediate application concurrency problems

5
Outline

Oracle’s Testing Lifecycle for Database and Applications

SQL Performance Analyzer (SPA): Overview

Database Replay (DBReplay): Overview

New in 11g Release 2 and Best Practices

Real-world Customer Case Studies

Conclusion

6
SQL Performance Regressions: Challenges

SQL performance regressions: #1 cause of poor system performance

SQL performance can regress due to many changes - DB upgrades, patch-

sets, optimizer statistics refresh, schema, parameter, hardware, etc.

Changes need to be tested thoroughly to avoid regressions

Traditional testing techniques suffer from many limitations: large workloads

(100K SQL), expensive, partial workload capture & manual point solutions

As a result, no testing or sometimes limited testing done in production

SQL • Proactively detects ALL SQL regressions BEFORE

deploying actual change
Performance • Provides integrated, comprehensive, and end-to-end
Analyzer (SPA) solution

7
 Oracle Real Application Testing: SPA
SQL Plans + Run-time Stats
Pre-Change Trial

SQL Workload
Compare SQL Analysis Report
STS
SQL Plans + Run-time Stats Performance
Post-Change Trial

• Test and predict impact of system changes on SQL query performance

• Analyze performance changes for improvements and regressions
• Comprehensive performance analysis and reporting
• Re-execute SQL queries in the given environment
• End-to-end solution: STS, SQL Plan Baselines, and SQL Tuning Advisor

8
SPA Report (Example)

3
1

9
When to use SPA?
• Testing database upgrades and patch-set releases*
– 9.2/10.1 10.2 or 11g releases
– 10.2.0.x 10.2.0.y or 11g releases
• SPA supports testing in Oracle Database Releases 10.2 and 11g
– Optimizer statistics refresh
– Database parameter changes
– Database schema changes (e.g., add/drop indexes)
– Implementation of tuning recommendations
– I/O subsystem changes (e.g., ASM, Database Machine)
• SPA handles trials in a manner that does not change database data
– Hence can be used for testing in production/standby environments

• Across many releases of Oracle and for upgrades

SPA Provides • On test, standby, and production environments
• Extended to home-grown scripts, third-party testing
Broad Testing tools, etc.
Coverage • Supports most applications - EBS, SAP, Siebel, home-
grown, etc.

*MOS Note: 560977.1 10

Outline

Oracle’s Testing Lifecycle for Database and Applications

SQL Performance Analyzer (SPA): Overview

Database Replay (DBReplay): Overview

New in 11g Release 2 and Best Practices

Real-world Customer Case Studies

Conclusion

11
 Oracle Real Application Testing
Database Replay

• Database load and performance Production Test

testing with real production Clients Replay Driver
workloads
– Production workload characteristics
such as timing, transaction
dependency, think time, etc., fully
maintained
• Test and measure transaction
throughput improvements
• Identify application scalability and
concurrency problems with new
features
• Remediate issues pre-production Storage Storage
for risk-free change
Analysis &
• Supports migrations from Oracle Capture Process Replay
Reporting
9iR2 and 10gR2

• *MOS 560977.1: Real Application Testing for Earlier

Releases 12
 Workload Replay Architecture

• Replay captured workload

Replay Clients
• Replayed operations see
the same data and
perform the same work
…
• Preserve timing and
File 1
concurrency Timing Preservation
characteristics Re-mapping
• Same number of user File 2
connections
• Replay Client …
• Multithreaded OCI Client Server Server … Server
• Drives multiple captured Commit Order Synchronization File N
processes
Sequence Replay
• Scalable Architecture Background
Metadata
• Interprets capture into
sequence of OCI calls
Replay Files
• Functional replay
 Database Replay - Supported Changes

Client Client Client

…
Changes
Unsupported
Middle Tier

Changes Supported
•Database Upgrades, Patches
•Schema, Parameters Recording of
External Client
•RAC nodes, Interconnect Requests
•OS Platforms, OS Upgrades Storage
•CPU, Memory
•Storage
•Etc.

14
Outline

Oracle’s Testing Lifecycle for Database and Applications

SQL Performance Analyzer (SPA): Overview

Database Replay (DBReplay): Overview

New in 11g Release 2 and Best Practices

Real-world Customer Case Studies

Conclusion

15
New Techniques for Avoiding SQL Regressions

Create central “SPA System” for all testing

• Many database releases
• Test, production, standby databases

Minimize production impact

• Use scoped or private session testing where possible

Leverage Oracle Active Data Guard (Read-only

Physical Standby) for testing
• Use idle standby resources for testing
• Provides full and current dataset

16
 New Techniques for Avoiding SQL Regressions
Prod Standby DB
Central SPA System

Prod 1 Test 1

11g SPA System

Prod N • SQL workload (STS) only Test N

• No application schema/data
necessary

What is SPA system? Benefits

• Remote test executes SQL workload (STS)
• Performs reporting/analysis
• Any 11g database, preferably latest release
• Not mandatory except for testing pre-11g
upgrades or 10.2.0.x  10.2.0.y
• Same or higher version than the target
database being tested
• Use latest software for analysis
• Can be used as repository, helps persist
results across database refreshes
Tips
• Use logon triggers to set environment or
to make change on remote system

17
 Database Replay Enhancements
New in Oracle Database 11g Release 2

Earlier restrictions removed with support for

• Shared Server configuration
• Streams “apply” workload

Replay filter support to target sub-set workload

• Similar to existing capture filters – Include/Exclude
• Service, Program, Action, Module, etc.
• API support only in current release

Integration of SPA and Database Replay

• Allows SQL-centric analysis when using Database Replay
• Simultaneously captures SQL workload into two different STS during
workload capture and replay
• SPA Report built from the two STS captured helps understand workload
drift
• Uses STS Compare functionality to highlight new, missing, top SQL,
changes in execution plans, #SQL executions etc.

19
 Oracle Database Testing Accelerator
Leverage DB Workload captures

• Create synthetic test scripts for

Oracle Database testing
– Import transactions from DB
Replay capture to create scripts
– Import from custom SQL Script
– Create DB test scripts manually
• Extend scripts in OpenScript
through GUI or Java Code view
• Run scripts in Oracle Load
Testing to emulate concurrent
Virtual Users
• Leverage integration with EM
Diagnostics to analyze database
performance under load

20
Outline

Oracle’s Testing Lifecycle for Database and Applications

SQL Performance Analyzer (SPA): Overview

Database Replay (DBReplay): Overview

New in 11g Release 2 and Best Practices

Real-world Customer Case Studies

Conclusion

21
 David Mitchell
Senior Vice President, OVUM

“Oracle Real Application Testing reduces the time

required to test changes by as much as 80%, lowers
testing costs by as much as 70%, mitigates risks by
reducing the number of unexpected outages, and
improves the quality of service for their IT operations.”

Source: Oracle Real Application Testing – business agility through superior testing, Jan 2008 22
 Case Study
EM RAT Real Application Testing

BUSINESS CHALLENGE ORACLE SOLUTION

• EM Enterprise Manager RAT Real Application
• Leading oil company operating in 10 countries in
Central and Eastern Europe Testing enabled:
• Business drivers for RAT • More efficiency in Testing using RAT's „db replay
– Cost reduction in application testing via feature“
centralization in IT and less dependency on • Safeguarding mission critical business process via
external developers via better testing and avoiding negative runtime
– Less performance problems in version impact in production
upgrades • Faster deployment on system enhancement
– Accelerated db infrastructure requests
implementation change
• Planned additional usage of RAT's „SQL
• Oracle Partner Remedios supported implementation performance Analyzing feature“ to filter out SQL-
of RAT
level problems in an early stageprior to production
runtime impacts
RESULTS

• 50% cost saving in Application Testing using Enterprise Manager RAT

• Modernized Testing methods on less costs with higher reputation of IT team and best and stable support for
missions critical business processes
• „Oracle Real Application Testing provides a unique tool that enables us to significantly improve our speed in rolling
out technology, and reduces our costs associated with critical testing.” – Erzsébet Szirjákné Birkás, MOL Expert of
System Programming

23
 Arup Nanda
Senior Director
Starwood Hotels & Resorts
Worldwide

“The Database Replay and SQL Plan Management

features alone are worth upgrading to Oracle
Database 11g” 11g "
 Sean McCown
Contributing Editor, Infoworld

“Oracle Database 11g will be welcome relief to DBAs

responsible for testing changes against a production
workload. The way that Data Guard and Real Application
Testing work together is sheer poetry, and no doubt most
shops will quickly come to rely on the combination”
Outline

Oracle’s Testing Lifecycle for Database and Applications

SQL Performance Analyzer (SPA): Overview

Database Replay (DBReplay): Overview

New in 11g Release 2 and Best Practices

Real-world Customer Case Studies

Conclusion

26
 Conclusion

• SPA enables businesses to

safely test and deploy system
changes using real workloads
• Real Application Testing is
much more than only a one-
time upgrade toolkit • 232% ROI over 4 years*
• Increases staff productivity – • <12 months payback
less firefighting period *
• More focus on strategic
planning and execution
• Capital expenditure savings

*Based on POC and business case from a Fortune 500 Insurance Company 27
Resources
• Dedicated ½ day workshops for Oracle Customers and Partners
– Contact ruxandra.radulescu@oracle.com for more information
• Oracle Database 11g Manageability and Real Application Testing
– http://www.oracle.com/technology/products/manageability/database/
• Oracle 11g Education (http://education.oracle.com)
– Oracle 11g: New features for Administrators
– Oracle 11g: Administration Workshop I and II
– Oracle 11g: Change Management Overview Seminar
• Oracle-By-Example Series on Oracle 11g
– http://www.oracle.com/technology/obe/11gr1_db/index.htm
• Oracle Database 11g Documentation
– http://www.oracle.com/pls/db111/homepage
• Metalink Notes
– 445116.1: Using the workload capture and replay in 11g
– 443580.1: 11g New Feature Database Replay
– 443720.1: 11g Using Snapshot Standby Database
– 455889.1: SQL Performance Analyzer
– 560977.1: Real Application Testing Now Available for Earlier Releases

28
29