40 Ways To Speed Up Your Upgrade And Decrease Downtime

Randy Giefer
Solution Beacon, LLC

1 Introduction
The goal of this paper and related presentation is to assist existing Oracle Applications DBAs and
Administrators in identifying the new features of Release 11i that will affect how they accomplish
their day-to-day administrative tasks. It is unrealistic to attempt for this to be the ultimate,
comprehensive reference for these items, but a summary level description of each is provided to
aid the reader. Additional references will be noted throughout the document.

Many changes will be occurring to this document, as new features, tips & tricks, and lessons
learned are discovered over time. Updated copies of this document can be found on our web site
(www.soltuionbeacon.com), or by requesting a copy via email from rgiefer@solutionbeacon.com

1.1 The Author

Randy Giefer has over 18 years of practical business experience in the Information Technology
and systems integration arena, with experience in the energy, airline, mortgage banking,
automotive manufacturing and commercial and federal finance industries. This business
experience is coupled with over 7 years of experience in the implementation and administration
of Oracle Financial (Commercial and Federal/Public Sector) and Manufacturing software
applications in the role of an Applications Database Administrator and Systems Architect. In
addition to his Oracle Applications experience, he has over 10 additional years of Oracle database
development and administration. His project experience includes systems architecture, database
architecture, installation, upgrades, performance tuning, development, testing, database
administration and production systems management. In addition to past management roles, he
has also served as a Practice Director for a consulting services company, as well as founding and
presiding over his own successful corporations, The DBA Domain and Solution Beacon, LLC.
He can be reached for further questions and comments by email at rgiefer@solutionbeacon.com
or by phone at (651) 994-0782.

1.2 Company
Solution Beacon, LLC (www.solutionbeacon.com) is a leading provider of expertise and
knowledge for information systems and database integration challenges. By providing
unparalleled industry experience, professionalism and attention to quality, Solution Beacon
delivers the most comprehensive and complete solutions available in a quick and efficient
manner.

2 Abstract
This paper enumerates over 40 ways to speed up a Release 11i upgrade and decrease the
amount of extensive system downtime. The 40 ways include time proven methods, tips,
and tricks to ensure that the upgrade is as fast and efficient as possible. .
3 Executive Overview
As a result of the extensive technical and functional changes introduced in
Release 11i, upgrading to Release 11i from prior versions can be an effort that
requires extensive system downtime. This downtime period is often a duration
that is unacceptable to the business. This presentation will demonstrate over
40 ways to Speed Up Your Upgrade And Decrease Downtime by sharing with the
attendees time-proven methods, tips, and tricks for ensuring that your upgrade
is as fast and efficient as possible.

Slide. Performance Tuning

There Is Always A Bottleneck!
Tune All Components
Server(s)
Network
Client PCs
People

Slide. Baseline Education

“One of the biggest factors contributing to the performance of the upgrade
process, as well as post-upgrade or post- implementation performance is a
properly fed Cost-Based Optimizer.”

Slide. Baseline Education

Oracle Applications Release 11i uses the cost-based optimization (CBO) approach
in choosing the most effective way to execute SQL statements.

Slide. Cost Based Optimizer Basics

Definition:
Cost Based Optimizer
Determines the most efficient way to execute a SQL statement based on
statistical data stored in the data dictionaries.

Rule Based Optimizer

A fixed ranking system to determine the most efficient access method when
executing a SQL statement.

Slide. Cost Based Optimizer Basics

How Does CBO Work?
The optimizer creates a set of potential execution plans for the SQL
statement based on available access paths and hints.
The optimizer estimates the cost of each execution plan based on statistics in
the data dictionary.
The optimizer compares the costs of the execution plans and chooses the one
with the lowest cost.
Slide. Cost Based Optimizer Basics
How Does CBO Work?
CBO Care
init.ora parameters
CBO Feeding
Gathering statistics

Slide. CBO Care

Oracle Applications 11i require particular init.ora parameters to be set up for the
CBO use:
To check current init.ora parameters settings by running
$FND_TOP/sql/AFCHKCBO.sql script
OR
from sqlplus query v$parameter table:
select name, value from v$parameter;

Slide. CBO Care

Recommended init .ora Parameter Settings:
optimizer_features_enable = 8.1.7
optimizer_mode = CHOOSE
_optimizer_undo_changes = FALSE
_optimizer_mode_force = TRUE
db_file_multiblock_read_count = 8
optimizer_max_permutations = 2000

Slide. CBO Care

Recommended init.ora Parameter Settings:
_complex_view_merging = TRUE
_sort_elimination_cost_ratio = 5
_use_column_stats_for_function = TRUE
_like_with_bind_as_equality = TRUE
_or_expand_nvl_predicate = TRUE
_push_join_union_view = TRUE

Slide. CBO Care

Recommended init.ora Parameter Settings:
_table_scan_cost_plus_one = TRUE
_push_join_predicate = TRUE
_fast_full_scan_enabled = FALSE
_ordered_nested_loop = TRUE
optimizer_percent_parallel = 0
query_rewrite_enabled = TRUE
compatible = 8.1.7

Slide. CBO Care

Two Often-Overlooked Parameters:
 optimizer_index_caching = 90
(indicates to the CBO how often index blocks are cached in the Buffer Cache
default 0)

optimizer_index_cost_adj = 10
(tells the CBO how expensive index scans are in relation to full- table scans
default 100)

Slide. CBO Feeding

Generating Statistics in 11i
Need To Keep Statistics Current !
u FND_STATS package is used to generate statistic (Not ANALYZE, Not
DMS_STATS)

u Sample FND_STATS package procedures :

GATHER_INDEX_STATS
BACKUP_SCHEMA_STATS
RESTORE_TABLE_STATS
VERIFY_STATS

Slide. CBO Feeding

Examples:
Run FND_STATS package from SQL*Plus

SQL > exec fnd_stats.gather_schema_statistics (‘AR’)

( For a specific schema )
SQL > exec fnd_stats.gather_schema_statistics (‘ALL’)
( For all schemas )
SQL > exec fnd_stats.gather_schema_statistics (‘GL’, ‘GL_JE_LINES’)
( For one table )

Slide. CBO Feeding

Generating Statistics in 11i
Using the System Administrator responsibility run the following concurrent
programs to generate statistics:
Gather Table Statistics
Backup Table Statistics
Restore Table Statistics
Gather Schema Statistics
Gather Column Statistics
Analyze All Index Column Statistics
Gather All Column Statistics
Slide. Gather Table Statistics

SlidSSAdlie

Slide. Gather Table Statistics

Gathers table statistics for the specified table
Gathers all related index statistics by default
If backup_flag is set to BACKUP, exports the old statistics before gathering the
new statistics
Old statistics are stored in FND_STATTAB table
Uses GATHER_TABLE_STATS procedure

Slide. Backup Table Statistics

Slide. Backup Table Statistics
Stores the statistics of the given table unto the FND_STATTAB table.
Backs up the related index and column statistics by default.
Uses different statistics identifiers
Can keep different versions of backup with different identifiers.
Uses BACKUP_TABLE_STATS procedure

Slide. Restore Table Statistics

Slide. Restore Table Statistics
Restores the previously backed up table statistics from a given statistics identifier.
Imports index and column statistics associated with the specified table
Uses RESTORE_TABLE_STATS procedure

Slide. Gather Schema Statistics

Slide. Gather Schema Statistics
Gathers specified schema level statistics
n Backs up existing statistics
Creates the histogram for the specified columns in the FND_HISTOGRAM_COLS
tables
Populates default statistics for all the interface tables as defined in the
FND_EXCLUDE_TABLE_STATS table
Uses GATHER_SCHEMA_STATS procedure
Slide. Gather Column Statistics

Slide. Gather Column Statistics

Gathers the column statistics for all columns specified in the SEED data table
FND_HISTOGRAM_COLS
Gathers the column statistics for a specified column_name in a given table
Backs up statistics into the FND_STATTAB table before gathering statistics
Uses GATHER_COLUMN_STATS procedure

Slide. CBO Feeding

Now Obsolete Programs:
Analyze All Index Column Statistics
Gather All Column Statistics
Slide. CBO Feeding
Gather Stats on Partitioned Tables
Pass ‘PARTITION’ in the Granularity parameter
OR
From SQL*Plus
SQL> begin
fnd_stats.gathe r_table_stats (
ownname => ‘APPLSYS’
tabname => ‘WF_ITEM_ACTIVITY_STATUSES’,
granularity => ‘PARTITION’);
end;

Slide. CBO Tools, Tips and Tricks

How to determine when statistics were last gathered?
Run FND.STATS.VERIFY_STATS to check when statistics were last gathered.
Example:
SQL> set server output on
SQL> set long 10000
SQL> exec fnd_stats.verify_stats
(‘QP’, ‘QP.QP_LIST_LINES’);

Slide.CBO Tools, Tips and Tricks

Use coe_stats.sql script provided by Oracle support
Verifies statistics for all tables owned by installed apps modules
Generates COE_STATS.TXT report and coe_fix_stats.sql script

n For more info see MetaLink Note 156968.1

Slide. CBO Tools, Tips and Tricks

Use bde_last_analyzed.sql script from Oracle support
Validates statistics for all tables
Use it when bad overall database performance is noticed
Generates bde_last_analyzed.txt report
For more info see Metalink Note 163208.1

Slide. CBO Tools, Tips and Tricks

How to determine when statistics were last gathered?
Run the following sql query to for individual tables:
select table_name, num_rows, blocks, avg_row_len,
to_char (last_analyzed ‘MM/DD/YYYY HH24:MI:SS’)
where table_name in from dba_tables
(‘TABLE1’, ‘TABLE2’, ….’TABLEn’);

Slide. CBO Tools, Tips and Tricks

How to determine when statistics were last gathered?
Run the following sql query to check indexes:
 select index_name, num_rows, distinct_keys “DISTINCT”, leaf_blocks,
clustering_factor “CF”, avg_leaf_blocks_per_key “AVG_LB”
from dba_indexes
where table_name in (‘TABLE1’, ‘TABLE2’, ….’TABLEn’);

Slide. CBO Tools, Tips and Tricks

When Should You Gather Statistics?
The right schedule depends on your data and how often it changes
Recommendation gather the CBO statistics at least once per week
Gather statistics more frequently on the tables with high number of
transactions

Slide. CBO Tools, Tips and Tricks

When Should You Gather Statistics?
Never gather CBO stats for SYS or SYSTEM data dictionary objects
Gather statistic after:
a large amount of data has been entered into your system
an upgrade
applying mini packs or maintenance packs

Slide. CBO Tools, Tips and Tricks

When Should You Gather Statistics?
Gather statistics during low system load periods, when number of active
users is minimum
Why?
When object statistic are refreshed all references to that object become
invalid in the shared pool or library cache, requiring a new hard parse for
each sql statement referencing the object

Slide. 40 Ways - To Speed Up Your Upgrade To Reduce Critical Downtime

Not All Of These Methods May Be Applicable To Your Situation
You Are The Best Judge
"Hypothetical Situation"
Some Options May Have Limited ROI

Slide. Don’t Upgrade. Re- implement

ERP2
Allows Business To Change Data, Setups
Incorrect
Incomplete
Bad Data
What to do with existing data?

Slide. You’ve Decided To Upgrade – Now What?

Three Keys To Successful Performance During A Release 11i Upgrade
P3 !!!
 P3 - Planning
P3 - Preparation
P3 – Practice, Practice, Practice

Slide. #2 P3 – Planning (Planning, Preparation and Practice)

Plan To Prepare
Plan to Practice
Plan for Adequate Resources
“If you can’t afford extended downtime, then plan on spending $”

Slide. #2 P3 – Planning (Planning, Preparation and Practice)

Plan For At Least One Dedicated Upgrade DBA
This Is Not: a:\upgrade
Install Is Easy, Don’t Be Mislead
Why Train Staff To Be Upgrade Experts?
Plan On Extensive Communications
Plan On Executive Involvement (Early)

Slide. #2 P3 – Planning (Planning, Preparation and Practice)

This Upgrade Is A Team Effort
Set Key Dates, Targets, Work Backwards
Start Determining Your Maximum Downtime Now
What Is Your $/day?
Assume Worst Case?

Slide. #2 P3 – Planning (Planning, Preparation and Practice)

Plan To Multi- Task During Each Upgrade Pass
Need To Get The Most Information (Data) From Each And Every Upgrade
Iteration!!!!
Capture System Info
Capture The Executed SQL
Use STATSPACK

Slide. #3 P3 – Preparation (Planning, Preparation and Practice)

Acquire Hardware
Acquire Resources
Develop/Evaluate Testing Plans
Identify/Document:
Extensions
Customizations
Setups

Slide. #3 P3 – Preparation (Planning, Preparation and Practice)

Order Latest Certified Software
Get Latest Oracle Release Notes, Alerts
 Download Latest Performance Patches
Download RDA & TUMS

Slide. #3 P3 – Preparation (Planning, Preparation and Practice)

System
Space
MAX_EXTENTS -> Unlimited Extents
Auto Extend ON
Gather Space Utilization Information
Before & After Each Upgrade
Freespace
Extents
Chained Rows
Fragmentation

Slide. #4 P3 – Practice (Planning, Preparation and Practice)

Four Outcomes of Practice:
Minimized Execution Time
Minimized Think Time
Minimized Scheduling Time
Minimized Human Error

Slide. #4 P3 – Practice (Planning, Preparation and Practice)

Minimized Execution Time
Backup, Experiment, Restore
Tackle Big Time Consumers, Most ROI
As A Team, Decide Those To "Invest In"
Minimized Think Time
Documentation (logs, checklists, etc.)
Scripts For Automation
Minimized Scheduling Time
Coordinate All Team Members

Slide. #4 P3 – Practice (Planning, Preparation and Practice)

Minimized Human Error
Who is your Drill Sergeant?
Turns a “lowly private” upgrade teams into a “soldier” upgrade team.
Repetition, Repetition, Repetition
Long Cycles For Upgrade
End Result: Decreases Human Error

Slide. #4 P3 – Practice (Planning, Preparation and Practice)

First Iteration Upgrade
Don't Worry About Customizations
Determine Maximum Downtime Now
Slide. #4 P3 – Practice (Planning, Preparation and Practice)
Document Every Step!
Date
Activity
Start / Stop

Slide. #4 P3 – Practice (Planning, Preparation and Practice)

Timings:
AD_TASK_TIMING
adt<session ID>.lst --> $APPL_TOP/admin/<ORA_SID>/out
$AD_TOP/admin/sql/adtimrpt.sql
Document When SQL Statements Start/Stop
Check MetaLink For Performance Patches

Slide. #5 Register Your 10.7/11.0 Extensions Properly

Do Properly Beforehand
Fix It The Right Way
Why re-register after?

Slide. #6 Do Test Upgrades On Production Hardware

Hopefully New Hardware
Test All Components Of New System
Safety Net / Fallback
Critical For Timing Data!

Slide. #7 Re-Evaluate Your Architecture

Some sites could now use single-node instead of multi- node
"Balance perceived performance tuning opportunities, availability, redundancy vs.
higher maintenance and architectural complexity"
JS, 2002
KISS - Keep It Simple and Supportable

Slide. #8 Involve Your SA During Test Upgrades

Monitor I/O, CPU, Memory
Utilities: top, iostat, vmstat, and sar
Document When Peaks/Hotspots Occur
Review With Upgrade Assistant Spreadsheet
Make The SA Responsible
This is too much for a DBA to do!

Slide. #9 Upgrade Your Hardware

Use Upgrade As An Excuse
Buy More - CPU, Memory and Disk
CPUs = parallelism
RAM = 2 X CPU
More is better
Faster is better
 11i Requires More Resources
Why Buy After Go- live?

Slide. #10 Utilize Solid State Disk

Eliminate I/O Wait Time
0 Latency, 0 Wait
Candidates
REDO Logs
Rollback
Temporary
Indexes
Key Upgrade Tables
V$FILESTAT

Slide. #11 Lease/Rent Bigger Hardware For The Upgrade

Beg, Borrow or Steal
(The stealing option is not recommended by Solution Beacon as it has a
limited payoff!)
From Hardware Vendor/Supplier
From ASP/Integrator

Slide. #12 Leave New Module Implementations For Post-Upgrade

Keep The 11i Upgrade An Upgrade
No Additional Modules
Phase II
KISS

Slide. #13 Upgrade Database Before 11i Upgrade

Keep The 11i Upgrade An 11i Upgrade
Spreads The Downtime Across Multiple Outages
Take Advantage of 8i Features Prior To Upgrade
Forces Upgrade of Tools Infrastructure Ahead Of Time (e.g. Backup)

Slide. #13 Upgrade Database Before 11i Upgrade

Post-Upgrade Support
(Focus on 11i, Not 8i Issues)
Extensive custom code?
Evaluate 9i for PL/SQL Impact

Slide. #14 Gather Statistics Before 11i Upgrade

Saves Having To Do It During Upgrade
Must Upgrade Database Before 11i Upgrade

Slide. #15 Convert to LMT Before Upgrade

Advantages of Locally-Managed Tablespaces
Better Space Management Due To:
 Uniformed Extent Sizes
Reduced Data Dictionary Access
Reduced Fragmentation (Uniform)
Better Management of Te mporary Space

Slide. #15 Convert to LMT Before Upgrade

SQL> CREATE TABLESPACE local_auto DATAFILE
2 '/u02/.../local_auto.dbf' SIZE 1M reuse
3 EXTENT MANAGEMENT LOCAL AUTOALLOCATE;

Export/Import Locally, Uniform

Apps Uses a Hybrid LMT Mode
Must Use the Oracle Upgrade Script

Slide. #16 Defragment Your Database

Reduce/Eliminate Chained Rows
Rebuild Indexes
Optimize Your Current Database For An Upgrade
Cleanup Before
Crap In, Crap Out

Slide. #17 Make Sure You Have Adequate Freespace

Need Plenty Of Freespace After Upgrade (Minimum 20%)
Not Just Enough To Accommodate Expansion For End of Upgrade
Temp Tables Used During Upgrade

Slide. #18 Perform Category 1 & 2 Before The Downtime

Don't Wait, Start Now
The first minute of the cutover you want to be "starting" your upgrade - not waiting!
Carry A Big Stick - No Last Minute Journal Entries!!!
Run Manual Steps In Parallel (Many CPUs, Many People)

Slide. #19 Reduce Data Volume, Purge

Benefits of Purging Before Upgrading:
Reduce AutoUpgrade and AutoPatch run time
Reduce upgrade related backup and copy times

Slide. #19 Reduce Data Volume – Purge

Will purging really help my upgrade?
Run AutoUpgrade and AutoPatch and analyze data in AD_TASK_TIMING
Look for applications with the most run time.
Match up with Oracle Purges

Slide. #19 Reduce Data Volume – Purge

Most Common Oracle 10.7 Purges
Schema Purge
 AP AP PO Purge
AR Call New Archive and Purge
BOM Delete Item Information
BOM Purge Standard Cost History
GL Archive and Purge
HR Payroll
INV Transaction Purge
OE Sales Orders
FA Asset Depreciation
PA Projects
PO AP PO Purge
WIP WIP Transactions
* For information on specific purges see OAUG Fall 2001 paper: An Online
Archive Solution Using Standard Purge Programs By Ziyad Dahbour

Slide. #19 Reduce Data Volume – Purge

Examples – AutoUpgrade and AutoPatch

10.7 to 11i Upgrade Category 3 & 4 Outage Time

Elapsed Time
Source DB Release Application Size (GB) Hours Days Workers
Vision Demo 11.5.2 2 18 0.8 2
Example 1 11.5.4 Mfg and Fin 70 37 1.5 6
Example 2 11.5.5. Project Acct 97 101 4.2 8
Example 3 11.5.5 Fin heavy AP 200 61 2.5 10
Example 4 11.5.5 Payroll 70 71 3.0 10

Note: Total outage time will typically be more than 2X the Run time of AutoUpgrade
plus AutoPatch.

Slide. #19 Reduce Data Volume – Purge

Example 1:Upgrade Time 1.5 day w/ 11.5.4 and 70 GB
 Conclusion: Purge OE before Upgrade Or Bifurcate

Slide. #19 Reduce Data Volume – Purge

Example 2: Upgrade Time 4.2 Days w/ 11.5.5 and 97 GB

Other PA Other
11% 72% 4%
PA
AP 92%
2%
AP
8% FND
2%
GL
9%

Conclusion: Purge PA before Upgrade

Slide. #19 Reduce Data Volume – Purge

Example 3: Upgrade Time 2.5 Days w/ 11.5.5 and 200 GB
GL AP
53% 37%

Other
10%
GL
1%

AP
Other 53%
46%

Conclusion: Purge AP before Upgrade

Slide. #19 Reduce Data Volume – Purge

Example 4: Upgrade Time 2.5 Days w/ 11.5.5 and 70 GB
 Other
25% Other PAY
PAY 47% 23%
35%
AP
9%

PER
WF 10%
BEN
9%
4% AD
GL AP
7% 9%
22%

Conclusion: Purge Payroll before Upgrade

Slide. #19 Reduce Data Volume – Purge stimating Outage Reduction

AutoUpgrade + AutoPatch Time by Application

Application Data (GB) Hours* Hours/GB Workers Release
OE 4.5 19.3 4.33 6 11.5.4
OE 9.5 41.7 4.37 8 11.5.3
PA 57.3 92.4 1.61 8 11.5.5
AP 76.2 32.5 0.43 10 11.5.5
AP 6.0 1.8 0.30 8 11.5.5
PAY 16.1 16.1 1.00 10 11.5.5
* Hours = (RunTime% by Application) X Total Elaspsed Time
 Slide. #19 Reduce Data Volume – Purge Etimating Total Outage Reduction
Purge AP and GL - Before: 300 GB After: 140 GB

Total Saved
Upgrade Component Hours Hours Comments
Backups (5) 25 hours 25 10 40% reduction
Restore to new server 12 hours 12 5 40% reduction
Upgrade DB from 7.3.4.5 to 8.1.7.1 and patch 8 0
11.5.5 AutoUpgrade and AutoPatch 52 16 Cut AP run time to 50%
11.5.6 Family pack and Developer 6i Patch 8 18 0
Gather schema stats 11 0
Apply Current Minipack patches 15 0
Total Hours 141 31
Total Days 5.9 1.3

Conclusion: Reduce Outage by > 1 Day

Slide. #19 Reduce Data Volume – Purge Implementation (continued)

Time Frame:
1 – 3 months to production
Start early, before 11i soaks up all resources!

Slide. #20 Reduce Data Volume – Bifurcate

An alternative for the OE – OM upgrade
What is it?
A way to upgrade OE to OM with limited down time
How does it work?
Orders are upgraded in two steps, during the upgrade and post upgrade
Selection: Open orders + Orders closed in the last N days.
Series of patches to apply at various times

Slide. #20 Reduce Data Volume – Bifurcate

An alternative for the OE – OM upgrade
From Patch 1697526 Readme
WARNING !!! WARNING !!! WARNING !!! WARNING !!! WARNING !!
IMPORTANT
---------
Please keep in mind, if you elect to defer the upgrade of
Inactive orders, that reports, forms and other user interface
tools will not be able to access the inactive orders until
they are upgraded.

Functional issues to consider before using Bifurcation:

RMA’s related to deferred orders
Open invoices related to deferred orders

Slide. #20 Reduce Data Volume – Bifurcate

An alternative for the OE – OM upgrade
Source of information
Oracle White Paper:
Order Management Release 11i Upgrade Instructions
November, 2001
Author: Swami Sukumaran

Slide. #21 Reduce Data Volume – Archive

To get user buy in on purging you will need an Archive solution. Considerations:
One time purge or ongoing?
Frequency of ongoing purges?
User access to data?
Upgrade path?
Speed of implementation?

Slide. #21 Reduce Data Volume – Archive

Archive Options
Oracle Purge - Archive Support
Tape or Online Pre-purge Copies
Data Warehouse
Build it Yourself
Vendor Packages
BitbyBit
OuterBay
Xoriant

Slide. #21 Reduce Data Volume – Archive

 70.00

60.00

50.00
Data in GB

40.00

30.00

20.00

10.00

-
Jun-01
Feb-01

Feb-02
Apr-01
Dec-00

Aug-01

Dec-01
Oct-00

Oct-01

Slide. #21 Reduce Data Volume – Archive Post 11i Considerations: Performance Decay
 6.0 550

5.5 500

5.0 450
Daily Average Run Time (Minutes)

Daily Total Run Time (Hours)

4.5 400

4.0 350

3.5 300

3.0 250

2.5 200

2.0 150

1.5 100

1.0 50

Jul-01
Mar-01

Feb-02
May-01

Jun-01

Jan-02
Nov-01

Dec-01
Oct-01
Sep-01
Jul-01
Mar-01

Feb-02
May-01

Jun-01

Jan-02
Nov-01

Dec-01
Oct-01

Aug-01
Apr-01
Sep-01
Aug-01
Apr-01

Concurrent Program run times 1st Year on 11i

Average Run Time: Increase 43% Total Run Time: Increase 78%

Slide. #21 Reduce Data Volume – Archive Post 11i Considerations: Re-Implementation
– WW Mfg

300

250

200
Data in GB

150

100

50

0
Jun-01

Aug-01

Oct-01
Feb-01

Feb-02
Apr-01

Apr-02
Dec-00

Dec-01
 Conclusion: Archive early in 11i life-cycle

Slide. #21 Reduce Data Volume – Archive

Why Purge and Archive Before 11i Upgrade?
Upgrade Outage Reduction
Big benefits for OE, AP, PA, PAY, others
Reduce backup time
Upgrade Project Benefits
Less storage for 11i Development and Test
Faster upgrade for each test cycle
Reduce impact of 11i data growth
Stabilize performance
Minimize storage growth

Slide. #22 Clean Up Your Data!

Reduce Upgrade Failures
Pre Cursor For Category 1 & 2
Run Those Scripts Weekly, Now!
Data Fix Scripts Created As Part of Testing

Slide. #23 Increase db_block_buffers

Upgrade Is Batch, Not OLTP
Use Multiple init.ora Files
One for Upgrade
One for Post Upgrade OLTP

#24 Increase sort_area_size

#25 Increase TEMP

Slide. #26 Change RBS Configuration For Batch

Increase The Size And Extents
Reduce The Number
Remember, This Isn't OLTP

Slide. #27 Optimize Batch Commit Size

Match the Batch Commit size to rollback segment size
"Push The Envelope" During Testing
Find The Limit
Make Notes For Each
Practice!

Slide. #28 Use RBO (Select Jobs)

Use On Select Upgrade Scripts
ALTER SESSION...
Candidates:
apxenvt1.sql
 apxalloc.sql
apxacct1.sql/apxacct2.sql
apgllink.sql
porevdat.sql

Slide. #29 Use Custom Indexes (Select Jobs)

Use On Select Upgrade Scripts
Not As Bad As It Sounds
Create an index specifically just for an upgrade script, then drop it later.

CREATE INDEX ap_chrg_allocations_n99 ON ap_chrg_allocations_all

(ABS(allocated_amount));

Slide. #30 Manipulate CBO Parameters (Select Jobs)

Use On Select Upgrade Scripts
OPTIMIZER_INDEX_CACHING
Values: 80-90
OPTIMIZER_INDEX_COST_ADJ
Values: 10-50
ALTER SESSION

Slide. #31 Optimize The Number Of Workers

Determine The Optimal Number Via Experimentation (Not Trail and Error)
Some Cases, 2 1/2 and 3 Times the # of CPU's
See All Workers Running?
Old Theory: # CPU’s +1 = # workers
Watch Restarts, Failures, Dependencies
Issues With SQL*Net?

Slide. #32 Use adctrl To Restart Failed Workers

Don't Assume AutoUpgrade Is Always Correct
Saves Research Time

Slide. #33 Merge Patches With admrgpch

Eliminates Duplicate Driver Statements Within A Patch
Eliminates adpatch Overhead
Startup
Loading Patch Revision Info
Saves apx. 5 mins For Each Patch
Decreases Human Errors
Can Be A Lot Of Work
Prerequisite Patches Required
Slide. #34 Run adpatch Non-Interactively
Non-Interactive Mode
Create defaults file
adpatch defaultsfile=/.../xyz.txt interactive=no ...
Script the patch process

Slide. #34 Run adpatch Non-Interactively

Use a Larger Commit Size
Practice, Practice, Practice!
Benefits:
Repeatable
Decreases Human Errors (Especially at 4:00am)
Decreases Chance of Missing a Driver

Slide. #35 Minimize Re-Linking / Re-Compiles

Save Relinking Until End
Form Re-Generations
Reports
Flexfields
Messages

Slide. #36 Use A Pre-Patched APPL_TOP

Build It Ahead Of Time
Use From Test System
If Multi-Node, Definitely
Be Sure To Test The Upgrade With The Pre-Built APPL_TOP!

Slide. #37 Delay Patching

Do some patches under after critical downtime (e.g. online help patches)
Subsequent Weekends/Downtime

Slide. #38 Use FNDLOAD / FNDSLOAD To Load AOL Data

For Your 11i Extensions/Customizations:
Flexfield Setup Data
Concurrent Program Definitions
Request Groups
Lookup Values
Profiles
Menus (FNDSLOAD)

Slide. #39 One Final Practice Upgrade

The Week Before Production
Dress Rehearsal
Picks Up Recent Data From Production – Clean Data?

Slide. #40 Performance Tune Your DBAs!!!

 Reduce Stress And Pressure Of Upgrade Team
Fresh Minds Make Better Decisions
Less Human Error
No Last Minute Changes
Stay With The Plan
Follow The Drill Sergeant!

Slide. Summary
“Care and Feed” The CBO
P3
Plan
Prepare
Practice, Practice, Practice
Follow The Drill Sergeant!