SFOracle RACOracle

Sr. System Engineer

1.

Oracle
How do I get the most out of my existing systems?

How do I ensure constant database availability?

How can I better protect our mission critical data?

How do I manage data growth without spending too much on storage?

2.


1. Getting it right with Storage Foundation Oracle HA & RAC Choosing the right levels of availability for Oracle Implementation Best Practices: Server, Network, Storage Roadmap 2. Featured Technology: Database Dynamic Storage Tiering 3. Customer Case Study: Amadeus 4. Q&A

3.

Oracle
SINGLE INSTANCE HA /CLUSTERING RAC ARCHITECTURES

Manual fail-over Simple set-up Hours to restore

Automated fail-over Active/passive or N+1 Minutes for fail-over

Highest availability Shared resources Fastest fail-over

Recommended Architectures 4.

:
SF HA for Oracle (single instance Oracle) SF CFS HA w/ (single instance Oracle) SF RAC (Oracle RAC)

Import disk group Mount file system Start database Resolve transactions (replay logs) Update IP Clients Reconnect

Import disk group Mount file system Start database Resolve transactions (replay logs) Update IP Clients Reconnect

Import disk group Mount file system Start database Resolve transactions (replay logs) Update IP Clients Reconnect

Resolving database transactions (replaying redo logs) to recover failed transactions during fail-over is a necessary step in any HA architecture. Log replay is likely to account for the bulk of recovery time.

5.

RTO Oracle
EXPENSE
ORACLE REAL APPLICATION CLUSTERS (RAC) AUTOMATED FAILOVER USING CLUSTERING MANUAL RECOVERY

5 MIN

15 MIN Recovery Time

25 MIN

35 MIN

Disclaimer Above times are illustrative rules of thumb; actual recovery time is dependent on many factors 6.


1. Getting it right with Storage Foundation Oracle HA & RAC Choosing the right levels of availability for Oracle Implementation Best Practices: Server, Network, Storage Roadmap 2. Featured Technology: Database Dynamic Storage Tiering 3. Customer Case Study: Amadeus 4. Q&A

7.

:
or ?

Decision Factors
Capacity Planning Load Testing Performance SLA Benchmarks Cost

Workload Patterns / Msg Overhead # of DB instances in cluster? Maintenance Points of server & storage admin

Best Practice: use identical node configuration Use identical CPU speeds and RAM Use identical number of NICs and HBAs
Predictable Capacity and HW Discovery

Use identical OS version and chip architecture Use identical time for server in the same site/cluster 8.

:
Low Latency Transport (LLT) or UDP (in 11g) Port aggregation handled by LMX/LLT For redundancy
Use multiple dedicated GigE links Set Auto Negotiate and Full Duplex == True Use multiple switches Use multiple Ethernet cards

Non-routable networks (10.x.x.x/ 176.x.x.x/ 192.x.x.x/) CRS heartbeat must use one of LLT interfaces Use VCS PrivNIC/Multi PrivNIC to make private interface HA Use NICS with identical speeds Dont use Cross Over Cables for interconnect

9.

:
Storage Array Best Practices
Use two storage arrays Adequate capacity does not equal adequate performance Configure appropriate stripe width and depth Use SCSI-3 PGR compliant storage especially for RAC Isolate DRL disks from Data disks

Switching Best Practices

Two independent fabrics

HBAs Best Practices

Configure to support the required I/O Bandwidth and IOPS

Multi-Pathing Best Practices

Is strongly recommended for both Oracle HA & RAC Dynamic Multi-pathing included in SF Oracle & SFRAC

10.

: (cont.)
Volume Manager
Mirror across two storage arrays, especially with host based mirrors Separate Oracle Recovery Structures from database files Separate Redo logs and put on fastest storage (i.e. RAID 1+0) Be aware that physical disks may be shared by multiple servers/applications Use Consistent and Meaningful Device Naming Conventions Use Snapshots to clone databases for testing or reporting

File System
Always use ODM Use Cluster File System for fast fail over in non-RAC environments Place database and CRS binaries in local file system Create separate file systems for binaries, data, redo, archive, temp etc.. With RAC, always use CFS for redo and archived logs rather then local Storage Checkpoints for quick restore or creating test instances Implement Tiered Storage, its easy with Dynamic Storage Tiering Balance file system extents automatically to improve performance 11.

Agenda
1. Getting it right with Storage Foundation Oracle HA & RAC Choosing the right levels of availability for Oracle Implementation Best Practices: Server, Network, Storage Roadmap 2. Featured Technology: Database Dynamic Storage Tiering 3. Customer Case Study: Amadeus 4. Q&A

12.

Storage Foundation News for Oracle & RAC

5.0 MP3
Usability, performance, resilience improvements Platform Support (RHEL5 & SLES 10) Thin storage optimization Faster storage migrations Installation and configuration improvements ODM,CIO enabled with Storage Foundation & Cluster File System DB DST for Oracle RAC Virtual DR Readiness Testing Support for Oracle DataGuard with SF RAC I/O Fencing enhancements iSCSI Support
13.

* Subject to change

Agenda
1. Getting it right with Storage Foundation Oracle HA & RAC Choosing the right levels of availability for Oracle Implementation Best Practices: Server, Network, Storage Roadmap 2. Featured Technology: Database Dynamic Storage Tiering 3. Customer Case Study: Amadeus 4. Q&A

14.


Traditional /financial /FY07 /FY06 /Prev /FY07 DST /financial /FY06 /Prev

File System Volume

eg. EMC DMX

Multi Volume File System V1 V2 V1 V2

eg. EMC DMX RAID5

V3

V1

V2

eg. EMC DMX RAID0+1

eg. EMC Clariion

Concept #1: Volume Concept #2:

Set: Set of volumes carved out of heterogeneous

multi-vendor storage hardware

MVFS: Single file system namespace over multiple tiers

15.

DST (DB DST)

Provides DB level abstraction by hiding storage & config complexity Maps DB objects to files Uses file level metadata Does not require root privilege Enables extent balancing
Sample DB DST Relocation Policies Archive relocation:
Policy to move archive logs /opt/VRTS/bin/dbdst_file_move -S $ORACLE_SID -o archive(n) -c storage_class:days

Oracle Partition Move:

Partitioned tables make good candidates for seasonal relocation $ /opt/VRTS/bin/dbdst_partition_move -S $ORACLE_SID -T table_name \ -p partition_name -c class

Tablespace Relocation:
Infrequently accessed tablespace -> slow storage, active fast storage $ /opt/VRTS/bin/dbdst_tbs_move -S $ORACLE_SID -t tablespace \ -c class

External File Relocation:

Store binary objects (eg. pictures) in a FS and store only the pathname in db /opt/VRTS/bin/dbdst_file_move -S $ORACLE_SID -o externalfile \ -f listfile -c storage_class:days

16.

Extent Balancing File System capability of DST

Improve Performance Evenly distribute file extents among component volumes within a tier Chunk-size is configurable; can be specified in placement policy Individual storage tiers may have different chunk-size from others Easy to manage
FileFile Extents

V1

V2

V3

TierX

17.

Adding Storage Class, Volumes & Policies

18.

Create Tablespace and Table

19.

Extent Balanced File System

20.

File System Extent Map

21.

Case Study

22.

23.

Amadeus by the numbers

764 Airlines storing flight schedules 502 Airlines bookable through Amadeus 145 Amadeus Alta Reservation Airlines 32000 Airline Sales Offices 22 Car Rentals & 36000 Rental Locations 184 Tour Operators 94000 Travel Agency Locations 255 Hotel Chains, 76814 Hotel Properties 106 Rail Operators 17 Cruise Lines 217 Markets served 3 Main Locations
Madrid (Head Office) Nice (Development), Erding (Operations)

400+ Million Transactions per Day 8700+ Transactions per Second peak 3+ Million Bookings per Day 75+ Million Passenger Name Records (PNR) <0.3 Second Response Time 99.98% Uptime

IT Service Centres in London & Sydney 8000+ Employees, 95 Nationalities

24.

Amadeus Unix Database Tier

Clusters Biggest Cluster Databases Database Size Oracle ver Servers SAN OS Storage Foundation File System 35 (2 Nodes/cluster) 2*64 cores 35 (Production), 300 (Test) Between 50 GB & 2 TB 9i & 10g (RAC & non RAC) 19 (Superdomes) EMC DMX (Production), HP XP (Test) HP-UX 11iv2, Red Hat 4.6 SF 4.1/5.0, SF RAC 4.1 VxFS & VxCFS with ODM

25.

Classical Test DB refresh

a sic y ph y lize l a k n e we rso e p

py l co

FVT

APP FVT

DEV

APP DEV

OBO2 8 weeks OBO1 8 weeks

DES

APP DES

PROD
Redo Log

Backup Backup B SMG Restore K

RMAN Restore PIT Recovery Customization Scrubbing / Cleanup

SUP

APP SUP

PDT

APP PDT

Base FS Base FS
UAT

APP UAT

Long time to copy (2-3 days) Long outage for weekly refreshes (24 hrs min) Storage usage, 8 x OBO (80 TB) Bound to one cluster / no isolation

MIG

APP MIG

SKL

APP SKL

File System based Oracle DB

26.

8-week test cycle with weekly refreshes

OBO1 preparation

Week 1

Week 2

Week 3

Week 4

Week 5

Week 6

Week 7

Week 8

Week 1

Week 2

Week 3

OBO2 preparation

27.

File System Storage Checkpoint based Test Database refresh

FVT Ckpt

APP FVT

lize a n rso e p

DEV Ckpt

APP DEV

OBO2 8 weeks OBO1 8 weeks

DES Ckpt

APP DES

PROD
Redo Log

Backup Backup B SMG Restore K

RMAN Restore PIT Recovery Customization Scrubbing / Cleanup

SUP Ckpt

APP SUP

PDT Ckpt

APP PDT

Base FS Base FS
UAT Ckpt

APP UAT

Quick & Easy Add only~ 30% space to OBO

Bound to one cluster / no isolation

MIG Ckpt

APP MIG

SKL Ckpt FS based Oracle DB FS checkpoint based Oracle DB

APP SKL

28.

Tiered Landscape
we ek ly

8-week cycle
PROD
Redo Log

FVT Ckpt

APP FVT

(Internal only)

OBO2 OBO1 8 weeks

8 weeks

DEV Ckpt

APP DEV

Lvl1

Backup Backup B SMG Restore K

Restore
Base FS Base FS

DES Ckpt

APP DES

SUP Ckpt

APP SUP

Instant snapshot
Lvl 2 Lvl 2 Base 2 Base1

PDT Ckpt

APP PDT

(External)

Lvl2

UAT Ckpt

APP UAT

Instant snapshot
MIG Ckpt

APP MIG

(SLA)

Isolation achieved

Lvl 3 Lvl 3 2 Base Base1

Lvl3

SKL Ckpt

APP SKL

29.

Integration into VCS

Weekly refreshes of FS checkpoints:
hares offline SBRTST_Oracle_FVT sys drcobt73 hares action SBRTST_Mnt_OBO_ORA_sbrora1 Create_FSCKPT Refresh_FSCKPT Remove_FSCKPT hares online SBRTST_Oracle_FVT sys drcobt73 -sys drcobt73

8-week synch of OBOs using instant snapshots

hares action SBRTST_Mnt_OBO_ORA_sbrora1 VXSNAP_MAKE VXSNAP_SPLIT VXSNAP_DEPORT -actionargs SKL sys drcobt73

hares action SBRTST_Mnt_OBO_ORA_sbrora1

-actionargs SKL sys drcobt73 VXSNAP_IMPORT VXSNAP_JOIN VXSNAP_REATTACH

30.

Tiered Landscape
Weekly cycle
si By
PHY 1

FVT Ckpt

Stop managed recovery - Create checkpoints -Resume managed recovery

APP FVT

(Internal only)

DEV Ckpt

ca

APP DEV

Lvl1

Ph y

St a

PROD

nd

DES Ckpt

Redo Log

APP DES

Physic

al

SUP Ckpt

APP SUP

Stand B y Ph ys St an ica l d By

PDT Ckpt

APP PDT

(External)

PHY 2

Lvl2

UAT Ckpt

APP UAT

MIG Ckpt

APP MIG

(SLA)

Lvl3

PHY 3

SKL Ckpt

APP SKL

31.

Elegant DR rehearsal
PROD D&R
ly p Ap

PROD
Redo Log

Lo

Archiver

PHY DR

Arch Redo

REH Ckpt

APP REH

Stop managed recovery Create REH checkpoint Resume managed recovery Mount REH checkpoint Personalize REH instance Start REH instance Run REH traffic against REH instance Log shipment and log apply keeps ongoing during rehearsal

32.

Amadeus: Improved ROI Metrics Downtime Before 24hrs/week After 15 min/week 0 36TB 0

DR Downtime 7-10 days Storage Labor 110TB 3 days/week

33.

Achieving optimization of Oracle database High Availability is more than clustering and installing RAC Reducing costs is more than just settling for good enough
It involves proper HW/SW configuration HA demands thorough Implementation Planning HA needs operational discipline
Symantec Storage and Availability Management solutions for Oracle improves performance, availability and manageability of databases
34.

DOWNLOAD
Veritas Storage Foundation HA plug-in for Oracle EM
http://www.symantec.com/business/products/utilities.jsp?pcid=2245&pvid=208_1

Optimizing Storage Foundation Oracle HA White Paper

http://www.symantec.com/business/products/whitepapers.jsp?pcid=2245&pvid=2 08_1#

Database Dynamic Storage Tiering for Oracle White Paper

http://eval.symantec.com/mktginfo/enterprise/white_papers/bwhitepaper_hp_symantec_dynamic_storage_tiering.pdf

Copyright 2007 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.

35.

Thank You
Symantec and Symantec Vision are registered trademarks of Symantec in the U.S. and in other countries. The other company names or products mentioned are or may be trademarks of their respective owners.

36.