<Insert Picture Here>

Oracle Data Integration Strategy and Roadmap

Oracle Fusion Middleware Product Management
Agenda
Introduction to Oracle Data Integration
Business Drivers for Data Integration
Benefits from a Modern Data Integration Platform
Key Oracle Data Integration Products
Oracle Data Integration Solution
Oracle GoldenGate Overview
Data Integrator Overview
ODI & GG Together
Best of Breed Integration for Batch and Realtime Data Integration
Support any Type of Data Integration Use Case
Implementing Best-Practice Technical Pattern for Data Warehousing
Technical Details How it Works
Demonstration and Q&A (if available)

2
 Data Integration Portfolio
INTRODUCTION

3
Business Drivers for Data Integration
Essential Ingredient for Information Agility

Strategic Value of Data Integration

Consistency for major enterprise initiatives like BI, DW, & MDM
Common technical foundation platform across data silos
Central point for data governance, availability and controls

Key Data Integration Use Cases

BI, DW, and OLTP Data Integration & Replication
SOA, Enterprise Integration & Modernization
Migrations and Master Data Management
 Benefits from a Modern DI Platform
Data Integration is Infrastructure that enables Business Value

Design metadata-driven integration

Do More with Less Leverage skills & dictate patterns

Ensure continuous uptime

Compete Globally 24X7 Access data in real time

Use Data for Competitive Ensure the quality of your data

Advantage Actively govern most valuable asset

Automate and Adapt Business Expose data services for reuse

Processes Orchestrate processes using SOA
Key Data Integration Products
Real Time Data Replication DBMS High Availability
Changed Data Capture Disaster Tolerance

Heterogeneous E-LT & ETL OLAP Data Loading

High-speed Transformations Data Warehouse Loading

Comprehensive Integration Process Orchestration

ELT/ETL for Bulk Data Human Workflow
Service Bus Data Grid

Data Service Modeling Data Security/Redaction

XQuery Data Federation XA Compliance

Business Data / Metadata Time Series Reporting

Statistical Analysis Integrated Data Quality

Cleansing & Parsing High Performance

De-duplication Integrated w/ODI
Oracle Data Integration Solution
Best-in-class Heterogeneous Platform for Data Integration

Oracle Custom MDM Business Activity SOA

Applications Applications Applications Intelligence Monitoring Platforms

Comprehensive Data Integration Solution

SOA Abstraction Layer
Process Manager Service Bus Data Services Data Federation

Oracle Data Integrator Oracle GoldenGate Oracle Data Quality

ELT/ETL Real-time Data Data Profiling

Data Transformation Log-based CDC Data Parsing

Bulk Data Movement Bi-directional Replication Data Cleansing

Data Lineage Data Verification Match and Merge

Storage Data Warehouse/ OLTP OLAP Cube Flat Files Web 2.0 Web and Event
Data Mart System Services, SOA

7
 Data Integration Portfolio
GOLDENGATE

8
 Oracle GoldenGate Overview
Enterprise-wide Solution for Real Time Data Needs

Standardize on Single
Disaster Recovery,
Data Protection Standby Technology for Multiple Needs
(Open & Active)
Deploy for Continuous
Zero Downtime Availability and Real-time Data
Migration and
Upgrades Access for Reporting / BI
Log Based, Real-
Time Change Data Operational
Capture Reporting
Reporting Database
OGG

ETL

ODS EDW
ETL
Highly Flexible
Heterogeneous EDW
Source Systems
Real-time BI Fast Deployments
Lower TCO & Improved ROI
Query Offloading

Data Distribution

9
 How Oracle GoldenGate Works
Modular De-Coupled Architecture
Capture: committed transactions are captured (and can be filtered) as
they occur by reading the transaction logs.
Trail: stages and queues data for routing.
Pump: distributes data for routing to target(s).
Route: data is compressed,
encrypted for routing to target(s).
Delivery: applies data with transaction integrity,
transforming the data as required.

Trail Trail
Capture Pump Delivery
LAN/WAN
Internet
TCP/IP

Source Target
Database(s) Bi-directional Database(s)

10
 Data Integration Portfolio
DATA INTEGRATOR

11
 Oracle Data Integrator Enterprise Edition
Optimized E-LT for High Performance, Productivity and Low TCO

Legacy
Sources

E-LT Transformation Any Data

vs. E-T-L Warehous
e
Application
Sources Declarative Set-based design

Change Data Capture Any

Planning
System
OLTP DB Hot-pluggable Architecture
Sources

Pluggable Knowledge Modules

12
12
 How ODI Works: E-LT Architecture
High Performance, Flexible, Lightweight Architecture
Key Architecture Benefits: 100% Java, Open APIs, fast E-LT

Packaged Business Intelligence

Application & Data Warehouse
ODI
Agent
ODI Agent may be
deployed in any part
of the architecture
LKM
LKM
IKM
IKM
JKM A
A
D
B
B C$_0
C$_0

I$
I$ E$ (Errors)
C$_1
LKM
LKM

File
C
C IKM
IKM CKM
CKM
RKM

Extract-Load
Extract-Load Transform
Transform Check-Load
Check-Load

13
 Data Integration Portfolio
ODI & GG TOGETHER

14
Oracle Data Integration Solution
Best-in-class Heterogeneous Platform for Data Integration

Oracle Custom MDM Business Activity SOA

Applications Applications Applications Intelligence Monitoring Platforms

Comprehensive Data Integration Solution

SOA Abstraction Layer
Process Manager Service Bus Data Services Data Federation

Oracle Data Integrator Oracle GoldenGate Oracle Data Quality

ELT/ETL Real-time Data Data Profiling

Data Transformation Log-based CDC Data Parsing

Bulk Data Movement Bi-directional Replication Data Cleansing

Data Lineage Data Verification Match and Merge

Storage Data Warehouse/ OLTP OLAP Cube Flat Files Web 2.0 Web and Event
Data Mart System Services, SOA

15
 Best-of-Breed Data Integration
Heterogeneous, Real-time, Non-Invasive, High Performance E-LT,
and Low Hardware Costs

Traditional ETL + CDC Oracle E-LT + Real-time

Invasive Capture on OLTP systems Continuous feeds from operational

using complex Adapters
Transformations in ETL engine on
expensive middle tier servers
Bulk load to the data warehouse with
large nightly/daily batch
Extract
systems
Non-invasive data capture
Thin middle tier with transformations
on the database platform (target)
Mini-batches throughout the day or
bulk processing nightly

Trickle

GG+ ODI
GG+ ODI
Xform Xform Bulk
Lookup Lookup
Data Data

Staging Load Heterogeneous

 Support Any Type of Data Integration
Best of Breed means using the Right Tools for the Job!

Analytical

OLTP Query / Report

ODS EDW
ODS
OLTP
OLTP

Operational

OLTP OLTP OLTP OLTP OLTP OLTP

Old New OLTP Heterogeneous
Heterogeneous Heterogeneous
ODI-EE Integration with GoldenGate
Non-invasive Data Capture combined with ODI ELT strengths
Key Benefits:
1. Eliminate Overhead no need for DB API overhead on the Source, or the
invasiveness of the ODI J$ objects on the Source system,
2. Automate GoldenGate automation of GG deployment directly from ODI GUI
3. Provide Common DW Pattern supplies a common pattern for mini-batch style
(non-real-time) DW aggregate loads

Source Systems ODI Staging & Target

Replicated ODI J$
Source DBs Target EDW
Source Tables Tables

Transactional Replicated ODI CDC Target

RDBMS Source Tables Framework Tables
Source J$
Tables

Generate all ODI CDC infrastructure Execute end-to-end CDC

Generate all GG deployment files ODI

18
 Oracles Data Integration Joint Solution
Best-of-Breed and Proven

Oracle Data Integrator

Oracle GoldenGate
Enterprise Edition

Technology Differentiators:
Lowest latency and E-LT architecture
highest throughput; for best performance of
Performance high data volume
non-invasive, low
overhead transformations

De-coupled Knowledge Module

Extensible & architecture; multiple architecture for
Flexible deployment styles; extensibility and flexible
open and extensible connectivity

Maintain SOA-native,
transactional integrity; integrated with Fusion
Enterprise
resilient against MW to fit future
interruptions and enterprise architectures
failures
 ODI & GG Together
TECHNICAL DETAILS
 Overview of the ODI KM Framework
Pluggable Knowledge Module Architecture
Reverse Journalize Load Check Integrate Service
Engineer Metadata Read from CDC From Sources to Constraints before Transform and Move Expose Data and
Source Staging Load to Targets Transformation
Services
ion
Reverse g rat Gate
e n
Int lde e! W
/ Go her W S W
w is S S

Staging Tables
Load Integrate
Services
CDC
Check Target Tables
Journalize
Sources
Error Tables

Benefits Leverage Database Optimizations:

Native SQL; Native Functions; Native Loads; Native Journaling / CDC
Tailor to an organizations existing best practices
Ease administration work
Reduce cost of ownership

21
ODI CDC in a Nutshell
A General Framework for Change Capture on Source DBs

Automatic w/JKMs Consumers

Journal Tables
Capture Services Consumption Services ExtendWindow() PurgeJournals()

Create Capture Process LockSubscriber() UnLockSubscriber()

View: CONT View: CUST View: EMP

Start/Stop Capture Process CONTID

I003
CUSTNAME
Thomas S.
CUSTID
C003
CUSTID
C002
CUSTNAME
YAHOO
EMPID
E003
EMPID
E001
ENAME
Joe Celko

Subscription Services Journal Tables

Subscription Services
Manage Consistency Sets J$CONT J$CUST J$EMP

Register/Un-register Subscriber
CONTID WID CUSTID WID EMPID WID
RegisterSubscriber() I002 - C002 20 E001 -
I003 20 E001 20

Consumption Services
AddTableToConsistencySet() E001 19

Capture Services
Consumption Views CDC_SET
CDC_SET_SUBSCRIBER
SUBSCRIBER MIN_WID MAX_WID
CDC000 FUSION_BI 10 10

Process

Process

Process
Capture
Capture

Capture
Consumption Operations CDC000 PILLAR_HCM 9 9

Extend Window
Source Data
Lock/Unlock Subscriber Table: CONT Table: CUST Table: EMP
CONTID CUSTNAME CUSTID CUSTID CUSTNAME EMPID EMPID ENAME

Purge Operations I001

I002
Vijay R.
Raghu M.
C003
C002
C001
C002
AT&T
YAHOO
E003
E003
E001
E002
Joe Celko
Albert Einstein
I003 Thomas S. C003 C003 GOOGLE E002 E003 John Doe
 Overview of the Integration
Using ODI & OGG Together

Transactional RDBMS Staging DB Target DB

Source Tables Target Tables
Replicated
Source Tables

ODI CDC
Framework

ODI
Interfaces

Datapump
Extract
WAN Replicat
Source trail Staging trail
files files
 Scenario: Analytics & Reporting

Transactional RDBMS Staging DB1 Target DB

Source Tables Target Tables
Replicated
Source Tables

ODI CDC
Framework

Datapump

WA Replicat
N
Extract Staging trail
files Historic
Datapump Replicat Staging DB2 Analytics/
Source trail
files WA Replicated Reporting
N Source Tables
Staging trail
files

Realtime
Reporting
 Step-by-step How It Works

Transactional RDBMS Staging DB Target DB

Source Tables Target Tables
 1. Replicated tables created with
Common Format Designer (using ODI)

Transactional RDBMS Staging DB Target DB

Source Tables Target Tables
Replicated
Source Tables
 2. Start Capturing Changed Data in
Source (OGG Extract process)

Transactional RDBMS Staging DB Target DB

Source Tables Target Tables
Replicated
Source Tables

ODI CDC
Framework

Extract

Source trail
files
 3. Initialize Staging and Target Data
(with ODI or optionally, OGG)

Transactional RDBMS Staging DB Target DB

Source Tables Target Tables
Replicated
Source Tables

ODI CDC
Framework ODI
ODI Interfaces
Interfaces

Extract

Source trail
files
 4. Start Replication / Propagate
Changes to Target DB (OGG and ODI)

Transactional RDBMS Staging DB Target DB

Source Tables Target Tables
Replicated
Source Tables

ODI CDC
Framework

ODI
Interfaces

Datapump
Extract
WA Replicat
N
Source trail Staging trail
files files
Simplified & Productive Development
DEMONSTRATION

30
31