Oracle Data Integration: Improving Information Agility

Mark Rabne
Senior Regional Sales Consultant Mark.Rabne@Oracle.Com

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle.

Oracle Confidential

Agenda

Why Unified Information Management is key to Information Agility Oracles Data Integration Offering & Strategy How Unified Information Management impacts IT and the Business Customer Case Studies Summary

2009 Oracle Corporation

Oracle Confidential

Business Challenges Managing Information

Requires Accessible, Up-to-date, and Trusted Information
Q: Why cant I see the reporting results that I need for all my inventory? Q: Why are my applications referring to last weeks retail sales numbers? Q: Why do I have so many duplicate copies of my customer data? Why isnt it accurate?

Accessible Information
Ensure information is connected, available, reliable, and secure. Connect information across mixed sources and targets.

Up-to-date Information
Integrate bulk data in realtime across data warehouses, data marts, business processes, MDM applications, BI applications.

Trusted Information
Ensure information consistency, quality, and authoritative data for improved, actionable business insight.

2009 Oracle Corporation

Oracle Confidential

IT Challenges to Unifying Information

What is it costing you to unify your data?
Custom Analytics Business Intelligence Enterprise Performance
Data Replication Data Warehousing Data Federation Data Hubs SQL Custom Data Access

Custom Reporting

Packaged Applications

Data Migration

Data Silos

Data Marts

Batch Scripts

Java

OLTP & ODS Systems

Oracle Data Warehouse, Data Mart PeopleSoft, Siebel, SAP Custom Apps

Files Excel XML

OLAP

Fragmented Data Silos

Slow Performance

Out of sync

Poor Data Quality

Missed SLAs

2009 Oracle Corporation

Oracle Confidential

Agenda

Why Unified Information Management is key to Information Agility Oracles Data Integration Offering & Strategy How Unified Information Management impacts IT and the Business Customer Case Studies Summary

2009 Oracle Corporation

Oracle Confidential

Where Does Data Integration Fit?

Essential Ingredient for Information Agility

SOA

Business Intelligence

Data Integration

Enterprise Data Warehousing

Master Data Management

Oracle Confidential

Oracle Data Integrator Enterprise Edition (ODI-EE)

Optimized E-LT for improved Performance, Productivity and Lower TCO

Legacy Sources

E-LT Transformation vs. E-T-L

Application Sources

Data Warehouse

Declarative Set-based design

Real Time Change Data Capture
Planning System

OLTP DB Sources

High Speed Batch Data Movement

Application

Pluggable Knowledge Modules

2009 Oracle Corporation

ODI-EE is part of ODI Suite, which is part of Fusion Middleware

Oracle Confidential

10

Oracle Data Quality

Ensure Data Quality as Part of the Integration Process

ODI EE Better Data Visibility Tighter Data Control Greater Data Accuracy

Visual Data Quality Tools

Oracle Data Profiling Metadata Profiling of Source & Target

Oracle Data Quality Duplicate Detection, Matching & Merging

2009 Oracle Corporation

Oracle Confidential

11

Oracle Data Integration Suite

Comprehensive solution for Data Integration

Comprehensive Optimized Performance with E-LT Improved productivity Lower TCO Open, Java-based, SOA native

MDM, Business Intelligence, BAM, SOA Platforms, Applications

Data ELT/ETL

Data Migration

Data Federation

Data Replication

Data Governance

Oracle Data Integration Suite

Process Manager Data Integrator Human Workflow Data Services Master Data Service Bus Data Profiling Data Grid Data Quality

Heterogeneous Data Sources

2009 Oracle Corporation

Oracle Confidential

12

Unified Information Strategy from Oracle

Complete, Integrated Platform from Storage to Scorecard
Business Process Management

Application and Data Services

Transaction Processing Services

Business Intelligence Services

Content Management Services

Collaboration Services

Information Management

Oracle BI Foundation

OLAP

OLTP

Oracle Data Integration Suite

ETL/ELT CDC Data Federation Data Services Data Quality Master Data

Oracle Data Warehousing

Appliance Mining Partitioning Modeling SQL Tooling RAC

OLTP

Data Warehouse OLAP Cube Content Mgmt File System Real-Time Data Feed

2009 Oracle Corporation

Oracle Confidential

13

Agenda

Why Unified Information Management is key to Information Agility Oracles Data Integration Offering & Strategy How Unified Information Management impacts IT and the Business Customer Case Studies Summary

2009 Oracle Corporation

Oracle Confidential

14

Improving Information Agility

Oracle Data Integration as part of the Fusion Middleware family

Unique Design Advantage

Complete Integrated Best of Breed Hot Pluggable

Agile and Intelligent for Business Efficiency and Innovation for IT

2009 Oracle Corporation

Oracle Confidential

15

Reduced IT Costs
Lower TCO for integration and migration projects

Unique Design Advantage

Consolidate and Integrate

Agile and Intelligent for Business Efficiency and Innovation for IT

Eliminate Custom Code

Improve Performance

2009 Oracle Corporation

Oracle Confidential

16

Reduced IT Costs
Lower TCO for integration and migration projects

Consolidate and Integrate

Reduced IT costs through consolidation and data migration

Eliminate Custom Code

Improve Performance

2009 Oracle Corporation

Oracle Confidential

17

Data Warehousing Using ODI

Reduces costs by consolidating Data silos

Load Transform Capture Changes

Incremental Update Data Integrity

Aggregate Export

Cube

Analytics

Heterogeneous sources Heterogeneous targets Any Data Warehouse Flexible data loading
Incremental load Changed data capture

2009 Oracle Corporation

Operational

-------------

Data Warehouse

Cube

Cube

Metadata

Slowly changing dimensions Data integrity and consistency Data lineage

Oracle Confidential

18

Data Migration Using ODI

Bulk Load Data To Ease Application Migration

Initial bulk load CDC for synchronization

Transformation to new application format

CDC for loopback synchronization

CDC

-------------

CDC

Bulk-load historical data to new application Transform source format to target Synchronize new and old applications during overlap time Capture changes in a bidirectional way (CDC)

2009 Oracle Corporation

Old Applications

New Application

Metadata

Oracle Confidential

19

Verizon
Reduces costs through consolidation

Challenges
Streamline and automate business

Results Using Oracle Data Integration

Projected cost savings of $400M in 3

processes around new hire on boarding

Provide better business insight and manage

Years
Significantly reduced the hardware footprint

FiOS supply chain

Lower the cost of rewriting SAP, PeopleSoft

through combined SOA+ ODI

ODI + SOA helped to build out a shared

& Legacy interfaces (about 209) through custom code

Lower the cost of IT infrastructure from

services layer which helped provide greater flexibility

Consolidate database (DB2, Sybase,

multiple systems

Informix to Oracle)
Consolidate ERP (PeopleSoft HR/Payroll,

Decommissioning SAP),
In addition the program helped to

decommission Cystal and Cognos and adopt Oracle BIEE

2009 Oracle Corporation

Oracle Confidential

20

Eliminate Custom Code

Improved flexibility and declarative design lowers implementation costs

Consolidate and Integrate

Reduced development costs through declarative design Pre-integrations (Knowledge Modules) for faster time to value

Eliminate Custom Code

Improve Performance

2009 Oracle Corporation

Oracle Confidential

21

Declarative Set-based Design The WHAT

Provides significant reduction in custom coding
Conventional ETL Design

ODI provides Declarative Setbased Design, simplifies number of steps Benefits: Significantly reduce the learning curve Streamline access to non-IT pros Shorten implementation times High performance every time

ODI Declarative Design 1

Define

Automatically Generate What Dataflow You Want

Define How: Built-in Templates

2009 Oracle Corporation

Oracle Confidential

22

100+ Pluggable Knowledge Modules The HOW

For Faster Time to Value, Key to Flexibility, Modularity, and Extensibility Knowledge Modules Architecture
Reverse Journalize Load Check Integrate Service

Engineer Metadata
Reverse

Read from CDC Source

From Sources to Staging

Constraints before Load

Transform and Move to Targets

Expose Data and Transformation Services

W W S S W S

Staging Tables

Load CDC
Journalize

Integrate Check
Error Tables
Target Tables

Services

Sources

Sample out-of-the-box Knowledge Modules

SAP/R3 Siebel Log Miner SQL Server Triggers Oracle DBLink JMS Queues Check MS Excel TPump/ Multiload Oracle Merge Siebel EIM Schema Oracle Web Services DB2 Journals DB2 Exp/Imp Oracle SQL*Loader Check Sybase Type II SCD DB2 Web Services

Benefits Leverage Database Optimizations:

Native SQL; Native Functions; Native Loads; Native Journaling / CDC Tailor to an organizations existing best practices Ease administration work Reduce cost of ownership

2009 Oracle Corporation

Oracle Confidential

23

Financial Case Study: Interbank

Reduced Development Costs for ETL

Challenges
Improve loading of data warehouse Being able to correctly profile each client

Results Using Oracle Data Integration

Increases development productivity by at

least 50%
Increases market share by 50% by being

based on the risk

Improve the integration of banking systems,

collection systems, and front office systems

Modernize batch exchanges and EDI

able to correctly profile each client based on the risk

Produces data interfaces that are better

interfaces toward real time

documented and easier to understand

Executes loading process that used to take

eight hours in less than two hours despite dramatic increases in data complexity of the transformations

2009 Oracle Corporation

Oracle Confidential

24

Improve Performance
Real-time data warehousing for real results

Consolidate and Integrate

High volume performance for bulk data movement through ELT Real-time feeds, improved overall responsiveness

Eliminate Custom Code

Improve Performance

2009 Oracle Corporation

Oracle Confidential

25

Optimized Data Loading through E-LT

The key to improved performance and reduced costs
Conventional ETL Architecture
Extract Transform Load

E-LT provides flexible architecture for optimized performance Benefits:

Lookup

Next Generation Architecture

Leverage Set-based transformations Improved performance for loading, no network hop Takes advantage of existing hardware

E-LT
Extract Load Transform Transform

2009 Oracle Corporation

Oracle Confidential

26 26

Changed Data Capture

Enables Real-time Data Warehousing

Change Data Capture provides real-time updates of mixed data sources without impacting performance Benefits: Improved performance for extraction delta based Operates in near-real time Timeliness of data

2009 Oracle Corporation

Oracle Confidential

27 27

Overstock.com
Real time Enterprise Data Warehouse

Challenges
Wanted to enable sales, finance, marketing

Results Using Oracle Data Integration

Having access to key business metrics

and merchandising teams to have access to near real-time data so that they could make timely, more intelligent business decisions.
Wanted to know at any point in time if

in real-time is no longer a fantasy. In short, Oracle Data Integrator gave us the ability to make better decisions and better manage our bottom line.
Found a way to ensure that data warehouse

company performance is meeting the target metrics.

Needed a data integration product that could

was constantly updated.

Even highly complex transformations are

handle our high-volume loading and transformation requirements in near real time.

automated
Supporting several terabytes of data stored

in the enterprise warehouse, and millions of daily transactions

2009 Oracle Corporation

Oracle Confidential

28

Business Benefits of Unified Information Management

Agile and Intelligent for Business

Unique Design Advantage

Adapt to dynamic conditions

Agile and Intelligent for Business Efficiency and Innovation for IT

Reduce Risk, Ensure Compliance

Improve Business Insight

2009 Oracle Corporation

Oracle Confidential

29

Improve Agility for your Data

Re-use information for improved business performance

Adapt to dynamic conditions

Connect SOA + Data Integration for improved flexibility and agility of business processes

Reduce Risk, Ensure Compliance

Improve Business Insight

2009 Oracle Corporation

Oracle Confidential

30

Bulk Data Services

Provides business processes integration and improved re-use

Oracle SOA Suite

Decouple business apps from underlying data-centric applications

Oracle BPEL Process Manager

Business Activity Monitoring

Web Services Manager Business Rules Engine Enterprise Service Bus

Recombine existing services and app components instead of new development

Benefit

Oracle Data Integrator

ODI Knowledge Modules Transform. Services Changed Data Capture Data Services

Enables efficiencies in transformation, data loading, data synchronization, all at improved performance

ODI Connectivity Framework

Bulk Data Processing

2009 Oracle Corporation

Oracle Confidential

31

Ross Stores
Data Warehouse Agility for Less

Challenges
Hard to consolidate consistent, accurate

Results Using Oracle Data Integration

75% improvement of Business Process

inventory information across over 800 stores

Lack of visibility to errors to the business Higher cost and time to delivery of new value

Execution
Together with Oracle SOA Suite (Oracle

Current state was complex, didnt scale, and

BPEL PM) provided business optimization, process visibility, exception handling

Seamless data integration from

difficult to manage
Non-standard approach required coding

paradigm, inconsistent error handling

heterogeneous sources leveraging SOA helped to create a more flexible, standards based integration approach
Closed loop processing using BAM and Data

Integration combined eliminated ordering/replenishing inventory errors by reducing inaccurate data

2009 Oracle Corporation

Oracle Confidential

32

Impact of Clean Data

Reduced risk and ensure compliance

Adapt to dynamic conditions

Improve data quality, invoke data cleansing rules, data profiling

Reduce Risk, Ensure Compliance

Improve Business Insight

2009 Oracle Corporation

Oracle Confidential

33

Data Quality Lifecycle

Ensure Data Quality as Part of the Integration Process

Data Quality and Profiling provide cleansing of data as part of integration process

Integration Process
Sources

Target

Static Check

Benefits:
Risk reduction from bad data

Flow Check Data Quality Process

Errors

Errors

Corrections

Error Recycling

Compliance where information flows

Improves visibility, business insight
Human Workflow

BPEL Process Manager

Trusted data
2009 Oracle Corporation Oracle Confidential

34

Leading Online Job Placement Firm in North America

Whats the Return on investment for Clean Data?

Challenges
Integrate data across multiple systems,

Results Using Oracle Data Integration

Increased overall revenue by $2M in one

including legacy acquisition infrastructure.

Better profile and detect issues with data

month by improving data quality

Saved $1M in acquisition: company had bad

among multiple systems

Remove duplicates and eliminate errors

data
Increased direct/inside sales productivity Increased number of sales (more calls per

associated with better identifying customer data

day), avg order (better targeting, messaging)

Increased cross-selling: Cross-divisional lead

sharing
280% revenue growth in 3 years

2009 Oracle Corporation

Oracle Confidential

35

Improving Business Insight with Data Integration

Improved Data Lineage better trusted data

Adapt to dynamic conditions

End-to-end data lineage - ensure pervasive reach of BI/EPM applications to multiple source systems

Reduce Risk, Ensure Compliance

Improve Business Insight

2009 Oracle Corporation

Oracle Confidential

36

Improve Visibility of Unified Data

Oracle Data Integrator EE & Oracle BI

Planning & Budgeting

Analytic Dashboards

Oracle BI EE, Essbase & ODI EE

Metadata-driven Data Lineage

Planning & Budgeting

Data Warehouse/Data Mart

2009 Oracle Corporation

Oracle Confidential

37

Lower TCO for Performance Management

Using Oracle Data Integration

Oracle OnDemand, Oracle Hyperion Planning, Oracle Data Integrator

By using Oracle Data Integrator, we were able to extract, transform, and aggregate financial information from two distinct systems and swiftly load data into our Hyperion Planning application.

By automating the data integration process and eliminating manual steps, we have saved time and resources, greatly reduced the risk of error associated with manually entry, and have more confidence about the data critical to our key planning and decision making processes.
.

2009 Oracle Corporation

Oracle Confidential

38

Summary: Improving Information Agility

Using Oracle Data Integration

Unique Design Advantage

Agile and Intelligent for Business

Efficiency and Innovation for IT

2009 Oracle Corporation

Oracle Confidential

39

Next Steps

1. Unify enterprise information for improving information agility 2. Leverage E-LT and Change Data Capture for realtime BI/DW 3. Align business demands and IT requirements for data initiatives Ask What If questions 4. Assess the existing and future needs of data across your enterprise 5. Leverage Oracle Partners and Oracle Professional Services

Oracle Confidential

Additional Customer Examples

Global Case Studies

Order Replenishment, Online Catalog J. Crew

Business Challenges
Order replenishment for online catalog system Define new corporate standard for data integration Better integrate data across 175 retail stores and 52 outlet chains

IT Challenges
Replace antiquated home-grown solution of scripts Teradata bulk data support, and help integrate data across SAP, Red Prairie Inventory systems

Oracle Data Integration Solution Benefits

Simplified the end-to-end data integration with all their core IT systems and database platforms to reduce the total cost of ownership of their order replenishment and cataloging Benefited from the Oracle advantage in batch processing, bulk data transformation for increased performance Successfully leveraged SAP and Teradata integration to phase out legacy systems and ultimately reduce costs

Sales and Marketing Data Warehouse Nestle

Business Challenges
Hard to consolidate consistent, accurate inventory Enable the marketing and CRM team to better target information across over 800 multiple 50 countries its campaigns and activities across store chains Lack of visibility toreporting challenges, in light of the Anticipate future errors to the business Higher cost and time to volume of production data strong increase in the delivery of new value

IT Challenges
Manage a data warehouse for operational transactions and marketing information Lack of high-value-added actionable data on clients, orders, stock, etc.

Oracle Data Integration Solution Benefits

Provided 100GB of actionable data to the business users Replaced weekly insertion with daily recording of change in status with time imprint for all clientsallowing for more effective client follow-up, for example Improved response times by transferring reporting processes to the data warehouse Improved reactivity for restocking requirements thanks to daily updates on stock quantity and value for each store Provided overall view of the status of business processes, resulting in improved user communication

For More Information

Quote Attribution Title, Company

Get Started
Visit the Oracle Fusion Middleware 11g web

Resources
Datasheet:

site at http://www.oracle.com/fusionmiddleware11g
Oracle Data Integration on oracle.com

http://www.oracle.com/products/middlewar e/odi/docs/odiee-datasheet.pdf
Blog:

www.oracle.com/goto/odi
Oracle Fusion Middleware on OTN

http://blogs.oracle.com/dataintegration
Technical information available at:

http://otn.oracle.com/middleware

http://www.oracle.com/technology/product s/oracle-data-integrator/index.html
Data Integration Events

http://www.oracle.com/events

2009 Oracle Corporation

44

Hands-On ODI Workshop Columbus, OH October 6th Contact: mark.rabne@oracle.com

Session Evaluation
Please rate this session now.
Thank You!!

Information Management

Oracle Confidential

Questions

2009 Oracle Corporation

Oracle Confidential

47