Demystifying Data Integration for the Cloud

An Oracle White Paper November 2012

Demystifying Data Integration for the Cloud

Oracle Data Integration

Demystifying Data Integration for the Cloud

Table of Contents
Introduction .................................................................................... 3 Zero Downtime Consolidation into Private Cloud .............................. 4 Benefits of Private Cloud ............................................................. 4 Data Integration for Zero Downtime Database Consolidation ........ 5 Continuous Availability and Disaster Tolerance for Cloud ................. 6 Eliminate planned downtime during maintenance and upgrades.... 7 Eliminate unplanned downtime .................................................... 7 Maximize Availability and Performance ........................................ 7 Integrating Data between On-Premise and Public Cloud ................... 8 Loading Bulk Data in the Cloud .................................................... 8 Real-Time Data Synchronization between On-Premise and Cloud 8 Integration between On-Premise and Cloud ................................. 9 Summary...................................................................................... 10 Learn More ................................................................................... 10

Demystifying Data Integration for the Cloud

Introduction
Cloud computing is a vision that is increasingly turning to reality for many companies. Enterprises, both small and big, are evaluating cloud computing and, in increasing numbers, are moving their IT infrastructure to the cloud. As a matter of fact, Forrester Research 1predicts that, by the year 2020, enterprises will be investing more than $241 billion in cloud computing each year thats six times what theyre spending today. But where does that leave our current investments for information management solutions, our existing relational data stores, data warehouses, business intelligence systems, and business applications that consume data? What impact will cloud have on the world of connecting your data sources?

Figure 1: Benefits of Cloud Computing

The benefits of Cloud range from lower data center costs, to significantly reduce environmental impact, to the ability to capture more of the opportunities that markets present through increased agility in resource deployment and dramatically reduced time to market. While the promised benefits of cloud computing can be immense, achieving them requires much more than simply connecting via an adapter to a software-as-a-service (SaaS) offering. These may be useful steps towards moving to a cloud computing blueprint, but on their own do not deliver cloud computing for the whole enterprise and its associated benefits. Instead, what is required is to look at ways of transforming the way we think of our current information management architectures and develop an enterprise-wide strategy for cloud computing.

Source: April 21, 2011 / Sizing The Cloud Understanding And Quantifying The Future Of Cloud Computing by

Stefan Ried, Ph.D. and Holger Kisker, Ph.D. with Pascal Matzke, Andrew Bartels, and Miroslaw Lisserman

Demystifying Data Integration for the Cloud

In this paper well outline three important areas where data integration plays a crucial role, and how Oracles data integration offering enables these critical steps: Zero downtime consolidation into private cloud Continuous availability and disaster tolerance for cloud Data integration between on-premise and public cloud applications

Zero Downtime Consolidation into Private Cloud

Before we discuss the importance and best practices for minimizing downtime for private cloud consolidation, it is helpful to evaluate the benefit of implementing a private cloud environment, particularly the value of consolidation for the IT operations.

Benefits of Private Cloud

A private cloud is an efficient way to reap the benefits of cloud computing as it gives the flexibility to deploy not only software-as-a-service but also platform-as-a-service and infrastructure-as-a-service (hardware, networking etc) within enterprise firewall. This enables companies to leverage the private cloud environment for all systems and data stores they have in house, rather than a few critical business systems they may choose to use on a public cloud. For example, implementing private database services enables IT departments to consolidate servers, storage, and database workloads onto a shared hardware and software infrastructure, and benefit all users across the enterprise. Databases deployed on a private cloud offer compelling advantages in cost, quality of service, and agility by providing ondemand access to database services in a self-service, elastically scalable, and metered manner. Secondly, many public clouds provide little or no availability or performance service-level agreements, and there are potential data security risks. In contrast, private clouds enable IT departments to have complete control over the performance and availability service levels they provide, and can easily enforce data governance regulations and auditing policies. Building a private cloud requires the transformation and optimization of the IT infrastructure, and that is typically executed in two steps: rationalization, architecture optimization.
Rationalization

IT rationalization determines the best use of IT services and reduces nonproductive redundancy throughout the enterprise. IT departments should rationalize their technology architecture by standardizing their service portfolio and technology stack. Through standardization, the IT environment becomes much more homogenous, which makes it easier to manage. It also reduces costs and complexity and increases agility.
Architecture Optimization

All layers of the technology stack must support service-level objectives and growth requirements. Scalability, availability, data security, and datacenter management are only as strong as the weakest link. Balanced technology architectures employ virtualization, consolidation, and management automation

Demystifying Data Integration for the Cloud

to meet business requirements. Virtualization, for example, transforms the typical server-to-application silo model to a multi-tenancy model. The key to virtualization is not necessarily the underlying technology, but rather the capability to abstract resources requested by the business from resources fulfilled by IT.
Shared Services

IT departments can leverage shared services to reduce costs and meet the demands of their business users, but there are many operational, security, organizational, and financial aspects of shared services that must be managed to ensure effective adoption. Consolidation is vital to shared services, as it allows IT to restructure resources by combining multiple applications into a cohesive environment. Consolidation goes beyond hard cost savings; it s implifies management, improves resource utilization, and streamlines conformity to security and compliance standards. Consolidation can be done at different levels such as server consolidation, storage consolidation, database consolidation, schema consolidation etc. They each bring different benefits. Database consolidation reduces the number of operating system installations and improves overall manageability.

Data Integration for Zero Downtime Database Consolidation

For private cloud architectures, consolidation of your databases and data stores is an important step to take to be able to receive the full benefits of cloud computing. Oracle GoldenGate, with its real-time data integration capabilities, and Oracle Data Integrator, with its bulk data movement and transformation capabilities, provide you with the ability to seamlessly consolidate your data. One of the key hurdles in consolidating systems is the impact on business operations. For missioncritical systems especially any downtime can be detrimental to the business. Oracle GoldenGates real time data integration capabilities minimize any impact on business operations during the consolidation and migration effort. Through real-time, bi-directional data movement and synchronization Oracle GoldenGate allows immediate switchover from their existing system to the new environment in the Cloud. This can be implemented when the new system is ready without ever denying access to the application. Since the source application can continue to operate, the IT teams have the flexibility to test the system as long as they need to. Furthermore, Oracle GoldenGates bidirectional replication capabilities enable companies to keep the old system in synch with the new one after the switchover, which provides failback option to the old system if there is any issue in the new environment. This is a great method for minimizing risk during the consolidation effort. For example, Fedex is in the process of implementing a private database cloud architecture. They are rehosting approximately 480 databases in 5 data centers onto a standard architecture with Oracle RAC running on Redhat Linux. The company has been using Oracle GoldenGate to consolidate database schemas without interrupting operations. In addition they are leveraging Oracle GoldenGate for zero downtime database maintenance. Oracle Data Integrator and Oracle GoldenGate can both be used for migrating to Oracle Database or Oracle Exadata from database appliances such as Teradata, IBM Netezza and other major systems.

Demystifying Data Integration for the Cloud

Oracle GoldenGate offers zero downtime migration from all major open system databases and mainframes including HP NonStop, IBM DB2 (for z, for i, for LUW), Teradata, Sybase ASE, SQL Server.

Figure 2: Oracle GoldenGates bidirectional real -time data replication capabilities for consolidation

Continuous Availability and Disaster Tolerance for Cloud

Businesses are facing increased pressure to meet end-users expectations for 24/7 system availa bility and to surpass competitors service levels. But at the same time its one of the key concerns for organizations trusting their applications to run in Cloud environments. Can their business applications operate with an acceptable level of continuity, even under a range of potentially damaging or interrupting conditions? What are the procedures that avoid or minimize the risk of data loss in the event of a disaster or major system outage? Typically, companies might first consider traditional disaster recovery products and practices. But business-critical applications requiring high-availability targets, such as fast recovery time objectives and strict recovery point objectives, traditional disaster recovery technologies alone are not likely to be sufficient, because with these solutions database recovery is typically required, which extends the failover time. Continuous availability and disaster tolerance of critical data systems is more than after -the-fact data recovery. Continuous availability prevents customers from experiencing downtime or the data loss effects of a disaster. In addition, it improves system availability when faced with not only unplanned outages but also planned outages and performance-related issues that impact the end-users perception of availabilityregardless of whether the system is down or available.

Demystifying Data Integration for the Cloud

Oracle GoldenGates real -time transactional data replication capabilities enable companies to achieve continuous availability for their private cloud environments. GoldenGate offers continuous availability in 3 key solution architectures:

Eliminate planned downtime during maintenance and upgrades

Similar to the solution described in the consolidation section, Oracle GoldenGate enables the IT teams to upgrade, migrate, or maintain their systems without impacting business operations. IT teams can eliminate planned downtime for their cloud environment by using GoldenGate to synchronize their systems with a back-up system in real time. During maintenance the application can be switched over to the backup system which is fully in synch and immediately available for transactions. After the maintenance, GoldenGate can update the primary system with the new transactions so it can take over the transaction processing without data loss. This enables IT teams to do their upgrade and maintenance activities without time pressure and thorough testing.

Eliminate unplanned downtime

Once the migration is completed ensuring maximum availability for the cloud environment is critical. Real-time database replication can provide hot standby systems that significantly improve recovery time for mission-critical environments; delivering up-to-the-second data to the standby system and enables immediate switchover if an outage occurs. It is also important to have the ability to initiate realtime data capture from the standby database to update the primary system, once it is online, with any new data processed by the standby system. As mentioned above, Oracle GoldenGate offers real-time bidirectional data replication that enables standby systems that are immediately ready to take over. The standby system can be used for reporting/queries or testing etc. as it is open for users.

Maximize Availability and Performance

Active-active, bidirectional database replication allows for the highest availability and performance on critical systems, because it enables load balancing by continuously and multi-directionally moving data between two or more active and heterogeneous databases. During an outage the applications can use the available databases. Oracle GoldenGates real -time bidirectional replication solution works between different regions, and offers robust, automated conflict detection and resolution capabilities. This replication architecture can be used for both consolidation into the cloud environment as well as to maintain continuous availability after the migration. Active-Active replication enables phased migration, which is preferred method for highly mission-critical environments where application switchover should not cause any downtime. As public cloud users demand better availability SLAs from their cloud service providers, investing in continuous availability solutions become more important for service providers as well. E-Dialog is a great example for this trend. eDialog is a software as a service provider for advanced email and multichannel marketing solutions. They decided to migrate to Oracle Exadata as the foundation for their new data infrastructure to scale better and cost efficiently. For migration to Exadata they selected Oracle GoldenGate as their SaaS platform could not tolerate any downtime. Oracle GoldenGate

Demystifying Data Integration for the Cloud

provided a phased migration option where it synchronized the old databases with Exadata for 6 months and was able to test the new environment during this time very thoroughly.

Integrating Data between On-Premise and Public Cloud

One fact remains with the emergence of the cloud computing phenomenon. While businesses are making new economic investments in public cloud computing, they continue to rely heavily on their existing on-premise IT investments. This strategy of straddling between the two infrastructure types poses its own set of challenges. To integrate data between public cloud and private data center solutions, organizations face two major challenges. First, they must be able to rapidly load data to get cloud applications up and running in a timely fashion. Second, that data must be kept synchronized often in real time, or near real time.

Loading Bulk Data in the Cloud

One of the first challenges that many organizations face when implementing cloud-based architectures is being able to load terabytes of data from their network into a cloud application. This requires accessing information from heterogeneous sources and then bulk loading this data consistently acros s the firewall to a cloud environment. To deliver data between firewalls, the bulk load solution needs to support modern web-services style integrations which can be delivered via HTTP, also known as dataaccess services. For example, an organization that is moving its customer relationship management (CRM) functionality in to a SaaS CRM application would need to do an upfront bulk upload of all the customer data currently residing in various applications both packaged and custom applications into the SaaS CRM application. In addition to the ability of moving data, in bulk, at extreme performance, the underlying data integration platform also needs comprehensive connectivity to interact with the existing on-premise business applications. Oracle Data Integrator and Oracle GoldenGate can connect your on-premise enterprise systems and the cloud environment by moving data in bulk or as real-time transactions across geographies. In addition, Oracle Data Integrator offers data services for service-oriented architecture and business process management solutions that enable agile application delivery and shared data services for organizations. For companies that decide to deploy private cloud architectures, and decide to put a standardized middleware platform in place as part of platform as a service (PaaS), Oracles data integration portfolio provides comprehensive capabilities from bulk to real-time data movement, transformations, data quality, data services and data federation.

Real-Time Data Synchronization between On-Premise and Cloud

Loading bulk data to the cloud environment for instantiation solves only one part of the integration problem. In many cases, the application data in the cloud needs to be synchronized with the data in the private IT datacenter to enable continuous data access for related business operations. Continuing with the previous example, the customer order information that lives in a SaaS CRM system should now be

Demystifying Data Integration for the Cloud

accessible to order fulfillment systems and customer helpdesk so that the order can be processed immediately and representatives can support customers with an integrated view of the customer. The need for integration between public and on-premise datacenters is not necessarily unidirectional. The data that lives in the primary enterprise systems may need to be shared with the cloud systems as well to enable accurate processing. Using the same example as above, the SaaS CRM application may need to be synchronized with the order fulfillment system to have the order status information up-todate for CRM application end users. The same principle applies to the systems in the private cloud. Continuous data synchronization between legacy enterprise systems and the private cloud infrastructure may be needed to support business operations. Keeping public cloud data in sync with the systems in the private datacenter in an efficient and nonintrusive way can seem a daunting task. In addition, companies need to use solutions that conform to open standards in web services or JMS to be able to move data through firewalls and support bidirectional replication between public cloud and private data sources. Finally the replication solution needs to maintain transaction integrity to ensure access to reliable data with accurate business context, and provide guaranteed data delivery to support business-critical systems. Oracle GoldenGate provides capabilities to distribute data in real time across all on-premise and private cloud environments; enabling access to data consistently across geographically dispersed data centers. Both Oracle GoldenGate and Oracle Data Integrator can feed data into JMS messaging systems.

Integration between On-Premise and Cloud

For public cloud architectures, Oracle offers a complete and extensible set of integration options. Specifically for Oracle Cloud, Oracle SOA Suite excels at integrating on-premise systems with Oracle cloud based services for application and service level integrations. For example, it can integrate with the Oracle Messaging Cloud Service using REST APIs, and with Oracle Fusion Applications services using SOAP APIs. Furthermore, it supports best practices for application service level integration with cloud services, including service virtualization, mediation, result set caching, service aggregation, process orchestration, centralized security policy management etc. This complements the data integration features to provide one of the most comprehensive on-premise to cloud service integration offerings in the industry. For data loading and transforming in the cloud, Oracle Data Integrator supports a knowledge module for Salesforce.comnow available on AppExchange. Other third-party knowledge modules are being developed by customers and partners every day. All of these data integration solutions, like Oracle SOA, are built using the technology underpinnings of Oracle Fusion Middleware which include key elements for managing, securing, extending, and deploying both enterprise and cloud applications. Oracle Fusion Middleware is also the centerpiece of Oracle Cloud Platform Services. Oracle Cloud offers a broad portfolio of software as a service applications, platform as a service, and social capabilities, all on a subscription basis. Oracle Cloud delivers instant value and productivity for end users, administrators, and developers alike through functionally rich, integrated, secure, enterprise

Demystifying Data Integration for the Cloud

cloud services. With Oracle Cloud, you get enterprise-grade application and platform services based on best-in-class business applications and the industry's leading database and application server, managed by experts with over a decade of cloud delivery experience.

Summary
Cloud computing brings many benefits to organizations; from agility, to operational cost savings to reduced time to market. Data integration technology plays a vital role in migrating data simply, efficiently, and reliably to the cloud; they are a necessary ingredient of any platform-as-a-service strategy because they support cloud deployments with data-layer application integration between onpremise and cloud environments of all kinds. Oracles complete and best -of-breed data integration offering, led by Oracle Data Integrator and Oracle GoldenGate products, address unique challenges in deploying and managing cloud infrastructures: 1) Eliminating downtime and minimizing risks for private cloud consolidation 2) Maintaining continuous availability and high availability for cloud infrastructures 3) Integrating and synchronizing data between on-premise systems and the cloud environment Oracle Data Integration products complete and best-of-breed offering provide the standards-based, reliable platform for migrating to and managing data movement for cloud environments.

Learn More
To learn more about Oracle Data Integration products and Oracle Cloud goto: www.oracle.com/goto/dataintegration and www.oracle.com/cloud

10

Demystifying Data Integration for the Cloud October 2012 Author: Dain Hansen, Irem Radzik Contributing Authors: Julien Testut, Gene Eun Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA 94065 U.S.A. Worldwide Inquiries: Phone: +1.650.506.7000 Fax: +1.650.506.7200 oracle.com

Copyright 2012, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error -free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchant ability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark licensed through X/Open Company, Ltd. 0611