Professional Documents
Culture Documents
Borkur Steingrimsson, Rittman Mead Consulting Oracle Open World 2010, San Francisco, September 2010
T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
Introduction
Brkur Steingrimsson Principal Consultant at Rittman Mead Consulting Blog ! http://www.rittmanmead.com/blog/ 15 years of dabbling in Oracle technology ! Discoverer ! DW ! OWB / ODI ! OBIEE ! GG
Introduction
Brkur Steingrimsson Principal Consultant at Rittman Mead Consulting Blog ! http://www.rittmanmead.com/blog/ 15 years of dabbling in Oracle technology ! Discoverer ! DW ! OWB / ODI ! OBIEE ! GG
T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
Agenda
Introduction to GoldenGate Technical Things ! Installation ! Platforms / Services ! Development ! Deployment Demo ! Interface; command line & GUI ! DML & DDL ! Log files GoldenGate and ODI
Oracle GoldenGate
Oracle GoldenGate
Supported Oracle Versions ! 9i Release 2 ! 11g Release 2 CDC ! Real-Time ! Log-based ! Large data volume ! Low-impact and footprint DML and DDL replication ETL - E-LT ! Filter ! Mapping ! Transformations
T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com
Technical Things
Local installation of Oracle GoldenGate on each participating box ! Oracle GoldenGate ! Protocols: TCP/IP ! Managers default listen on port 7809 Movement on data can be encrypted Windows can run the Manager as a service ! Create scripts to start up on Unix Increasing number of platforms supported ! iOS4 support for iPad is probably in the making ;) EXTRACT and REPLICAT groups are created and configured to move data around
Availability
Zero-downtime operations ! Enable uninterrupted business operations during system upgrade, migration, and maintenance activities. Disaster recovery and data protection ! Immediate failover with up-to-the-minute data to minimize recovery time ! Deploy with Oracle Database across database versions or operating systems ! Non-Oracle environments. Data distribution ! Synchronize data for distributed applications in real time across geographies for reliable access to timely data. Query offloading ! Ensure high performance for production systems while still supporting necessary read-only activities by replicating data between heterogeneous sources and targets.
T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com
Operational reporting
! Offload reporting activity from production databases to lower- cost secondary systems with current data for real-time reporting. Oracle GoldenGate 11g is certified to support major Oracle applications, including Oracle E-Business Suite, JD Edwards, PeopleSoft and Siebel CRM, for operational reporting solutions.
There is no magic!
There is no magic!
But it is quite DBA centric - so perhaps there is some magic?
T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com
Building Blocks
Extract groups Replication groups Trails Maps Files parameter files scripts and the OBEY command
Installation
Each node will have the Oracle GoldenGate software installed ! Download the platform-specific zip files from EDelivery.oracle.com - Operating system - 32bit or 64 bit - 10.4 or the new 11g release - Database vendor & version ! Unzip the downloaded file in to a new directory Create default directories to store our metadata Configure the Manager on each node ! Which port (default is 7809) ! Service name ! Windows: Configure to run as Service & Install Event Messages Bobs your uncle
T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com
Configuration
The GGSCI utility (GoldenGate Command Interpreter) is our first entry point to the installation ! Command-Line interface ! Configure the Manager process ! Manage extractions and replications ! Edit parameters - Global - extract / replicat specific ! Help command - Often with examples ! Run scripts ! View statistics ! ... and then some
T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com
port 7809
port 7809
Initial Load
We want to have special loads that get all the data sources & targets in synch We can filter, transform and map the source to target or just pass the data through directly SOURCEISTABLE keyword allows us to by-pass trail files on extract SPECIALRUN keyword makes for a one-time run and not a continuous process BULKLOAD tells the replicate process to interface with SQL*Ldr
Initial Load
We want to have special loads that get all the data sources & targets in synch We can filter, transform and map the source to target or just pass the data through directly SOURCEISTABLE keyword allows us to by-pass trail files on extract SPECIALRUN keyword makes for a one-time run and not a continuous process BULKLOAD tells the replicate process to interface with SQL*Ldr
GGSCI (winxpvm) 49> send extract oow_ex1, showtrans Sending SHOWTRANS request to EXTRACT OOW_EX1 ... Oldest redo log le necessary to restart Extract is: Redo Log Sequence Number 39, RBA 10782224 -----------------------------------------------------------XID: 5.28.1323 Items: 564442 Extract: OOW_EX1 Redo Thread: 1 Start Time: 2010-09-17:18:35:02 SCN: 0.1402704 (1402704) Redo Seq: 39 Redo RBA: 10783248 Status: Running GGSCI (winxpvm) 50> send extract oow_ex1, skiptrans 5.28.1323 FORCE Sending skiptrans request to EXTRACT OOW_EX1 ... Transaction [XID 5.28.1323, Redo Thread 1, Start Time 2010-09-21:18:35:02, SCN 0 .1402704 (1402704)] skipped.
T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com
DDL Support
DDL operations are written to the redo log ! CREATE / DROP ! ALTER ! RENAME ! GRANT (etc) ! Database Import Various parameters control the replication process ! DDLOPTIONS ! DDLERROR ! DDLSUBST ! PURGEDDLHISTORY ! PURGEMARKERHISTORY
DDL Support
tables clusters indexes functions packages procedure tablespaces roles sequences synonyms triggers types views materialized views users
DDL Support
Mapping occurs between source and target ! Renaming of base objects and derived objects ! Only propagate certain DDL actions - EXCLUDE | INCLUDE - OPTYPE - OBJTYPE - OBJNAME - INSTR[WORDS] - INSTRCOMMENTS[WORDS]
DDL & INCLUDE UNMAPPED & OPTYPE alter & OBJTYPE table & OBJNAME users.tab* & INCLUDE MAPPED OBJNAME * & EXCLUDE MAPPED OBJNAME "temporary.tab*"
LOCAL_TEMP_DIR : C:\TEMP SRC_LSCHEMA : GG_TEST_SRC SRC_DB_USER : oow_gg SRC_DB_PASSWORD : password1 SRC_OGG_PATH : C:\GOLDENGATE\GOLDENGATE_SRC STG_HOST : winxpvm.rmcvm.com STG_MANAGER_PORT : 7910 STG_OGG_PATH : C:\GOLDENGATE\GOLDENGATE_STG COMPATIBLE : 10
This readme file provides a detail of the different steps to be executed manually in order to complete the setup of the CDC using OGG. UPLOAD FILES TO SOURCE MACHINE The files for the source machine have been generated in C:\TEMP/ODIS_to_ODIT1/src These files must be copied on the source machine, to the following location C:\GoldenGate \GoldenGate_Src using the same directory structure. For instance, files contained in C:\TEMP/ODIS_to_ODIT1/src/dirprm should end up in C:\GoldenGate \GoldenGate_Src/dirprm UPLOAD FILES TO STAGING MACHINE The files for the staging machine have been generated in C:\TEMP/ODIS_to_ODIT1/stg These files must be copied on the target machine, to the following location C:\GoldenGate \GoldenGate_Tgt using the same directory structure. For instance, files contained in C:\TEMP/ODIS_to_ODIT1/stg/dirprm should end up in C:\GoldenGate \GoldenGate_Tgt/dirprm RUN THE SOURCE OBEY FILE Connect to the source machine using a command line tool such as cmd on Windows or a shell on UNIX/Linux and execute the following command: C:\GoldenGate\GoldenGate_Src/ggsci paramfile C:\GoldenGate\GoldenGate_Src/diroby/ODISS.oby GENERATE THE DEFINITION FILE Connect to the source machine using a command line tool such as cmd on Windows or a shell on UNIX/Linux and execute the following command: C:\GoldenGate\GoldenGate_Src/defgen paramfile C:\GoldenGate\GoldenGate_Src/dirprm/ODISD.prm COPY DEFINITION FILE Copy the definition file C:\GoldenGate\GoldenGate_Src/dirdef/ODISC.def from the source machine to C:\GoldenGate\GoldenGate_Tgt/dirdef/ODISC.def on the staging machine. RUN THE STAGING OBEY FILE Connect to the staging machine using a command line tool such as cmd on Windows or a shell on UNIX/Linux and execute the following commands: C:\GoldenGate\GoldenGate_Tgt/ggsci paramfile C:\GoldenGate\GoldenGate_Tgt/diroby/ODIT1T.oby Sunday, 26 September 2010
Conclusion
Oracle GoldenGate offers very reliable, zero down-time movement of data Useful for ! Replication ! Migration ! DR ! Data distribution ! CDC Relatively straight forward to set up ! But highly customizable ! Hundreds of parameters and settings to play with ! Secure and Encrypted ... and Fun :)
Questions?