Professional Documents
Culture Documents
ODI Trainer
Blogger at http://www.rittmanmead.com/blog/
Email : jerome.francoisse@rittmanmead.com
Twitter : @JeromeFr
Worlds leading specialist partner for technical Providing our customers targeted expertise; we are a
excellence, solutions delivery and innovation in company that doesnt try to do everything only
Oracle Data Integration, Business Intelligence, what we excel at
Analytics and Big Data
Founded on the values of collaboration, learning,
70+ consultants worldwide including 1 Oracle ACE integrity and getting things done
Director and 3 Oracle ACEs
Comprehensive service portfolio designed to
support the full lifecycle of any analytics solution
Set-based transformation
Pipeline of transformations
Dataflow processes
RDDs
in-Memory
pages_and_posts_details.saveAsTextFile("/user/
val logs_base = sc.textFile(raw_logs) flatMap { logs_base_page_schemaRDD.registerAsTable("logs_
mrittman/top_10_pages_and_author4")
case rowRegex(host, base_page")
identity, user, time, request, status, size,
referer, agent) => }
val page_count = sql("SELECT request_page,
count(*) as hits FROM logs_base_page GROUP BY }
Seq(accessLogRow(host, identity, user, time, request_page").registerAsTable("page_count")
request, status, size, referer, agent))
Hard to maintain
No Governance
ODI
Files - Logs
Files
API BigData SQL
Flume OLH/OSCH
Sqoop Sqoop
NoSQL Enterprise
Database DWH
Hive Hive
HBase HBase
HDFS HDFS
OLTP
Database
info@rittmanmead.com www.rittmanmead.com @rittmanmead 17
Import Hive Table Metadata into ODI Repository
1
- Can be temperamental with tables using non-standard SerDes;
make sure JARs registered
3
OdiOSCommand "-OUT_FILE=/tmp/
C___Calc_Ratings__Hive___Pig___Spark_.out" "-ERR_FILE=/tmp/
C___Calc_Ratings__Hive___Pig___Spark_.err" "-WORKING_DIR=/tmp"
/usr/lib/spark/bin/spark-submit --master yarn-client /tmp/
C___Calc_Ratings__Hive___Pig___Spark_.py --py-files /tmp/
pyspark_ext.py --executor-memory 1G --driver-cores 1 --
executor-cores 1 --num-executors 2
ODI
Files - Logs
Files
API BigData SQL
Flume OLH/OSCH
Sqoop Sqoop
NoSQL Enterprise
Database DWH
Hive Hive
HBase HBase
HDFS HDFS
OLTP
Database
info@rittmanmead.com www.rittmanmead.com @rittmanmead 27
Oozie
no additional footprint
ODI
Files - Logs
Files
API BigData SQL
Flume OLH/OSCH
Sqoop Sqoop
NoSQL Enterprise
Database DWH
Hive Hive
HBase HBase
HDFS HDFS
OLTP
Database
info@rittmanmead.com www.rittmanmead.com @rittmanmead 30
Oracle Big Data SQL
IKM Hive Append used for join and load into Hive target
ODI
Files - Logs
Files
API BigData SQL
Flume OLH/OSCH
Sqoop Sqoop
NoSQL Enterprise
Database DWH
Hive Hive
HBase HBase
HDFS HDFS
OLTP
Database
info@rittmanmead.com www.rittmanmead.com @rittmanmead 34
Missing?
Streaming Capabilities
Spark Streaming
Kafka
http://www.rittmanmead.com/2015/04/odi12c-advanced-
big-data-option-overview-install/
http://www.rittmanmead.com/2015/04/so-whats-the-real-
point-of-odi12c-for-big-data-generating-pig-and-spark-
mappings/
Blogs:
- www.rittmanmead.com/blog
Contact:
- info@rittmanmead.com
- jerome.francoisse@rittmanmead.com
Twitter
- @rittmanmead
- @JeromeFr
Blogs:
- www.rittmanmead.com/blog
Contact:
- info@rittmanmead.com
- jerome.francoisse@rittmanmead.com
Twitter
- @rittmanmead
- @JeromeFr
Jrme Franoisse | Sunday, Oct 25, 8:00am | Mark Rittman | Sunday, Oct 25, 2:30pm | Moscone
Moscone South 301
South 301
Empowering Users: Oracle Business Intelligence Oracle Data Integration Product Family: a
Enterprise Edition 12c Visual Analyzer [UGF5481] Cornerstone for Big Data [CON9609]
Edelweiss Kammermann | Sunday, Oct 25, 10:00am Mark Rittman | Wednesday, Oct 28, 12:15pm |
| Moscone West 3011
Moscone West 2022
A Walk Through the Kimball ETL Subsystems Developer Best Practices for Oracle Data
with Oracle Data Integration Solutions [UGF6311] Integrator Lifecycle Management [CON9611]
Michael Rainey | Sunday, Oct 25, 12:00pm | Jrme Franoisse | Thursday, Oct 29, 2:30 pm |
Moscone South 301 Moscone West 2022