You are on page 1of 10

Implementing Teradata Utilities in Informatica

By

Chandru Samuthiraraj (chandru.samuthiraraj@wipro.com) Oct 2009

Implementing Teradata Utilities in Informatica

Table of Contents

1. Introduction................................................................................................................3 2. !radata "ti#iti!s........................................................................................................3 3. $appin% that us!s &ump'(ast)oad..........................................................................* +. Sh!## Script ,or d!cidin% )oad!r yp!.......................................................................*. S!ssion #!.!# &rop!rti!s............................................................................................../ ........................................................................................................................................0 -. 1ata #oad and "sa%! o, )oad!r )o% (i#!...................................................................0 h! a2o.! 3u!ry on th! 4rror ta2#! shows that 115th row ,rom th! sourc! %ot r!j!ct!d 2!caus! o, In.a#id tim!stamp.......................................................................................10 /. Conc#usion................................................................................................................10

Confidential

Page 2

Implementing Teradata Utilities in Informatica

1. Introduction
In som! o, th! 4 ) oo#s6 Imp#!m!ntin% !radata uti#iti!s #i7! &ump6 (ast #oad !tc is a cha##!n%!. his docum!nt !8p#ains how to imp#!m!nt !radata uti#iti!s in In,ormatica 0.-.0. In,ormatica 0.-.0 supports .arious !nt!rpris!9wid! data int!%ration and data 3ua#ity so#utions inc#udin% data war!housin%6 data mi%ration6 data conso#idation6 data synchroni:ation6 data %o.!rnanc!6 mast!r data mana%!m!nt6 and cross9!nt!rpris! data int!%ration. !radata is a massi.!#y para##!# proc!ssin% syst!m runnin% a shar!d nothin% archit!ctur!. h! !radata 1B$S is #in!ar#y and pr!dicta2#y sca#a2#! in a## dim!nsions o, a data2as! syst!m wor7#oad (data .o#um!6 2r!adth6 num2!r o, us!rs6 comp#!8ity o, 3u!ri!s). !radata acts as a sin%#! data stor! that can acc!pt #ar%! num2!rs o, concurr!nt r!3u!sts ,rom mu#tip#! c#i!nt app#ications. Si%ni,icant ,!atur!s inc#ud!; "nconditiona# para##!#ism6 with #oad distri2ution shar!d amon% s!.!ra# s!r.!rs. Comp#!8 ad hoc 3u!ri!s with up to -+ joins. &ara##!# !,,ici!ncy6 such that th! !,,ort ,or cr!atin% 100 r!cords is sam! as that ,or cr!atin% 1006000 r!cords. Sca#a2i#ity6 so that incr!asin% o, th! num2!r o, proc!ssors o, an !8istin% syst!m #in!ar#y incr!as!s th! p!r,ormanc!. &!r,ormanc! thus do!s not d!t!riorat! with an incr!as!d num2!r o, us!rs.

2. Teradata Utilities
!radata o,,!rs c!rtain uti#iti!s that assists in data war!housin% mana%!m!nt and maint!nanc! a#on% with th! !radata <1B$S. Som! o, th!m ar! (ast)oad (ast48port &ump $u#ti)oad FastLoad:

Confidential

Page 3

Implementing Teradata Utilities in Informatica

his para##!# #oad uti#ity is d!si%n!d to mo.! #ar%! .o#um!s o, data co##!ct!d ,rom data sourc!s on chann!# and n!twor79attach!d c#i!nts into !mpty ta2#!s in th! !radata 1ata2as!.(ast)oad typica##y o,,!rs hi%h!r p!r,ormanc! #!.!#s than a standard app#ication pro%ram writt!n to #oad data to an !mpty data2as! 2!caus! data a##ocation6 con.!rsion6 mo.!m!nt6 and #oadin% ar! automatic and p!r,orm!d in para##!#. Features; Initia# ta2#! #oad. 1!#imit!d or ,i8!d ,ormat input. B!st ,or hu%! data .o#um!s. Constraints; h! tar%!t ta2#! must 2! !mpty. =o acc!ss (r!ad or writ!) is a##ow!d on th! tar%!t ta2#! whi#! #oadin%. On#y a sin%#! ta2#! can 2! #oad!d in on! jo2. FastExport:

(ast48port is a command9dri.!n uti#ity that us!s mu#tip#! s!ssions to 3uic7#y trans,!r #ar%! amounts o, data ,rom ta2#!s and .i!ws o, th! !radata <!#ationa# 1ata2as! $ana%!m!nt Syst!m (<1B$S) to a c#i!nt92as!d app#ication. !8port can 2! don! ,rom any ta2#! or .i!w to which you ha.! th! S4)4C acc!ss pri.i#!%!. h! d!stination ,or th! !8port!d data can 2!; > ,i#! on your chann!#9attach!d or n!twor79attach!d c#i!nt syst!m >n Output $odi,ication (O" $O1) routin! you writ! to s!#!ct6 .a#idat!6 and pr!proc!ss th! !8port!d data TPump: &ump is a hi%h#y para##!# uti#ity d!si%n!d to continuous#y mo.! data ,rom data sourc!s into !radata ta2#!s without #oc7in% th! a,,!ct!d ta2#!.
Confidential Page 4

Implementing Teradata Utilities in Informatica

&ump pro.id!s n!ar9r!a#9tim! data into your data war!hous!6 a##owin% you to maintain ,r!sh6 accurat! data ,or up9to9th!9mom!nt d!cision9ma7in%. ?ou can us! &ump to ins!rt6 updat!6 ups!rt6 and d!#!t! data in th! !radata 1ata2as!. Features: (ast6 sca#a2#! continuous data #oads. >##ows tar%!t ta2#!s to; @ Aa.! s!condary ind!8!s6 r!,!r!ntia# int!%rity6 constraints @ B! popu#at!d or !mpty. B!st ,or sma## data .o#um!s. ultiLoad: $u#ti)oad is a command9dri.!n para##!# #oad uti#ity ,or hi%h9.o#um! 2atch maint!nanc! on mu#tip#! ta2#!s and .i!ws o, th! !radata 1ata2as!. It is sp!cia##y d!si%n!d ,or hi%h9sp!!d 2atch cr!ation and maint!nanc! o, #ar%! data2as!s. It Supports up to ,i.! popu#at!d ta2#!s and &!r,orms 2#oc7 #!.!# op!rations a%ainst popu#at!d ta2#!s

3.

apping t!at uses TPump"FastLoad


ar%!t is a !radata ta2#!. Som!

In this mappin% Sourc! is a ,#at ,i#! and 48pr!ssions ar! us!d to Con.!rt Strin% to dat!.

Confidential

Page 5

Implementing Teradata Utilities in Informatica

#. $!ell $cript for deciding Loader T%pe


(or #oadin% a ta2#! in !radata w! can us! &ump or (ast)oad "ti#ity o, !radata. Bas!d on th! (!atur!s o, th!s! "ti#iti!s6 B! can say &ump is suita2#! ,or sma## data .o#um! and (ast)oad is suita2#! ,or hu%! data .o#um!. h! B!#ow Sh!## Script can 2! us!d ,or d!,inin% th! )oad!r Conn!ction in <untim!. hat is6 wh!n!.!r th! ,#at ,i#! %ot %!n!rat!d ,rom som! !n.ironm!nt6 th! Script wi## ta7! th! Count o, rows and i, th! count !8c!!ds *00006 th! Script wi## s!t th! )oad!r Conn!ction as (ast)oad or !#s! th! Conn!ction wi## 2! &ump.

LoaderConnection_S cript.txt

In th! a2o.! Script th! )oad #imit is d!c#ar!d as *0000. h!r! ar! ,i.! Input &aram!t!rs ar! us!d in th! Script. S4SSIO=C=>$4 h! s!ssion ,rom which th! mappin% is %oin% to run. &><>$C(I)4 )oad!r Conn!ction typ! wi## 2! sa.!d in this ,i#!. )<(C=>$4 h! Input ,#at ,i#! nam!. &"$&CCO==4C IO= h! nam! o, th! <!#ationa# Conn!ction ,or &ump. ()O>1CCO==4C IO= h! nam! o, th! <!#ationa# Conn!ction ,or (ast)oad. h! Script can 2! ca##!d ,rom a Command s!ssion or you can s!t th! &r!9 s!ssion9command in your s!ssion.

Confidential

Page 6

Implementing Teradata Utilities in Informatica

Presession_Cm d_Scri pt.txt

In th! a2o.! script6 D&$<oot1ir'Script(i#!s'data#oad'crC 1CconnCtyp!.7sh is th! dir!ctory and ,i#! path o, th! )oad!r Con!!ction d!cidin% script in which th! cod! is writt!n o, ca#cu#atin% th! num2!r o, r!cords in th! ,#at ,i#! and s!ttin% th! #oad!r conn!ction accordin% to th! row count. sCmCIS&C=> IE4CSC>SS4 C)<(CI=C is th! s!ssion nam!. his s!ssion is us!d ,or !8!cutin% th! mappin% and th! s!ssion #!.!# prop!rti!s ha.! to 2! s!t in this s!ssion as m!ntion!d 2!#ow in th! S!ssion #!.!# &rop!rti!s s!ction. D&$<oot1ir'BB&aram's.c'IS&C=> IE4'w,CSECCIS&C=> IE4C)<(CI=C.parm is th! dir!ctory and ,i#! path o, th! &aram!t!r ,i#! in which th! #oad!r conn!ction ,or th! sCmCIS&C=> IE4CSC>SS4 C)<(CI=C s!ssion wi## 2! stor!d dynamica##y 2y th! crC 1CconnCtyp!.7sh )oad!r Con!!ction d!cidin% script. his param!t!r ,i#! wi## 2! ca##!d ,rom th! sCmCIS&C=> IE4CSC>SS4 C)<(CI=C s!ssion to %!t th! #oad!r conn!ction usin% th! #oad!r conn!ction .aria2#! D)oad!rConn!ctionCsCmCIS&C=> IE4CSC>SS4 C)<(CI=C. &"$&CS4<EIC4CIS&C4 )C01 and ()O>1CS4<EIC4CIS&C4 )C01 ar! th! &ump and (#oad conn!ctions. On! o, th!s! conn!ctions wi## 2! s!t a%ainst th! D)oad!rConn!ctionCsCmCIS&C=> IE4CSC>SS4 C)<(CI=C .aria2#! usin% th! crC 1CconnCtyp!.7sh script ,or th! sCmCIS&C=> IE4CSC>SS4 C)<(CI=C s!ssion in th! w,CSECCIS&C=> IE4C)<(CI=C.parm param!t!r ,i#!. D&$<oot1ir' %t(i#!s's.c'IS&C=> IE4'#r,CSC>SS4 .dat is th! dir!ctory and ,i#! path o, th! ,#at ,i#!.

&. $ession le'el Properties


h! ,o##owin% st!ps ha.! to 2! don! in th! s!ssion #!.!#. $tep1: h! path and th! nam! o, th! &aram!t!r ,i#! ha.! to 2! %i.!n in th! &rop!rti!s &an!#.

Confidential

Page 7

Implementing Teradata Utilities in Informatica

$tep2: h! Sourc! ,#at ,i#! nam! and th! dir!ctory path ha.! to 2! %i.!n in th! Sourc! attri2ut! prop!rti!s o, th! $appin% &an!#.

$tep(: In th! ar%!t &rop!rti!s th! Conn!ction typ! has to 2! F)oad!r5 and th! .a#u! shou#d 2! th! )oad!rConn!ctionCGS!ssion nam!H. his )oad!r Conn!ction .a#u! wi## 2! ,!tch!d ,rom th! param!t!r ,i#!. h! param!t!r ,i#! stor!s th! conn!ction .a#u! dynamica##y usin% th! Sh!## Script

). *ata load and Usage of Loader Log File


>,t!r s!ttin% a## th! a2o.! prop!rti!s th! s!ssion can 2! run and th! data wi## %!t #oad!d into th! !radata tar%!t ta2#!. (or !.!ry !radata #oad6 On! )oad!r #o% ,i#! Bi## 2! %!t %!n!rat!d on th! sam! path o, th! Sourc! ,i#!.

Confidential

Page 8

Implementing Teradata Utilities in Informatica

)oad!r #o% ,i#! can us!d to .!ri,y th! num2!r o, rows r!ad ,rom th! sourc!6 num2!r o, rows #oad!d into th! tar%!t and th! num2!r o, rows #oad!d into th! 4rror ta2#!. !radata automatica##y %!n!rat!s an 4rror ta2#! ,or !.!ry #oad. h! r!cords which %ot r!j!ct!d on th! #oad wi## 2! #oad!d in to th! !rror ta2#! and th! r!ason ,or th! r!j!ction wi## 2! stor!d in a co#umn o, th! !rror ta2#!. Example Loader Log File: B!#ow is th! !8amp#! #oad!r #o% ,i#! which shows that 20000 rows ar! ,!tch!d ,rom th! sourc! and 0 rows #oad!d into th! !rror ta2#!. 4rror ta2#! nam! is 4<<O<C .4 CSC>SS4 0

Example Error Table:

Confidential

Page 9

Implementing Teradata Utilities in Informatica

h! a2o.! 3u!ry on th! 4rror ta2#! shows that 115th row ,rom th! sourc! %ot r!j!ct!d 2!caus! o, In.a#id tim!stamp.

+. Conclusion
A!nc! this docum!nt c#!ar#y !8p#ains th! !radata uti#iti!s and main#y ,ocus!s on imp#!m!ntin% th! !radata #oad uti#iti!s #i7! &ump and (ast)oad usin% In,ormatica and th! usa%! o, )oad!r #o% ,i#! and th! 4rror ta2#!s.

Confidential

Page 10

You might also like