Transformations Part 1 in Informatica

6/4/2015
krishnareddyoracleapps.blogspot.in/2012/07/transformationspart1ininformatica.html
TRANSFORMATIONSPART1ININFORMATICA
FILTERTRANSFORMATION
Activeandconnectedtransformation.
WecanfilterrowsinamappingwiththeFiltertransformation.WepassalltherowsfromasourcetransformationthroughtheFiltertransformation,
andthenenteraFilterconditionforthetransformation.AllportsinaFiltertransformationareinput/outputandonlyrowsthatmeettheconditionpass
throughtheFilterTransformation.
Example:tofilterrecordswhereSAL>2000
ImportthesourcetableEMPinSharedfolder.Ifitisalreadythere,thendontImport.
Insharedfolder,createthetargettableFilter_Example.KeepallfieldsasinEMPtable.
Createthenecessaryshortcutsinthefolder.
CreatingMapping:
1. Openfolderwherewewanttocreatethemapping.
2. ClickTools>MappingDesigner.
3. ClickMapping>Create>Givemappingname.Ex:m_filter_example
4. DragEMPfromsourceinmapping.
5. ClickTransformation>Create>SelectFilterfromlist.GivenameandclickCreate.Nowclickdone.
6. PassportsfromSQ_EMPtoFilterTransformation.
7. EditFilterTransformation.GotoPropertiesTab
8. ClicktheValuesectionoftheFiltercondition,andthenclicktheOpenbutton.
9. TheExpressionEditorappears.
10. Enterthefilterconditionyouwanttoapply.
11. ClickValidatetocheckthesyntaxoftheconditionsyouentered.
12. ClickOK>ClickApply>ClickOk.
http://krishnareddyoracleapps.blogspot.in/2012/07/transformationspart1ininformatica.html
1/35
6/4/2015
13. NowconnecttheportsfromFiltertotargettable.
14. ClickMapping>Validate
15. Repository>Save
CreateSessionandWorkflowasdescribedearlier.Runtheworkflowandseethedataintargettable.
Howtofilteroutrowswithnullvalues?
Tofilteroutrowscontainingnullvaluesorspaces,usetheISNULLandIS_SPACESFunctionstotestthevalueoftheport.Forexample,ifwewant
tofilteroutrowsthatContainNULLsintheFIRST_NAMEport,usethefollowingcondition:
IIF(ISNULL(FIRST_NAME),FALSE,TRUE)
ThisconditionstatesthatiftheFIRST_NAMEportisNULL,thereturnvalueisFALSEandtherowshouldbediscarded.Otherwise,therowpasses
throughtothenextTransformation.
Performancetuning:
FiltertransformationisusedtofilteroffunwantedfieldsbasedonconditionsweSpecify.
1. UsefiltertransformationasclosetosourceaspossiblesothatunwanteddatagetsEliminatedsooner.
2. Ifeliminationofunwanteddatacanbedonebysourcequalifierinsteadoffilter,TheneliminatethematSourceQualifieritself.
3. Useconditionalfiltersandkeepthefilterconditionsimple,involvingTRUE/FALSEor1/0
============================================================
ExpressionTRANSFORMATION:
2/35
6/4/2015
Passiveandconnectedtransformation.
UsetheExpressiontransformationtocalculatevaluesinasinglerowbeforewewritetothetarget.Forexample,wemightneedtoadjustemployee
salaries,concatenatefirstandlastnames,orconvertstringstonumbers.
UsetheExpressiontransformationtoperformanynonaggregatecalculations.
Example:Addition,Subtraction,Multiplication,Division,Concat,Uppercaseconversion,lowercaseconversionetc.
WecanalsousetheExpressiontransformationtotestconditionalstatementsbeforeweoutputtheresultstotargettablesorothertransformations.
Example:IF,Then,Decode
Thereare3typesofportsinExpressionTransformation:
Input
Output
Variable:Usedtostoreanytemporarycalculation.
CalculatingValues:
TousetheExpressiontransformationtocalculatevaluesforasinglerow,wemustincludethefollowingports:
Inputorinput/outputportsforeachvalueusedinthecalculation:Forexample:TocalculateTotalSalary,weneedsalaryandcommission.
Outputportfortheexpression:Weenteroneexpressionforeachoutputport.Thereturnvaluefortheoutputportneedstomatchthereturn
valueoftheexpression.
WecanentermultipleexpressionsinasingleExpressiontransformation.Wecancreateanynumberofoutputportsinthetransformation.
Example:CalculatingTotalSalaryofanEmployee
ImportthesourcetableEMPinSharedfolder.Ifitisalreadythere,thendontimport.
Insharedfolder,createthetargettableEmp_Total_SAL.KeepallportsasinEMPtableexceptSalandCommintargettable.AddTotal_SAL
porttostorethecalculation.
Createthenecessaryshortcutsinthefolder.
3/35
6/4/2015
CreatingMapping:
3. ClickMapping>Create>Givemappingname.Ex:m_totalsal
5. ClickTransformation>Create>SelectExpressionfromlist.GivenameandclickCreate.Nowclickdone.
6. LinkportsfromSQ_EMPtoExpressionTransformation.
7. EditExpressionTransformation.AswedonotwantSalandCommintarget,removecheckfromoutputportforbothcolumns.
8. Nowcreateanewportout_Total_SAL.Makeitasoutputportonly.
9. ClickthesmallbuttonthatappearsintheExpressionsectionofthedialogboxandentertheexpressionintheExpressionEditor.
4/35
6/4/2015
10. EnterexpressionSAL+COMM.YoucanselectSALandCOMMfromPortstabinexpressioneditor.
11. ChecktheexpressionsyntaxbyclickingValidate.
12. ClickOK>ClickApply>ClickOk.
13. NowconnecttheportsfromExpressiontotargettable.
15. Repository>Save
CreateSessionandWorkflowasdescribedearlier.Runtheworkflowandseethedataintargettable.
AsCOMMisnull,Total_SALwillbenullinmostcases.Nowopenyourmappingandexpressiontransformation.SelectCOMMport,InDefault
Valuegive0.Nowapplychanges.ValidateMappingandSave.
Refreshthesessionandvalidateworkflowagain.Runtheworkflowandseetheresultagain.
NowuseERRORinDefaultvalueofCOMMtoskiprowswhereCOMMisnull.
5/35
6/4/2015
Syntax:ERROR(Anymessagehere)
Similarly,wecanuseABORTfunctiontoabortthesessionifCOMMisnull.
Syntax:ABORT(Anymessagehere)
Makesuretodoubleclickthesessionafterdoinganychangesinmapping.Itwillpromptthatmappinghaschanged.ClickOKtorefreshthemapping.
Runworkflowaftervalidatingandsavingtheworkflow.
Performancetuning:
ExpressiontransformationisusedtoperformsimplecalculationsandalsotodoSourcelookups.
1. Useoperatorsinsteadoffunctions.
2. Minimizetheusageofstringfunctions.
3. Ifweuseacomplexexpressionmultipletimesintheexpressiontransformer,thenMakethatexpressionasavariable.Thenweneedtouseonly
thisvariableforallcomputations.
===================================================================
RouterTransformation:
Activeandconnectedtransformation.
ARoutertransformationissimilartoaFiltertransformationbecausebothtransformationsallowyoutouseaconditiontotestdata.AFilter
transformationtestsdataforoneconditionanddropstherowsofdatathatdonotmeettheCondition.However,aRoutertransformationtestsdatafor
oneormoreconditionsAndgivesyoutheoptiontorouterowsofdatathatdonotmeetanyoftheconditionstoadefaultoutputgroup.
Example:IfwewanttokeepemployeesofFrance,India,USin3differenttables,thenwecanuse3Filtertransformationsor1Router
transformation.
MappingAusesthreeFiltertransformationswhileMappingBproducesthesameresultwithoneRoutertransformation.
ARoutertransformationconsistsofinputandoutputgroups,inputandoutputports,groupfilterconditions,andpropertiesthatweconfigureinthe
Designer.
6/35
6/4/2015
7/35
6/4/2015
WorkingwithGroups
ARoutertransformationhasthefollowingtypesofgroups:
Input:TheGroupthatgetstheinputports.
Output:UserDefinedGroupsandDefaultGroup.WecannotmodifyordeleteOutputportsortheirproperties.
UserDefinedGroups:Wecreateauserdefinedgrouptotestaconditionbasedonincomingdata.Auserdefinedgroupconsistsofoutputportsand
agroupfilterCondition.WecancreateandedituserdefinedgroupsontheGroupstabwiththeDesigner.Createoneuserdefinedgroupforeach
conditionthatwewanttospecify.
TheDefaultGroup:TheDesignercreatesthedefaultgroupafterwecreateonenewuserdefinedgroup.TheDesignerdoesnotallowustoeditor
deletethedefaultgroup.Thisgroupdoesnothaveagroupfilterconditionassociatedwithit.IfalloftheconditionsevaluatetoFALSE,theISpasses
therowtothedefaultgroup.
Example:FilteringemployeesofDepartment10toEMP_10,Department20toEMP_20andresttoEMP_REST
SourceisEMPTable.
Create3targettablesEMP_10,EMP_20andEMP_RESTinsharedfolder.StructureshouldbesameasEMPtable.
Createtheshortcutsinyourfolder.
CreatingMapping:
1.Openfolderwherewewanttocreatethemapping.
2.ClickTools>MappingDesigner.
3.ClickMapping>Create>Givemappingname.Ex:m_router_example
4.DragEMPfromsourceinmapping.
5.ClickTransformation>Create>SelectRouterfromlist.Givenameand
ClickCreate.Nowclickdone.
6.PassportsfromSQ_EMPtoRouterTransformation.
7.EditRouterTransformation.GotoGroupsTab
8.ClicktheGroupstab,andthenclicktheAddbuttontocreateauserdefinedGroup.Thedefaultgroupiscreatedautomatically..
9.ClicktheGroupFilterConditionfieldtoopentheExpressionEditor.
10.Enteragroupfiltercondition.Ex:DEPTNO=10
11.ClickValidatetocheckthesyntaxoftheconditionsyouentered.
8/35
6/4/2015
12.CreateanothergroupforEMP_20.Condition:DEPTNO=20
13.TherestoftherecordsnotmatchingtheabovetwoconditionswillbepassedtoDEFAULTgroup.Seesamplemapping
14.ClickOK>ClickApply>ClickOk.
15.Nowconnecttheportsfromroutertotargettables.
16.ClickMapping>Validate
17.Repository>Save
CreateSessionandWorkflowasdescribedearlier.RuntheWorkflowandseethedataintargettable.
Makesuretogiveconnectioninformationforall3targettables.
9/35
6/4/2015
SampleMapping:
DifferencebetweenRouterandFilter:
Wecannotpassrejecteddataforwardinfilterbutwecanpassitinrouter.RejecteddataisinDefaultGroupofrouter.
========================================================
============================================================
10/35
6/4/2015
SORTERTRANSFORMATION
ConnectedandActiveTransformation
TheSortertransformationallowsustosortdata.
Wecansortdatainascendingordescendingorderaccordingtoaspecifiedsortkey.
WecanalsoconfiguretheSortertransformationforcasesensitivesorting,andspecifywhethertheoutputrowsshouldbedistinct.
WhenwecreateaSortertransformationinamapping,wespecifyoneormoreportsasasortkeyandconfigureeachsortkeyporttosortinascending
ordescendingorder.WealsoconfiguresortcriteriathePowerCenterServerappliestoallsortkeyportsandthesystemresourcesitallocatesto
performthesortoperation.
TheSortertransformationcontainsonlyinput/outputports.AlldatapassingthroughtheSortertransformationissortedaccordingtoasortkey.The
sortkeyisoneormoreportsthatwewanttouseasthesortcriteria.
SorterTransformationProperties
1.SorterCacheSize:
ThePowerCenterServerusestheSorterCacheSizepropertytodeterminethemaximumamountofmemoryitcanallocatetoperformthesort
operation.ThePowerCenterServerpassesallincomingdataintotheSortertransformationBeforeitperformsthesortoperation.
Wecanspecifyanyamountbetween1MBand4GBfortheSortercachesize.
Ifitcannotallocateenoughmemory,thePowerCenterServerfailstheSession.
Forbestperformance,configureSortercachesizewithavaluelessthanorequaltotheamountofavailablephysicalRAMonthePowerCenter
Servermachine.
Informaticarecommendsallocatingatleast8MBofphysicalmemorytosortdatausingtheSortertransformation.
2.CaseSensitive:
TheCaseSensitivepropertydetermineswhetherthePowerCenterServerconsiderscasewhensortingdata.WhenweenabletheCaseSensitive
property,thePowerCenterServersortsuppercasecharactershigherthanlowercasecharacters.
3.WorkDirectory
DirectoryPowerCenterServerusestocreatetemporaryfileswhileitsortsdata.
4.Distinct:
Checkthisoptionifwewanttoremoveduplicates.Sorterwillsortdataaccordingtoalltheportswhenitisselected.
11/35
6/4/2015
Example:SortingdataofEMPbyENAME
SourceisEMPtable.
CreateatargettableEMP_SORTER_EXAMPLEintargetdesigner.StructuresameasEMPtable.
CreatingMapping:
3.ClickMapping>Create>Givemappingname.Ex:m_sorter_example
4.DragEMPfromsourceinmapping.
5.ClickTransformation>Create>SelectSorterfromlist.GivenameandclickCreate.Nowclickdone.
6.PassportsfromSQ_EMPtoSorterTransformation.
7.EditSorterTransformation.GotoPortsTab
8.SelectENAMEassortkey.CHECKmarkonKEYinfrontofENAME.
9.ClickPropertiesTabandSelectPropertiesasneeded.
10.ClickApply>Ok.
11.Dragtargettablenow.
12.ConnecttheoutputportsfromSortertotargettable.
13.ClickMapping>Validate
14.Repository>Save
Makesuretogiveconnectioninformationforalltables.
SampleSorterMapping:
12/35
6/4/2015
PerformanceTuning:
Sortertransformationisusedtosorttheinputdata.
1. Whileusingthesortertransformation,configuresortercachesizetobelargerthantheinputdatasize.
2. ConfigurethesortercachesizesettingtobelargerthantheinputdatasizewhileUsingsortertransformation.
3. Atthesortertransformation,usehashautokeyspartitioningorhashuserkeysPartitioning.
=============================================================
RANKTRANSFORMATION:
Activeandconnectedtransformation
TheRanktransformationallowsustoselectonlythetoporbottomrankofdata.ItAllowsustoselectagroupoftoporbottomvalues,notjustone
value.
Duringthesession,thePowerCenterServercachesinputdatauntilitcanperformTherankcalculations.
RankTransformationProperties:
CacheDirectorywherecachewillbemade.
Top/BottomRankasperneed
NumberofRanksEx:1,2oranynumber
CaseSensitiveComparisoncanbecheckedifneeded
RankDataCacheSizecanbeset
RankIndexCacheSizecanbeset
PortsinaRankTransformation:
Ports
NumberRequired Description
13/35
6/4/2015
1Minimum
Porttoreceivedatafrom
anothertransformation.
1Minimum
Portwewanttopasstoother
transformation.
notneeded
canusetostorevaluesor
calculationstouseinan
expression.
Only1
Rankport.Rankiscalculated
accordingtoit.TheRankport
isaninput/outputport.We
mustlinktheRankportto
anothertransformation.
Example:TotalSalary
RankIndex
TheDesignerautomaticallycreatesaRANKINDEXportforeachRanktransformation.ThePowerCenterServerusestheRankIndexporttostore
therankingpositionforEachrowinagroup.
Forexample,ifwecreateaRanktransformationthatranksthetopfivesalariedemployees,therankindexnumberstheemployeesfrom1to5.
TheRANKINDEXisanoutputportonly.
Wecanpasstherankindextoanothertransformationinthemappingordirectlytoatarget.
Wecannotdeleteoreditit.
DefiningGroups
Ranktransformationallowsustogroupinformation.Forexample:Ifwewanttoselectthetop3salariedemployeesofeachDepartment,wecan
defineagroupforDepartment.
Bydefininggroups,wecreateonesetofrankedrowsforeachgroup.
WedefineagroupinPortstab.ClicktheGroupByforneededport.
WecannotGroupByonportwhichisalsoRankPort.
1)Example:FindingTop5SalariedEmployees
EMPwillbesourcetable.
CreateatargettableEMP_RANK_EXAMPLEintargetdesigner.StructureshouldbesameasEMPtable.JustaddonemoreportRank_Index
tostoreRANKINDEX.
14/35
6/4/2015
CreatingMapping:
3. ClickMapping>Create>Givemappingname.Ex:m_rank_example
5. CreateanEXPRESSIONtransformationtocalculateTOTAL_SAL.
6. ClickTransformation>Create>SelectRANKfromlist.GivenameandclickCreate.Nowclickdone.
7. PassportsfromExpressiontoRankTransformation.
8. EditRankTransformation.GotoPortsTab
9. SelectTOTAL_SALasrankport.CheckRtypeinfrontofTOTAL_SAL.
10. ClickPropertiesTabandSelectPropertiesasneeded.
11. TopinTop/BottomandNumberofRanksas5.
12. ClickApply>Ok.
13. Dragtargettablenow.
14. ConnecttheoutputportsfromRanktotargettable.
16. Repository>Save
2)Example:FindingTop2SalariedEmployeesforeveryDEPARTMENT
Openthemappingmadeabove.EditRankTransformation.
GotoPortsTab.SelectGroupByforDEPTNO.
GotoPropertiestab.SetNumberofRanksas2.
ClickApply>Ok.
Mapping>ValidateandRepositorySave.
Refreshthesessionbydoubleclicking.Savethechangedandrunworkflowtoseethenewresult.
15/35
6/4/2015
RANKCACHE
SampleRankMapping
WhenthePowerCenterServerrunsasessionwithaRanktransformation,itcomparesaninputrowwithrowsinthedatacache.Iftheinputrowout
ranksaStoredrow,thePowerCenterServerreplacesthestoredrowwiththeinputrow.
Example:PowerCentercachesthefirst5rowsifwearefindingtop5salariedEmployees.When6throwisread,itcomparesitwith5rowsincache
andplacesitinCacheisneeded.
1)RANKINDEXCACHE:
Theindexcacheholdsgroupinformationfromthegroupbyports.IfweareUsingGroupByonDEPTNO,thenthiscachestoresvalues10,20,30
etc.
AllGroupByColumnsareinRANKINDEXCACHE.Ex.DEPTNO
2)RANKDATACACHE:
ItholdsrowdatauntilthePowerCenterServercompletestherankingandisGenerallylargerthantheindexcache.Toreducethedatacachesize,
connectOnlythenecessaryinput/outputportstosubsequenttransformations.
AllVariableportsifthere,RankPort,AllportsgoingoutfromRANKTransformationsarestoredinRANKDATACACHE.
Example:AllportsexceptDEPTNOInourmappingexample.
====================================================================
TRANSACTIONCONTROLTRANSFORMATION:
PowerCenterletsyoucontrolcommitandrollbacktransactionsbasedonasetofrowsthatpassthroughaTransactionControltransformation.A
transactionisthesetofrowsboundbycommitorrollbackrows.Youcandefineatransactionbasedonavaryingnumberofinputrows.Youmight
wanttodefinetransactionsbasedonagroupofrowsorderedonacommonkey,suchasemployeeIDororderentrydate.
InPowerCenter,youdefinetransactioncontrolatthefollowinglevels:
Withinamapping.Withinamapping,youusetheTransactionControltransformationtodefineatransaction.Youdefinetransactionsusingan
expressioninaTransactionControltransformation.Basedonthereturnvalueoftheexpression,youcanchoosetocommit,rollback,or
continuewithoutanytransactionchanges.
Withinasession.Whenyouconfigureasession,youconfigureitforuserdefinedcommit.Youcanchoosetocommitorrollbackatransaction
iftheIntegrationServicefailstotransformorwriteanyrowtothetarget.
Whenyourunthesession,theIntegrationServiceevaluatestheexpressionforeachrowthatentersthetransformation.Whenitevaluatesacommit
row,itcommitsallrowsinthetransactiontothetargetortargets.WhentheIntegrationServiceevaluatesarollbackrow,itrollsbackallrowsinthe
transactionfromthetargetortargets.IfthemappinghasaflatfiletargetyoucangenerateanoutputfileeachtimetheIntegrationServicestartsanew
transaction.Youcandynamicallynameeachtargetflatfile.
16/35
6/4/2015
PropertiesTab
OnthePropertiestab,youcanconfigurethefollowingproperties:
Transactioncontrolexpression
Tracinglevel
EnterthetransactioncontrolexpressionintheTransactionControlConditionfield.ThetransactioncontrolexpressionusestheIIFfunctiontotest
eachrowagainstthecondition.Usethefollowingsyntaxfortheexpression:
IIF(condition,value1,value2)
TheexpressioncontainsvaluesthatrepresentactionstheIntegrationServiceperformsbasedonthereturnvalueofthecondition.TheIntegration
Serviceevaluatestheconditiononarowbyrowbasis.ThereturnvaluedetermineswhethertheIntegrationServicecommits,rollsback,ormakesno
transactionchangestotherow.
WhentheIntegrationServiceissuesacommitorrollbackbasedonthereturnvalueoftheexpression,itbeginsanewtransaction.Usethefollowing
builtinvariablesintheExpressionEditorwhenyoucreateatransactioncontrolexpression:
TC_CONTINUE_TRANSACTION.TheIntegrationServicedoesnotperformanytransactionchangeforthisrow.Thisisthedefaultvalueof
theexpression.
TC_COMMIT_BEFORE.TheIntegrationServicecommitsthetransaction,beginsanewtransaction,andwritesthecurrentrowtothetarget.
Thecurrentrowisinthenewtransaction.
TC_COMMIT_AFTER.TheIntegrationServicewritesthecurrentrowtothetarget,commitsthetransaction,andbeginsanewtransaction.
Thecurrentrowisinthecommittedtransaction.
TC_ROLLBACK_BEFORE.TheIntegrationServicerollsbackthecurrenttransaction,beginsanewtransaction,andwritesthecurrentrow
tothetarget.Thecurrentrowisinthenewtransaction.
TC_ROLLBACK_AFTER.TheIntegrationServicewritesthecurrentrowtothetarget,rollsbackthetransaction,andbeginsanew
transaction.Thecurrentrowisintherolledbacktransaction.
Ifthetransactioncontrolexpressionevaluatestoavalueotherthancommit,rollback,orcontinue,theIntegrationServicefailsthesession.
MappingGuidelinesandValidation
UsethefollowingrulesandguidelineswhenyoucreateamappingwithaTransactionControltransformation:
IfthemappingincludesanXMLtarget,andyouchoosetoappendorcreateanewdocumentoncommit,theinputgroupsmustreceivedata
fromthesametransactioncontrolpoint.
TransactionControltransformationsconnectedtoanytargetotherthanrelational,XML,ordynamicMQSeriestargetsareineffectiveforthose
targets.
YoumustconnecteachtargetinstancetoaTransactionControltransformation.
YoucanconnectmultipletargetstoasingleTransactionControltransformation.
YoucanconnectonlyoneeffectiveTransactionControltransformationtoatarget.
17/35
6/4/2015
YoucannotplaceaTransactionControltransformationinapipelinebranchthatstartswithaSequenceGeneratortransformation.
IfyouuseadynamicLookuptransformationandaTransactionControltransformationinthesamemapping,arolledbacktransactionmight
resultinunsynchronizedtargetdata.
ATransactionControltransformationmaybeeffectiveforonetargetandineffectiveforanothertarget.Ifeachtargetisconnectedtoan
effectiveTransactionControltransformation,themappingisvalid.
EitheralltargetsornoneofthetargetsinthemappingshouldbeconnectedtoaneffectiveTransactionControltransformation.
ExampletoTransactionControl:
Step1:Designthemapping.
Step2:CreatingaTransactionControlTransformation.
IntheMappingDesigner,clickTransformation>Create.SelecttheTransactionControltransformation.
Enteranameforthetransformation.[ThenamingconventionforTransactionControltransformationsisTC_TransformationName].
Enteradescriptionforthetransformation.
18/35
6/4/2015
ClickCreate.
ClickDone.
Dragtheportsintothetransformation.
OpentheEditTransformationsdialogbox,andselectthePortstab.
SelectthePropertiestab.Enterthetransactioncontrolexpressionthatdefinesthecommitandrollbackbehavior.
GotothePropertiestabandclickonthedownarrowtogetintotheexpressioneditorwindow.LatergototheVariablestabandType
IIF(EMpno=7654,)selectthebelowthingsfromthebuiltinfunctions.
IIF(EMPNO=7654,TC_COMMIT_BEFORE,TC_CONTINUE_TRANSACTION)
Connectallthecolumnsfromthetransformationtothetargettableandsavethemapping.
SelecttheMetadataExtensionstab.CreateoreditmetadataextensionsfortheTransactionControltransformation.
ClickOK.
19/35
6/4/2015
Step3:Createthetaskandtheworkflow.
Step4:Previewtheoutputinthetargettable.
==================================================================
SOURCEQUALIFIERTRANSFORMATION:
ActiveandConnectedTransformation.
TheSourceQualifiertransformationrepresentstherowsthatthePowerCenterServerreadswhenitrunsasession.
Itisonlytransformationthatisnotreusable.
DefaulttransformationexceptincaseofXMLorCOBOLfiles.
TasksperformedbySourceQualifier:
Joindataoriginatingfromthesamesourcedatabase:Wecanjointwoormoretableswithprimarykeyforeignkeyrelationshipsbylinkingthe
sourcestooneSourceQualifiertransformation.
FilterrowswhenthePowerCenterServerreadssourcedata:IfweIncludeafiltercondition,thePowerCenterServeraddsaWHEREclauseto
theDefaultquery.
Specifyanouterjoinratherthanthedefaultinnerjoin:IfweincludeaUserdefinedjoin,thePowerCenterServerreplacesthejoininformation
SpecifiedbythemetadataintheSQLquery.
Specifysortedports:Ifwespecifyanumberforsortedports,the
PowerCenterServeraddsanORDERBYclausetothedefaultSQLquery.
Selectonlydistinctvaluesfromthesource:IfwechooseSelectDistinct,thePowerCenterServeraddsaSELECTDISTINCTstatementtothe
defaultSQLquery.
CreateacustomquerytoissueaspecialSELECTstatementforthePowerCenterServertoreadsourcedata:Forexample,youmightusea
Customquerytoperformaggregatecalculations.TheentireabovearepossibleinPropertiesTabofSourceQualifiert/f.
SAMPLEMAPPINGTOBEMADE:
20/35
6/4/2015
21/35
6/4/2015
SourcewillbeEMPandDEPTtables.
CreatetargettableasshowedinPictureabove.
Createshortcutsinyourfolderasneeded.
CreatingMapping:
3. ClickMapping>Create>Givemappingname.Ex:m_SQ_example
4. DragEMP,DEPT,Target.
5. RightClickSQ_EMPandSelectDeletefromthemapping.
6. RightClickSQ_DEPTandSelectDeletefromthemapping.
7. ClickTransformation>Create>SelectSourceQualifierfromList>GiveName>ClickCreate
8. SelectEMPandDEPTboth.ClickOK.
9. Linkallasshowninabovepicture.
10. EditSQ>PropertiesTab>OpenUserdefinedJoin>GiveJoinconditionEMP.DEPTNO=DEPT.DEPTNO.ClickApply>OK
11. Mapping>Validate
12. Repository>Save
SQPROPERTIESTAB
1)SOURCEFILTER:
WecanenterasourcefiltertoreducethenumberofrowsthePowerCenterServerqueries.
Note:Whenweenterasourcefilterinthesessionproperties,weoverridethecustomizedSQLqueryintheSourceQualifiertransformation.
Steps:
1. IntheMappingDesigner,openaSourceQualifiertransformation.
2. SelectthePropertiestab.
3. ClicktheOpenbuttonintheSourceFilterfield.
4. IntheSQLEditorDialogbox,enterthefilter.Example:EMP.SAL)2000
5. ClickOK.
Validatethemapping.Saveit.Nowrefreshsessionandsavethechanges.Nowruntheworkflowandseeoutput.
2)NUMBEROFSORTEDPORTS:
Whenweusesortedports,thePowerCenterServeraddstheportstotheORDERBYclauseinthedefaultquery.
Bydefaultitis0.Ifwechangeitto1,thenthedatawillbesortedbycolumnthatisatthetopinSQ.Example:DEPTNOinabovefigure.
22/35
6/4/2015
IfwewanttosortasperENAME,moveENAMEtotop.
Ifwechangeitto2,thendatawillbesortedbytoptwocolumns.
Steps:
1. IntheMappingDesigner,openaSourceQualifiertransformation.
2. SelectthePropertiestab.
3. EnteranynumberinsteadofzeroforNumberofSortedports.
4. ClickApply>ClickOK.
3)SELECTDISTINCT:
IfwewantthePowerCenterServertoselectuniquevaluesfromasource,wecanusetheSelectDistinctoption.
JustchecktheoptioninPropertiestabtoenableit.
4)PREandPOSTSQLCommands
ThePowerCenterServerrunspresessionSQLcommandsagainstthesourcedatabasebeforeitreadsthesource.
ItrunspostsessionSQLcommandsagainstthesourcedatabaseafteritwritestothetarget.
Useasemicolon()toseparatemultiplestatements.
5)USERDEFINEDJOINS
EnteringauserdefinedjoinissimilartoenteringacustomSQLquery.However,weonlyenterthecontentsoftheWHEREclause,nottheentire
query.
Wecanspecifyequijoin,leftouterjoinandrightouterjoinonly.WeCannotspecifyfullouterjoin.Tousefullouterjoin,weneedtowrite
SQLQuery.
Steps:
23/35
6/4/2015
1. OpentheSourceQualifiertransformation,andclickthePropertiestab.
2. ClicktheOpenbuttonintheUserDefinedJoinfield.TheSQLEditorDialogBoxappears.
3. Enterthesyntaxforthejoin.
4. ClickOK>AgainOk.
JoinType
Syntax
EquiJoin
LeftOuterJoin
RightOuterJoin
DEPT.DEPTNO=EMP.DEPTNO
{EMPLEFTOUTERJOINDEPTON
DEPT.DEPTNO=EMP.DEPTNO}
{EMPRIGHTOUTERJOINDEPTON
DEPT.DEPTNO=EMP.DEPTNO}
24/35
6/4/2015
6)SQLQUERY
Forrelationalsources,thePowerCenterServergeneratesaqueryforeachSourceQualifiertransformationwhenitrunsasession.Thedefaultqueryis
aSELECTstatementforeachsourcecolumnusedinthemapping.Inotherwords,thePowerCenterServerreadsonlythecolumnsthatareconnected
toanotherTransformation.
Inmappingabove,wearepassingonlySALandDEPTNOfromSQ_EMPtoAggregatortransformation.Defaultquerygeneratedwillbe:
SELECTEMP.SAL,EMP.DEPTNOFROMEMP
ViewingtheDefaultQuery
1. OpentheSourceQualifiertransformation,andclickthePropertiestab.
2. OpenSQLQuery.TheSQLEditordisplays.
3. ClickGenerateSQL.
4. TheSQLEditordisplaysthedefaultquerythePowerCenterServerusestoSelectsourcedata.
5. ClickCanceltoexit.
Note:IfwedonotcanceltheSQLquery,thePowerCenterServeroverridesthedefaultquerywiththecustomSQLquery.
WecanenteranSQLstatementsupportedbyoursourcedatabase.Beforeenteringthequery,connectalltheinputandoutputportswewanttousein
themapping.
Example:Asinourcase,wecantusefullouterjoininuserdefinedjoin,wecanwriteSQLqueryforFULLOUTERJOIN:
SELECTDEPT.DEPTNO,DEPT.DNAME,DEPT.LOC,EMP.EMPNO,EMP.ENAME,EMP.JOB,EMP.SAL,EMP.COMM,EMP.DEPTNO
FROMEMPFULLOUTERJOINDEPTONDEPT.DEPTNO=EMP.DEPTNOWHERESAL>2000
25/35
6/4/2015
WealsoaddedWHEREclause.WecanentermoreconditionsandwriteMorecomplexSQL.
Wecanwriteanyquery.WecanjoinasmanytablesinonequeryasRequiredifallareinsamedatabase.Itisveryhandyandusedinmostofthe
projects.
ImportantPoints:
WhencreatingacustomSQLquery,theSELECTstatementmustlisttheportnamesintheorderinwhichtheyappearinthetransformation.
Example:DEPTNOistopcolumnDNAMEissecondinourSQmapping.
SowhenwewriteSQLQuery,SELECTstatementhavenameDNAMEfirst,DNAMEsecondandsoon.SELECTDEPT.DEPTNO,
DEPT.DNAME
Oncewehavewrittenacustomquerylikeabove,thenthisquerywillAlwaysbeusedtofetchdatafromdatabase.Inourexample,weused
WHERESAL>2000.NowifweuseSourceFilterandgiveconditionSAL)1000oranyother,thenitwillnotwork.Informaticawillalwaysuse
thecustomqueryonly.
MakesuretotestthequeryindatabasefirstbeforeusingitinSQLQuery.Ifqueryisnotrunningindatabase,thenitwontworkinInformatica
too.
AlsoalwaysconnecttothedatabaseandvalidatetheSQLinSQLqueryeditor.
====================================================================
STOREDPROCEDURETRANSFORMATION:
PassiveTransformation
ConnectedandUnconnectedTransformation
Storedproceduresarestoredandrunwithinthedatabase.
AStoredProceduretransformationisanimportanttoolforpopulatingandMaintainingdatabases.Databaseadministratorscreatestoredproceduresto
AutomatetasksthataretoocomplicatedforstandardSQLstatements.
UseofStoredProcedureinmapping:
Checkthestatusofatargetdatabasebeforeloadingdataintoit.
Determineifenoughspaceexistsinadatabase.
Performaspecializedcalculation.
Dropandrecreateindexes.Mostlyusedforthisinprojects.
26/35
6/4/2015
DataPassesBetweenISandStoredProcedureOneofthemostusefulfeaturesofstoredproceduresistheabilitytosenddatatothestoredprocedure,
andreceivedatafromthestoredprocedure.TherearethreetypesofdatathatpassbetweentheIntegrationServiceandthestoredprocedure:
Input/outputparameters:ParameterswegiveasinputandtheparametersreturnedfromStoredProcedure.
Returnvalues:ValuereturnedbyStoredProcedureifany.
Statuscodes:StatuscodesprovideerrorhandlingfortheISduringaworkflow.Thestoredprocedureissuesastatuscodethatnotifieswhetherornot
thestoredprocedurecompletedsuccessfully.Wecannotseethisvalue.TheISusesittodeterminewhethertocontinuerunningthesessionorstop.
SpecifyingwhentheStoredProcedureRuns
Normal:Thestoredprocedurerunswherethetransformationexistsinthemappingonarowbyrowbasis.Wepasssomeinputtoprocedureandit
returnssomecalculatedvalues.Connectedstoredproceduresrunonlyinnormalmode.
PreloadoftheSource:Beforethesessionretrievesdatafromthesource,thestoredprocedureruns.Thisisusefulforverifyingtheexistenceof
tablesorperformingjoinsofdatainatemporarytable.
PostloadoftheSource:Afterthesessionretrievesdatafromthesource,thestoredprocedureruns.Thisisusefulforremovingtemporarytables.
PreloadoftheTarget:Beforethesessionsendsdatatothetarget,thestoredprocedureruns.Thisisusefulfordroppingindexesordisabling
constraints.
PostloadoftheTarget:Afterthesessionsendsdatatothetarget,thestoredprocedureruns.Thisisusefulforrecreatingindexesonthedatabase.
UsingaStoredProcedureinaMapping:
1. Createthestoredprocedureinthedatabase.
2. ImportorcreatetheStoredProceduretransformation.
3. Determinewhethertousethetransformationasconnectedorunconnected.
4. Ifconnected,maptheappropriateinputandoutputports.
5. Ifunconnected,eitherconfigurethestoredproceduretorunpreorpostsession,orconfigureittorunfromanexpressioninanother
transformation.
6. Configurethesession.
StoredProcedures:
ConnecttoSourcedatabaseandcreatethestoredproceduresgivenbelow:
CREATEORREPLACEproceduresp_agg(in_deptnoinnumber,max_saloutnumber,
min_saloutnumber,avg_saloutnumber,sum_saloutnumber)
As
Begin
selectmax(Sal),min(sal),avg(sal),sum(sal)intomax_sal,min_sal,avg_sal,sum_sal
fromempwheredeptno=in_deptnogroupbydeptno
End
/
CREATEORREPLACEproceduresp_unconn_1_value(in_deptnoinnumber,max_saloutnumber)
As
27/35
6/4/2015
Begin
Selectmax(Sal)intomax_salfromEMPwheredeptno=in_deptno
End
/
1.ConnectedStoredProcedureT/F
Example:TogiveinputasDEPTNOfromDEPTtableandfindtheMAX,MIN,AVGandSUMofSALfromEMPtable.
DEPTwillbesourcetable.CreateatargettableSP_CONN_EXAMPLEwithfieldsDEPTNO,MAX_SAL,MIN_SAL,AVG_SAL&
SUM_SAL.
WriteStoredProcedureinDatabasefirstandCreateshortcutsasneeded.
CreatingMapping:
3.ClickMapping>Create>Givename.Ex:m_SP_CONN_EXAMPLE
4.DragDEPTandTargettable.
5.Transformation>ImportStoredProcedure>GiveDatabaseConnection>Connect>Selecttheproceduresp_aggfromthelist.
28/35
6/4/2015
6.DragDEPTNOfromSQ_DEPTtothestoredprocedureinputportandalsotoDEPTNOportoftarget.
7.Connecttheportsfromproceduretotargetasshownbelow:
29/35
6/4/2015
8.Mapping>Validate
9.Repository>Save
CreateSessionandthenworkflow.
Giveconnectioninformationforalltables.
GiveconnectioninformationforStoredProcedurealso.
Runworkflowandseetheresultintable.
2.UnconnectedStoredProcedureT/F:
AnunconnectedStoredProceduretransformationisnotdirectlyconnectedtotheflowofdatathroughthemapping.Instead,thestoredprocedureruns
either:
Fromanexpression:Calledfromanexpressiontransformation.
Preorpostsession:Runsbeforeorafterasession.
Methodofreturningthevalueofoutputparameterstoaport:
Assigntheoutputvaluetoalocalvariable.
30/35
6/4/2015
AssigntheoutputvaluetothesystemvariablePROC_RESULT.(SeeLater)
Example1:DEPTNOasinputandgetMAXofSalasoutput.
DEPTwillbesourcetable.
CreateatargettablewithfieldsDEPTNOandMAX_SALofdecimaldatatype.
WriteStoredProcedureinDatabasefirstandCreateshortcutsasneeded.
CreatingMapping:
3.ClickMapping>Create>Givename.Ex:m_sp_unconn_1_value
5.Transformation>ImportStoredProcedure>GiveDatabaseConnection>Connect>Selecttheproceduresp_unconn_1_valuefromthelist.
ClickOK.
6.StoredProcedurehasbeenimported.
7.T/F>CreateExpressionT/F.PassDEPTNOfromSQ_DEPTtoExpressionT/F.
8.EditexpressionandcreateanoutputportOUT_MAX_SALofdecimaldatatype.
9.OpenExpressioneditorandcallthestoredprocedureasbelow:ClickOKandconnecttheportfromexpressiontotargetasinmappingbelow:
10.Mapping>Validate
11.RepositorySave.
GiveconnectioninformationforStoredProcedurealso.
31/35
6/4/2015
PROC_RESULTuse:
Ifthestoredprocedurereturnsasingleoutputparameterorareturnvalue,wethereservedvariablePROC_RESULTastheoutputvariable.
Example:DEPTNOasInputandMAXSalasoutput:
:SP.SP_UNCONN_1_VALUE(DEPTNO,PROC_RESULT)
Ifthestoredprocedurereturnsmultipleoutputparameters,youmustcreatevariablesforeachoutputparameter.
Example:DEPTNOasInputandMAX_SAL,MIN_SAL,AVG_SALandSUM_SAL
asoutputthen:
1.CreatefourvariableportsinexpressionVAR_MAX_SAL,VAR_MIN_SAL,VAR_AVG_SALandiVAR_SUM_SAL.
2.CreatefouroutputportsinexpressionOUT_MAX_SAL,OUT_MIN_SAL,OUT_AVG_SALandOUT_SUM_SAL.
3.CalltheprocedureinlastvariableportsaysVAR_SUM_SAL.
:SP.SP_AGG(DEPTNO,VAR_MAX_SAL,VAR_MIN_SAL,VAR_AVG_SAL,PROC_RESULT)
Example2:
DEPTNOasInputandMAX_SAL,MIN_SAL,AVG_SALandSUM_SALasO/PStoredProceduretodropindexinPreLoadofTargetStored
ProceduretocreateindexinPostLoadofTarget
DEPTwillbesourcetable.CreateatargettableSP_UNCONN_EXAMPLEwithfieldsDEPTNO,MAX_SAL,MIN_SAL,AVG_SAL&
SUM_SAL.
WriteStoredProcedureinDatabasefirstandCreateshortcutsasneeded.Storedproceduresaregivenbelowtodropandcreateindexon
target.Makesuretocreatetargettablefirst.StoredProcedurestobecreatedinnextexampleinTargetDatabase:
32/35
6/4/2015
CreateorreplaceprocedureCREATE_INDEX
As
Begin
Executeimmediate'createindexunconn_deptonSP_UNCONN_EXAMPLE(DEPTNO)'
End
/
CreateorreplaceprocedureDROP_INDEX
As
Begin
Executeimmediate'dropindexunconn_dept'
33/35
6/4/2015
End
/
CreatingMapping:
3.ClickMapping>Create>Givename.Ex:m_sp_unconn_1_value
5.Transformation>ImportStoredProcedure>GiveDatabaseConnection>Connect>Selecttheproceduresp_aggfromthelist.ClickOK.
6.StoredProcedurehasbeenimported.
7.T/F>CreateExpressionT/F.PassDEPTNOfromSQ_DEPTtoExpressionT/F.
8.EditExpressionandcreate4variableportsand4outputportsasshownbelow:
34/35
6/4/2015
9.CalltheprocedureinlastvariableportVAR_SUM_SAL.
10.:SP.SP_AGG(DEPTNO,VAR_MAX_SAL,VAR_MIN_SAL,VAR_AVG_SAL,PROC_RESULT)
11.ClickApplyandOk.
12.Connecttotargettableasneeded.
13.Transformation>ImportStoredProcedure>GiveDatabaseConnectionfortarget>Connect>SelecttheprocedureCREATE_INDEXand
DROP_INDEXfromthelist.ClickOK.
14.EditDROP_INDEX>PropertiesTab>SelectTargetPreLoadasStoredProcedureTypeandincalltextwritedrop_index.ClickApply>Ok.
15.EditCREATE_INDEX>PropertiesTab>SelectTargetPostLoadasStoredProcedureTypeandincalltextwritecreate_index.ClickApply>
Ok.
16.Mapping>Validate
17.Repository>Save
GiveconnectioninformationforStoredProceduresalso.
AlsomakesurethatyouexecutetheprocedureCREATE_INDEXondatabasebeforeusingtheminmapping.Thisisbecause,ifthereisno
INDEXontargettable,DROP_INDEXwillfailandSessionwillalsofail.
35/35

Transformations Part 1 in Informatica

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Transformations Part 1 in Informatica

Uploaded by

Copyright:

Available Formats

6/4/2015

You might also like