You are on page 1of 13

SoftwareTestingHelp

SoftwareTestingCompleteGuide
ASimple12StepsGuidetoWriteanEffectiveTest
SummaryReport[withaSampleReportforDownload]

Search

HowtoTestSmarter:ExploreMore,DocumentLess

ETLvs.DBTestingACloserLookatETLTesting
Need,PlanningandETLTools
PostedIn|DatabaseTesting|LastUpdated:"March17,2016"
Softwaretestinghasvarietyofareastobeconcentrated.Majorvarietiesarefunctionalandnonfunctional
testing.Functionaltestingistheproceduralwaytoensuresothatthefunctionalitydevelopedworksasexpected.
Nonfunctionaltestingistheapproachbywhichthenonfunctionalaspectslikeenhancedorperformanceatan
acceptedlevelcanbeensured.
ThereisanotherflavoroftestingcalledDBtesting.Dataisorganizedinthedatabaseintheformoftables.For
businesstherecanbeflowswherethedatafromthemultipletablescanbemergedorprocessedontoasingle
tableandviceversa.

ETLtestingisoneanotherkindoftestingthatispreferredinthebusinesscasewhereakindofreportingneedis
soughtbytheclients.Thereportingissoughtinordertoanalyzethedemands,needsandthesupplysothat
clients,businessandtheendusersareverywellservedandbenefited.
Whatyouwilllearninthistutorial?
Inthistutorialyouwilllearnwhatisdatabasetesting,whatisETLtesting,differencebetweenDBtestingand
ETLtesting,andmoredetailsaboutETLtestingneed,process,andplanningwithrealexamples.
WehavealsocoveredETLtestinginmoredetailsonbelowpage.Alsohavealookatit.
=>ETLTesting/DataWarehouseTestingTipsandTechniques

DBTestingvs.ETLTesting
MostofusarelittleconfusedoverconsideringthatbothdatabasetestingandtheETLtestingaresimilarand
same.Thefactistheyaresimilarbutnotsame.

DBtesting:
DBTestingisusuallyusedextensivelyinthebusinessflowswheretherearemultipledataflowsoccurringinthe
applicationfrommultipledatasourcesontoasingletable.Thedatasourcecanbeatable,flatfile,applicationor
anythingelsethatcanyieldsomeoutputdata.Inturntheoutputdataobtainedcanstillbeusedasinputforthe
sequentialbusinessflow.HencewhenweperformDBtestingthemostimportantthingthathastobecapturedis
thewaythedatacangettransformedfromthesourcealongwithhowitgetssavedinthedestinationlocation.
SynchronizationisonemajorandtheessentialthingthathastobeconsideredwhenperformingtheDBtesting.
Duetothepositioningoftheapplicationinthearchitecturalflow,theremightbefewissueswiththedataorDB
synchronization.Hencewhileperformingthetesting,thishastobetakencareasthiscanovercomethepotential
invaliddefectsorbugs.
Example#1:
ProjectAhasintegratedarchitecturewhereparticularapplicationmakesuseofdatafromseveralother
heterogeneousdatasources.Hencetheintegrityofthesedatawiththedestinationlocationhastobedonealong
withthevalidationsforthefollowing:
Primaryforeignkeyvalidation
Columnvaluesintegrity
Nullvaluesforanycolumns
WhatisETLTesting?

ETLtestingisaspecialtypeoftestingthattheclientwantstohaveitdonefortheirforecastingandanalysisof
theirbusiness.Thisismostlyusedforthereportingpurposes.Forinstanceiftheclientsneedtohavereportson
thecustomerswhouseorgofortheirproductbasedonthedaytheypurchase,theyhavetomakeuseoftheETL
reports.
Postanalysisandreporting,thisdataisdatawarehousedtoadatawarehousewheretheoldhistoricalbusiness
datahastobemoved.
Thisisamultipleleveltestingasthedatafromthesourceistransformedintomultipleenvironmentsbeforeit
reachesthefinaldestinedlocation.

Example#2:
WewillconsideragroupAdoingretailcustomerbusinessthroughashoppingmarketwherethecustomer
canpurchaseanyhouseholditemsrequiredfortheirdaytodaysurvival.Hereallthecustomersvisitingare
providedwithauniquemembershipidwithwhichtheycangainpointseverytimetheycometopurchasethings
fromtheshoppingmarket.Theregulationsprovidedbythegroupsaythatthepointsgainedexpireeveryyear.
Anddependingupontheirusage,themembershipcanbeeitherupgradedtoahighergradememberor
downgradedtoalowergrademembercomparativelytothecurrentgrade.After5yearsofshoppingmarket
establishmentnowmanagementislookingforscalinguptheirbusinessalongwithrevenue.
Hencetheyrequiredfewbusinessreportssothattheycanpromotetheircustomers.
Indatabasetestingweperformthefollowing:
1)Validationsonthetargettableswhicharecreatedwithcolumnswithlogicalcalculationsasdescribedinthe
logicalmappingsheetandthedataroutingdocument.
2)ManipulationslikeInserting,updatinganddeletionofthecustomerdatacanbeperformedontheanyenduser
POSapplicationinanintegratedsystemalongwiththebackenddatabasesothatthesamechangesarereflected
intheendsystem.
3)DBtestinghastoensurethatthereisnocustomerdatathathasbeenmisinterpretedoreventruncated.This
mightleadtoseriousissueslikeincorrectmappingofcustomerdatawiththeirloyalty
InETLtestingwecheckforthefollowing:
1)Assumingthereare100customersinthesource,youwillcheckwhetherallthesecustomersalongwiththeir
datafromthe100rowshavebeenmovedfromthesourcesystemtothetarget.Thisisknownasverificationof
Datacompletenesscheck.
2)Checkingifthecustomerdatahasbeenproperlymanipulatedanddemonstratedinthe100rows.Thisis
simplycalledasverificationofDataaccuracycheck.
3)Reportsforthecustomerswhohavegainedpointsmorethanxvalueswithintheparticularperiod.

ComparativestudyofETLandDBtesting
ETLandDBtestinghavefewoftheaspectsdifferingwithinthemselvesthatismoreessentialtobeunderstood
beforeperformingthem.Thishelpsusinunderstandingthevaluesandsignificanceofthetestingandthewayit
helpsthebusiness.
Followingisatabularformthatdescribesthebasicbehaviourofboththetestingformats.

DBTesting

ETLTesting

Primary
goal

Dataintegration

BIReporting

Applicable
place

Inthefunctionalsystemwherethe
businessflowoccurs

Externaltothebusinessflowenvironment.input
isthehistoricalbusinessdata

Automation
tool

QTP,Selenium

Informatica,QuerySurge,COGNOS

Business
impact

Severeimpactscanleadasitisthe
integratedarchitectureofthebusiness
flows

Potentialimpactsasinwhentheclientswantsto
havetheforecastingandanalysistobedone

Modelling

EntityRelationship

Dimensional

used
System

OnlineTransactionProcessing

OnlineAnalyticalProcessing

Data
Nature

Normalizeddataisbeingusedhere

Denormalizeddataisbeingusedhere

WhyshouldthebusinessgoforETL?
PlentyofbusinessneedsareavailableforthemtoconsiderETLtesting.Everybusinesshastohavetheirunique
missionandthelineofbusiness.Allbusinesshastheirproductlifecyclewhichtakesthegenericform:

Itisveryclearthatanynewproductentersthemarketwithatremendousgrowthinsalesandtillastagecalled
maturityandthereafteritdeclinesinsales.Thisgradualchangewitnessesadefinitedropinbusinessgrowth.
Henceitismoreimportanttoanalyzethecustomerneedsforthebusinessgrowthandotherfactorsrequiredto
maketheorganisationmoreprofitable.Soinreality,theclientswanttoanalyzethehistoricaldataandcomeup
withsomereportsstrategically.

ETLtestplanning
OneofthemainstepsinETLtestingisaboutplanningthetestthatisgoingtobeexecuted.Itwillbesimilarto
thetestplanforthesystemtestingthatisusuallyperformedexceptfewattributeslikerequirementsandtest

cases.
Heretherequirementsarenothingbutamappingsheetthatwillhavekindofmappingbetweendatawithin
differentdatabases.AsweareawarethattheETLtestingoccursinmultiplelevels,therearevariousmappings
neededforvalidatingthis.

Mostofthetimethedatabeingcapturedfromthesourcedatabasesisnotdirectly.Allthesourcedatawillhave
thetablesviewfromwherethedatacanbeused.
Examples:Followingisanexampleofhowthemappingscanbeprovided.ThetwocolumnsVIEW_NAMEand
TABLE_NAMEcanbeusedtorepresenttheviewsforreadingdatafromthesourceandthetableintheETL
environmentrespectively.Itisadvisabletomaintainnamingconventionthatcanhelpuswhileplanningfor
automation.Genericnotationthatcanbeusedisjustprefixingthenameoftheenvironment.

MostsignificantthinginETLisaboutidentifyingtheessentialdataandthetablesfromthesource.Thenext
essentialstepisthemappingoftablesfromsourcetotheETLenvironment.
Followingisanexamplehowthemappingbetweenthetablesfromthevariousenvironmentscanberelatedfor
theETLpurpose.

Theabovemappingassumesthedatafromthesourcetabletothestagetable.Andfromthenontothetablesin

EDWandthentoOLAPwhichisthefinalreportingenvironment.Henceatanypointoftime,data
synchronizationisveryimportantfortheETLsake.

CriticalETLneeds
AsweunderstandETListheneedforforecasting,reportingandanalysingofthebusinessinordertocapturethe
customerneedsinamoresuccessivemanner.Thiswillenablethebusinesstohavehigherdemandsthanthepast.
HerearefewofthecriticalneedswithoutwhichETLtestingcannotbeachieved:
1. DataandtablesidentificationThisisimportantastherecanbemanyotherirrelevantandunnecessary
datathatcanbeofleastimportancewhenforecastingandanalysingthecustomerneeds.Hencethe
relevantdataandthetableshavetobeselectedbeforestartinguptheETLworks.
2. MappingsheetThisisoneofthecriticalneedswhiledoingETLworks.Mappingoftherighttablefrom
thesourcetothedestinationismandatoryandanyproblemsorincorrectdatainthissheetmightimpactthe
wholeETLdeliverable.
3. Tabledesignsanddata,columntypeThisisnextmajorstepwhenconsideringthemappingofsource
tablesintothedestinedtables.Thecolumntypehastomatchwiththetablesatboththeplacesetc.
4. DatabaseaccessMainthingistheaccesstothedatabasewhereETLgoeson.Anyrestrictionsonthe
accesswillhaveequivalentimpact.

ETLreportingandtesting
ReportinginETLismoreimportantasitexplainsanddirectstheclientsthecustomerneeds.Bythistheycan
forecastandanalysetheexactcustomerneeds
Example#3:
Acompanywhichmanufacturessilkfabricwantedtoanalyseontheirannualsales.Onreviewoftheirannual
sales,theyfoundduringthemonthofAugustandSeptembertherewastremendousfallinsaleswiththeuseof
thereporttheygenerated.Hencetheydecidedtorolloutpromotionalofferlikeexchange,discountsetc.,that
enhancedtheirsales.

BasicissuesinETLtesting
TherecanbenumberofissueswhileperformingETLtestinglikethefollowing:
1. Eithertheaccesstothesourcetablesortheviewswillnotbevalid.
2. Thecolumnnameandthedatatypefromthesourcetothenextlayermightnotmatch.
3. Numberofrecordsfromthesourcetabletothedestinedtabledmightnotmatch.
Andtheremightbemanymore..
FollowingisasampleofmappingsheetwheretherearecolumnslikeVIEW_NAME,COLUMN_NAME,
DATA_TYPE,TABLE_NAME,COLUMN_NAME,DATA_TYPE,andTRANSFORMATIONLOGIC
present.
Thefirst3columnsrepresentthedetailsofthesourcedatabaseandthenext3arethedetailsfortheimmediate
precedingdatabase.Thelastcolumnisveryimportant.Transformationlogicisthewaythedatafromthesource
isreadandstoredinthedestineddatabase.ThisdependsonthebusinessandtheETLneeds.

PointstorememberwhileETLtestplanningandexecution
ThemostimportantthinginETLtestingisloadingofdatabasedontheextractioncriteriafromthesourceDB.
WhenthiscriterionisinvalidorobsoletethentherewillbenodatainthetabletoperformETLtestingthatreally
bringsinmoreissues.
FollowingarefewofthepointstobetakencarewhileETLtestplanningandexecution:
#1:Dataisbeingextractedfromtheheterogeneousdatasources
#2:ETLprocesshandlingintheintegratedenvironmentthathavedifferent:
DBMS
OS
Hardware
Communicationprotocols
#3:Necessityinhavingalogicaldatamappingsheetbeforethephysicaldatacanbetransformed
#4:Understandingandexaminingofthedatasources
#5:Initialloadandtheincrementalload
#6:Auditcolumns
#7:Loadingthefactsandthedimensions

ETLToolsandtheirsignificantusage
ETLtoolsarebasicallyusedtobuildandconvertthetransformationlogicbytakingdatafromthesourceinto
anotherapplyingthetransformationlogic.Youcanalsomaptheschemasfromthesourcetothedestination
whichoccursinuniqueways,transformandcleanupdatabeforeitcanbemovedintothedestination,alongwith
loadingatthedestinationinanefficientmanner.
ThiscansignificantlyreducethemanualeffortsasthemappingcanbedonethatisusedforalmostalloftheETL
validationandtheverification.
ETLtools:
1. InformaticaPowerCenterisoneofthepopularETLtoolsthatisintroducedbytheInformatica
Corporation.Thishasverygoodcustomerbasecoveringwideareas.Themajorcomponentsofthetoolare
itstoolsforclientsandtherepositorytoolsandtheservers.Toknowmoreaboutthetoolpleaseclickhere
2. IBMInfosphereInformationServerIBMwhoisthemarketleaderintermsofComputertechnology
hasdevelopedtheInfosphereInformationserverthatisusedfortheInformationIntegrationand
Managementintheyear2008.Toknowmoreaboutthetoolpleaseclickhere
3. OracleDataIntegratorOracleCorporationhasdevelopedtheirETLtoolinthenameofOracle
DataIntegrator.TheirincreasingcustomersupporthasmadethemupdatetheirETLtoolsinvarious
versions.Toknowmoreaboutthetoolpleaseclickhere
MoreexampleswithusageofETLtesting:
ConsideringsomeAirlineswhichwanttorolloutpromotionsandofferstoattractthecustomersstrategically.
Firstlytheywilltrytounderstandthedemandsandneedswiththecustomersspecifically.Inordertoachievethis
theywillrequirethehistoricaldatapreferablytheprevious2yearsdata.Usingthedatatheywillanalyzeand
preparesomereportsthatwillbehelpfulinunderstandingthecustomersneeds.

Thereportscanbeoffollowingkind:
1. CustomersfromregionAwhotravelstoregionBoncertaindates
2. CustomerswithspecificagecriteriontravelstocityXX
Andtherecanbemanyotherreports.
Analyzingthesereportswillhelptheclientsinidentifyingthekindofpromotionsandoffersthatwillbenefitthe
customersandatthesametimecanbenefitbusinesswherethiscanbecomeaWinWinsituation.Thiscanbe
easilyachievedbytheETLtestingandreports.
Inparallel,theITsegmentfacesseriousDBissuethathasbeennoticedthathasstoppedmultipleservicesinturn
hasthepotentialtocauseimpactsinthebusiness.Oninvestigationitwasidentifiedthatsomeinvaliddatahas
corruptedfewdatabasesthatneededtobecorrectedmanually.
Intheformercase,itisETLreportsandtestingthatwillberequired.
WhereasthelattercaseiswheretheDBtestinghastobedoneproperlytoovercomeissueswithinvaliddata.

Conclusion:
HopetheabovetutorialhasprovidedasimpleandclearoverviewofwhatETLtestingisandwhyithastobe
donealongwiththebusinessimpactsorbenefitstheyyield.Thisdoesnotstophere,butitcanextendtoset
foresightingrowthinbusiness.
Abouttheauthor:ThistutorialiswrittenbyNagarajan.Heisatestleadwithover6yearsofsoftwaretesting
experienceinvariousfunctionalareaslikeBanking,Airlines,andTelecomintermsofbothmanualand
automation.
Pleaseletusknowyourthoughts/questionsincommentsbelow.
54
Like

24

63
Share
Tweet

17

TheBestSoftwareTestingTrainingYou'llEverGet!

17comments
#1Surbhi
niceexplanation.doyouhaveETLtestingtrainingaswell?
#2Sathish
NicearticleofETLTesting.
#3KunjalGandhi
Awesomeexplanation!!
#4VenkateswaraRaoKotta
VeryNiceArticle
#5NjunekiJayne
Ihaveenjoyedreadingthisarticle,veryeducative.
#6rinku
thankuforthevaluableinformation
#7Nagarajan
@Surbhi,Sathish,VenkateswaraRaoKotta,NjunekiJayne
Thanksforyourcomments.YoucanalwaysgetbacktousregardinganyclarificationsfromSoftware
Testing.
Nagarajan
#8Nagarajan
@Surbhi,
WearePlanningtoconducttrainingonETLtestingaswell.Willkeeppostedonourtrainingscalendar..
Pleasekeepchecking!!
Nagarajan
#9Boxfish
UsefulArticle,ThankyouforSharing.
Ithinkyourwebsitehaveaproblemwithhtmllink.ididntseeanyimagesonthispage,fyi.
#10Nandhini
Verynicelyandneatlyexplained.Thanks
#11MananderSingh
Nicepost!!!
IamconfusedwithmyworkthatIamworkingformycompany.
Myjobistovalidatetheexcelreportsforcorrectnessandcompletenessagainstdatabaseentries
manualy

WhatIthoughtitsnotatesting.
ButafterreadingyouarticleIthoughtitsakindofETLtesting.
Canyouexplainalittle.
#12Tester
Hi,Ihaveonesuggestionforyourwebsite.Ifoneleftclicksonanyimageonwebpageitgetsopeninthe
entirepageandweneedtogobacktothepage.eg.Checkpagehttp://www.softwaretestinghelp.com/etl
testingvsdbtesting/.Itwouldbemuchbetterifyourimagewillnotrespondtoleftclickandrather
providethisoptioninrightclick.
#13Neelakanta
NiceExplanation
#14PiyushDubey
Thanksforsuchagoodexlplanation.AreyouconductingETLtestingalso.Ifyesthenplzupdatemeon
piyushdubey5@gmail.com.
#15Vasu
Whichoneisbestofthesetwoetlanddbtesting.Justsuggestne.
#16Saritha
ThisarticleprovidedrequiredinfoforfurtherprogressinETLTestingininitialstage
#17mallesh
FullyclarifiedaboutETLtestingandDataWarehousethankyouverymuchforyoureffortstohelp
others..

LeaveaComment
Name
Mail
Website(Optional)

Submit

LatestArticles!
58BestMobileTestingToolsYouCantLiveWithout
WhatisTechnicalDebtandWhyQATestersShouldbeConcernedAboutIt?
TheBeginnersGuidetoWebApplicationPerformanceTestingusingWAPTPro
ThisScenarioExplainsHowImportantItistoDocumenttheFrequentlyEncounteredErrors
3StrategiesforDealingwithaBlockerDefect

HowtoImproveSoftwareQualityUsingContinuousIntegrationProcess
ContinuousDelivery:HowtoHavetheReliableSoftwareReleasestoProductionatAnyTime

FollowUs!

BestOnlineTraining

Adv

Categories
AgileTesting
AutomationTesting
BasicsofSoftwaretesting
BugDefecttracking
CareerinSoftwareTesting
CookieTesting
DatabaseTesting
GameTesting
General
GUITesting
Howtobeagoodtester
HPQualityCenter
InstallationTesting
ISOstandards
ISTQBCertification
LoadRunnerTutorials
ManualTesting
MobileTesting
QAcertifications
QAforum
QAleadership
QAteamskills
QATestengineersPayscale
QTPTutorials
Qualityassurance
Questions&answers
Reviews
Securitytesting
SeleniumTutorials
Softskillsfortesters
SoftwareJobOpenings
SoftwareTestingBooks
SoftwareTestingEvents
softwaretestinglinks
SoftwareTestingResume
SoftwareTestingTemplates
SoftwareTestingTools
SoftwareTestingTraining
TestManagementTools
TestPlanTemplate
Teststrategy
TestervsDeveloper
Testingbestpractices
TestingConcepts
TestingInterviewquestions
TestingLifecycle

TestingMethodologies
TestingNews
TestingSkillImprovement
TestingTipsandresources
TestingWebServices
Typesoftesting
WebTesting
Winrunner
WriteForUs
Search

STARTHERE!
HomeAboutus|Sitemap|Contactus|FindJobs|Directory|Affiliates|Advertise
GetFREEUpdates
TestingRESOURCES
Allarticlesarecopyrightedandcannotbereproducedwithoutpermission.
QATestingTRAINING
PremiumeBook
FREEeBooks 20062015SoftwareTestingHelpReadourCopyrightPolicy|PrivacyPolicy
|LinktoUs ALLArticles
ContactUs
WhatReadersSayAboutUs
YourFeedback!

DownloadFreeeBook+GetBlog
Updates
YourEmail:

Help&Tutorials
ISTQBStudyGuide
ISTQBPremiumStudyGuide
FreeQATraining
FreeSeleniumTraining
FreeQTPTraining
FreeQCTraining
HPLoadRunnerTutorials
JIRATutorials
101+InterviewQuestions