W ?

addow PeadsLrong ConsulLlng

2011
1
8egresslon 1esL lannlng and LxecuLlon for ul and uaLa
1CICS
opular ueflnlLlons of 8egresslon 1esLlng 1
Common SLraLegles for SelecLlng 8egresslon 1esL SulLes 1
SuggesLed SLraLegy for 8egresslon 1esLlng 8eporLs 2
1he SelecLlon of 1esL Cases for 8egresslon 1esLlng 2
Sample SelecLlon of 1esL Cases for Cne 8l Screen 2
WhaL's needed for L1L (daLa Lable load) 8egresslon 1esL lannlng 4
Popular Definitions of kegress|on 1est|ng
few meanlngs of regresslon LesLlng from Lhe experLs"
1 1esLlng LhaL ls performed afLer maklng a funcLlonal lmprovemenL or repalr Lo a reporL lLs
purpose ls Lo deLermlne lf Lhe change has regressed oLher aspecLs of Lhe reporL or daLa
2 repeLlLlon of LesLs lnLended Lo show LhaL Lhe sofLware's behavlor ls unchanged excepL as
requlred by change Lo Lhe sofLware or daLa
3 1esLlng conducLed for Lhe purpose of evaluaLlng wheLher or noL a change Lo Lhe sysLem has
lnLroduced a new fallure

1he flgure below shows sLages of regresslon LesLlng LhaL mlghL be necessary afLer changes Lo reporLs or
daLa


ommon Strategies for Selecting Regression Test Suites

W ?addow PeadsLrong ConsulLlng
2011
2

ketest n|gh r|or|ty k|sky Use Cases ] 1est Cases Choose basellne LesLs Lo rerun by rlsk
heurlsLlcs mosL rlsk Lo daLa reporL or dashboard when falllng
2 ketest|ng 8y rof||e Choose basellne LesLs Lo rerun by allocaLlng Llme / C resources ln
proporLlon Lo operaLlonal proflle
3 ketest|ng Changed Segments Choose basellne LesLs Lo rerun by assesslng code or daLa changes
ketest Us|ng Lx|st|ng 1est Cases Choose basellne LesLs Lo rerun by analyzlng dependencles and
relaLlonshlps wlLh changed or added code

Suggested Strategy for kegress|on 1est|ng keports

RecommenJeJ: comblne four common xtruteglex from ubove.
ny one of Lhe exlsLlng regresslon LesLlng sLraLegles above may be good buL ln Lhe real world a
comblnaLlon of Lhe sLraLegles may be a beLLer declslon
lL ls assumed LhaL flrsL we LesL Lhe change (flx) lLself by runnlng all relaLed 1C 1hen for regresslon Leslng
1 30 of Lhe allowed regresslon LesL Llme run LesL cases represenLlng Lhe mosL rlsky funcLlons ln
parLlcular Lhose LhaL may be affecLed by new or changed code ComponenLs for deflnlng
prlorlLy one conslder buslness rlsk and frequency of uslng Lhls scenarlo by Lhe cusLomer
2 30 of allowed regressslon LesL Llme ConLlnuous cycle regresslon LesLlng for runnlng Lhe
exlsLenLlng regresslon LesL sulLe
3 20 of allowed regresslon LesL Llme LxploraLory 1esLlng uo noL forgeL Lo properly documenL
Lhe resulLs of exploraLory LesLlng s a mlnlmum updaLe Lhe LesL log lf you do noL llke Lhe
sound of 'exploraLory LesLlng' use Lhe Llme ln Lhe schedule Lo lmprove your undersLandlng of
Lhe requlremenLs Lhe sysLem and your loglcal and archlLecLural coverage of appllcaLlon by 1C
?ou musL allocaLe Llme and resources for Lhls Lask anyway)

30 30 and 20 ls a common alloLmenL oLhers make work beLLer ?ou can play Lhls around as you
llke MosL lmporLanLly all your exlsLlng 1C wlll run you wlll have Lhe prlorlLy for runnlng 1C and you wlll
allocaLe Llme for keeplng your 1C sulL under conLlnuous lmprovemenL
1he Se|ect|on of 1est Cases for kegress|on 1est|ng
O 8equlres knowledge on Lhe bug flxes and how Lhey affecL Lhe sysLem
O lncludes Lhe area of frequenL defecLs
O lncludes Lhe area whlch has undergone many and / or recenL code changes
O lncludes Lhe area whlch ls hlghly vlslble Lo Lhe users
O lncludes Lhe core feaLures of Lhe producL whlch are mandaLory requlremenLs of Lhe cusLomer

Samp|e Se|ect|on of 1est Cases for Cne 8I Screen
1he followlng LesL cases were run on anoLher 8l pro[ecL ll buLLons dropdowns fllLers eLc were
LesLed durlng a 30 mlnuLe perlod

W ?addow PeadsLrong ConsulLlng
2011
3
Step
Descr|pt|on Lxpected kesu|ts
My Cred|ts
Screen
verlfy LhaL porLfollos llsLed ln SelecL
orLfollo dropdown
ll my CredlLs
My rlmary and backup credlLs
My 1eam / racLlce CredlLs
ll CusLom orLfollos
ll porLfollos are llsLed and all credlLs are
llsLed from each porLfollo

verlfy llnd CredlL funcLlon ny credlL wheLher wlLh or wlLhouL CuS daLa
should dlsplay wlLh company name for
selecLlon

verlfy LhaL all credlL daLa are dlsplayed for
a sampllng of porLfollos
lrom SelecL orLfollo selecL one
ll credlLs are dlsplayed for LasL uay wlLh
reasonable daLa

verlfy LhaL vlew CredlL uaLa vallablllLy
llnk works correcLly
1he resulL should be a dlsplay of credlL daLa
LhaL ls avallable for all credlLs among all Lhe
users porLfollos llsLlng when lasL CuS daLa ls
avallable
user should be able Lo download Lhe
compleLe llsL of credlLs ln Lxcel

verlfy erlod dropdown for a sampllng of
LasL uay 7 day eLc
Lach Llme perlod ls dlsplayed and all credlLs
for selecLed porLfollod are llsLed ln Lhe
resulLlng dlsplay 1he same number of credlLs
are llsLed for each Llme perlod from each
porLfollo

verlfy LhaL uownload llnk works correcLly 1he downloaded Lxcel spreadsheeL should
conLaln Lhe credlL daLa conLenLs of Lhe screen
whlch was dlsplayed when Lhe download llnk
was selecLed (daLa for Lhe selecLed porLfollo
and 1lme perlod)

verlfy LhaL rlnL works correcLly 8eporL from screen prlnLs Lo ul wlLh 8uslnes
daLa and Llme porLfollo name and Llme
perlod

verlfy Lhe 1ear SheeL uownload and rlnL
faclllLles
Lach of Lhe Lables ln Lhe 1ear SheeL should be
dlsplayed ln Lxcel 1he enLlre 1ear SheeL
should be prlnLed ln ul formaL

verlfy LhaL cllcklng a credlL name resulLs
ln dlsplay of Lhe 1ear SheeL
verlfy LhaL a mouseover of an lndlvldual
credlL Lhen cllck resulLs ln dlsplay of Lhe 1ear
SheeL for Lhe selecLed credlL
1he 1ear SheeL should be dlsplayed wlLh each
secLlon (Summary Lable MarkeL Llneup CuS
LqulLy and CredlL roflle)

verlfy LhaL each fleld on Lhe page ls
sorLable
Lach fleld should be sorLable ln ascendlng and
desendlng order

verlfy navlgaLlon from currenL page Lo
nexL and from currenL page Lo prevlous
page
1he nexL page or prevlous page of credlLs
should be dlsplayed

verlfy LhaL Lhe number of credlLs llsLed aL
Lhe Lop lefL of Lhe page ls correcL
1he number (ex Showlng 1 20 of 330) ls
correcL use Lhe uownload funcLlon Lo creaLe
a spreadsheeL wlLh all credlLs verlfy Lhe
counL by means of Lhe number of rows

verlfy user Culde funcLlon 1he user Culde should be dlsplayed ln a
separaLe wlndow and should be prlnLable

W ?addow PeadsLrong ConsulLlng
2011
4

verlfy Lhe lllLer by 8eglon Currency
CredlL 1ype and ClC SecLor works
1he numnber and names of reglons currency
and CredlL 1ype for each selecLlon should be
reasonable

Jhat's Needed for L1L (data tab|e |oad) kegress|on 1est |ann|ng
fLer a daLa load or daLa Lable change of any klnd Lhe followlng quesLlons should be pursued ln order Lo
plan for LesLlng
1 Whlch L1L processes were changed and whaL new or changed loglc was lmplemenLed?
2 Whlch sLored procedures and maLerlallzed vlews were changed whaL were Lhe changes Lo
loglc?
3 Whlch Lables vlews and relaLed flelds were changed?
4 Whlch Lable relaLlonshlps were changed (prlmary forelgn keys naLural keys)?
3 WhaL daLa source Lo LargeL mapplngs were changed?
8egresslon LesLlng ls lmporLanL for daLa load processses wheLher accompllshed Lhrough lnformaLlca or
L/SCL

1o plan for L1L regresslon LesLs C should ldenLlfy how Lables relaLe Lo each oLher (le Lhrough
examlnaLlon of a daLa model) and use Lhe knowledge along wlLh user speclflcaLlons Lo deLermlne
exacLly whaL daLa warehouse daLa should be ldenLlcal or changed across LesL L1L runs leavlng ouL
changeprone values such as surrogaLe keys 1he meLhod provldes Lools for qulckly deLecLlng and
dlsplaylng dlfferences beLween Lhe new L1L resulLs and Lhereference resulLs from an earller daLe ln
summary manual work for LesL seLup ls reduced Lo a mlnlmum whlle sLlll ensurlng an efflclenL LesLlng
procedure

WhaL should be regresslon LesLed for L1L sofLware ls Lhe resulL of Lhe enLlre L1L run or ln oLher words
Lhe obLalned slde effecLs noL [usL lndlvldual funcLlon values 8uL even when Lhe daLa ls flxed ln Lhe lnpuL
sources for Lhe L1L some Lhlngs may change

lor example Lhe order of feLched daLa rows may vary ln Lhe relaLlonal model ddlLlonally aLLrlbuLes
obLalned from sepoeoces may have dlfferenL values ln dlfferenL runs Powever acLual values for
surrogaLe keys asslgned values from sequences are noL lnLeresLlng whereas lL lndeed ls lnLeresLlng how
rows are connecLed" wlLh respecL Lo prlmary key/forelgn key palrs lor lnsLance lL ls noL lnLeresLlng lf
an lu aLLrlbuLe has been asslgned Lhe value 1 or Lhe value 23 from a sequence WhaL ls lmporLanL ls
LhaL any oLher aLLrlbuLe LhaL ls supposed Lo reference has Lhe correcL value lurLher Lhe resulLs Lo
compare from an L1L run have a hlghly complex sLrucLure as daLa ln several Lables has Lo be compared

CpLlonally Lhe user may speclfy [olns Lables and columns Lo lnclude/lgnore ln Lhe comparlson
C dlff process can Lhen generaLe Lhe socalled tefeteoce tesolts an offllne copy of Lhe uW
conLenL query orserles of querles can be run agalnsL Lables before Lhe change has been lmplemenLed
l Lake snapshoL of Lable (ex uay week) Lhen exporL Lo LxL flle for lnpuL Lo Lxcel or MS ccess
Whenever Lhe L1L sofLware has been changed Lhe reference resulLs can be compared wlLh Lhe currenL
resulLs called Lhe test tesolts and any dlfferences wlll be polnLed ouL ln Lhe use case descrlbed above
Lhe Lunlng Leam can Lhus use Lxcel or MS ccess as laborsavlng regresslon LesLlng Lools Cuerles can be
easlly seL up ln MS ccess Lo compare earller daLa wlLh new daLa Lo deLermlne lf resulLs are as expecLed

LesLer wlll Lyplcally exerclse Lhe followlng Lasks

W ?addow PeadsLrong ConsulLlng
2011
3
1) wrlLe a SCL expresslon LhaL [olns Lhe relevanL Lables and selecLs Lhe columns Lo compare
2) verlfy LhaL Lhe query ls correcL and lncludes everyLhlng needed ln comparlsons
3) execuLe Lhe query and wrlLe Lhe resulL Lo a flle
4) wrlLe an appllcaLlon or exporL Lo Lxcel / ccess Lo compare resulLs and polnL ouL dlfferences

lurLher Lhe user can exerclse Lhe followlng Lasks for each new L1L verslon
3) 8un Lhe new L1L sofLware
6) run Lhe query from sLep 1 agaln
7) sLarL Lhe appllcaLlon from sLep 4 Lo compare Lhe resulLs from sLep 6 Lo Lhe flle from sLep 3
Lven Lhough much of Lhls could be auLomaLed lL would Lake even more work Lo seL Lhls up
1hus Lo seL up regresslon LesLlng manually Lakes hours maybe days lnsLead of mlnuLes

Compar|ng Data 8efore and After Changes

When comparlng daLa Lhere should be Lwo daLa seLs Lo conslder 1he resulLs of an L1L run (Lhe
tefeteoce tesolts) and Lhe currenL resulL (Lhe test tesolts) before and afLer MeLhods Lo deLermlne
dlfferences perform Lwo Lasks when runnlng a LesL 1) LxporL daLa from Lhe uW and 2)
compare Lhe LesL resulLs Lo Lhe reference resulLs and polnL ouL any dlfferences found

slmple buL effecLlve and efflclenL sLraLegy Lo reLesL baslc funcLlonallLy ls Lo sLore source daLa seLs and
resulLs from successful runs of Lhe code and compare new LesL resulLs wlLh prevlous runs When dolng a
regresslon LesL lL ls much qulcker Lo compare resulLs Lo a prevlous execuLlon Lhan Lo do an enLlre daLa
valldaLlon agaln