8/7/13

IBM

Manage time and resources better by scheduling automated tests

English Signin(orregister)

Technicaltopics

Evaluationsoftware

Community

Events

SearchdeveloperWorks

Managetimeandresourcesbetterbyschedulingautomatedtests
VaibhavRangare(vrangare@in.ibm.com),SystemSoftwareEngineer,IBM Summary:Schedulingatestinanautomationsystemisanimportantpartofexecutingthetaskatspecificdateandtime.Itnotonlyhelpsyoubemore efficientanduselabresourcesmoreeffectively,buthelpsyoumonitorresults.Thisarticledescribestheconsiderations,thecomponentsofascheduling subsystem,thefunctionalitythatitshouldinclude,andhowtodeveloponebyusingtheJavalanguage. Date:21Jun2011 Level:Intermediate PDF:A4andLetter(128KB|12pages)GetAdobeReader Alsoavailablein:Chinese Activity:13983views Comments:0(View|AddcommentSignin) Averagerating(3votes) Ratethisarticle Advantagesandchallengesofaddingschedulingtoautomatedtesting Atestingautomationsystemcanbeusedfortestingsoftware,systemhardware,andfirmwareacrossmultiplesoftwareplatform.Youcanuseautomation frameworksthataredesignedtoperformregressiontestingtocreatetasks,executethem,andmonitorthetestresultsagainstdifferentlabresources.These frameworksworkforbothcommandlineandgraphicaluserinterfaceexecution.Animportantpartofanautomationsystemistheprocessofschedulinga task. Thisarticlegivesyouabasicunderstandingofthearchitectureoftheautomationsystemthatisintegratedwithaschedulingmechanismtoencourageyou todesignanddevelopanautomationframeworkwithaschedulingcapability.Italsogivesyouanoverviewofhowtodeveloptheschedulingcomponent byusingJavalanguage. Advantagesofschedulingintestautomation Schedulingcanbeviewedasthesubsystemofanautomationsystem.Includingtheschedulingcapabilityintheautomationframeworkenablesyouto scheduleanytaskatdaily,weekly,ormonthlyintervalsandtoscheduleittorepeatifnecessary.Youcanalsousetheschedulertoselectanylabresource toperformthetestregression.Thisreduceshumaninteractionandexecutesautomaticallyatthefuturetimespecified.Thescheduleralsohelpsyoumanage timeandresourcesfortheregressiontestandhelpsinimprovingtestperformance,thussuccessrates. Challenges Gatheringinformationabouttheexecutionenvironment Thereisquiteoftenadifficultyingettingthelabresourcethatisinaproperstatetoperformthetaskandavailablewhenyouneedtouseitfor testing.Itisimportanttocheckinadvance. Geographicallydistributedofteams Teamsthatareusingtheautomationsystemmightbeworkingfromdifferentcountries.Theirtimezonesmightbedifferentthanthetimezoneofthe system,wheretheautomationserverisrunning.Thetimeanddateofeachandeveryschedulemustbeinsynchronizationwiththetimeofthe automationsystem. Supplyingnecessaryinformationonthefly Sometimes,itisnecessarytoprovidesomeadditionalinformation,suchasbuildandsandboxlevel,atthetimeofexecution.Theschedulershouldbe abletocopeupwiththechangesmadebyusersattheverylastmomentofexecution. Workinginconjunctionwithothercomponentsoftheautomationsystem Schedulingisthesubsystemoftheautomationsystem,soitshouldintegratewiththerestofthecomponentsoftheautomationsystemtoproducethe desiredresult.Anykindofmiscommunicationorinadequatecoordinationamongthecomponentsmightresultinaserioushardwareprobleminthe labresourceundertest. Satisfyingeachandeveryteam'sschedulingneeds Everyteamhasitsownautomationneeds.Someteamsmightneedtoexecutetasksdaily.Thechallengeistounderstandtherequirementsfromallof theteams,anddevelopaschedulingmechanismthatcopesupwithallofthesediverserequirements. Architectureoftheautomationsystem Thetestautomationsystemhastwomajorcomponents: Theuserinterface Thegraphicaluserinterface Thecommandlineinterface Thetestengine Thegraphicaluserinterface(GUI)providesaneasywayofmanagingthetests.Italsohelpsincreatingthetestregressionthatrunsagainstdifferentlab

www.ibm.com/developerworks/rational/library/manage-time-schedule-automated-tests/

1/7

8/7/13

Manage time and resources better by scheduling automated tests

resourcestotesttheimagesinstalledonthem. Thecommandlineinterfaceisanalternativewayofexecutingtests.Inthis,theusershavetodoalittlemoreworktocreateandmanagetheirtestsandtest regression,becausecreationandmanagementofthetestsandtestregressionrequiresfrequentuserinteractions. ThetestengineacceptsthetestsandtestregressionsformtheGUIorcommandlineinterface,orboth,andrunsthemagainstthelabresources. ThecomponentsofanautomationsystemareshowninFigure1.Itincludestheuserinterface,thetestengineandthesystemundertest.Thesubsections thatfollowdescribethefunctionalityofthesecomponents Figure1.Architectureofanautomationsystem

Theuserinterface ThisinterfacecanbeawebbasedGUIorcommandline.Awebbasedinterfacecanbedevelopedusinganywebtechnology.Acommandlineinterface canbeusefulforapplicationsthatdon'tusebrowsers.Asapartoftestautomationsystem,theuserinterfaceplaysaveryimportantrole.Itistheplaceto startwithtestautomation.Differentuserinterfacescomeswithdifferentfeatures.Someofthemincludemaintainingtheuserworkareasinaformatfor teams.Thesearethemaintasksofauserinterface: 1. Provideauserfriendlyinterface 2. Provideaneasywayofcreatingtestsandtestregressions 3. Providetestexecutionenvironmentsfortests 4. Showthetestresults 5. Assistinmanagementofresources Testengine Thetestengineistheheartofthetestautomationsystem,becauseitrunsthetestsagainstdifferentlabresources.Itisatypeofsoftwarethatisusedtotest software,hardware,oracompletesystem.Selectionofasuitabletestengineisveryimportant.Itdependsonthetypeofteststhatyouwanttoautomate. Differenttestenginesprovidedifferentfunctionality.Forexample,IBMRationalBuildForgeandtheopensourceSTAXengineprovidethefeatures ofparallelandsequentialexecutionoftests.Sometimes,thetestautomationframeworkcontainsthetestengine,whichsupportstheautomationthatthe frameworkhandles.Thesearemaintasksofatestengine: 1. Execute(run)testsortestregressionsontheapplicationsorimages 2. Repeatthetest 3. Collectthetestresults 4. Analyzethetestresults 5. Monitortheexecution 6. Sendemailnotificationstothetestersaboutanyfailures Automationprocess Microcodedevelopmentformultipleplatformsrequirescollectingsourcecodefromthedevelopersandbundlingthecodeintoaunit.Afterthebundleis installedonthelabresource,itrequiresmultipleteststovalidateitscorrectness.Automationofsuchtestsreducesthetimethatittakestesterstoperform theirtests,sotheycanconcentratemoreonthemicrocodedevelopment. Obviously,automationneedstoreducethetimeandeffortrequiredformanualinteractionwithtests.Testingisdoneeitherbyrunningscriptsorrunning commands.Thescriptscanbewritteninanylanguage.Thetestengineshouldbecapableofinterceptingthesescriptcallsandperformingthetest. Selectionoftheframework,alongwiththetestengine,isanimportantpartoftheautomationprocess.Therearevariousframeworksavailablethatcanbe usedinthetestautomationsystem,suchastheopensourceSoftwareTestingAutomationFramework(STAF),oryoucancreateyourown. Manyorganizationsareworkingonautomationbycreatingtheirownautomationframeworkandtestengine.Afteryouhavethesetwocomponentsofan automationsystem,youcanthinkaboutusingtheminanautomationprocesstoperformthesetasks: 1. Createandmanagetestsandtestregression,usingtheuserinterface 2. Submittheteststorunontheapplicationorimage 3. Executethetestsontheapplicationorimage(testengine) 4. Notifythetesterforanyattentionrequired 5. Generateandanalyzethetestresult(testengine)

www.ibm.com/developerworks/rational/library/manage-time-schedule-automated-tests/

2/7

8/7/13

Manage time and resources better by scheduling automated tests

6. Sendemailtothetesterifthetestfails Automationsystemwithscheduling TheautomationsystemwiththeschedulingcapabilityisshowninFigure2. Figure2.Automationsystemwithscheduler

Itincludesonemorecomponentcalledthescheduler.Thepurposeofthescheduleristoidentifythetypeofthetaskthatyouwanttoexecute.Ifitisofthe scheduletype,theschedulerwillinstructthetestenginetoscheduleitotherwise,itwillgiveitdirectlytothetestengineforexecution.Youcanschedule theexecutionthroughtheuserinterface. Automationframeworkbuildsontheschedulingcapability Theautomationframework,whichincludestheschedulingfeature,mightrequirerunningtheautomationserveronasystem.Usersfromdifferentteams mightbeatdifferentgeographiclocations,whichcanresultinwrongschedulingtimes.Theschedulershouldbeabletorecognizethetimezonedifference andschedulethetasksaccordingtouserrequests. TheautomationframeworkcanhelpyoucreateschedulesbasedontheoptionsgiveninFigure3.Youcanalsospecifythedateandtimeoftheschedule andrepeatthescheduleifnecessary.ThescheduledescribedinFigure3schedulesthetaskdailyandcontinuesitforthreedays.Italsoshowsthedatesthat itwillrunthetest.Theuserinterfacecanalsoprovideoptionstoselectthesystemundertestandtasksduringthepreparationofschedule(notshowninthe screencapture). Figure3.AutomationframeworkwiththeschedulerUI

ImplementingtheschedulingfeatureusingJavatechnology Javaprovidesthetimerframework,whichmakesiteasyfortheprogrammertoscheduleatask.Thisincludesjava.util.Timerandjava.util.TimerTask classes.TimerandTimerTaskworktogetherandprovidetheschedulingcapability.Timeristheclassthatyouusetoscheduleatask.Taskshouldbean instanceofTimerTask.Toscheduleatask,youmustcreateTimerTask.TheTimerTaskimplementsarunnableinterfacethatcanbeusedtocreateathread ofexecution.Afterthetaskiscreated,youcanscheduleitbyusingTimerinstance.Thetimerusesthefollowingmethodstohelpwithscheduling: voidschedule(TimerTaskTTask,longwait) voidschedule(TimerTaskTTask,longwait,longrepeat) voidschedule(TimerTaskTTask,DatetargetTime) voidschedule(TimerTaskTTask,DatetargetTime,longrepeat) Testenginegeneratesthetestresultandanalyzeit voidscheduleAtFixedRate(TimerTaskTTask,longwait,longrepeat) voidscheduleAtFixedRate(TimerTaskTTask,DatetargetTime,longrepeat) Themethodclearlytellshowtoscheduleatask.Toprovideevenmoreflexibility,youcanusertheobserverpatterntoprovideawrapperaroundtheoneof theschedulingmechanisms.Usingthisapproach,italsoprovidesinterthreadcommunication,whichinformstheschedulerbynotifyingitwheneverany newschedulecomesintoexistence.ThisapproachisbasedonJavatechnology.Youcancomeupwithyourownimplementationofscheduling mechanism,whicheverisbestsuitedforyourautomationframework. Figure4describestheinteractionamongtheschedulingcomponents.Wheneversomeoneaddsaschedule,theschedulerautomaticallynotifiestheexecutor

www.ibm.com/developerworks/rational/library/manage-time-schedule-automated-tests/

3/7

8/7/13

Manage time and resources better by scheduling automated tests

ortestlaunchercomponenttoexecutethetest.Oncethescheduleisgiven,ithastoberemovedfromthescheduledlistandallowthenextscheduledtestto runatthespecifiedtime. Figure4.Interactionamongschedulingcomponents

Cancelinganddeletingatask Note: YoucanusetheTimerclassinJavatocancelorevendeletethescheduledtaskwitheitherofthesemethods: voidcancel()//cancelthetimerthread intpurge()//deletethecancelledtaskfromtimersqueue Frameworkofanautomationsystemwithschedulingcapability Thefollowingframeworkdescriptionisoneofthesolutionsthatcanhelpyouunderstandtheworkingofautomationframework.Aspreviouslymentioned, theautomationsystemincludestheuserinterface,thetestengine,theandsystemundertest.Theautomationframeworkcontainsthefollowingbasic components: Theuserinterface,whichcanbewebbased Thedatabasetostorethetasksthatyouwanttoexecute Aprocessthatcontinuouslychecksthedatabasetogetthetask Aschedulerthattakesthetaskfromtheprocessmentionabove Atestenginethatexecutionthetaskprovidedbythescheduler Figure5describesthecompleteinteractionamongthecomponentsofautomationsystem: 1. Theuseraddsthetasktobeexecuted,withthescheduleddateandtimeandthesystemundertest. 2. Thisinformationgetsstoredinthedatabase. 3. Thedaemonrunstheprocesscontinuouslytogetsthejobfromthedatabase,handitovertothescheduler,andthenaddinformationthatincludesthe testofsystemhealth. 4. Theschedulerthenforwardsthejobtothetestexecutionengine,whichinteractswiththesystemundertest. Figure5.Completeinteractionamongthecomponentsofautomationsystem

Theotherresponsibilityofthewebframeworkistogettheresultofthetestandanalyzeit.Theschedulerhelpsyoumanagethefollowingaspectsofan automationsystem: Testregression,whichcontainsthetasksthatyouwanttoexecuteagainstanysystemundertest Systemundertest Timeanddateofexecutionofthetestregressionandrepeats,ifrequired Performancemeasurementasanaddedbenefit

www.ibm.com/developerworks/rational/library/manage-time-schedule-automated-tests/

4/7

8/7/13

Manage time and resources better by scheduling automated tests

Theautomationhelpsyounotonlybemoreefficientwiththetimethatyouspendonrepetitivetesting,butalsohelpsyouanalyzetheresultsoftests.Italso helpsyoufocusmoreonthemicrocodedevelopment,insteadofspendingmostofthetimeintesting.Oncethedeveloperdonewiththemicrocode development,itistheoneclickprocesstoperformtheoveralltestingofthebuildbymanagingandschedulingtheregressiontest.Therefore,itreducesthe humaninteractionandhumanerrorsduringthemanualtesting. Theschedulingcapability,ontheotherhand,reducesthesubmissionofjobsatrepeatedintervals.ThegraphinFigure6showstheperformance improvementoftheoveralltestingofthenewestbuildonasystemundertest.Thebarsrepresentthetestingdoneonadaily,weekly,ormonthlybasis, whichispossibleonlywiththehelpofschedulingcapabilityofthetestautomationframework. Figure6.Performancemeasurementbargraphs

TodigdeeperintothepossibilitiesandwhatRationalsoftwareoffersfortestingandforautomation,seetheResourcessection.

Resources Learn Digdeeper: SchedulingrecurringtasksinJavaapplications:IntroducingasimplegeneralisationoftheJavalanguage'sTimerclassbyTomWhite (developerWorks,November2003) ChoosingatestautomationframeworkbyMichaelKelly(developerWorks,Nov2003) Automationforthepeople:ReceiveimmediatefeedbackwitheverysourcecodechangebyMichaelKelly(developerWorks,Nov2006) AdvancingtowardtestautomationthrougheffectivemanualtestingbyBobLevyandDennisElenburg(developerWorks,Jun2005) ChoosingatestautomationframeworkbyMichaelKelly(PDF,undated) SevenStepstoTestAutomationSuccessbyBretPettichord(June2001) EffectiveSoftwareTestAutomation:DevelopinganAutomatedSoftwareTestingToolbyKanglinLiandMengqiWu(Sybex,2004) SoftwareTestAutomation:EffectiveUseofTestExecutionToolsbyMarkFewsterandDorothyGraham(AddisonWesleyProfessional, 1999) CheckintoRationaltestingandautomationsoftware(partiallisting): RationalQualityManager,awebbasedcentralizedtestmanagementenvironmentthatprovidesacollaborativeandcustomizablesolutionfor testplanning,workflowcontrol,trackingandmetricsreporting. AllRationalqualitymanagementsolutions,aportalpage. RationalBuildForge,whichprovidesbuildandreleasemanagementcapabilitiestoaccelerateandstreamlinesoftwareandsystemsdelivery. VisittheRationalsoftwareareaondeveloperWorksfortechnicalresourcesandbestpracticesforRationalSoftwareDeliveryPlatformproducts. StaycurrentwithdeveloperWorkstechnicaleventsandwebcastsfocusedonavarietyofIBMproductsandITindustrytopics. AttendafreedeveloperWorksLive!briefingtogetuptospeedquicklyonIBMproductsandtools,aswellasITindustrytrends. WatchdeveloperWorksondemanddemos,rangingfromproductinstallationandsetupdemosforbeginnerstoadvancedfunctionalityfor experienceddevelopers. Improveyourskills.ChecktheRationaltrainingandcertificationcatalog,whichincludesmanytypesofcoursesonawiderangeoftopics.Youcan takesomeofthemanywhere,anytime,andmanyofthe"GettingStarted"onesarefree. Getproductsandtechnologies ChecktheTrialsandDemospageforRationalsoftware. EvaluateIBMsoftwareinthewaythatsuitsyoubest:Downloaditforatrial,tryitonline,useitinacloudenvironment,orspendafewhoursinthe SOASandboxlearninghowtoimplementserviceorientedarchitectureefficiently. Discuss Participateinthediscussionforum. JointheRationalsoftwareforumstoaskquestionsandparticipateindiscussions.

www.ibm.com/developerworks/rational/library/manage-time-schedule-automated-tests/

5/7

8/7/13

Manage time and resources better by scheduling automated tests

ShareyourknowledgeandhelpotherswhouseRationalsoftwarebywritingadeveloperWorksarticle.You'llgetworldwideexposure,RSS syndication,abylineandabio,andthebenefitofprofessionaleditingandproductiononthedeveloperWorksRationalwebsite.Findoutwhatmakes agooddeveloperWorksarticleandhowtoproceed. FollowRationalsoftwareonFacebook,Twitter(@ibmrational),andYouTube,andaddyourcommentsandrequests. AskandanswerquestionsandincreaseyourexpertisewhenyougetinvolvedintheRationalforums,cafs,andwikis. ConnectwithotherswhoshareyourinterestsbyjoiningthedeveloperWorkscommunityandrespondingtothedeveloperdrivenblogs. Abouttheauthor

VaibhavRangareisanintegration,test,anddeploymentengineerintheIBMSoftwareLabs,inIndia.Heiscurrentlyworkingonanautomationprojectas partoftheTestToolsDevelopmentteam.HereceivedhisMTechinComputerScienceandInformationTechnologyfromIITKharagpurIndiain2003. Close[x]

developerWorks:Signin
IBMID: NeedanIBMID? ForgotyourIBMID? Password: Forgotyourpassword? Changeyourpassword Keepmesignedin. ByclickingSubmit,youagreetothedeveloperWorkstermsofuse.
Submit

Cancel

ThefirsttimeyousignintodeveloperWorks,aprofileiscreatedforyou.Selectinformationinyourprofile(name,country/region,andcompany)is displayedtothepublicandwillaccompanyanycontentyoupost.YoumayupdateyourIBMaccountatanytime. Allinformationsubmittedissecure. Close[x]

Chooseyourdisplayname
ThefirsttimeyousignintodeveloperWorks,aprofileiscreatedforyou,soyouneedtochooseadisplayname.Yourdisplaynameaccompaniesthe contentyoupostondeveloperWorks. Pleasechooseadisplaynamebetween331characters.YourdisplaynamemustbeuniqueinthedeveloperWorkscommunityandshouldnotbeyour emailaddressforprivacyreasons. Displayname: (Mustbebetween331characters.)

ByclickingSubmit,youagreetothedeveloperWorkstermsofuse.
Submit

Cancel

Allinformationsubmittedissecure. Averagerating(3votes) 1star 2stars 3stars 4stars 1star 2stars 3stars 4stars

www.ibm.com/developerworks/rational/library/manage-time-schedule-automated-tests/

6/7

8/7/13

Manage time and resources better by scheduling automated tests

5stars
Submit

5stars

Addcomment: Signinorregistertoleaveacomment. Note:HTMLelementsarenotsupportedwithincomments.

Notifymewhenacommentisadded1000charactersleft

Post

Bethefirsttoaddacomment
Printthispage Sharethispage FollowdeveloperWorks

About Help Contactus Submitcontent

Feeds

Reportabuse Termsofuse IBMprivacy IBMaccessibility

Faculty Students BusinessPartners

www.ibm.com/developerworks/rational/library/manage-time-schedule-automated-tests/

7/7