Date: <revision date> Index Index Revision History Introduction Goal of Project and eature !eam Primary !estin" #oncerns Primary !estin" ocus References Personnel !estin" $chedule eature History eatures: iles and %odules: iles &ist: Re"istry' INI $ettin"s: $etu( Procedures: De)installation Procedures Data*ase $etu( and Procedures Net+or, Domain-!o(olo"ies #onfi"uration Procedures Performance %onitorin" #ounters $etu( And #onfi"urations .(erational Issues /ac,u( Recovery Archivin" %onitorin" .(erational Pro*lem 0scalation-Alert %ethods $co(e of !est #ases Acce(tance #riteria 1ey eature Issues !est A((roach Desi"n 2alidation Data 2alidation API !estin" #ontent !estin" &o+)Resource !estin" $etu( !estin" %odes and Runtime .(tions Intero(era*ility Inte"ration !estin" #om(ati*ility: #lients #om(ati*ility: $ervers /eta !estin" 0nvironment-$ystem ) General #onfi"uration 3ser Interface Performance 4 #a(acity !estin" $cala*ility $tress !estin" 2olume !estin" International Issues Ro*ustness 0rror !estin" 3sa*ility Accessi*ility 3ser $cenarios /oundaries and &imits $(ecial #ode Profilin" and .ther %etrics !est 0nvironment .(eratin" $ystems Net+or,s Hard+are Template page2 of 14 %achines5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 Gra(hics Ada(ters55555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 0xtended and 0x(anded %emory /oards5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 .ther Peri(heral55555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 $oft+are 3ni6ue !estin" #oncerns or $(ecific eatures Area /rea,do+n eature Name $u* eature .ne5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 su* 7575555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 su* 7585555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 su* 7595555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 $u* eature !+o5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 $u* eature !hree :etc5;55555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 $(ec Revie+ Issues !est !ools $mo,e !est :acce(tance test' *uild verification' etc5; Automated !ests %anual !ests Re"ression !ests /u" /ashes /u" Re(ortin" Plan #ontin"encies 0xternal De(endencies Headcount Re6uirements Product $u((ort !estin" $chedule Dro( Procedures Release Procedures Alias-Ne+s"rou(s and #ommunication #hannels Re"ular %eetin"s eature !eam %eetin"s Project !est !eam %eetin"s eature !eam !est %eetin"s Decisions %a,in" Procedures Notes 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page 3 of 14 Revision History irst Draft: <author> <date> <*rief descri(tion of chan"es> Introduction $in"le sentence descri*in" the intent and (ur(ose of the test (lan5 or exam(le B!his test (lan addresses the test covera"e for the CCC release of the /AR area of feature ooD5 Goal of Project and eature Team %ission statement and "oal of overall (roject team5 %ission statement and "oal of s(ecific feature team5 !his section is used to set the sta"e for testin"Es (lans and "oals in relation to the feature team and (rojectEs "oals5 Primary Testing !oncerns A statement of +hat the main critical concerns of the test (lan are5 An itemiFed list' or short (ara"ra(h +ill suffice5 Primary Testing ocus A short statement of +hat items testin" +ill focus on5 !he testin" concerns a*ove state +hat testin" is +orried a*out5 ocus indicates more of a methodolo"y ) a statement of ho+ those concerns +ill *e addressed via focus5 References document name location test (lan test (lan location (roject s(ecifications (roject s(ec location feature s(ecification feature s(ec location develo(ment docs on feature dev doc location *u" data*ase 6ueries location for raid 6ueries test case data*ase 6ueries location for test case 6ueries schedule documents location for schedule documents *uild release server location of *uild releases source file tree location of source file tree other related documents other locations Personnel Pro"ram %ana"er: name and email Develo(er: name and email !ester: name and email Testing "c#edule /rea, the testin" do+n into (hases :ex5 Plannin"' #ase Desi"n' 3nit 4 #om(onent !ests' Inte"ration !ests' $ta*iliFation' Performance and #a(acity !unin"' ull Pass and $hi((in"; ) and ma,e a rou"h schedule of se6uence and dates5 Ghat tas,s do you (lan on havin" done in +hat (hasesH !his is a *rief' hi"h level 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page 4 of 14 summary ) just to set ex(ectation that certain com(onents +ill *e +or,ed on at certain times ) and to indicate that the (lan is ta,in" (roject schedule concerns into consideration5 Include a (ointer to more detailed feature and team schedules here5 eature History A history of ho+ the feature +as desi"ned' and evolved' over time5 It is a "ood idea to *uild this history u( as test (lans "o5 !his "ives a "ood feel for +hy the current release is focusin" on +hat it has done5 It also serves a "ood frame+or, for +here (ro*lems have *een in the (ast5 A (ara"ra(h or t+o is (ro*a*ly sufficient for each dro(' indicatin" ) ori"inal intent' feed*ac, and successes' (ro*lems' resolutions' thin"s learned from the release' major issues dealt +ith or discovered in the release5 /asically' this section is a mini (ost)mortem5 It is eventually finishes +ith a statement re"ardin" the develo(ment of the s(ecific version5 It is often hel(ful to u(date this history at each milestone of a (roject5 eatures$ !his section "ives a *rea,do+n of the areas of the feature5 It is often useful to include in this section a (er area statement of testin"Es thou"hts5 Ghat ty(e of testin" is *est used for each areaH Ghat is (ro*lematic a*out each areaH Has this area had a (ro*lem in the (ast5 Iuic, statements are all that is need in this list5 N.!0: this is only here as a hi"h level summary of the features5 !he real meat is in the area *rea,do+n5 !his is a tad redundant in that res(ect555 iles and %odules$ Include in this section any files' modules and code that must *e distri*uted on the machine' and +here they +ould *e located5 Also include re"istry settin"s' INI settin"s' setu( (rocedures' de)installation (rocedures' s(ecial data*ase and utility setu(s' and any other relevant data5 1. Files List: filename (ur(ose location on machine 2. Registry, INI Settings: settin"7 (ur(ose $ettin"7 (ossi*le values settin" 8 (ur(ose $ettin" 8 (ossi*le values 3. Setup Procedures: 75 *lah 85 *lah 4. De-instlltion Procedures 75 *lah 85 *lah !. Dt"se Setup nd Procedures 75 *lah 85 *lah 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page & of 14 #. Net$or% Do&in'(opologies )on*igurtion Procedures 75 *lah 85 *lah +. Per*or&nce ,onitoring )ounters Setup -nd )on*igurtions 'perational Issues Is the (ro"ram *ein" monitored-maintained *y an o(erational staffH Are there s(ecial (ro*lem escalation' or o(erational (rocedures for dealin" +ith the feature-(ro"ram-areaH .. /c%up 0. Reco1ery 12. -rc3i1ing 11. ,onitoring 12. 4pertionl Pro"le& 5scltion'-lert ,et3ods "cope of Test !ases $tatement re"ardin" the de"ree and ty(es of covera"e the testin" +ill involve5 or exam(le' +ill focus *e (laced on (erformanceH Ho+ a*out client v5s5 server issuesH Is there a lar"e class of testin" covera"e that +ill *e intentionally overloo,ed or minimiFedH Gill there *e much unit and com(onent testin"H !his is a *i" s+ee(in" (icture of the testin" covera"e ) "ivin" an overall statement of the testin" sco(e5 (cceptance !riteria Ho+ is BGood 0nou"h !o $hi(D defined for the (rojectH or the featureH Ghat are the necessary (erformance' sta*ility and *u" find-fix rates to determine that the (roduct is ready to shi(H )ey eature Issues Ghat are the to( (ro*lems-issues that are recurrin" or remain o(en in this test (lanH Ghat (ro*lems remain unresolvedH Test (pproac# 13. Design 6lidtion $tatements re"ardin" covera"e of the feature desi"n ) includin" *oth s(ecification and develo(ment documents5 Gill testin" revie+ desi"nH Is desi"n an issue on this releaseH Ho+ much concern does testin" have re"ardin" desi"n' etc5 etc55 14. Dt 6lidtion Ghat ty(es of data +ill re6uire validationH Ghat (arts of the feature +ill use +hat ty(es of dataH Ghat are the data ty(es that test cases +ill addressH 0tc5 1!. -PI (esting 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page * of 14 Ghat level of API testin" +ill *e (erformedH Ghat is justification for ta,in" this a((roach :only if none is *ein" ta,en;H 1#. )ontent (esting Is your area-feature-(roduct content *asedH Ghat is the nature of the contentH Ghat strate"ies +ill *e em(loyed in your feature-area to address content related issuesH 1+. Lo$-Resource (esting Ghat resources does your feature useH Ghich are used most' and are most li,ely to cause (ro*lemsH Ghat tools-methods +ill *e used in testin" to cover lo+ resource :memory' dis,' etc5; issuesH 1.. Setup (esting Ho+ is your feature affected *y setu(H Ghat are the necessary re6uirements for a successful setu( of your featureH Ghat is the testin" a((roach that +ill *e em(loyed to confirm valid setu( of the featureH 10. ,odes nd Runti&e 4ptions Ghat are the different run time modes the (ro"ram can *e inH Are there vie+s that can *e turned off and onH #ontrols that to""le visi*ility statesH Are there o(tions a user can set +hich +ill affect the run of the (ro"ramH &ist here the different run time states and o(tions the (ro"ram has availa*le5 It may *e +orth+hile to indicate here +hich ones demonstrate a need for more testin" focus5 22. Interoper"ility Ho+ +ill this (roduct interact +ith other (roductsH Ghat level of ,no+led"e does it need to have a*out other (ro"rams )) B"ood nei"h*orD' (ro"ram co"niFant' (ro"ram interaction' fundamental system chan"esH Ghat methods +ill *e used to verify these ca(a*ilitiesH 21. Integrtion (esting Go throu"h each area in the (roduct and determine ho+ it mi"ht interact +ith other as(ects of the (roject5 $tart +ith the ones that are o*viously connected' *ut try every area to some de"ree5 !here may *e su*tle connections you do not thin, a*out until you start usin" the features to"ether5 !he test cases created +ith this a((roach may du(licate the modes and o*jects a((roaches' *ut there are some areas +hich do not fit in those cate"ories and mi"ht *e missed if you do not chec, each area5 22. )o&pti"ility: )lients Is your feature a server *ased com(onent that interacts +ith clientsH Is there a standard (rotocol that many clients are ex(ected to useH Ho+ many and +hich clients are ex(ected to use your featureH Ho+ +ill you a((roach testin" client com(ati*ilityH Is your server suited to handle ill)*ehaved clientsH Are there su*tleties in the inter(retation of standard (rotocols that mi"ht cause incom(ati*ilitiesH Are there non)standard' *ut +idely (racticed use of your (rotocols that mi"ht cause incom(ati*ilitiesH 23. )o&pti"ility: Ser1ers Is your feature a client *ased com(onent that interacts +ith serversH Is there a standard (rotocol su((orted *y many servers that your client s(ea,sH Ho+ many different servers +ill your client (ro"ram need to su((ortH Ho+ +ill you a((roach testin" server com(ati*ilityH Is your client suited to handle ill)*ehaved or non)standard serversH Are there su*tleties in the inter(retation of standard (rotocols that mi"ht cause incom(ati*ilitiesH Are there non)standard' *ut +idely (racticed use of (rotocols that mi"ht cause incom(ati*ilitiesH 24. /et (esting Ghat is the *eta scheduleH Ghat is the distri*ution scale of the *etaH Ghat is the entry criteria for *etaH Ho+ is testin" (lannin" on utiliFin" the *eta for feed*ac, on this 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page + of 14 featureH Ghat (ro*lems do you antici(ate discoverin" in the *etaH Gho is coordinatin" the *eta' and ho+H 2!. 5n1iron&ent'Syste& - 7enerl Are there issues re"ardin" the environment' system' or (latform that should "et s(ecial attention in the test (lanH Ghat are the run time modes and o(tions in the environment that may cause difference in the featureH &ist the com(onents of critical concern here5 Are there (latform or system s(ecific com(liance issues that must *e maintainedH 2#. )on*igurtion Are there confi"uration issues re"ardin" hard+are and soft+are in the environment that may "et s(ecial attention in the test (lanH $ome of the classical issues are machine and *ios ty(es' (rinters' modems' video cards and drivers' s(ecial or (o(ular !$REs' memory mana"ers' net+or,s' etc5 &ist those ty(es of confi"urations that +ill need s(ecial attention5 2+. 8ser Inter*ce &ist the items in the feature that ex(licitly re6uire a user interface5 Is the user interface desi"ned such that a user +ill *e a*le to use the feature satisfactorallyH Ghich (art of the user interface is most li,ely to have *u"sH Ho+ +ill the interface testin" *e a((roachedH 2.. Per*or&nce 9 )pcity (esting Ho+ fast and ho+ much can the feature doH Does it do enou"h fast enou"hH Ghat testin" methodolo"y +ill *e used to determine this informationH Ghat criterion +ill *e used to indicate acce(ta*le (erformanceH If modifications of an existin" (roduct' +hat are the current metricsH Ghat are the ex(ected major *ottlenec,s and (erformance (ro*lem areas on this featureH 20. Scl"ility Is the a*ility to scale and ex(and this feature a major re6uirementH Ghat (arts of the feature are most li,ely to have scala*ility (ro*lemsH Ghat a((roach +ill testin" use to define the scala*ility issues in the featureH 32. Stress (esting Ho+ does the feature do +hen (ushed *eyond its (erformance and ca(acity limitsH Ho+ is its recoveryH Ghat is its *rea,(ointH Ghat is the user ex(erience +hen this occursH Ghat is the ex(ected *ehavior +hen the client reaches stress levelsH Ghat testin" methodolo"y +ill *e used to determine this informationH Ghat area is ex(ected to have the most stress related (ro*lemsH 31. 6olu&e (esting 2olume testin" differs from (erformance and stress testin" in so much as it focuses on doin" volumes of +or, in realistic environments' durations' and confi"urations5 Run the soft+are as ex(ected user +ill ) +ith certain other com(onents runnin"' or for so many hours' or +ith data sets of a certain siFe' or +ith certain ex(ected num*er of re(etitions5 32. Interntionl Issues #onfirm localiFed functionality' that strin"s are localiFed and that code (a"es are ma((ed (ro(erly5 Assure (ro"ram +or,s (ro(erly on localiFed *uilds' and that international settin"s in the (ro"ram and environment do not *rea, functionality5 Ho+ is localiFation and internationaliFation *ein" done on this (rojectH &ist those (arts of the feature that are most li,ely to *e affected *y localiFation5 $tate methodolo"y used to verify International sufficiency and localiFation5 33. Ro"ustness Ho+ sta*le is the code *aseH Does it *rea, easilyH Are there memory lea,sH Are there (ortions of code (rone to crash' save failure' or data corru(tionH Ho+ "ood is the (ro"ramEs recovery +hen these (ro*lems occurH Ho+ is the user affected +hen the 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page , of 14 (ro"ram *ehaves incorrectlyH Ghat is the testin" a((roach to find these (ro*lem areasH Ghat is the overall ro*ustness "oal and criteriaH 34. 5rror (esting Ho+ does the (ro"ram handle error conditionsH &ist the (ossi*le error conditions5 Ghat testin" methodolo"y +ill *e used to evo,e and determine (ro(er *ehavior for error conditionsH Ghat feed*ac, mechanism is *ein" "iven to the user' and is it sufficientH Ghat criteria +ill *e used to define sufficient error recoveryH 3!. 8s"ility Ghat are the major usa*ility issues on the featureH Ghat is testin"Es a((roach to discover more (ro*lemsH Ghat sorts of usa*ility tests and studies have *een (erformed' or +ill *e (erformedH Ghat is the usa*ility "oal and criteria for this featureH 3#. -ccessi"ility Is the feature desi"ned in com(liance +ith accessi*ility "uidelinesH #ould a user +ith s(ecial accessi*ility re6uirements still *e a*le to utiliFe this featureH Ghat is the criteria for acce(tance on accessi*ility issues on this featureH Ghat is the testin" a((roach to discover (ro*lems and issuesH Are there (articular (arts of the feature that are more (ro*lematic than othersH 3+. 8ser Scenrios Ghat real +orld user activities are you "oin" to try to mimicH Ghat classes of users :i5e5 secretaries' artist' +riters' animators' construction +or,er' airline (ilot' shoema,er' etc5; are ex(ected to use this (ro"ram' and doin" +hich activitiesH Ho+ +ill you attem(t to mimic these ,ey scenariosH Are there s(ecial niche mar,ets that your (roduct is aimed at :intentionally or unintentionally; +here mimic real user scenarios is criticalH 3.. /oundries nd Li&its Are there (articular *oundaries and limits inherent in the feature or area that deserve s(ecial mention hereH Ghat is the testin" methodolo"y to discover (ro*lems handlin" these *oundaries and limitsH 30. 4pertionl Issues If your (ro"ram is *ein" de(loyed in a data center' or as (art of a customerJs o(erational facility' then testin" must' in the very least' mimic the user scenario of (erformin" *asic o(erational tas,s +ith the soft+are5 /ac,u( Identify all files re(resentin" data and machine state' and indicate ho+ those +ill *e *ac,ed u(5 If it is im(erative that service remain runnin"' determine +hether or not it is (ossi*le to *ac,u( the data and still ,ee( services or code runnin"5 Recovery If the (ro"ram "oes do+n' or must *e shut do+n' are there ste(s and (rocedures that +ill restore (ro"ram state and "et the (ro"ram or service o(erational a"ainH Are there holes in this (rocess that may ma,e a service or state deficientH Are there holes that could (rovide loss of data5 %imic as many states of loss of services that are li,ely to ha((en' and "o throu"h the (rocess of successfully restorin" service5 Archivin" Archival is different from *ac,u(5 /ac,u( is +hen data is saved in order to restore service or (ro"ram state5 Archive is +hen data is saved for retrieval later5 %ost archival and *ac,u( systems (i""y)*ac, on each otherJs (rocesses5 Is archival of data "oin" to *e considered a crucial o(erational issue on your featureH If so' is it (ossi*le to archive the data +ithout ta,in" the service do+nH Is the data' once archived' readily accessi*leH 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page - of 14 %onitorin" Does the service have ade6uate monitorin" messa"es to indicate status' (erformance' or error conditionsH Ghen somethin" "oes +ron"' are messa"es sufficient for o(erational staff to ,no+ +hat to do to restore (ro(er functionalityH Are the Khear*eatK counters that indicate +hether or not the (ro"ram or service is +or,in"H Attem(t to mimic the scenario of an o(erational staff tryin" to ,ee( a service u( and runnin"5 3("rade Does the customer li,ely have a (revious version of your soft+are' or some other soft+areH Gill they *e (erformin" an u("radeH #an the u("rade ta,e (lace +ithout interru(tin" serviceH Gill anythin" *e lost :functionality' state' data; in the u("radeH Does it ta,e unreasona*ly lon" to u("rade the serviceH %i"ration Is there data' scri(t' code or other artifacts from (revious versions that +ill need to *e mi"rated to a ne+ versionH !estin" should create an exam(le of installation +ith an old version' and mi"rate that exam(le to the ne+ version' movin" all data and scri(ts into the ne+ format5 &ist here all data files' formats' or code that +ould *e affected *y mi"ration' the solution for mi"ration' and ho+ testin" +ill a((roach each5 42. Specil )ode Pro*iling nd 4t3er ,etrics Ho+ much focus +ill *e (laced on code covera"eH Ghat tools and methods +ill *e used to measure the de"ree to +hich testin" covera"e is sufficiently addressin" all of the codeH Test .nvironment What are the requirements for the product? They should be refected in the breadth of hardware confguration testing. 41. 4perting Syste&s Identify all operating systems under which this product will run. Include version numbers if applicable. 42. Net$or%s Identify all networks under which this product will run. include version numbers if applicable. 43. :rd$re Identify the various hardware platforms and confgurations. %achines Gra(hics Ada(ters This includes the requirements for single or dual monitors. 0xtended and 0x(anded %emory /oards .ther Peri(heral eripherals include those necessary for testing such as !"#$%&' printers' modems' fa(es' e(ternal hard drive' tape readers' etc. 44. So*t$re 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page 1/ of 14 Identify software included with the product or likely to be used in con)unction with this product. *oftware categories would include memory managers' e(tenders' some T*$s' related tools or products' or similar category products. 0ni1ue Testing !oncerns or "pecific eatures &ist s(ecific features +hich may re6uire more attention than others' and descri*e ho+ testin" +ill a((roach these features5 !his is to serve as a sort of Bhot listD5 (rea 2rea3do4n !his is a detailed *rea,do+n of the feature or area ) and is *est done in an outline format5 It is useful as a tool later +hen *uildin" test cases5 !he outline of an area can "o on 6uite lon"5 3sually it starts +ith a menu *rea,do+n' and then continues on +ith those features and functionalities not found on any menu in (articular5 4!. Feture N&e $u* eature .ne su5 161 eature testin" a((roach matrix: this +ill re(eat for each su*item' includin" any class of testin" relevant to any item5 Put in NA if not a((lica*le5 &ocation of this matrix in the hierarchy determines sco(e5 or exam(le' data validation rules "lo*al to anythin" under $u* eature .ne should "o under B$u* eature .neD5 Inheritance should *e im(lied5 #lass Info AutoH %an5H Desi"n 2alidation Data 2alidation 2alid data 4 ex(ected results :e5"5 Bal(hanumericD; Invalid data 4 ex(ected results :e5"5 Bno LME' L-E or LNE; Ho+ to validateH API !estin" Ghat are the APIEs ex(osedH Ghat are the (ermutations of callin" these APIEs :order' s(ecific ar"s' etc5;H #ontent !estin" Ghat content exercises this featureH Ghat content does this feature (roduce' modify or mana"eH &o+)Resource !estin" Ghat resource dimensions to testH Ghat to do +hen resource is lo+H $etu( !estin" Ghat ty(es of setu(sH Ho+ to confirm feature after a setu(H %odes 4 Runtime .(tions Ghat modes and runtime o(tions does this haveH Ghat should *e tested durin" these modesH Ghat are ex(ected results in different modesH Intero(era*ility Ghat do +e intero(erate +ithH Do +hat action +ith itH Inte"ration !estin" Ghat do +e inte"rate +ithH Do +hat action +ith itH #om(ati*ility: #lients Ghat clientsH Doin" +hat actionsH #om(ati*ility: $ervers Ghat serversH Doin" +hat actionsH 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page 11 of 14 /eta !estin" 0nvironment-$ystem Ghat environmental issues a((ly to thisH Ghat to do to ex(oseH #onfi"uration Ghat environmental issues a((ly to thisH Ghat to do to ex(oseH 3ser Interface Ghat are the interface (ointsH Ho+ to exercise themH Performance Ghat are the tar"et (erformance dimensionsH Ghat +ill you do to exercise theseH #a(acity Ghat is the tar"et ca(acityH Ghat +ill you do to test thisH $cala*ility Ghat is the tar"et scale' and ho+H Ghat +ill you do to test thisH $tress Ghat dimensions do you (lan on stressin"H Ghat is ex(ectationH Ho+ +ill you stress itH 2olume !ests Ghat actions +ill *e included in volume testsH International Ghat are the international (ro*lems of this itemH Ro*ustness Ghat ro*ustness :crashes' corru(tion' etc5; errors are antici(atedH Ho+ +ill you loo, for themH 0rror !estin" Ghat are the relevant error conditions that the (ro"ram ex(ectsH Ghat are the error situations you (lan on simulatin"H 3sa*ility Ghat are the usa*ility issues a*out this itemH Accessi*ility Ghat are the accessi*ility issues a*out this itemH 3ser $cenarios Ho+ +ould a user ty(ically use this itemH Ghat tests +ill you do to simulate user scenariosH /oundaries and &imits Ghat are the *oundary conditions surroundin" this itemH Ghat are the limits of this itemH %ax 2aluesH %inimum 2aluesH $(ecial #ode Profilin" and .ther %etrics $chedule GhenH #ode Paths and $e6uencesH Ghat are the different +ays to invo,e or activate this itemH Ghat are thin"s you can do just *efore this item that are su((osed to chan"e the +ay it o(eratesH Ghat should N.! chan"e the +ay it o(eratesH su5 162 su5 163 $u* eature !+o $u* eature !hree :etc5; 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page 12 of 14 Test !ase "tructure Ghere +ill test cases *e storedH Ghat is the namin" schemeH Ghat is the or"aniFin" structureH Ho+ do test cases correlate to the test (lanH R0#.%%0ND0D: !he test case structure follo+s the area *rea,do+n structure hi"hli"hted a*ove5 !est cases +ill *e stored in the !#% data*ase5 !#% +as chosen *ecause it su((orts ar*itrary de(ths of hierarchy' and *ecause it $I& *ased ) allo+in" a "reat deal of flexi*ility in re(ortin"' data*ase mana"ement' etc5 In !#%' the left (ane holds the hierarchy' the ri"ht (ane holds instances of test cases5 !he left (ane +ill follo+ the hierarchy throu"h all the levels of feature detail' and then add one more level to ex(ress test class ty(es5 or exam(le' one mi"ht see the follo+in" hierarchy in the left (ane: Gord 0ditin" ormat ont !y(eface Data 2alidation 0rrors /oundaries &imits 0tc5 $tyle 0tc5 Para"ra(h !est cases can exist at any level' *ut it is recommend that they *e entered at the terminal level of the hierarchy so that they can *e easily associated +ith similar classes of testin"5 Ghen a "iven level is selected' the list of test cases associated +ith it is sho+n in the ri"ht (ane5 !his hierarchy allo+s one to avoid needin" to follo+ num*erin" schemes :+hich are a (ane to maintain and or"aniFed;' allo+s you to ex(ress a testJs location in the tree +ith a Kitem5item5item5itemK namin" convention' and allo+s one to determine to +hat de"ree different classes of tests are covered for each feature5 It is recommended that if a class is *ein" s,i((ed that an ex(lanatory entry *e (laced in the ri"ht (ane justifyin" +hy tests of that class are not relevant5 !he intent is to ma,e this document drive the creation and evaluation of the test cases5 "pec Revie4 Issues Indicate location and method *ein" used for re(ortin" (ro*lems a"ainst the s(ecification and desi"n5 Test Tools &ist +hatever test tools +ill *e used' or need to *e +ritten' and for +hat (ur(ose5 It is often *est to (oint to an external location for more details' as tools usually re6uire an entire (lan and architectural summary of their o+n5 "mo3e Test 7acceptance test8 5uild verification8 etc69 !he smo,e test determines +hether or not the *uild is "ood enou"h to *e su*mitted to testin"5 !his section "ives a statement of +hat the *asic smo,e test consists of' ho+ it is desi"n' and ho+ it +ill *e (erformed5 A (ointer to suite locations is hel(ful here too5 (utomated Tests 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page 13 of 14 Ghat de"ree of automation +ill *e used testin" this areaH Ghat (latform-tools +ill *e used to +rite the automated testsH Ghat +ill the automation focus onH Ghere are the automated tools' suites and sources chec,ed inH %anual Tests Ghat sorts of items +ill *e tested manually rather than via automationH Ghy is manual testin" *ein" chosen over automationH Ghere are the manual tests defined and locatedH Regression Tests Ghat is your "eneral re"ression strate"yH Are you "oin" to automateH Ghere are the re"ressions storedH Ho+ often +ill they *e re)runH 2ug 2as#es Ghat is your strate"y for *u" *ashesH Ho+ manyH Ghat "oalsH Ghat incentivesH Ghat areas are tar"etted to *e *ashedH /y +hoH 2ug Reporting Ghat tool:s; +ill *e used to re(ort *u"s' and +here are the *u" re(orts locatedH Are there any s(ecial (ieces of information re"ardin" cate"oriFation of *u"s that should *e re(orted here :areas' ,ey+ords' etc5;H Plan !ontingencies Is there anythin" that may re6uire testin"Es (lans to chan"eH /riefly descri*e ho+ you (lan to react to those chan"es5 .xternal :ependencies Are there any "rou(s or (rojects external to the team that are de(endent on your feature' or that your feature is de(endent onH Ghat testin" (ro*lems and issues does this createH Ho+ are delivera*les from external "rou(s "oin" to *e tested and confirmed +ith your o+n featureH Gho are the individuals servin" as (rimary contact and liaison in this relationshi(H Headcount Re1uirements Ho+ many (eo(le +ill it re6uire to im(lement these (lansH Are there currently enou"h (eo(le on staff ri"ht no+H Ghat effect +ill hirin" more or less (eo(le have :sli( in schedule' 6uality' or somethin" elseH;5 Product "upport Ghat as(ects of this feature have *een a (ro*lem for su((ort in the (astH Ho+ are those (ro*lems *ein" addressedH Ghat as(ects of this feature +ill li,ely cause future su((ort (ro*lemsH Ho+ are those (ro*lems *ein" resolvedH Ghat testin" methodolo"y is *ein" used to (revent future su((ort (ro*lemsH Ho+ is information *ein" sent to su((ort re"ardin" (ro*lems and functionality of the featureH Testing "c#edule /rea, the testin" do+n into (hases :ex5 Plannin"' #ase Desi"n' 3nit 4 #om(onent !ests' Inte"ration !ests' $ta*iliFation' Performance and #a(acity !unin"' ull Pass and $hi((in"; ) and ma,e a rou"h schedule of se6uence and dates5 Ghat tas,s do you (lan on havin" done in +hat (hasesH !his is a *rief' hi"h level summary ) just to set ex(ectation that certain com(onents +ill *e +or,ed on at certain times ) and to indicate that the (lan is ta,in" (roject schedule concerns into consideration5 Include a (ointer to more detailed feature and team schedules here5 :rop Procedures Define the methodolo"y for handin" off the code *et+een Develo(ment and !estin"5 7<-8=-8<7> 8>?7@98A@5doc Revision 7 Template page 14 of 14 Release Procedures Descri*e the ste()+ise (rocess for "ettin" the (roduct from the net+or, testin" version to ready)to)shi( master dis,ette sets5 (lias;<e4sgroups and !ommunication !#annels &ist any email aliases and +hat they are for5 &ist any *ulletin *oards' ne+s"rou(s' or other communication (rocedures and methodolo"ies here5 Regular %eetings or each meetin"' +hen' +here' +hat is "eneral a"enda5 4#. Feture (e& ,eetings 4+. Pro;ect (est (e& ,eetings 4.. Feture (e& (est ,eetings :ecisions %a3ing Procedures Gho revie+s decision (oints for the follo+in" sorts of thin"s: *uild a((roval' *u" tria"e' feature si"n off' test (lan si"n off' develo(ment desi"n si"n offH Ghat is the (rocess for the various decisionsH <otes 75 Areas in Red:; are fla""ed as such to ex(ress s(ec items that are non)existin"' incom(lete' unclear' or *ein" considered as issues5 85 Areas in /lue:; are fla""ed as such to ex(ress an im(ortant issue that needs definition and-or resolution5 7<-8=-8<7> 8>?7@98A@5doc Revision 7