EDUCATIONAL RESEARCH An Introduction DAVID MoKAY COMPANY, INC. ‘Nee York ‘ante oF contents Experineer Biss Sai Versus Weak Pspesinental Treatments Experimental Desane and Stout! Acayis Teka Saale Group Dans "The On Shot Ce Sady Ore Grog Prt Pastiet Design Censol-Growp Desiews vith Rordom Acizment rss Forte Conte-Group Design Breen ostet Contr Deg with Matching Poscesenly Cootok Gre Desizn Sslemen Croup Design| (Quas:Expericatal Designs StasGroup Comparion Design Nenequfvaleat Coal Group Design ‘Time Series Desa Other Expeiental Desks Measurement of Cange 46, Eocenriousn Resunc ase Devens What e Eaentional Rear and Derlopment? The R&D Cue Prof Seton Liteesture Review ‘Panning Derlopnentof the Pretininary Form of dhe Pract Preliminary Fad Test ond Product Revision [Moin Field Tes ane Proc Revisn (Opeationl Held Test ond Fal Product Revision Prabione and less Heveatonn! dt “esting Verne Pols ‘Realism Venus Prtnene ter Lessons ‘Opportunies to Saacatonat R&D 1, Prensa Rastawen Dasa Toei Steps fn Seving Standard Tes Dakine Scoring of Standard Test Seri Uretrctured oe SellDevsaps eases Recorsing Research Data Uso ofthe Compute Data Prosmsing ‘When wo Uae the Compote for State! Sains Rn oH m a ny a9 a Ms 0 0m2 EDUCATIONAL BESEARCH 16 uarren etd tan bo mcr at eat 1 DidethTal Psflee he Mesrrettof Atiermen G ese, (56, STs lepers he se ath 8 Doman Intedetn teh duis mate cages nae 2. Campbell to Sloan Cogent sad Co Smet Degs for Reser Teng nN, Cove (0). Hendon evar on Tos Chicte: Raed MMaty, 0 The cede | Educational Research and Development* eatonal research ant developer (eines calle rear [test devdopast) appears to be the mst promising sragy We 5e¥ or improving efcaion.Beczese resesteh and develop I ee ly new f ecuason, we wl done the term ant show haw fe rs from educational research, mich inthe past nos comsdeel by tony to The best web lor avin ou eh. ‘Bacatioral earch sna cevlopaaat (K & D) isa process wed 10 develop and valdace eats prodets® The steps of tls poss are tly reeed io a5 "the RA B cycle" whlch cos of studying te Bodin prsinnt to Ue prodict ta be develops evlping he ved ch thse Sng, ld sing tn the tg whore twill bese svenesly, and reviing it correct te deine Jounin tae eld sing stage Tn mace sawous programs of RAD, Ui cycle Is eosin he Gk test te Ite that the pie? mets ee iy defied ebjectives. a cots, he pul of educational eeteh nett Seep prods font rather to alssover new keowiedge (brough baie wstreh) ef 9 Janes spec questions about pacttalprblens (thvoieh applied = Sere). Of cour, my np sere project ole develope of ional prods or example, ina peje ences with coparing feiss of two meds for teaching veadg the sche a edap ators at inerparte each meshed eserves als ate not aval. Typically, however, thew nates ave developed 1 Water B Dn, Mero Kale, Dilp tan, Mow Cay mont of Maer aan ont te Comea4 EDUCATIONAL RESEARCH ‘and rtd ony We Ue point where they can be uso teat the In fto's hypothe, For ths rts it i try rare for apie ea esearch ro ld prods that are ready for operational we inthe Arto they have many iportact cntitions to make to ee Iaslc ant applied vexurch are peerally peur ruthcolenios er devel ing new proets that can be wed ie Ge sles. To ail rece sarily the researcher often shal comping poorly det Imgrove, er incomplete predars ta dtesmine wach is fs ual ‘his meshoology geetlly preeesapgcve oe Inordatve reall an 4 best blag: abo inprvement in edeaton at low rate, Eve ‘hey ee abisioed,pesitne fares ae usally sez ony im a ‘ial serse ad have eo practical sigeicnee for the repuae cleo, voter deiciney of maty Lac aed applied reach andes = tat ‘Siaans Chey sey re too removed tron th typi] caaromn to duck dtoct test upon educations practice. Tt true, of ext, ‘sic and aplled research produce the findings that ste realy 1o inprove facaionalpratin Honeer, the gap ietmon these sed educational practic soften ao great that many echlars have 8 lletine worthwile basi and applied reacacch prublas without roving the chal ene whi. aveatos and resetrchers have been sceking a way to bridge the g ‘tween reearch end prac fur many yews, This freely the tsiavan of eluate W & D, I tabs the findives seocated by tnd applied reeeeh and wes them te bul tse aes tat ‘ead or operational use the chet, We bel emphasis bee hoe ‘hat educteral R&D is not sbsitate for bask or apa Teeateh, All hee srscarh satgiesbasi, appl a! We & Dae aque ‘ring about edorational change, In fae, R & D inreven the pote Snpsct of basic and eplied resents findings open schol peice easly them feta aebleedsatonal pode We Rao crcie 1 raider hs ager we sal cas each of he mala | ‘in the R& D cycle, The specific R & 1 cycle which will be presented was | dag thal of he Tess Edenton Praga ae Fer Wa | Tater fr Bahco! Resesih snd Deepens ty wee ‘utc aaa Te Fos Lbwatey iene of ean ab esa ed by he US, Os Eten on ae ene ingrown: heh 8" To Tsar Hones Pron eon ‘products called “minicourses,” which are designed io improve teachers” ore dase EDUCATONAL RESEARCH AND DEVELOPMENT as Since wil ing the Svat of or fest minum o The tate the RED cove, we wil rly dove tere be cera of th prt ach iene isos aews 1 tof ter xing Inlet recrvie ee nrc sling. Diets Une the teacher Twig tine introduced oa nde fii trom sl Tt Alls re fir: ented ad erated ian strana ian The tle ‘heme ais Sennett int "ode fm” tat by stim oft Tet caro tsion ondaced by adel tae, Ties Ue ince {Banach on i wica be tempts apy the eile tat have teen presets the leon tu sal ey of pup ad ecards fhe laton on vee. Tete afr the eso, the tres ws fhe eda, loans etention Gate speci ie bo ating team Tis oa i called even nen” cate the aula ca ‘um sialon esl down inne ad momber of papi Having en Abd eat the videotape esc fr lon ttisher {an the sae lemon and release ellowig yf sete sal gape The won Was ecered ‘on cig, ond aga ‘wt od rleates is perornance tamed afer te lear cee feted The acer tee pecans ty Ihe pest souece of Intnl ton sil eae, nicach aed resh “Te maj ep in he WB D je eveip mince ae at Seles 1, Reseach and information callectng—nclres review of literature, ‘ouse Sequence an sal sae easly est 4. Devlep preliminary form of proct—inludes praration of i ‘sauctles] matsas, Handbooks ad evaluation Helen. 4 Prelinizary Geld teting Conducted in tom 1 wo 3 schools, wing (to 17 subjects, Interven, cbevaticnal apd quesionaive data ‘cllaced antalya 4. Bai prac cov Revision of prod as aggre hy the reliniaary ld et eeu 6, Mala Geld wsting—Condacted in § to 15 schots with 30 to 100 subjects. Quantistive data on subjects’ pronase 1nd poxcouse perloraence ace illected. Reculs are evaluated wis rape 10 ‘oure ojeces and ate compaced wth carta group data, when ‘appropriate. 1. Operatiual produc evsls—Revlson of product as suggested by smi fe tot esate 48, Operational Geld tstng—Canducte in 10 to 30 shoo invlvinga6 EDUCATIONAL RESEARCH. | GDUCATIONAL RESEARCH AND DEVELOPMENT a7 1 Does the propceed proluc: mee an feportant wcsons reed? 2 Ta thesttaof the ae saSlenty avaped otha ther Is rason 400200 mbes, Inks, arvana and eine dt te oben ed payed i 9. Mal pod ein evion ef pad en gre by pu: | alesse can ets foe fl rt I) fen pene rates vo tev sila Enon nd eer 10, Diaeniatin snd dtu Report on pod ot profesional | gerry te ais poset? ‘extn ond in orm Work wi pula whan sed] Na Cc rec be rnp i eae me? Sea anwtiniss Mester sittin te rote gay oar = Seer twas png ex to dee steve posit ar nae hi sure ef tn ssf (toned pep, se ax wana | fuk ela, Sha uss erly ic ver Mle nae bic bon tsath Bch aly ety for apertnethe | cn and at avinle perl por Convent acer ‘Stat. Ath ec ofthe te sos wil be dead tn tall we | lnon props hve fot segs ciara: (0) the tac = ‘should point cut here that set of these st» ate she dd in ny Id what toda mest ofthe time, rather than being gives the oppartaaity fearon! vere poets Tak pricy tae of Sep 6 main | apace yond Lecce talaizess () mot eeaig ten Bo ‘Seld testing, in which quantitative data are collected to determine whether -leachers with vague generalities, such as “individualige your instrtc- fe prtaet mh urmance jee Tsgartt he R&D cpa | bet fal ts os vos specs taal aed rns ‘eal he nea an appl earch pect. Than aan ot | Ml (8) sent tacos Ia cece ede octal (0 ‘ity ated fo ean! rocwch me wc a endutag KD nn ng poem prvi ie ec ote eat Pees ess chancon peemnce Far misouse waned to oe Be wrstne eg vac onag pee PRODUCT SELECTON 4 Befote the educaslonal R & D proces can be apple, Is ncesary to ducribo ae specially as possi the ecectioalprofuet that ‘Ye developed. This description should dude (2) ex overall macative eserpton of the peoposed product (2) a tentve cutie of wat te ‘product WH code and bow i wil be aed, and aust inpetant, (3) & pete sstment af the jpesves of the pad. In the ate of ce ‘fot such a cho miniceus, the cbjctvs shuld state the specie performance eels to be achieved by texchers empleing Ue nurse that 4 che musber of tees they wl demoastate exch shall within @ given tine per. Tire caee the nature ofthe product wil change substncally during the develpment proces. This due not reas, brevet, Chat the inital Plsoning aboot taken lightly, Tie planing oxides te fat pon which Intec revisor are bul Withous cael planing et the Sar, the Mette of bailing good preter 86 much refed, ‘Siae very fom well developed products are svalase i education the developer hasan aos unlimited range of posibe products hat Be eae \deeop: However, tere area rorber of erer that he con agp fos lecting an aren fo workin. The criteria for product slecton eed st the Far Weat aborsory inclu the fllonag? Lurezarune eeview “Once he a of the ecard hs ten ett em fie tear view ender tele resesh Scns a ‘sirmation pinto te len develo. sn tse ot ees pape eesti vow eter at of Knol nit oof coer However B&D prt, oer mtn beaver with Norte kouledg m b 1 te prot be its fo deeb. Elie ce oo Kinsbns os tachi els eared Sjenietng Gekaen io canreo uctans need © ood Tar ut et mnie Ti tie ventas to Mc | etacve Questing Tirontny Tov” See Misra 1 wm Bit pout evap by the Teer Eesti Prgms oy i cmc te ce fve, The pune of Ov rae eae rere tha! cb ed deve » ane faris Trt for hing tears Rewarch for areas wanted: yey om om edi insing ad gt hoes Gat ew wv wee se tn thy myelin thal etiqese more ein. For camped ha ng sacar whip etek ft cho persas DUCATIONAL FESEARCH ance i a eatin tchnique or developing new classroom still, Age Uther sectvetechnigee ie provide « model ofthe skills to be learned Interestingly, recarch fncings indicate thatthe precenco of a cupavane |S aot necrsary Uy Lea about teacher iaproveent whe modeling std wiectape feebck ave provided In faci, Tuckman aod Olver ound that sipervisorfeoback ted 10 a ctbiage in teaches” ented Invi ever amon nora in the cvertion apposite thst reco by the supervisor Wet many educators elev thst the superior fe ntessry elaneat in texningfenchers This example demonsietes| ‘pinion aed prevailing pracice are often poor guides for developing Chufa products that work ss de) ave Intend to. ‘Or seeand iterate review vas cancer with qaetoning and cusion kil, We fcust tnt wenateh im his aren exis bask Stevens” (1012) study of high schol laser Steven fund that ‘Birds of teacher! questions reuired students to reall acts rather to think about facts. Furthermore, teachers talked two thirds of the cussion tine, thus allowing stents to participa only one third of tee. Sst Sndings have been cbtaimed in more recent statin? 6 sppesr: that even though they nave know abot the prevalencs of ‘testable teaching peveices for a long tine, educators have aot fede in bringing abot weeded improvements In teicher eat Sis We decides that cna gals of Minkonrse 1 would be ta teacher tale and corespondingy to incre stent clk ane 10 the percentage of tacher’ thought quests. "nthe ost phace of the Kieratee review, it was necstary to identify specie tecriqaes that teachers cond une to accomplish thee goal, Ae ‘hough afew teseatchstadis wore peron, it was aso meaty Cr ‘8 Co she considera attention tthe opinion ane! eepeience of pete done. or eximple, Gromer acvoca'es meal teaching, sages ‘Thich were ineluded in Minicouse I, bathe provera no evdence on that SMB Gr, Te Fs 9 Msn an Fer Yi oe Aen cant eure sinters” intent date urns Seb Uses s etc st Beck o Tach «Fan ht Soa eo of Roto! Palo ge 08,397 i So nn tt Mos of Fc Iara" Thre Cale STATA ae a thre, Bemus a FL eth J Pe Langage of 1h, arom Neu sks Paces Cage ray Cis Cae "5, cn, eo Intense er Cann Anca Bagh (oid; Invtog Aedes Firoy, Rosch sa patin (Pa Xan) EAE arb. ed "ana he Or! Oust Ay I Sed oa a sary Channon (Uae eal dneitin Cnn Sate Calg, 8) EDUCATIONAL RESEARCH AND DEVELO?AENT ffectivener.® Since er Ite eld experience with hat met of the stategie bing about improved clas diseusion, they tne inclated in the na form ef the course Tnleviews and direct Geld Diervations have alo been vsefulsupplenerts to dhe esearch eratare In providing swith 2 fosndation of Eaowiege upon which to develop geen eduetinal product. For example, in Miniccure 5, which le 2am ‘Gemed vith mathematics toring sls, we could fel no research dings ‘esarding what cecars betweee pupil and teacher ia the (ypcal tering Sequence. Ta order to partially All Wis gap, the laboratory seat observers dnt a number of chsstoons to study tuteing, Ineracions beeen teacnara and pape. We leemed oom thse obeeraions that he uel tutoring contact tween the tence end the indvidoal pupil was ery “ea, averaging oly 15 veeoads. The content of these tutoring cortacis sugested that ie ceacher typically gave the pupil an aswer ce pointed ‘ut bis enter and dhen moved on. Eftcts to gulde the pupil tard the ‘entiation of his eres or fo Gevelop understanding’ of mathematieal ‘ezneapis and problmcelving procedure: were rare. Although they were ‘ot callectad fn a tightly contelled recnreh coting,thezo data di pro- ‘ide ws wth bai information about the nature of malemates tutoring fn the intermediate rades ard suggested to us tht teas could profit tom learning 2 tering sequence Tx which the pup! gue toward Aiscovey of his ener: and understanding of mathematical eanceps ssl protlen sling procelurs. Tn developirg an educational product wsiag the R&D approach she ‘seater wiloien have ccrain questions that cannot be ansnered by ‘eerting to prtvent research, Thuy be researcher wil inde hell {o exrty cut oor of more rmallicale sive prior to coveleeing the reaver Alco 26 we shall pat out inthe next sections, the R&D cle ‘emits coveral parasites te collect raearch data and te rorian the red, These phises ofthe R 8 D cycle ean be ase to ansmer pressing ‘earch questions luvalved ia the construction and use of Lie predic. PuNsInic Once he hes completed his sevew of the teratue and calecied other ertinentinferation, the investigator proceeds othe planing sep ofthe R&D gee ‘erhaps the most important aspect of penning a research based! adcae tional produc i the statement cf the specic ebjectives to be achieved by tho product. A frogusat citi of isting eluceional practices LP in He ToL Fe to Qin Yorks te Frat«0 EDUCATIONAL RESEARCH Chat objectives or citi ave avslable to joie thie eectveees ‘Now caren prensa alten rconeione for Cee ebony fn tat, eaten Halon, and acepiance by exces and ten | ‘Yet naa missing a sate of he programs oetives a terms of stueut outcomes. For example, an objective of a socal stucles proguan | baht be stat a, "At eos 45 pore: of the encenta who compe the rogram wl tr acre of 90 oc beter on in ean vers imap sil” Soc uen-bted chjacivee enable ecto fodeterniag ip quantaie rns wheter the progam "mara" Objective po ‘ite te est ts for deeping an fstrctioal pees, since the ‘roxram cane eld tested and revised nt ik meets Hs objects ee | tier specication of elicatenaloateomer—or Urhavaral obetes, a they area ealed—requrs camera ail on the pat of the devel. pert In same ways developing a beaviorlcbjetve for an cba Dolut iimiar to developing «god criterion i a nesarch stay. ‘Deeg the planing phase, beavis objets are mally stale serve loc. or erampl, inthe inital planning of Misiouae tne of ovr abectives stated that ater the etre mt teachers won incre thei 3 ef thoueht aestiens in a aetsslon situation. We a aot have suicen: knowilse It he planning ple, toh, to soeclly the pereatge ofthat gestion Sha we would expect teachers to ak Inner forthe courte to be enseret ftv, Ae we proceeded thragh ‘he R&D ele and scunclated recat data, we ware able ose ‘Ee statement ofthe behavioral ojetve to Tht tok fe fllonig form: “Givens tmisue lacus ison at lent bal ofall quedo {Ske by teachers wl be cased as though: gestions. Tai reo ‘PIT be et hy a east 75 poet of teachers wha complete Atco 1.” ‘Snncher important cement ofthe plannieg pisses eine of the rooney, manpower, aod tne recur to develop the prot. Geneally Sop rscurees are ete lo cary et sige R&D project. Our “prensa ben thatthe cov of developing single sncowse, wich ovis abut 5 hrs of stacey I in ssen of 100000, & aloe {Srvcson project wll cst sveral milion dae, Manpower nde are eovitenbie, too. The cevsepment of 4 minicar rqeites a8 AVeE {ior ances of protesional work 5) matemses of eal wer fe $0 mancweeks of profucten work. In contra, cst esa Projects lvelve sal sn of remy lcs lm thas» tho las, Bed ihe fers of single ingest with perhops « few purine ‘adate mettnnta) In R&D ory voles caret plana i dey the Iovetigstare may find that thet tenure have ren ox Before the md at a ie nar oh Lr be a Mag Papago Ea! Reh EDUCATIONAL SESEARCH AND DEVELOPMENT a role: as been flly developed. Planning is necessary ia err to ane Alpate needed materials, profesional help, and Geld tet alts, Conld- eatin ofthe Gd at sits particlely important when testing fe dene Inthe school, which generally are receptive to letting Oly at cata ines sf the year. For example, ifthe provct is rey for testing in June, ene fay have to wat unl September or October unless the product can be tested daving a sumer school sein, Asp school administrators genet- ally rule few months pri atice before aresing to have ele schoo ‘Alnough the R'& D specialist ust devote » considerable amount of fine to inital plansing, the planning function iy neve really ended, As ‘werk progresses hee Likely to cscoversoveral ores in which fil plan- ing was isuficiont ot in ere. Replanning mux! then be done. Nonethe- ese it wise to evote major effort to bung a sound inital plan. A ‘gpd plan can hep the ivestigstor avd much wasted work dur Iter ‘imses of the R & D eve DDEVELOPNENT OF THE PRELIMINARY FORA OF THE PRODUCT Alter the into planing has been completed, the next major step in the RUD epee ie ling «preliminary form of the edueatoral pod tit that cam fel tested In the case of Minicnrse 1, this involved a ‘wide range of tasks. Scripis describing the specie skis that teaches are to leara were writen for each ivsttrtonsl sequence, The scripts ‘were thea producsd oa videoape and ete t Include lps showing the lls being teed in asso auatins, Prospective wale tarhers were Iecated, observed, and tained to condact model lesan designed to Tar ‘ther strate the minicoure kil, Tie madel lessons wore then recorded ‘a videotape wai edt. tencher handbook designd to supplement the ierotaped instructional lssons was drafted, evsel, and prlaed. A set (Of forms for the tener fo ee in eetevaation of Hie microeach and ‘etesch lessons wis develope’ and pring. Questionnaires and jnierlew ‘guldes tobe nse inthe preininary fed test were developed, an labora Ate staff members were tralaed ia thele ws. At port peincile that held be coserved in developing the peeisinary form of ap edational Dreduct i to strctare the proict ao as to permit obtaining ss mich Tedback as possbla From the eld test Ths the preliminary form she incade many mote procecres for evaluation dha wil be iced inthe ‘nal product The render who has sitemoted & major eeeational K&D efor wil sealize that hese steps, stated so simply, ae far frm simple to carey out ‘The fuvestigator must expect my fale start ad setbacks n developinga EDUCATIONAL RESEARCH ew education procct However, since th ates prcedares involved in protuct development vary grealy depending upen the rate of the product, are fe ite specie guidance that con be ven sn thie phase of We R&D cycle However, one pola Cat apples ia snot R & D work in edecaton that the investigator should sive trom the outset W de ‘vl products that are tally vady for use tn the sehoos Paral dele ped pcs tors the eal preetitoner ta make additions and change {order to uo he product. Since fw schools are eqipped to make such adjastment, a parially developed product carnot be aed eetvely and [soften badly misuned [PRELMINARY FIELD TEST AND PRODUCT EVISON. he parpese ofthe preliminary fl tet is to oben am intial quai tative valuation of the now educations prodact, For tho aniouree tise evakatie is based primary upon the fedback of a omall soup of teachers wn take the ours plus the olzevatous of laboratory pe ‘sone who cerdinte the feld test. As rl fom four to eight teachers Fave been suficint forthe preliminary hel ts, since the erga of this eelantion i pen qualitative appa of corsa content rate tha ‘qumntitative appratal of come ovteomes Tal phases ofthe R & D crcl involving poet ershation, i ne portant fo establish eld sites sina to those in which the product wi De used when ii uly €evelopa. Ts diferent type off site is wed ‘the ivertigater feces the problem of generlieg findings cbtained it re telling to snotier. For example, Minicoarae 1 war dsiged We teed by elementary school teachers curing that toglor echo ay. | ‘Therefore the preliminary foul test was canied cut wilh ix teachers fiom ewo'eleneatay schools, Intend of this pice, we ale ave Invite the teachers and scr of thele sets 10 Burs) 1 fake the course, pears on a speed-up basis. The major probe with the prcedave fe that we might have ebtaned a very untenlisie nsprestion of the corse. Elements of the course Wat be 30 Jem in a laboratory sectng might create ave when sed i the schools ‘sig an adverse eect on Ue course oaeoses “Phieushout te prelimiuay eld west of Mincurse 1, two Bald eps | esertaesffom the Ishoratory werked closely withthe’ al teaches be ‘order to obtain a mach texcher feenick an abuevatonal dal ak ose ‘ble Each tracker wat interviowed individually tree times during the ‘eld test. Tats interviow focused apen specs problems and cour do- Siciacien wa wel as sogestion for improvezeal A te ead of the coarse, ‘cca tae completed questionnaire resaring the course ard pte IDUCATIONAL NESEARCH AND DEVELOPMENT ey pate in sroupdiscassion with Inberatoy pesonna. I addin o these formal contacts, exch teacher Bad Enforual eatacts Wi oe of he Iab= ratory represestatives each day. "Tne to citnin esti fdbac fom teachers dering the pol ‘nary Held test can crate a problem. Obtaining the neces feccback seals inthe teacher's seeing a reat eal of atertion frm the invest (ates, This attention can produce a Hawthorne efect, whch will lead the developer to overstate the etlectiventss of bis podec.™ Tas, tae developer mist sve toward a delicate baler Ia which feedback a= tained without ging the participating teschers se unde amount of Ic teoten. ‘Observing the participating teachers near the end of the preliizary field et of Minicourse 1 zeyealed that the ‘cachers mere generally anable ose the course sks eetively eit In tel egular Cassocms oc La their micrearh lessons. Thos, ftom the standpant of bringing about ‘pecie changes inthe lao ebaeor of thee teaches tbe Felis ‘ary form ofthe couse wan flat, End couce intarvies sd que onnaire sbisind fm thee teachers, howerer, indies that they per ‘ved the course as being very efactlve and at providing thers with a ‘lea deal of Delp Iu ingroving Civ teaching. Tse resp suagest ‘hat teachers asa group ate not highly critical and are Uikely tobe chat table in thir eration of new efational practices. More general, ‘weave found that lobal ratings are of Ile valu a evaluating ‘decatioonl edeeation program whics teachers We a2 a ‘Standard of comparison. Neverthees, our experience has made clear the hanger of making user judgiaea the bast for measuring the suczs of aa educatinal product. Edweatieat products shuld have ebjectves tat ate fuchet in terme of terminal sehaviors and choad be evatted om the ‘ss of tei sucess te bringing about these terminal behaviors. Hos rer, although we pat Lite tr fa glbal teacher evshation, we sly Iuavly inthe preliminary feld test upon specific teacher feedback in Idbine 10 deveop and inprove cur educational pradets. We obtained ‘any speliccriclsms an sasesions ding the prliniaay Del test Df Minfconse 1 that led cecly to changes acd improvemsen's Im the ‘une structure. In fac, threaghoet the ceelopment cys, oUF mala metertomeetic edi ntme EDUCATIONAL RESEARCH serfs f rehing sites ones Suonnnncas ny i cone i we epi) Siete eee eer ad Sea eee pet ee eae ae “MAIN FIELD TEST AND PRODUCT REVISION i “The purpose of the main eld text fa the minleurse R&D eye to dnwrmine wheter the edcational product ender development meets preformance objectives. Ceneally an exprimeatal design is used to ame ‘eth eatin th ae of Ml 14 leon Pt ‘design was t2ed fo determine mbethertescbecs would (Grae thet er of dumden shi About $0 nacre par Uepaied ll ‘experimen, Shorty before the course began, each teacher was asked 10 feomdect a 2C-ninate elsession In bee eguar czsscom, and this le fomson wre vdectaped. After the coocee as completed, each Yeacher_| ‘gain conducted a 2Omipute videstaped dteusen. Each videotape was ‘viewed by trsined raters who made quantitative observation of teacher! ‘seo te obi ad Lehavice pater presente inthe minicourse Sate random ores, he rales Desens the major ings ofthis experinent to deter the eflective= nes of Minjcours |. Mest ofthe changes in tescher an ete behavior brought about by Minicouse 1 ere not only statistically significant, bat sve ao signicent in ther iolicaion for educational race. Although ‘Acontrol group ws nt ond in te ein fel tat, cobseqaent studies ba Indicated that teachers who take the couse take substantially larger fas aa teachers who either donot have the coure © nl eee some fore of minimal teacnent ‘nation tothe primary purpase ofthe main fed test, which 10 Aotermine the succes af the evr proact In meeting Hs objects, the ‘eendary purpose it collect information that exn be wed We improve ‘he coun i a aest revision, Therefore, questionssze and interview data shouldbe obtained fm ll pateipante i the min ald tt 1 the mia eld test findings indicate that the new produc fas sub- stanly short of meting its objectives, tg wecesary to revise the prod ‘oe apd conduct another main fel west. This cycle of fed testing and ‘even would contin un che peofuct meets the minum perfonse ‘ance objectives set fori. Tn practice the protact would probably be ‘abandoned if substantial pregtss were not msde i the send main eld feet SDUCATIONAL RESEAECH AND DEVELOPMENT as ates Page is newer Comimt ‘Mea Mae ent Mae ts tr od 2 ember fees eater oe eine ow ot 4 Teg of pu reps Sere (sd on mia Bedpe spac komt (6 Lah tae pee ‘gente ne oi Snes ore and pote). 18) Bak Be rapes of ted coc fiat ole tor Se no renee ee Dest ott deise Pane d Geo woe fateh yack ate pa ase a Ties ote te om ‘See (hat en She Sinisetpe st omapel, Sat ast Fry pate ne ‘eosin ne pag a Froperen of ace tne EL tac nw an Pent eb eee pity a i et a a en EBs e ‘OPEHATIONAL FELD TEST AND FINAL PRODUCT REVISION ‘The puspote of the oeralonal eld test i to determine whecher a9 ‘dualonal producti aly weady for use i the cna withoot the pes ‘noe of tbe developer o® bie sta Ta order tbe fly realy for operaey EDUCATIONAL RESEARCH tional ue, the package mast be complete se oroughly tested ia every respec. In dhe ease ofthe minicurse, all maverals needed to coordina ‘the ssn ste normally tial out ring the preimivary and main fl tees, However since (nec Bd tots are condcted by Laboratory pee ened, a susioctory test of how well the total erurse package works nfs bw cannot be ebtaued. The operational fel ts setup and Cerda hy regs choo pera and eld ety apres ‘regular operational wt. Feodback fom both Ue coordinators and (aches taking the course are coleted by meno of questionnaires which aro stalled la the Laboratory, The msia ase of thee data fo determine ‘cher the couse porkige ie complete Ttervlewers foc om parts of | The core that fa todo their job or on serie hate minded $a trder to make the operation of the couse oarer cr mere eBective. P= soe and yanicurse vicolpes te aot bean daring the opeatonal feat est [Alter the peratioal eld tit is compose and the data have bee nae Iyaet, «final revision of the total conse package is caved ont. In the se of te miscoure progea, che Laboratory auhes« Goal revision of [BT scrips and prted maeras and twas these aver 10 2 commetcal Dablsher for aa! prodution. The couse then sold or rented oscil for opeatcesl ate in thee nverice taining programe!” During ops. tional use ofthe course, the publisher surpies course cordaates with ‘valuation quetonuares and Interview fens sp thatthe Laboratcy can ‘alta ¢comtioor syprasl of tbe course's efectiveress ant eam ene Uy new problems thst aise in ie operatiral use. The tal ten, weve js tantly a quality control provedure and would net be reznded as further feld testing of be couse. PROBLEMS AND ISSUES IN EDUCATIONAL R 2 D Since thee vas vitully xo gublance available on Une edscatinal RAD proces when the Fur West Taboriney suri fs prgtane In 1, wa necsary’for us to deve ou procedures s we went slog. 1s 6 doing we uncovered soverl problems and isaes that aro base. (0 the R&D approsch, Since these lsues ace likely tobe faced by ober Investiatars who do educatlal restarch and development, we wll de serine some of them Aecrsing vers pal ‘Te Best problem conceas law far the educational developer should fn bring the preimizary form of a predict inolving an expensive acum eg mace come by Macalles hun ee, ey Pa aad row St, verde Ne Je DUCATIONAL RESEARCH AND DEVELOPMENT ar ‘cempones, Sich 28 igtcutional Alms or tapes. The development of the ftininary fore of Miricourse 1 presenta ge with 49 setresting Tere. On the one hone, it was devirable to spond ac lite moazy as lle or inital development, sie the fedback bined ort dhe frelininy fel test would ast sacely al for extemve revision. Om the ether hard, 2 poorly developed Se of materials might produce poo? fonts eventhough the dese dering the develapmect ere cone. ‘The mst defensible reskten of this lemma i te ot me ef hein tial develope effort into & siaple product that makes mau se fearing prinaples, that's © theeescally sound protic Lisle or 20 ‘Gort should he devoted! to such anise comeing minor erate ‘areata, bailing attractive chorts where cade ones would serve the Purpose, or reshotlag uation pice foouge Leause of poor cmere ‘work. Th sunmary, oor strategy calls for ping the essendals ont Dest ‘Sloe! enc ding everything ese st cheaply sna quickly a= powibie. In fis, this approoch ancurts to ¢Togcal application end extoasion of digs of May ad Lunsdaioe (1988) and oie ceseachess who have Studied the leaning ovteaes of auctovisual media (Our experince indicates thatthe slacational developer wil not nd this an ey roed to fellow. Media specialise on ike development team, such as artis, acters, and television production pesoanel may aoo1y {rent prewre to ‘raprove the nonessmtat pret of the pro TT hese prssures are not contlled by the educational developer, he wit se an increasing proportion of is sources acing into essen polish, (8 oore subtle recson for essing forts co apply polish daring the Seveipent cyte that the developer sieks fo this presse, be may dwell create a monstec he is unable to detoy. Jt very diet to seep A polohed product oven if Geld tet data ineat that i aot achieving lis objectives. Fst the developer has spent a gvet deal of snevey which ‘be oes tt want to anit has tenn wie, Sean, she pct Tonks ‘pet and knows that most consumes of educational products ae mt ‘ery cononred with hard evidence en efloctivenee, Fal, even thoygh ‘ap educatioa] proche’ fails to achieve iis ebjetve, it easy to te ‘delle that is probably beste, o surely no worse, than the competing, ‘materi curently fm se Reotom vor pence ‘The second question of development strategy bas only recanly been reselvod in the minicurse program. This question concerts the exent Lo ‘hich eGuestcsl developers should werk with "real tench ia “eal” dasrooms who are teacteg “rel” Jetons. Our work with mo! lessons SM A May and A.A Lama (lt) Laoning om Fle Sw Hain Yale ery Pn‘quently excouter Ios schol administzetors who want to dovelop the dun misicorses. Phere are probaly very few schol dss that he reources oattupt a develypment tak o this magnlte. Tt aU ih tie tht enjr efestional developers programs should be { orpunzatens such ae the razon laboratories a4 resarch and velpment center, which have the perenne, euipment, and supp for such work, LOFFORTUNTIES 70 D9 EDUCATIONAL 8 & D ‘We nave ales sensed te considerate recat ete to nt ven angle esol KD pret. ie Hgnly ney tat radiate etdent wll beable to Sind the foal and manpower 2 to complete a majer R&D project. Tn fat, educational Re D be the aides of most seen dts; ing realize eres ‘pont edocs Ie 8 D ert ere eenemscal ery when 4 nen po | {5 developed at one place and thea distiToutedaationally® Tae ae | umber of ways. though Chat the studene can parcipate inthe K&D process. The stent might deine very imited procbet to devel, par- ularly i ideas not vole experive materiale sed regis 36 moe than an bur or two af instruction, For example, a sal team of grachate |, stuleas ight develop 2 new gue Co lnpcove pups’ sk wing umber systems other than bse 10.1 proven ere, sich & wold prebably have = mh gestr impact upon educate Ua the ‘ypicl decor dissertation, Pechaps move fosble, the stent might Jirate a product thst hs alta been deeoped so we itn a Ine of ‘eefback n kl afin Also, mach research Is nee te determine sects of teaching lle on stent tebawor. or sample, tees ‘waned to ak more thought quarto, what cts wil this change Baye fs student bevioe? The rafanie stent could we aller part of min curse to vestigate such questins 1p tie ae hal ieee oramizions exeved in elon: tional R&D. Tf he nds an organization developing a product in ‘es Ineesied, te stout ang be able vo arvange withthe orgie tion tonse the predict na research project. Net nly might the developers ‘lp the student formulate his projec, but the sndoat may be able Jc the organisation answer a poesing riers potlem, "Eight nd LD. Cary, ‘Racrcdh and Dedgrn Stgt Th Co en Sah he Dero iprneal anat 3? a 8h EDUCATIONAL RESEARCH | EDUCATIONAL RESEARCH AND DEVELOPNENT a ANNOTATED REFERENCES Kosei. at J 84 sam Gaon Tee). "Ketsech an Deep feet Tovscl te Ingroveet of eaten” Jawa of eprint! Ba Bay 27 (isc), 11s), The nk sone ene] Arr vere terns ep lt sle toward sed devdopmant peje. The pecs ‘hes vinta promis padre tikes She mathenatee ‘Site of he Sibel Murai Sade Gros the acces cen of {he AAAS Cassin on Scns duran, nd contr ed trod Sem The role so cnet dang be and ppd racech {Hat cud te wed os de fnan of an K & D roe Poured of te AIRLIE HOUSE Cuatenace, "The Pull cet is 8 ‘Ms Eautical R 8 D." Jost of Revarch wnt Detldyment ie Bacon, 2 (se), 190. ts journal tne, x ci ns das hele of RADIintergse stout wirteealInprevene aes port es pre tue vastinenpoteten hose rnc, appa Psat and 2D, to htt toe nership tay est erat a am engl Cases ‘pe eof yarssr tent ace ho dncmes 1 sana say oppose {e dowlopirg slats! potion. Tye sdet = reommenel sey een al ore she Ridaed B "Devebying ihe Die Educaoat K&D" Theory feo Prue 6 (1957, 1896. Toe tlhe desc he RD proven wy Srudwes Resin Libercy for Eston Rew snl Devhgmes. ‘The da eects of dK 8 D pecs we Itt di, ef poe fro produc uly vera, sac aay a sa ae “The sere mens eK & D auld copa he prove ceed Aer wi jee ti ane “Taney, Lay 7m ond On dey and i Shel” ean Laar- thi, 35 (100), 2-27. ‘Th sate iss fhe reer eve of K AD Inti a hw nda ie tearing to ne RD tong sost eventos] "Hs tus sampolartved inaucton an sap of Sow Ee fapecitecten NED Sicisny and chain