CHAPTER 7: EXPERIMENTAL RESEARCH Experiment research - research in which at least one independent variable is deliberately manipulated by the researcher.

Experimental variable - the independent variable that is deliberately manipulated by the researcher. THE MEANIN !" EXPERIMENTAL #ESI N

Experimental $esi%n - the structure by which variables are built into the experiment. The experimental design includes: ! independent variable(s) ! dependent variable(s) ! intervening/control variable(s) ! when and how the dependent variable(s) are measured | experimental designs are often diagramed to indicate arrangement of variables and measurements Upper part of diagram !ower part of diagram design is written out design is diagramed symbolically

" random assignment to groups #$% #&% #' participant groups ()% (* % (+ experimental variables ,$% ,&% ,' observations/data collection | experimental design helps the researcher understand and interpret results of the experiment

-xperimental )ariables Experimental variables - are independent variables that the researcher is able to deliberately manipulate or control. .owever-not all independent variables are experimental (e.g. intact groupings of students). Experimental treatments - the number of levels of the experimental independent variable. /n education% experimental variables are usually assigned only a few levels-more levels become increasingly difficult to control and interpret. S&b'ect - someone who participates in an experiment.

CRITERIA "!R A (ELL)#ESI NE# EXPERIMENT

$. A$e*&ate experimental c+ntr+l - the research design and variables are controlled well enough to permit clear interpretation of results. 0ay include randomi1ation of sub2ects or controlling of extraneous variables. &. Lac, +- arti-icialit. - the experiment is conducted in such a way to permit application of results to the real world. '. /asis -+r c+mparis+n - the research design permits comparison between variables to determine what% if any% effects are due to the experimental variable(s). 0ay use a control group for comparison of effects. 3. A$e*&ate in-+rmati+n -r+m the $ata - the data generated must be sufficient and of the proper type to allow hypothesis testing or responding to research 4uestions. 5. 0nc+ntaminate$ $ata - the data should accurately reflect the experiment effects and not reflect errors in measurement or procedure. 6. N+ c+n-+&n$in% +- relevant variables - the effects of variables other than the experimental variable(s) should not be allowed to have a significant effect on the dependent variable. The effects of such confounding variables must be controlled or separated through experimental design or statistical procedures. 6. Representativeness - the sub2ects used in an experiment must be representative of the population to which results will be generali1ed. +areful selection and/or assignment of sub2ects helps assure representativeness. 7. Parsim+n. - the experimental design should be as simple as possible while still permitting accurate interpretation of results. -xperimental validity Internal vali$it. - the basic minimum of control% measurement% analysis% and procedures necessary to ma8e the results of an experiment interpretable. External vali$it. - the ability to generali1e experimental results to a larger population. | the researcher must be careful to maintain a reasonable balance between internal validity and external validity. Designs that secure one type of validity to an extreme may jeopardize the other type of validity. The purpose of the experimental research should determine the extent to which internal and external validity are controlled. T1PES !" EXPERIMENTAL #ESI NS Pretest - a measure or test given prior to an experimental treatment.

P+sttest - a measure ta8en after an experimental treatment has been applied. P+sttest)!nl. C+ntr+l r+&p #esi%n

! sub2ects randomly assigned to two or more groups ! one or more of the groups receive an experimental treatment while one group--the control group--receives no treatment ! upon conclusion of the experimental treatment% all groups are measured on the dependent variable under study ! efficient design ! easy to administer ! no pretesting re4uired ! individual sub2ects need not be identified " #$ --------9 ($ --------9 ,$ " #& --------9 (& --------9 ,& " #8 --------9 (8 --------9 ,8 (8 experimental treatments)

" #8:$ ---------------------9 ,8:$ (control group) | if no control group (i.e. two/more experimental groups only) is used% this becomes a posttest-only randomized groups design Pretest)P+sttest C+ntr+l r+&p #esi%n

! sub2ects randomly assigned to two or more groups ! all groups pretested prior to experimental treatment ! one or more of the groups receive an experimental treatment while one group--the control group--receives no treatment ! upon conclusion of the experimental treatment% all groups are measured on the dependent variable under study P may used pretest scores as a control P may measure gain scores as dependent variables " #$ ------9 ,$ ------9 ($ ------9 ,' " #& ------9 ,& ------9 (& ------9 ,3 " #8 ------9 ,&8-$ ---9 (8 ------9 ,&8 " #8:$ ----9 ,&8:$ ----------------9 ,&(8:$) | if no control group (i.e. two/more experimental groups only) is used% this becomes a pretest-posttest randomized groups design S+l+m+n "+&r) r+&p #esi%n

! combination of the pretest-posttest and posttest-only control group designs using two group for each of these designs ! therefore% this design contains two experimental and two control groups ! both experimental groups receive the same experimental treatment ! only one control and one experimental group is pretested ! sub2ects are assigned randomly to all four groups ! the effects of the experimental treatment are determined by comparing the posttest scores of the experimental and control groups | allows the researcher to determine possible effects of or interaction due to pretesting (by comparing results of pretested and non-pretested groups) " #$ ------9 ,$ ------9 ( ------9 ,& " #& ------9 ,' --------------------9 ,3 (+ontrol ;$) " #' -------------------9 ( --------9 ,5 " #3 -----------------------------------9 , < (+ontrol ;&) =,T-: 0ore experimental treatments may be used but two groups (pretest > no pretest) must be used for each treatment. "act+rial #esi%ns "act+rial $esi%ns - experimental designs in which all levels of each independent variable are ta8en in combination with levels of the other independent variable(s). ! re4uires a minimum of two independent variables each having at least two levels--a & x & factorial ! may have various numbers of independent variables each with a different number of levels (e.g. & x 3% & x ' x 5--two teaching methods% three ability levels% and five grade levels% etc.) ! the total number of groups involved in the design is the product of the number of levels of each independent variable (& x ' has six different groups)--therefore% the number of groups increases rapidly as more variables/treatments are added ! at least one of the independent variables must be experimental? the others may be organismic or classification variables Cells - the groups that receive the various levels of the independent variables.

@dvantages of factorial designs: ! provide the economy of single designs rather than separate designs for each independent variable ! permit the researcher to investigate the interaction between the variables

! permits the researcher to manipulate or control more than one independent variable at a time--may include relevant factors as independent variables in the design (enhances control) Interacti+n - an effect on the dependent variable such that the effect of one independent variable changes over the levels of another independent variable. ! interaction is present if the 2oint effect of two independent variables is not e4ual to their separate (additive) effects "irst)+r$er interacti+n ) two independent variables interacting =,T-: +omplex factorial designs should be attempted with caution since increasing complexity: ! re4uires increasing large numbers of sub2ects ! ma8es interpretation of results more difficult or impossible Repeate$ Meas&res #esi%ns Repeate$ meas&res $esi%ns - experimental designs in which the same sub2ect is measured more than once on the same dependent variable. ! a series of measurements are made rather than one measurement on the dependent variable ! can incorporate any of the designs previously mentioned ! in its simplest form all the experimental treatments are administered to all the sub2ects ! observations from different sub2ects are independent of each other% but observations of the same sub2ect over time are =,T independent of each other A$ -----9 ($, -----9 (&, ---------------9 (8, A& -----9 ($, -----9 (&, ---------------9 (8, B ? An -----9 ($, -----9 (&, ---------------9 (8, A specific sub2ect (total n sub2ects) 8 number of treatments , measurement ta8en after each treatment C+&nterbalance$ #esi%ns C+&nterbalance$ #esi%ns - special cases of repeated measures in which the order of administering experimental treatments is varied according to some plan. ! all sub2ects (or groups) enter into all treatments% but in different orders

! allows the researcher to determine if one of the treatments differs significantly from the others Latin S*&are - counterbalanced design in which each number or letter appears only once in each column. -xample: $ ' & & $ ' ' & $

THREATS T! EXPERIMENTAL 2ALI#IT1 -xperimental design - should attempt to account for all possible threats to experimental validity to as great a degree as possible. ! random selection - avoids selection bias ! random assignment - enhances group e4uivalency ! controlling extraneous variables by including them as independent variables with 8nown values

/nternal )alidity $.Hist+r. - unanticipated events occurring during the progress of an experiment affect the dependent variable. &.Mat&rati+n - processes operating within the sub2ects as a function of time affect the dependent variable. '.Testin% - ta8ing one test (one treatment) has some effect on the outcome of subse4uent tests (treatments). 3.Instr&mentati+n - inconsistent use of the measurement instrument affects the dependent variable. 5. Statistical Re%ressi+n - an effect caused by the tendency for sub2ects selected on the basis of extreme scores to regress toward an average performance on subse4uent tests. <.#i--erential Selecti+n +- S&b'ects 3Selecti+n /ias4 - an effect observed when groups of sub2ects are not e4uivalent at the beginning of an experiment. 6.Experimental M+rtalit. - an effect due to sub2ects dropping out of an experiment on a nonrandom basis.

7.Selecti+n)Mat&rati+n Interacti+n - an effect of maturation not being consistent across groups due to some selection factor. -xternal )alidity $. Testin% Interacti+n - pretesting interacts with experimental conditions such that results cannot be generali1ed to a non-pretested population. &. Selecti+n /ias)Experimental Treatment Interacti+n - selection of (none4uivalent) intact groups interacts with the experimental treatment to produce results that would not have occurred in randomly selected groups (or in the general population). '. Experimental Arran%ement E--ects - an effect due to sub2ects 8nowing they are participating in an experiment (.awthorne -ffect). 3. M&ltiple)Treatment Inter-erence - an effect due to the carry over for sub2ects receiving two or more experimental treatments over time. The effects may not be generali1able to a single-treatment experiment.. /nterpreting "esults of -xperiments: RAN#!MNESS AN# REPRESENTATI2ENESS Ran$+m Selecti+n5Assi%nment - provides some assurance that a sample is representative of the population from which it was drawn. Cut - one must consider whether the population from which the sample was drawn is representative of a larger more general population.