Fall2004MastersComprehensive/PhDPreliminaryExam

September17,2004
MorningSession
Measurement
Measurement1.
(a)
Briefly define three types of validity as discussed in the Standardsfor
educationalandpsychologicaltesting.
(b)
Provide an example of a situation where construct validity methods might
be applied.
(c)
Describe research and/or statistical methods typically used in determining
or demonstrating construct validity.
(d)
Discuss limitations or problems commonly associated with applications of
construct validity.

Measurement2.
(a)
Whenassessingthequalityoftestitems,itiscommontocalculatethe
relationshipbetweengettingtheitemcorrectorincorrectandthetotaltest
score.Nametwocorrelationsthatsatisfythispurpose.
(b)
Compareandcontrasttheassumptionsofthesetwocorrelations.
(c)
Discusstheadvantagesanddisadvantagesofusingeachcoefficient.
(d)
Underwhatconditionsisonecoefficientdefinitelytobepreferredtothe
other?
Measurement3.
(a)
Produce and label a graph representing an item characteristic curve
resulting from the 3-parameter logistic item response theory model.
(b)
Briefly define each parameter in the model.
(c)
Produce a second graphical representation for which the values of all 3
parameters have increased in relationship to your initial graph.
(d)
Provide a practical interpretation of an increase in the value for each
parameter.

Measurement4.
(a)
Distinguishbetweenauniverseofadmissibleobservationsassociatedwith
aGeneralizabilitystudyandauniverseofgeneralizationassociatedwitha
DecisionstudyinGeneralizabilitytheory.
(b)
SupposeaninvestigatorconductsapxtxrGstudyusingnp=100persons,
nt=5essayitems,andnr=6raters.Whatvariancecomponentsmaybe
estimatedbysuch a design?
(c)
WhatwouldbetheinterpretationoftheGeneralizabilitycoefficientfrom
thisdesignandhowwoulditbecalculated?(Youdonothavetoactually
calculatethecoefficient.)
ResearchMethods
ResearchMethods1.
The analysis below represents the relationship between 3 predictors (X1, X2, and X3)
and a criterion variable (Y).
CorrelationCoefficients
YX1X2X3
Y1.0000.1965.1620.1389
X1.19651.0000.6484.0468
X2.1620.64841.0000.0038
X3.1389.0468.00381.0000
VariablesintheEquation
BSEBBetaTSigT
X2.069157.046243.0673901.495.1352
X3.644750.167312.1323393.854.0001
X1.184873.056871.1466433.251.0012
(Constant)10.825611.58250218.585.0000

In the context of the results above:

(a)
(b)
(c)
(d)

Discuss the difference in meaning and interpretation of B (raw) and

(Beta; standardized) regression weights.
Discuss how the equation weights above relate back to the correlation
matrix.
Discuss the predictor contributions in this equation.
How would you expect the equation weights for X2 and X3 to change if
X1 is removed from the analysis?

ResearchMethods2.
AstudywasmadeofasampleofEDMS645students.Thestudentsinthestudywereall
1stsemestergraduatestudentsandweredistributedacross7Universitydepartments.
Studentswererandomlydistributedto3groups.Overthefirst5weeksofasemester,
group1wasgiveneasyproblemassignments,group2receivedmoderatelydifficult
problemassignments,andgroup3receivedverydifficultassignments.Thegroupswere
thencomparedontheirselfratingsoftheusefulnessofstatistics.Noattemptwasmadeto
assessorcontrolforstudentdifferencesinmathbackground.Somestudentsreceived
specialstatisticstutoringduringthe1st5weeksofthesemesterunbeknownsttothe
researcher.
(a)
(b)
(c)
(d)

Discusspotentialthreatstointernalvalidityposedbythisstudy.
Describetwowaystodealwiththethreatsdiscussedina.
Discusspotentialthreatstoexternalvalidityposedbythisstudy.
Describetwowaystodealwiththethreatsdiscussedinc.

Fall2004MastersComprehensive/PhDPreliminaryExam
September17,2004
AfternoonSession
ResearchMethods,continued
ResearchMethods3.
A research design may include one or both of these random procedures: (i) random
sampling from some specified existing population and (ii) random assignment of
experimental participants to treatment levels within the experiment. Discuss how the use
of each of these procedures contributes to the meaningfulness and interpretation of the
research outcome.
ResearchMethods4.
SupposethataresearcherisdoingastudyoftheeffectsofArtisticTrainingon4thgrade
studentsinVirginia.TenschoolsvolunteerandshegetsagreementstohavetheArt
teachersprovidethetrainingthatshehasprepared.AtestofArtKnowledgeisalso
developedandwillbeadministeredatthebeginningandattheendofthearttraining.She
alsoknowsthatsheshoulduseacontrolgroup,buttheVirginiaschoolswillnotlether
randomlyselectschools.Sosheselectscontrolschoolsthatareassimilartothe10
experimentalschoolsaspossible,usingasetof3variables(averageSESofeachschool,
Statetestaverageforeachschool,andpercentageofnonnativeEnglishspeaking
studentsineachschool).Shewilltestallthe4thgradestudentsinthose
"control/comparable"schoolsatthesametimethatsheteststhestudentsfromthe
experimentalschools.
(a)
(b)
(c)
(d)

Whataretheriskswiththisapproachtofindingacomparisongroup?
Whatdoesshegainfromgettingacomparisongroupinthisway,
comparedtonothavingacomparisongroupofstudentsatall?
Whatstatisticalanalysisshouldsheusefortheresearchdesigndescribed?
Defendyourchoiceofanalysisanddiscusstheadvantagesand
disadvantagesoftheanalysisyourecommend.
Discusstheadvantagesanddisadvantagesoftheresearchdesigndescribed
comparedtousingrandomlyselectedschoolsastheexperimentaland
controlschools(i.e.,fullrandomselection).

Statistics
Statistics1.
Suppose that a directional z test is used with =.05 to test
Ho: 50
against
H1: > 50,
where = 10 and n = 25.
(a)
(b)
(c)

Sketch the sampling distributions of the mean for = 50 and for = 55 on

a common axis.
Label appropriate regions for Level of Significance; 1 ; ; and Power.
Explain how the sketch would change if the sample size was increased.

Statistics2.
Thereisalotofdiscussiongoingoninpsychologyabouttheusefulnessofclassical
hypothesistestingasanindicationoftheimportanceofresults.Ifonestudyrejectsthe
nullhypothesisatthe.01levelandanotherstudyrejectsatthe.05level,isthefirst
indicativeofastrongereffect?
(a)
Discusstheconcernwithdeterminingimportancefromclassicalhypothesis
testing?Howdosamplesizeandstandarderrorcontributetothis
discussion?
(b)
Whatindicesbesidesstatisticalsignificancecouldastatisticianutilizeto
describethemagnitudeofaneffect?

Statistics3.
In experimental applications, a two-way ANOVA with one experimental factor and one
blocking variable and an ANCOVA (or ordered regression analysis) with a covariate or
quantitative control variable are both ways of reducing experimental error.
(a)
(b)

Contrast the research situations in which each would be appropriate

Explain how each procedure leads to error reduction

Statistics4.
Aresearcherwondersifstudentsreportedstresslevels(measuredbyastressinventory)
variesoverthesemester.Sherandomlyselects15studentsfromeachofthe4main
campusesoftheUniversityofMarylandandasksthemtocompleteastressinventory
duringWeeks1,7,and14ofthesemester.Higherscoresindicategreaterstress.The
SPSSoutputfromthisstudyisbelow.
Mauchly's Test of Sphericityb
Measure: MEASURE_1
Epsilon
Within Subjects Effect Mauchly's W
TIME
.075

Approx.
Chi-Square
46.685

df
2

Sig.
.000

Greenhous
e-Geisser
.519

Huynh-Feldt
.523

Lower-bound
.500

Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is
Tests of Within-Subjects Effects
proportional to an identity matrix.
a. May
be used
to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in the
Measure:
MEASURE_1
Tests of Within-Subjects Effects table.
Type III Sum
b. Source
of Squares
df
Mean Square
F
Sig.
Design:
TIME InterceptSphericity Assumed
93.733
2
46.867
3.652
.035
Within Subjects Greenhouse-Geisser
Design: TIME
93.733
1.039
90.230
3.652
.069

Error(TIME)

(a)

(b)

Huynh-Feldt
Lower-bound
Sphericity Assumed
Greenhouse-Geisser
Huynh-Feldt
Lower-bound

93.733
93.733
487.600
487.600
487.600
487.600

1.045
1.000
38
19.738
19.863
19.000

89.663
93.733
12.832
24.704
24.549
25.663

3.652
3.652

Shouldheresearcherinferastatisticallysignificant(at=.05)TIME
effectbasedontheseresultsornot?Explainyouranswerinlightofall
relevantportionsoftheoutput,andspecificallymentionwhatdfandp
value(s)youbaseyouranswerupon.
Inthecontextofthisstudy,brieflydescribewhatthesphericity
assumptionisaboutandwhatadjustmentsaremadeifitisviolatedand
why.

.069
.071