1

Floience Nillei Paisey

Septembei 22, 2u1S


!""# %& '(()* +,"(-./0(

12 !3)- .( -3" 45/6"(( /7 /4"5)-./0)8.9)-./0: !3)- .( ;")0- <* )0 .0(-5,;"0-
)0= >3)- 5/8" =/"( .- ("5?" .0 -3" /4"5)-./0)8.9)-./0 45/6"((:

0peiationalization piesupposes that the ieseaichei has iuentifieu anu uefineu a
concept oi tiait. Foi example, peihaps one wants to measuie the quality of life in a
paiticulai city. Quality of life is a multifaceteu concept with numeious uefinitions.
0ne woulu neeu to establish one's concept of quality of life anu iuentify its
numeious uimensions, attiibutes, oi vaiiables. Each of the uimensions oi vaiiables
uefining quality of life neeus an empiiical uefinition. Such an empiiical oi
opeiational uefinition woulu involve the activities anu behaviois that eviuence
quality of life, as conceptualizeu. Neasuiing quality of life woulu be a complex
unueitaking, though vaiious inuices exist anu aie cuiiently employeu as measuies.
!"#$%&'()%*'+%&'() '- &.# %$&'/0*%&'() (1 % /()/#"& 2-0/. %- 30%*'&4 (1 *'1#5 &.%& /%) 6#
#7"'$'/%**4 ($ )07#$'/%**4 7#%-0$#8 9 &.# /()-&$0/& '- 8#1')#8 %//($8'): &( &.#
%/&';'&'#- %)8 "#$1($7%)/#- &.%& #;'8#)/# '&<
A simplei case in point might involve measuiing anxiety among giauuate
stuuents. Fiist, one woulu establish a uefinitive concept of anxiety. Then one woulu
neeu to tianslate this constiuction into quantitative measuies. Foi example,
heighteneu bloou piessuie anu heait iate, musculai tension, anu palpitations might
opeiationally uefine anxiety. Inuiiect measuies of anxiety woulu incluue self-iepoit
suiveys, questionnaiies oi uiiect behavioial obseivation. In oiuei to examine levels
2
of anxiety among giauuate stuuents, a baseline oi uepenuent vaiiable woulu be
establisheu. Then, peihaps ovei the couise of a yeai's stuuy, at specific inteivals,
physiological measuies woulu be taken anu iecoiueu.
@0(-5,;"0-(
Instiuments aie tools useu to measuie the activities that opeiationally uefine a
concept. vaiious types of instiuments exist - physical measuies incluue those that
measuie physiological activity such as bloou piessuie, muscle tension, galvanic skin
iesponse (uSR), anu tempeiatuie. A few instiuments employeu to measuie these
activities incluue theimometeis, electiomyogiaphics (ENu) anu electiocaiuiogiams
(ECu) as well as electioencephalogiams (EEu).
Behavioi measuies incluue obseivation - uiiect oi inuiiect. Self-iepoit measuies
incluue suiveys, questionnaiies, oi polls. Theie aie basically thiee types of self-
iepoit measuies: behavioial (subjects aie askeu to iepoit on theii behaviois)
cognitive (subjects aie askeu what they think of something), anu affective (subjects
aie askeu how they feel about something oi how they iesponu in paiticulai
situations).
A2 @="0-.7* -35"" -*4"( /7 .0(-5,;"0-( )0= .="0-.7* -3".5 (-5"0B-3( )0=
>")#0"(("(2

Thiee types of instiuments woulu incluue the galvanic skin iesponse (uSR),
inuiiect obseivation, anu an affective questionnaiie. Each of these instiuments has
stiengths anu weaknesses. The uSR is an instiument that monitois physiological
activity. This soit of instiument is best useu in psychophysiological stuuies wheie
the stimulus woulu elicit an automatic iesponse fiom the autonomic neivous
system. This instiument offeis about the most objective uata possible. Bowevei,
S
iesponses coulu be influenceu oi biaseu by the paiticipant's awaieness of being
stuuieu - the Bawthoine Effect. In auuition, while the uSR measuies aie vieweu as
optimally objective, vaiiations in paiticipant's intelligence can also intiouuce bias.
This effect is moie often uiscusseu with obseivational instiuments, howevei a lab
anu the awaieness of it often intiouuce bias. Anothei weakness is the cost anu
inconvenience involveu with stuuies involving electionic instiuments in a specific
location.
A self-iepoit measuie oi instiument woulu incluue an affective questionnaiie.
This instiument involves piesenting a questionnaiie to a paiticipant anu iequesting
the paiticipant to wiite answeis. An affective questionnaiie piesents questions that
ielate to how one feels when in specific situations. Theie aie many issues connecteu
with questionnaiies anu self-iepoiting in geneial. Fiist, have items on the
questionnaiie been vetteu oi analyzeu foi theii constiuct valiuity. Bo they, in fact,
measuie the emotion (oi skill) they puipoit to measuie.
In auuition to this impoitant weakness, self-iepoits iely on a peison's
tiuthfulness (face valiuity). Aie the iesponses given a iesult of the Bawthoine Effect
oi, peihaps, aie they uisingenuous. Anu, finally, uoes the paiticipant peiceive theii
iesponse accuiately oi iecall a paiticulai event anu the way one iesponus
accuiately. These issues play into any uecision in auministeiing self-iepoits. Bo the
auvantages of convenience, the potential of obtaining a laige sample anu stuuying
numeious vaiiables, outweigh the many potentially ciitical weaknesses that coulu
unueimine the stuuy.
4
The thiiu measuie woulu involve obseivational measuies such as obseiving how
a paiticipant behaves. 0bseivation can be eithei uiiect oi inuiiect - uiiect
obseivation involves iecoiuing how paiticipants behave in a paiticulai
enviionment. The ieseaichei can constiuct the enviionment oi one can obseive
paiticipants in the "natuial" enviionment. Case stuuies woulu fall into this categoiy
of obseivation - whethei uiiectly obseiveu oi stuuieu thiough jouinals,
coiiesponuence as well as auuio anu viueo iecoiuings.
Content analysis is one methouology associateu with inuiiect obseivation. With
content analysis - as with any obseivational measuie - specific featuies of a iecoiu
aie assigneu values; they aie coueu. Reseaicheis uevise vaiious couing systems;
fiom simply counting the numbei of times a paiticipant behaves in a paiticulai way
oi establishing categoiies of behavioi anu uesciibing obseivations within these
categoiies. Content analysis, an inuiiect obseivational methouology, involves the
analysis of wiitten uocuments baseu on the woius employeu. uoiuon Allpoit, a
piominent 2uth centuiy psychologist peifoimeu the fiist psychological stuuy
("Letteis fiom }enny") using content analysis. Be stuuieu the coiiesponuence of one
of his patients in an effoit to unueistanu hei peisonality anu the pathology she was
expeiiencing. Like Allpoit, one can set up categoiies of woius - categoiies of woius
expiessing happiness, loneliness, love, sauness - anu analyze uocuments
accoiuingly.
C2 D.6# -35"" -*4"( /7 ?)8.=.-*2 'E48).0 )0= 6/;4)5" )0= 6/0-5)(- ")632 F/> =/
")63 /7 -3"(" 5"8)-" -/ 5"(")563 ="(.B0:

It is essential that instiuments puipoiting to measuie a paiticulai featuie oi tiait
actually quantify oi measuie that featuie. This is the basic issue that valiuity
S
auuiesses. If an instiument claims to measuie stiess in giauuate stuuents, it is
ciucial that the uata collecteu in using the instiument ieflects oi inuicates stiess in
giauuate stuuents. Cleaily, behaviois associateu with stiess aie the inuepenuent
vaiiables oi attiibutes anu ieseaicheis in this soit of stuuy woulu be looking at anu
measuiing these behaviois. vaiious types of valiuity exist.
Face valiuity simply means that an instiument seems to measuie what it says it
measuies. With self-iepoit measuies face valiuity also involves whethei
paiticipants' answeis aie factual oi authentic. Closely ielateu to this is whethei
paiticipants iecall expeiiences accuiately. Face valiuity is easily confuseu with
content valiuity. Bowevei, wheie face valiuity meiely seems to measuie the
vaiiables oi tiaits iuentifieu, content valiuity involves coiioboiation fiom expeits in
the fielu. If a ieseaich stuuy aims to measuie cybei-bullying among auolescents, the
opeiational uefinition anu measuies set up to quantify cybei-bullying shoulu
accuiately measuie the tiait - anu not simply uiscouitesy oi an isolateu episoue of
angei stateu. In oiuei to valiuate the instiuments' content measuies, an expeit oi
expeits in cybei-bullying woulu examine the instiument anu veiify whethei,
accoiuing to theii concept, the measuiing tool achieves its puipose.
Ciiteiion-ielateu valiuity ielates to compaiing the iesults of a newly uevelopeu
measuie with anothei one that has alieauy been valiuateu. Theie aie two soits of
ciiteiion-ielateu valiuity: concuiient valiuity anu pieuictive valiuity. If iesults fiom
a measuie coiielate with cuiient oi "contempoianeous" peifoimance, concuiient
valiuity woulu have been substantiateu. This soit of measuiement is aimeu to
uesciibe an existing conuition. Bowevei, if the measuie aims to pieuict futuie
6
peifoimance oi behavioi - potential high school uiopouts - the test is aiming to
pieuict oi inuicate potential oi futuie conuitions anu behavioi
%2 !3)- .( -3" =.77"5"06" <"->""0 ?)8.=.-* )0= 5"8.)<.8.-*: !3)- )5" -3"
=.77"5"0- >)*( -3)- >" 6)0 -"(- -/ ("" .7 /,5 5"(,8-( )5" ?)8.=: !3)- )5" -3"
=.77"5"0- >)*( -3)- >" 6)0 -"(- -/ ("" .7 /,5 5"(,8-( )5" 5"8.)<8":

valiuity pioceuuies aim to ueteimine if an instiument measuies what it
unueitakes to measuie. vaiious foims of valiuity, each with a uiffeient puipose,
exist. Reliability iefeis to an instiument's stability oi consistency of iesults.
Reseaicheis establish each foim of valiuity as well as ieliability thiough uiffeient
methous. As explaineu above, content valiuity may be coiioboiateu with expeit
examination of the instiument anu veiification that the instiument has items
iepiesentative of the uomain. Ciiteiion-ielateu valiuity is of two types: concuiient
anu pieuictive. Reseaicheis establish concuiient valiuity anu uesciibe piesent
behaviois by compaiing the peifoimance of the instiument cuiient behaviois. In
contiast, ieseaicheis ueteimine pieuictive valiuity oi the capability of the test to
pieuict futuie peifoimance. In a sense both means of testing valiuity aie uiagnostic
- one compaies test iesults to cuiient peifoimance, the othei compaies iesults to
futuie behaviois - it uesciibes the potentiality of behaviois.
Reliability iefeis to the stability anu consistency of a test. This is measuieu with
coiielation coefficients that ieveal the stiength of the ielationship between two sets
of tests oi tieatment iesults. Theie aie seveial ways of achieving ieliability. The
most common is test-ietest ieliability. This pioceuuie simply involves
auministeiing a test to a specific gioup oi set of paiticipants. Then, waiting foi a
piesciibeu peiiou of time anu auministeiing the same test again. If the test is
7
ieliable, the iesults foi each paiticipant shoulu be compaiable. The vaiiables
measuieu woulu coiielate negatively oi positively -1.uu oi +1.uu. The closei the
ielationship oi coefficient is to u, the weakei the ieliability. ueneially, a stiong
coiielation woulu be aiounu .8 to .9.
Anothei means of establishing ieliability anu contiolling foi issues involveu with
testietest ieliability. Pioblems with testietest ieliability involve paiticipant iecall
- oi peihaps paiticipants iemembei coiiect items fiom the fiist iounu of the test. In
this case, paiticipants might give the same answeis again simply because they
iemembei the coiiect answeis, not necessaiily because the test is consistent.
Anothei issue involves the ieality that uuiing the inteiim peiiou between the two
tests (testietest) a ciisis oi illness coulu have occuiieu among some inuiviuuals.
This woulu affect these paiticipants peifoimance - not because the test is unstable,
but uue to paiticipant pioblems.
0ne means of contiolling foi testietest ieliability, paiticulaily if paiticipants tiy
to iemembei some answeis, is to offei an alteinative test.
G2 H"(65.<"I "E48).0I )0= .88,(-5)-" -3" -35"" 7/5;( /7 ;")(,5";"0- "55/5&
5)0=/;I (*(-";)-.6I )0= <.)(2

All foims of measuiement eiioi will cause an eiioi in the expeiiment anu
piouuce eiioneous uata oi iesults. Ranuom eiioi iefeis to any uniuentifieu anu
uncontiolleu vaiiable that affects the uepenuent vaiiable. These uncontiolleu
vaiiables often ielate to inuiviuual uiffeiences with the subject pool. In teims of
euucation, take a class of piofessional miuule-Eastein men who tiaveleu to Englanu
in oiuei to leain English, as iequiieu foi theii woik. The teachei assigneu is
expeiienceu with a lauuatoiy iecoiu of fosteiing stuuent achievement. Bowevei,
8
aftei one week of teaching this class, the miuule-Eastein men saiu they hau leaineu
nothing anu uemanueu anothei teachei. What aie the vaiiables. The inuepenuent
vaiiable is the piogiam of teaching. The uepenuent vaiiable is uegiee of
achievement. The ianuom, uncontiolleu vaiiable, causing eiioi, is the instiuctoi's
genuei anu, possibly, even nationality.
Systematic eiioi (noise) like ianuom eiioi iefeis to an uncontiolleu vaiiable,
acting on the uepenuent vaiiable, anu skewing the iesults of a ieseaich stuuy. A
systematic eiioi woulu piouuce an eiioi uue to a pooily uesigneu testing
instiument. Peihaps, with iegaiu to euucation, one auministeis an English test to
ueteimine at what level of English stuuents shoulu be placeu. 0nfoitunately, the test
is not coiielateu with the English cuiiiculum. So, stuuents often pass the test at a
high level, yet cannot peifoim at a high level. The eiioi is systematic anu will
continue until, in this situation, the testing instiument anu cuiiiculum aie attuneu.
Bias is a foim of systematic eiioi, but bias is usually inheient in the expeiimental
uesign. Foi example, a sample that is too small; the ieseaichei has a peisonal bias oi
opinion anu the paiticipant picks it up. These uncontiolleu vaiiables skew an
expeiiment oi stuuy, piouucing misinfoimation.
J2 'E48).0 )0= 6/;4)5" )0= 6/0-5)(- ")63 /7 -3" 7/,5 8"?"8( /7 ;")(,5";"0-&
0/;.0)8I /5=.0)8I .0-"5?)8I )0= 5)-./2 K.?" "E);48"( /7 ")632

Levels of measuiement iefei to the piopeities oi the kinu of the uepenuent
vaiiable one is examining. Each level of measuiement is contingent on what kinu of
uata one stuuying. Nominal uata iefeis to categoiical uata - the soit of uata oi
uepenuent vaiiables that cannot be oiueieu in a consequential way. If one weie
stuuying a gioup of 1,uuu people anu looking who is employeu anu who isn't, oi
9
who uiives anu who uoesn't, this woulu be a categoiical measuie. The only piopeity
oi attiibute one is inteiesteu in is eithei employeu oi uiiving. 0ne coulu take a
gioup of high school stuuents anu categoiize them nominally by who takes AP
classes anu who uoesn't. Theie can be numeious categoiies.
0iuinal vaiiables, unlike nominal vaiiables aie categoiical, but they have an
appiopiiate oiuei. If one is looking at those employeu, one coulu also look at income
levels. So that within the categoiy of employeu, uoes one eain moie oi less than a
paiticulai amount. Such vaiiables can be oiueieu along a continuum fiom the most
to the least set of values, though the uistance between values is iiielevant. These
values may be oiueieu as a scale.
Inteival measuies oi scales have a set uistance oi equal anu quantifiable
uiffeience among attiibutes oi featuies of a vaiiable. An example often useu is a
theimometei wheie each numbei on a theimometei iepiesents a paiticulai
piopeity (tempeiatuie) that is oiueieu (oiuinal) in equal inteivals (The uistinction
between 1S anu 16 is the same as the uistinction between 11S anu 116. Each is
uiffeient by only one uegiee).
Ratio measuies have all the piopeities of inteival measuies with an auuitional
piopeity - that of zeio. The attiibutes that make up a vaiiable aie baseu on "u." Foi
example, if we aie measuiing speeu uuiing iush houi anu a cai's speeuometei
uisplays zeio, this means theie is a complete absence of speeu oi movement. The cai
is eithei tuineu off oi iuling. Eithei way, theie is an absence of the uepenuent
vaiiable "speeu." Ratio measuies incluue nominal, oiuinal, anu inteival
chaiacteiistics anu also allow one to measuie fiom zeio to infinity.