EDUCATIONAL & PSYCHOLOGICAL

MEASUREMENT & EVALUATION SGDY 5063

LECTURER:
DR.S.KANAGESWARI A/P SUPPIAH SHANMUGAM

PREPARED BY :
KANMANI A/P PALAMALAI (823542)
KAVITHA A/P KUPUSAMY (823543)
 CRITERIA FOR EVALUATING ITEM, TEST AND
MEASUREMENT

7.1 EVALUATE MULTIPLE CHOICE

OPTIONS USING DISTRACTOR
ANALYSIS
7.2 EVALUATE TEST ITEMS USING ITEM
ANALYSIS
 Item Analysis
• The effort to improve individual questions after
they are used
• Process of examining answers to questions in order to
assess the quality of those items and of the test

• Items to be analyzed must be valid measures

of instructional objectives
• Items must be diagnostic
• Selecting and rewriting items on the basis of item
performance data improves effectiveness of items
and improves validity of
scores
 Purpose
• Improves items used again in later tests

• Eliminates ambiguous or misleading items in a

single test administration
• Increases instructors' skills in test
construction
• Identify specific areas of course content which
need greater emphasis or clarity
 Distracter Analysis

Distractor

 The distractor is analyzed in multiple

choice tests to determine whether it works
or not.

 A distractor is important in multiple choice

test so the used item serves a distractor
as defined as a distractor. (Haladyna &
Downing-1989)

https://languagetestitemanalysis.weebly.com/content/distractor-analysis
 1. Item Difficulty Index

 Percentage of students who answered a

test item correctly

p-value
• Occasionally everyone knows the answer

An unusual high level of success may be due

to:
a) Previous teacher

b) Knowledge from home; child’s background

c) Excellent teaching

d) Poorly constructed, easily guessed

• Low scores

– Is it the student’s fault for “not trying”?

a) Motivation level

b) Ability of teacher to get a point across

c) Construction of the test item

p = total who answered correctly
total taking the test

*p is the difficulty index

p = 22 get the correct answer
25 students take the test

p = ? 0.88

*p is the difficulty index

 p = 0.88

88% of the students got it right

high difficulty index
a) item was too easy
b) students were well taught

*p is the difficulty index

Sample Problem:
In a Math test administered by Mr.Keshu, seven
students answered word problem #1 correctly. A
total of twenty-five students took the test.

What is the difficulty index for word

problem #1?
 p = 0.28
low difficulty index
low difficulty level at p < 0.30

a) students didn’t understand the

concept being tested
b) item could be badly constructed
EXAMPLE 2
A B C* D
3 0 18 9

p = total who answered correctly

total taking the test

p= 18 = .60
30
 The item was moderately difficult.

60 % of the class got it right.

Option B ought to be modified or

replaced.
 2. DISCRIMINATION INDEX
• Arrange the papers from highest to lowest score.

• Separate the papers into upper group and lower

group.

• For each item , count the number in the upper group

and the number in the lower group.

• Record information
 Example for item X
(Class size = 30)
Options A B C* D
Upper 1 0 11 3
Lower 2 0 7 6

D= Number correct (upper) – Number correct (lower)

Number in either group ( if group sizes are unequal,
choose the higher number)
 D= 11 - 7 = .267
15

 Discrimination Index is .267, which is positive.

 More students who did well on the overall test

answered the item correctly than students who did
poorly on the overall test.

Difficulty index (p) = .60

Discrimination index (D) = .267
 Example for Item Y
(class size = 28)
Options A* B C D
Upper 4 1 5 4
Lower 1 7 3 3

p = total who answered correctly D= Number correct (upper) – Number correct (lower)
total taking the test Number in either group

p= 5 = .18 D= 4– 1 = .214
28 14

 Positive discrimination
 It is a difficult item
 18% of the class got it right.
 Distractor C and D attracted more students who did well on the test overall.
 These distractors modified agood item will be made even better.
 Example for Item z
(class size = 30)
Options A B*
Upper 3 4
Lower 0 10
C D
3 5
2 3

p = total who answered correctly D= Number correct (upper) – Number correct (lower)
total taking the test Number in either group

p= 14 = .467 (Moderately D= 4– 10 (negatively

difficult)
= - .40 discriminating)
30 15

 Negatively discrimination
 It is a moderately difficult.
 47% of the class got it right.
 Shoul be eliminated.
 MISKEYING
 Students who did well will select an option that is a distractor
rather than the option.
 Exp:
Who was the first astronaut to set foot on the moon?
A. John Glenn
B. Scott Carpenter
C. Neil Amstrong
D. Alan Sheppard
Analyzing the response
A B C D*
Upper half 1 1 9 2

 Most students in the upper half of the class fail to select the keyed
option.
GUESSING
AMBIGUITY