TUTORIAL 8

Question 1: Why is it important for a teacher to analyse each of the test item?
1. Fix marks for current class that just wrote the test
o find flaws in the test so that you can adjust the mark before return to
students
o

can find questions with two right answers, or that were too hard, etc., that
you may want to drop from the exam

2. More diagnostic information on students

o another immediate payoff of item analysis
Classroom level:
o

will tell which questions they were are all guessing on, or if you find a
questions which most of them found very difficult, you can reteach that
concept

CAN do item analysis on pretests to:

so if you find a question they all got right, don't waste more time on
this area

find the wrong answers they are choosing to identify common

misconceptions

can't tell this just from score on total test, or class average

Individual level:
o

isolate specific errors this child made

after you've planned these tests, written perfect questions, and now
analyzed the results, you're going to know more about these kids than
they know themselves

3. Build future tests, revise test items to make them better

o REALLY pays off second time you teach the same course

by now you know how much work writing good questions is

studies have shown us that it is FIVE times faster to revise items

that didn't work, using item analysis, than trying to replace it with a
completely new question

new item which would just have new problems anyway

--> this way you eventually get perfect items, the envy of your
neighbours

SHOULD NOT REUSE WHOLE TESTS --> diagnostic teaching means

that you are responding to needs of your students, so after a few years
you build up a bank of test items you can custom make tests for your class

know what class average will be before you even give the test
because you will know approximately how difficult each item
is before you use it;

can spread difficulty levels across your blueprint too.

4. Part of your continuing professional development

o doing the occasional item analysis will help teach you how to become a
better test writer
o

and you're also documenting just how good your evaluation is

useful for dealing with parents or principals if there's ever a dispute

once you start bringing out all these impressive looking stats parents and
administrators will believe that maybe you do know what you're talking
about when you fail students...

parent says, I think your "question stinks",

well, "according to the item analysis, this question appears to have worked
well "
(actually, face validity takes priority over stats any day!)

and if the analysis shows that the question does stink, you've already
dropped it before you've handed it back to the student, let alone the parent
seeing it.

Question 2: What is the item difficulty for the following situations?

a. Thirtty students take a test and only twenty answered item 1 correctly.
20/30 = 0.67
b. All the students in (a) answered item 2 correctly.
30/30 = 1
c. None of the ss answered item 3 correctly.
0/30 = 0

The item difficulty index is calculated as

percentage of the total number of correct responses to
the test item. It is calculated using the formula
P=R/T, where P is the item difficulty index, R is the
number of correct responses and T is the total number
of responses (which includes both correct and incorrect
responses). An item was considered difficult when the
difficulty index value was less than 30% and the item
was considered easy when the index value was greater
than 80%.

Si-Mui Sim, Rasiah RI. Relationship between item difficulty and

discrimination indices in true/false type multiple choice questions of
a para-clinical multidisciplinary paper. Ann Acad Med Singapore
2006; 35: 67-71