Michele Mund

Technology Assessment Project 2

Three clicker questions were designed to represent level one, two, and three questions
focusing on Johnny Appleseed. While analyzing the questions I looked at two main areas: the
difficulty index and the discrimination index. Using that information I considered the quality of
the questions and answers. I took into consideration any changes that I could make for future use
of these questions.
After the questions were delivered in class and discussion was opened up about the
questions I drew some conclusions about the familiarity of the material for the participants. I
was surprised that the level of familiarity with the subject was low. Students knew a few basic
facts, but did not seem to have a deeper knowledge. I believe that this affected the scores,
especially on the level one question.
The two main areas for assessing the effectiveness of the questions is the difficulty index
and the discrimination index. The difficulty index uses a range from 0 to 1. A score of 0 would
mean that every student got the question wrong and a score of 1 would mean that every student
got the question right. The closer the score is to 1 the easier the question. A difficulty index for
a level two question should be between 0.8 and 1. The score for a level three question should be
from 0.6 to 0.8. The score for a level four question should be between 0.4 and 0.6. The
discrimination index tells you how well the students that did well on the other questions did on
this question in comparison. The discrimination index ranges from -1 to 1. A score of zero is a
mix, some got it and some didnt. A negative score means that the question was confusing or
worded badly. In general you are looking for a score above a 0.3.
The first question was a level two question. The aim of a level two question is to
determine if students have a basic understanding of the material. There were 12 students that
participated, and 50% of the students answered this question correctly. The difficulty index for
this question was 0.5. I expected the difficulty index to be much higher, at least a 0.8. I would
have expected this score for a level four question. The discrimination index for this question was
0.64. This tells me that that did well on the other questions also did well on this question. I
think that improvement for this question is found in the wording of the answers and answer
choices.
The second question was a level three question. The goal of a level three question is to
determine if the student has a complete understanding of the material. Out of 12 students 33% of
the students answered this question correctly. The difficulty index for this question is 0.33. This
is what I would expect for a level two question. The discrimination index is 0.9. The students
that did well on the other questions also did well on this question.
The third question was a level four question. The goal of a level four question is to
determine if learning has surpassed what was presented. It determines if there is a deeper
understanding of the material. There were 25% of the students that answered the question
correctly. The difficulty index for this question is 0.25. This is lower than I would expect for a
level four question. This tells me that the question was hard. This could be because of a lack of
knowledge in regards to what a folk hero is and how to apply that to modern day. The
discrimination index is 0.58. The students that did well on the other questions also did well on
this question.
Overall the questions did fairly well. The discrimination index for all three of the
questions was above 0.3 indicating that the students that scored well on a question consistently
scored well on the other questions. The difficulty index scores were not as positive. The scores
for the level two and four questions indicated that they were more difficult than you would want
a question at that level to be. However, I do not think that the scores would have indicated this if
they had been taught the material. It would have provided them the additional information that
they would have needed to differentiate between the similar answers in the level two question.