Professional Documents
Culture Documents
Introduction
Planning a Test
Planning a Test
Blooms Taxonomy
Multiple Choice
Matching
SUPPLY TYPE
Short Answer
Structured Response
Free Response
Completion/Cloze
Essay
Application
MCQ
Short Answer
Essay
Structured Response (MCQ, TF, Likert
Scales)
Free Response (Essay, Interview Qs, Fill
in the blanks)
Performance
Synthesis
Analysis
Evaluation
MCQ
Short Answer
Essay
Free Response
Great Depth
Quick Scoring
Free
Response
Advantages
Provide full and
independent responses
gives a better
indication of the
competence and
intelligence of the test
Disadvantages
Limited Depth
Difficult to assess higher
levels of skills
Guessing/Memorization vs.
Knowledge
Disadvantages
Gap-filling
Matching
fat
young
dangerous
short
old
tall
thin
safe
True-False
Open
Questions
Error
Correction
Etc.....
(other Test
types)
Translation
Essay
Dictations
Sequencing/ordering of words /
phrases
Sentence construction
10
Multiple-Choice Tests
11
When to Use
To assess breadth of learning
To test a variety of levels of learning
When you have a large number of individuals
taking the test
When you have time to construct the test items
When time is limited for scoring
When it is not important to determine how well
individuals can formulate their own answer
When you want to prepare individuals for future
assessments that use a similar format
Multiple-Choice Tests
Students select the correct answer from
alternative responses. Each item has:
item stem
correct or keyed option
several distractor options
Format:
complete question
incomplete question
14
15
Constructing Items
MCQ Responses should:
Be plausible/logical/reasonable
Be grammatically consistent with the stem
Be approximately the same length or other logical order
(alphabetical, numerical, time series..)
Make alternatives mutually exclusive (i.e. do not overlap)
Provide only one correct answer
Avoid all or none of the above
Incorrect answers (distractors) should not be too obvious but
should be justifiably incorrect
When composing incorrect alternatives, include any
misconceptions that students are likely to hold
Do not lift a statement exactly from the textbook
16
Advantages
Multiple-choice items can provide:
versatile in measuring all levels of cognitive
ability,
highly reliable test scores,
scoring efficiency and accuracy,
objective measurement of achievement or ability,
a wide sampling of content or objectives,
a reduced guessing factor compared with truefalse items,
different response alternatives which can provide
diagnostic feedback.
(Ory & Ryan, 1993)
17
Limitations
Multiple-choice items:
are difficult and time-consuming to construct,
lead an instructor to favour simple recall of
facts,
place a high degree of dependence on the
students reading ability and instructors
writing ability, and
are particularly subject to clueing. (Students
can often deduce the correct response by
elimination.)
18
Objectives at Different
Levels
Level: Knowledge
19
20
21
Essay Questions
Provide
Allow
Essay Questions
Types of Essay Questions
Extended Response Question
Great deal of latitude on how to respond to a
question.
Example: Discuss essay and multiple-choice
type tests.
Restricted Response Question
More specific, easier to score, improved
reliability and validity
Example: Compare and contrast the relative
advantages of disadvantages of essay and
multiple choice tests with respect to:
reliability, validity & objectivity
23
24
Essay Questions
Advantages:
25
Essay Items/Questions
Disadvantages
May limit the sampling of material covered
Tends to reduce validity of the test
Subjective unreliable nature of scoring
halo effect good or bad students
previous level of performance
Written expression
Handwriting legibility
Grammatical and spelling errors
Time Consuming
26
27
Alternative Methods of
Assessment
Research/Term Papers
Research Reviews
Reports
Case Studies
Portfolios
Projects
Performances
Peer evaluation
Simulations
28
29
2.
Cue students to the material from texts and class that forms
the basis of your questions.
3.
4.
30
Tutorial Task
1.
31
Resources
1.
2.
3.
4.
5.
6.
7.
32
Further Reading
Svinicki, M. & McKeachie, W. J. (2011). McKeachie's teaching tips: Strategies,
research, and theory for college and university teachers. Belmont, CA:
Wadsworth, Cengage Learning.
University of Oregon, Teaching Effectiveness Program. Writing Multiple Choice
Questions that Demand Critical Thinking. Web site: h
ttp://tep.uoregon.edu/resources/assessment/multiplechoicequestions/mc4cr
itthink.html
University of Minnesota, Office of Measurement Services. Writing Multiple
Choice Items. Web site:
http://oms.umn.edu/fce/how_to_write/multiplechoice.php
University of Texas at Austin, Instructional Assessment Resources. Writing
Multiple Choice Items. Web site:
http://www.utexas.edu/academic/ctl/assessment/iar/students/plan/method
/exams-mchoice-write.php
33
REFLECTION
A good test needs an instrument of quality.
An instrument of quality needs items that are
of quality.
Items that are of quality need item builders
who are of quality.
DO YOU HAVE ALL THOSE QUALITIES?
34
ISL
MCQ: Writing Item
Develop as many effective choices as you
can, but research suggests three is
adequate.
Make sure that only one of these choices is
the right answer.
Vary the location of the right answer
according to the number of choices
Place choices in logical or numerical order.
Keep choices independent; choices should
not be overlapping.
(Haladyna, Downing &
Rodriguez, 2002)
35
ISL
MCQ: Writing Item
Phrase choices positively; avoid negatives such
as NOT.
Avoid giving clues to the right answer, such as:
36
ISL
MCQ: Writing Item