See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/304349657

Basic Principles of constructing multiple choice

questions

Article · July 2015

CITATIONS READS

0 3,023

3 authors, including:

Sitanshu Sekhar Kar Subitha Lakshminarayanan

Jawaharlal Institute of Postgraduate Medical… Jawaharlal Institute of Postgraduate Medical…
106 PUBLICATIONS 363 CITATIONS 66 PUBLICATIONS 170 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Dietary Practices and Barriers to Dietary Modification among Diabetics and Hypertensive in a Rural
Health Service Area of Puducherry - A Qualitative Study View project

Epidemiology of diabetes in rural pondicherry View project

All content following this page was uploaded by Sitanshu Sekhar Kar on 24 June 2016.

The user has requested enhancement of the downloaded file.

Medical E ducation

Basic principles of constructing Multiple Choice

Questions
Sitanshu Sekhar Kar1, Subitha Lakshminarayanan2, Mahalakshmy T2
1
Associate Professor, 2Assistant Professor, Department of Preventive and Social Medicine, JIPMER, Puducherry, India

Abstract
Multiple Choice Questions (MCQs) are one of the most important well established tools used for assessment and
selection processes in medical education. Designing good assessment tools requires time and effort and ultimately
determines the learning outcome. The quality of MCQ depends on the quality of items on the whole and presence
of efficient distractors. Flawed MCQs interfere with assessment process and therefore it is vital to develop reliable
and valid items that are free of such flaws. Basic criteria for constructing good quality MCQs include item content
that is tested and well-structured, flawless items. Three areas that are addressed in this article include general
guidelines of using MCQ inevaluation, principles of constructing effective MCQs, and constructing higher order
MCQs. Characteristics of effective MCQs are determined by the overall item, the stem and distractors. Interpretation
and problem-solving items like MCQs with data, diagrams and images, use of clinical or lab vignettes, formats like
Extended Matching items, Key feature questions and assertion reason questions require higher order thinking skills
like analysis, synthesis and evaluation. Significant commitment and technical knowhow is required to prepare MCQs
that are of good quality, reliable, able to test higher order thinking skills and consistent with curriculum objectives.
Constructing MCQs is an art which can be perfected with practice and it is essential for medical educators to be
skilled in effective test item writing to assess the learner’s knowledge.

Key words: Assessment, effective MCQs, higher order MCQs

Introduction over a broad coverage of concepts in a short span of time.

The last decade witnessed a change in assessment strategies
amongst medical teachers in India to accommodate some
degree of objectivity in written assessment.1 Multiple
choice questions (MCQs) have come to occupy a prominent
place in the student’s assessment process.2 It represents
one of the most important well established examination
tools widely used in assessment at the undergraduate and
postgraduate levels of medical examinations.3
Scoring is objective, reliable and quick using computers.
Though MCQs are useful in measuring factual recall they
can test higher order of thinking skills such as application,
analysis, synthesis and evaluation of knowledge.1,3 A few
other issues related to use of MCQs include guessing and
cueing effect by the student and lack of feedback. Faulty
items interfere with accurate and meaningful interpretation
of test scores and have an adverse impact on the assessment
process.1

MCQs are used most commonly for formative assessment and
selection processes. Appropriately constructed MCQ based
methods are efficient, objective, capable of discrimination
and can be combined with other assessment strategies to
contribute to a comprehensive student assessment strategy.4
Similar to other methods of assessment, they have their
advantages and limitations. A distinct advantage of using
MCQ is its ability to evaluate a large number of candidates
Basic structure of MCQs
The basic MCQ or ‘item’ is one in which the candidate
chooses one answer from a set of options provided (Single
best response type). An item consists of a ‘stem’ followed
by a number of options. Sometimes the stem is followed
by the ‘lead in question’. The correct answer in the list of
options is called a ‘key’ and the incorrect options are called
‘distractors’.3

Address for Correspondence: Dr. Sitanshu Sekhar Kar, Associate Professor, Dept. of Preventive and Social Medicine, JIPMER,
Puducherry,India- 605006. Email: drsitanshukar@gmail.com Received: 07.06.2015, Accepted: 22.07.2015

Indian Journal of Community & Family Medicine | Vol. 1 | Issue 02 | Jul-Dec, 2015 65
Kar S S: Basic principles of constructing multiple choice questions

ITEM Table 1. Types of Multiple Choice Questions (Hubbard

and Clemans - 1971)
Single best indicator of social STEM
development and well-being is Type of MCQ Example
a) Infant mortality rate DISTRACTORS
b) Maternal mortality rate Single or best Bengal gram is deficient in
response a) Lysine
c) 1-4 year mortality rate KEY b) Tryptophan
type
d) Under 5 mortality rate (Type A) c) Methionine
d) Leucine
Figure 1. Basic structure of an Item
Multiple True about vehicle transmission is:
Examiners constructing MCQs need to be aware of the completion 1. Common source of infection is often
different formats in order to decide the cognitive level of type traceable
(Type K) 2. Not always possible to isolate
performance required by the candidates. The MCQs are an
organism from vehicle
extensively used and time tested method of assessment 3. Many cases may occur simultaneously
of knowledge innational and international examination. 4. Higher the dose, higher the number
These are of different types as classified by Hubbard and of cases
Clemans (1971). a. 1, 2
b. 1, 2, 3, 4
Writing effective MCQs c. 1, 3, 4
d. 1, 2, 3
Designing good assessment tools requires time and effort
and ultimately determines the learning outcome. Merely Relationship Assertion: Measles vaccine should never
applying a particular tool is not enough; rather how well analysis type be given before the age of 9 months
that tool has been designed makes all the difference. A (Type E) BECAUSE
poorly written MCQ is likely to distort learning.5 Reason: The maternal antibodies
present in the child till 9 months of age
Constructing MCQs is an art which can be perfected with may neutralize the vaccine
practice. Characteristics of effective MCQs aredetermined a. Both Assertion and Reasoning are
by the overall item, the stem and distractors.Flaws related true, reason is correct explanation
to test-wiseness make it easier for some students to b. Both Assertion and Reasoning
answer the question correctly, based on their test-taking are true, reason is not the correct
skills alone; these flaws commonly occur in items that are explanation
c. Assertion is true, Reason is false
unfocused. Flaws related to irrelevant difficulty make the
d. Assertion is false, reason is true
question difficult for reasons unrelated to the trait that
e. Both Assertion and reason are false
is the focus of assessment.6 Flawed MCQs interfere with
accurate and meaningful interpretation of test scores and Extended A. Calcium B. Fluoride
negatively affect student pass rates.7 Therefore, to develop Matching C. Folic acid
reliable and valid tests, items must be constructed that are type D. Iron E. Vitamin A
free of such flaws. Guidelines that are often referred in the F. Vitamin B1 (thiamine)
context of writing MCQs include Haladyna’s guidelines8 G. Vitamin B6
and National Board of Medical Examiners (NBME) series.6 H. Vitamin B12 (cyanocobalamin)
I. Vitamin C
Two basic criteria for constructing good quality questions J. Vitamin D
include item content (covering critical areas) and well- K. Vitamin E
structured, flawless items.6 This article provides guidelines
that can be used in writing effective MCQs for selection For each child, select the appropriate
vitamin or mineral supplements.
examinations, continuing medical education materials,
1. A 1-month-old infant is brought to the
as well as routine assessment for medical student.Three physician for a well-child examination.
areas that are addressed in this article include: (a) general He has been exclusively breast-fed,and
guidelines of using MCQ in evaluation, (b) principles of examination shows normal findings.
constructing effective MCQs, and (c) constructing higher (select 2 supplements).
order MCQs. 2. A 6-year-old girl has cystic fibrosis.
She has been taking no medications.
(select 3 supplements).

Indian Journal of Community & Family Medicine | Vol. 1 | Issue 02 | Jul-Dec, 2015 66
Kar S S: Basic principles of constructing multiple choice questions

A. GENERAL GUIDELINES FOR USING MCQs IN
EVALUATION9 :
1. Scope of the examination and learning outcomes
being tested should be clear before formulating an
assessment strategy.
2. Cognitive levels being tested should be decided as the
format of MCQ may depend on this fact.
3. Sufficient number of items isrequired for a valid
MCQ examination that covers a specific topic; probably
60-100 items are optimal for an examination of 60-90
minutes duration.
4. Time allotment for examination needs due
consideration. Different item formats require different
time duration (single response - 40 seconds, whereas
case history type – 60 to 90 seconds).
5. Negative marking and guessing correction are
important issues that need to be addressed keeping
the objective of assessment in mind.
6. Fixation of pass level in MCQ examination is more
difficult than other formats. Though several formats
are available, MCQs function much better for the
purpose of ranking.
7. Validation process is an important step in formulation

Table 2. Checklist for constructing effective Multiple Choice Question items

Area DO’s DON’Ts

Content issues Base each item on important content area / learning Avoid opinion-based items.
outcome Avoid trick items.
Ensure each item is wholly independent of the
others
Frame items that are specific.
Choose items of appropriate level of difficulty
Writing the stem Ensure that the directions in the stemare very clear. Avoid negatives such as NOT or EXCEPT;
Include the central idea and common elements in if used ensure that the word appears
the stem capitalized and boldface.
Keep it clear, concise and unambiguous. Avoid double negatives.
If using a lead-in, it should clearly indicate how to Avoid vague expressions like fairly high,
answer the MCQ. considerably greater, etc.
Keep vocabulary simple. Avoid clues suggestive of the right answer

Writing the Keep choices independent/ mutually exclusive Avoid All-of-the-above.

choices/ Make sure that only one of these choices is the right None-of-the-above should be used
distractors answer. carefully.
Keep choices homogeneous in content (distractors Avoid negatives such as NOT.
are in the same category as the correctanswer). Avoid the use of specific determiners such
Make all distractors plausible. asalways, never, completely, and absolutely.
Vary the location of the right answer according to Avoid grammatical inconsistencies
the number of choices.
Arrange choices in logical or numerical order,
wherever appropriate.
Keep the length of choices about equal.
Formatting and Format the item vertically instead of horizontally. Avoid abbreviations
style issues All parts of the item should be in the same page.
Mix of optimum number of items of different levels
of difficulty.
Group similar formats together.
Edit and proof items.
Use correct grammar, punctuation, capitalization,
and spelling.
Keep it brief and minimize the amount of reading in
each item.

Indian Journal of Community & Family Medicine | Vol. 1 | Issue 02 | Jul-Dec, 2015 67
Kar S S: Basic principles of constructing multiple choice questions

of MCQs. Prevalidation (before the item appears in an
examination) and post validation (item analysis after
the test) are essential to ensure validity of the item.
8. Instructions to candidates on how to respond to
individual items, time allotment and negative marking,
if any, should be clearly outlined in the beginning of
the paper.
B. CONSTRUCTING EFFECTIVE MCQs: 1,8,6,10
Constructing multiple choice questions is a laboriously
acquired art, and it is much easier to advise how not
towrite them than to give a simple way of writing them.
The questionsshould be relevant, short, understandable
and discriminant.11 It is worth following a checklist
whileformulating MCQs (Table 2).
Prevalidation of the items can be done by discussion with
three or four experts on the relevance to learning outcome;
clarity, brevity and appropriateness of stem; plausibility of
all distracters and decision of the correct choice. It is also
important to estimate the level of cognition tested by the
item and the expected difficulty level. Construction of the
item can be verified by checking if it can be answered by
only reading the stem & lead‐in.
C. CONSTRUCTING HIGHER ORDER MCQs
MCQ shouldtest the appropriate level of learning and ensure
assessment of intended learner competence. Bloom’s
taxonomy ofcognitive learning,described as a hierarchy of
knowledge, comprehension, application,analysis, synthesis,
and evaluation, has been adopted for test development.7
Traditionally, items are classified by the cognitive processes
required to answer the question (eg, recall, interpretation,
problemsolving, comprehension, or reasoning). Recall
items are thought to test examinees’ knowledge of isolated
facts.Interpretation items require examinees to review
some information (often in tabular or graphical form) and
reach some conclusion(eg, a diagnosis). Problem-solving
items present a situation and require examinees to take
some action (eg, the next step in patient management).
Interpretation and problem-solving items are thought to
involve “higher order” skills, rather thanjust rote memory
of factual information.6
Clinical vignettes provide a good basis for questions at both
comprehension and application levels.10 Each item would
begin with the presenting problem of a patient, followed by
the history, physical findings, results of diagnostic studies,
initial treatment, subsequent findings, etc. Brief patient

Table 3. Constructing Higher order Multiple Choice Questions

Cognitive level Bloom’s Examples of MCQs

Taxonomical
domains
If P indicates prevalence, I: incidence and D: duration of illness, then the
relationship between prevalence and incidence is expressed as
a) P = I X D
b) I = P X D
Knowledge c) P = I/D
Lower order Comprehension d) P = D/I
Application Prevalence of a disease
a) Can only be determined by a cohort study
b) Is the number of new cases in a defined population
c) Describes the balance between incidence, mortality and recovery
d) Is the best measure of disease frequency in etiological studies
If drug prevents mortality but does not affect cure, then which of the following
will be true
a) Incidence will decrease
b) Incidence will increase
c) Prevalence will decrease
Analysis d) Prevalence will increase
Higher order Synthesis
Evaluation If the prevalence is very low as compared to the incidence for a disease, it
implies
a) Disease is very fatal and/or easily curable
b) Disease is nonfatal
c) Calculation of prevalence and incidence is wrong
d) Nothing can be said, as they are independent

Indian Journal of Community & Family Medicine | Vol. 1 | Issue 02 | Jul-Dec, 2015 68
 Kar S S: Basic principles of constructing multiple choice questions
vignettes or laboratory vignettes may be suitable for Basic
sciences. The overall structure of an item for such vignettes
can be depicted by an item template.6 Use of patient and
lab vignettes to assess application of knowledge has
several benefits like enhancing “face validity” of the exam
and focussing on important information rather than
trivia. It also helps to identify those examinees who have
memorized facts, but are unable to use that information
effectively.7,6
Solving MCQs with data, diagrams and images require
higher order thinking skills like analysis and synthesis
level. Extended Matching items are multiple-choice items
organized into sets that use one list of options for all items
in the set. This reduces the chances of recognition effect
(as more than one combination of vignette and option
exists) and also helps in testing application and problem
solving ability. Key feature questions and assertion reason
questions are examples of Evaluation type of questions.10
Item analysis
Item analysis is a means to ensure quality of MCQs
REFERENCES
1. Singh T, Gupta P, Singh D. Principles of Medical Education. First.
New Delhi: Jaypee Brothers.
2. Srinivasa DK, Adkoli BV. Multiple choice questions: how
to construct and how to evaluate? Indian J Pediatr. 1989
Feb;56(1):69–74.
3. Al-Rukban MO. Guidelines for the construction of multiple
choice questions tests. J Fam Community Med. 2006;13(3):125–
33.
4. Brady A-M. Assessment of learning with multiple-choice
questions. Nurse Educ Pract. 2005 Jul;5(4):238–42.
5. Singh T, Natu MV. Examination reform at the grassroots: teacher
as the change agent. Indian Pediatr. 1997 Nov;34(11):1015–9.
6. Case S, Swanson D. Constructing Written Test Questions For
the Basic and Clinical Sciences [Internet]. National Board of
Medical Examiners, Philadelphia; Available from: http://www.
nbme.org/pdf/itemwriting_2003/2003iwgwhole.pdf
Indian Journal of Community & Family Medicine | Vol. 1 | Issue 02 | Jul-Dec, 2015
View publication stats
and facilitate question banking. It provide a numerical
assessmentof item difficulty and item discrimination.
Itemdifficulty is determined from the percentage ofstudents
who answered each item correctly.Item discrimination
refers to the percentage differencein correct responses
between two groups of students (generally referring to
students in the top 25% and the lower 25%).7
CONCLUSION
MCQs can be used as a meaningful and effective tool
of evaluation in medical education. The quality of
MCQ depends on the quality of items on the whole and
presence of efficient distractors. Flawed MCQs interfere
with assessment process and therefore it is vital to
develop reliable and valid items must be constructed
that are free of such flaws. Significant time, effort and
commitment is required to prepare MCQs that are of good
quality, reliable, able to test higher order thinking skills
and consistent with curriculum objectives. It is essential
for medical educators to be skilled in effective test item
writing to assess the learner’s knowledge.
7. Collins J. Education techniques for lifelong learning: writing
multiple-choice questions for continuing medical education
activities and self-assessment modules. Radiogr Rev Publ
Radiol Soc N Am Inc. 2006 Apr;26(2):543–51.
8. Haladyna TM, Downing SM, Rodriguez MC. A Review of Multiple-
Choice Item-Writing Guidelines for Classroom Assessment.
Appl Meas Educ. 2002 Jul 1;15(3):309–33.
9. Ananthakrishnan N, Sethuraman K, Kumar S. Medical
Education: Principles and practice. Second. Pondicherry:
Alumni association of National Teacher Training Centre,
JIPMER, Pondicherry;
10. Singh T, Anshu. Principles of assessment in Medical Education.
First. New Delhi: Jaypee Brothers.
11. Lowe D. Set a multiple choice question (MCQ) examination.
BMJ. 1991 Mar 30;302(6779):780–2.
69