Criteria for MCQ1 evaluation -- 3000, FA.

2010

attribute Student provides Evaluation

Correctness • The Question 1. Look at student's answers
• five (5) answers – A1, .., A5, indicating which A1, ..., A5 and
one is the correct answer. (you should indicate 2. look at the rationales
exactly one answer as correct).
RA1, ..., RA5.
◦ Rationales and substantiation – RA1, ...,
RA5, for each of the five answers for why
3. Look at the references
the answer is correct or wrong. (Example 1: FA1, ..., FA5.
RA3: “Answer A3 is correct because ....”; or
“Answer A2 is wrong because ...”; example If any of RA1, ..., RA5 is missing,
2: (RA4) the answer A4 is correct according ==> reduce marks. ( E1 )
to the statement “such and such of <source
– e.g. Textbook or URL> ).
◦ References – FA1, ..., FA5. If any of references FA1, ..., FA5 is
▪ Where is the MCQ material quoted or missing, ==> reduce marks.( E2 )
inspired from. e.g.,
• URL of web site from which it is If the answer that is suggested as
quoted/inspired. Date of access of correct is clearly wrong ==> reduce
that web site; or
marks. ( E3 )
• book title and pages # -- all pages,
as applicable. Or,
• book title (including author name, If the question, or the answers, or
publisher, date), or the rationales/substantiation, or
• article citation (article title, author references are badly written ==>
name, venue (conf./journal), date, reduce marks. ( E4 )
location).
▪ the references should be specific – i.e.,
referencing a huge web site and nothing
else is not a strong reference. In such
cases there should be given specific
location of where the material is found
within that web site.

Note: providing the above items promotes

• learning (of the material) [acquire technical
skills]
• critical reading of the material
• research (by possibly reading of additional
sources, and by providing references).
• Presentation skills (by writing the question,
the answers, the rationale/substantiation, and
the references).
Difficulty • The main (primary) MCQ – PQ. Read the statement of comparison
• Two (2) similar MCQs – SQ1 and SQ2.2 (the which and the why).
1 MCQ stands for Multiple Choice Question.
2“You don't understand anything until you learn it more than one way.” Marvin Minsky quotes (American Electrical
engineer, Mathematician and Educator, b.1927) http://thinkexist.com/quotation/you_don-

Page 1 of 2
 • A statement of comparison in which the student
explains If the statement of 'which' is more
◦ which of the PQ, SQ1, and SQ2 is more difficult is missing ==> reduce
difficult and
◦ why it is more difficult.
marks. ( E5 )

Note: the creation of the 3 MCQs in conjunction with If the statement of 'why' it is more
the statement of comparison, promotes difficult is missing ==> reduce
• critical thinking marks. ( E6 )
• critical writing.
Relevance An argument as to Read the argument (the why and the
• why the MCQ is relevant to the course material what).
associated with the designated 10-day period.
For example: “the question is relevant to CH.4,
If the argument for 'why' is missing,
section 4.1, because ....” . Or, “this question is
especially referring to material in page 12 ==> reduce marks. ( E7 )
lower half, where it says “....<quote from page
12> ...” and page 17 3rd paragraph (where it says If the argument for 'what' is
“... <quote from page page 17> ...”. Part of the missing, ==> reduce marks. ( E8 )
'why' should definitely (but not necessarily only)
be which chapter of the book, or which talk the
MCQ is from.
• what the MCQ tests. For example: this question
tests knowledge of whether the student:
◦ knows rule R17 of the ACM code of ethics.
◦ Can distinguish between rules R17, R18,
and R24 of the IEEE code of ethics.

Note: The argument promotes

• critical thinking
• critical writing.
Clarity • The same MCQ but written in three (3) different Read the 3 MCQs.
ways – i.e., 3-same-MCQs. 3
• A statement that substantiates why one (and If 3-same-MCQs are not provided
which one) of the 3 MCQs is more clear than the
others – actually rank all 3 MCQs by clarity. ==> reduce marks. ( E9 )

Note: the creation of the 3 MCQs in conjunction with If statement is missing ==>
the statement of substantiation, promotes reduce marks. ( E10 )
• critical thinking
• critical writing.

t_understand_anything_until_you_learn_it/215044.html

3“You don't understand anything until you learn it more than one way.” Marvin Minsky quotes (American Electrical
engineer, Mathematician and Educator, b.1927) http://thinkexist.com/quotation/you_don-
t_understand_anything_until_you_learn_it/215044.html

Page 2 of 2