Warren Loyd E.

Pacana
BEE-4


ADVANTAGES OF OBJECTIVE TEST ITEMS
Survey a wide variety of material, in very little time
o SAMPLING --students can write 60 multiple choice in an hour but only 1 essay
o means they are generally more reliable
CAN'T BLUFF their way around the question
--> teachers hate giving 0 for
essay even when it doesn't say anything on topic
Writing skills, spelling and neatness don't get in child's way
my friend the dyslexic loves multiple choice....

by same token, spelling, neatness don't get in marker's way either
old Social Studies exams, 50% of the variance in student's mark could be accounted for
by spelling and mechanics even though these were not supposed to matter -- so
prejudicial
Can use statistical analysis on individual questions or groups of questions to identify:
strengths and weakness of test (before you hand it back)
specific strengths and weakness of individual students
topics that need to be retaught; areas where course needs work

Unbiased in sense that teachers' preconceptions of student's work can't influence
marking
Answer key is unambiguous so two observers come to same conclusion, unlike essay
where same essay can go from 0 to 100%

"there's that right-wing jerk arguing in favor of Hitler again"
Fairly useful as pre-tests
since it is fast and you're probably not looking for higher level thinking so much as
knowledge of the area
or diagnostic testing where you can reuse year after year because individual not class
Fast to mark:
so you can do large groups, especially when the teacher's time is limited

[but then again its a pain in the behind to write, so it balances out in the end]
Cheap to mark--> machine score --> why government uses them so much.

DISADVANTAGES OF OBJECTIVE TEST ITEMS
Most types are limited to factual recall
Not necessarily the case --> I will teach you how to get higher level thinking out of
objective questions

most people don't have the know-how to make objective items which go
beyond mere recall of facts even those who know how, often lack the time to
build good ones
o a higher order multiple choice can take 3 or 4 hours per question to write--> not
worth the trouble unless you know you're teaching this for 20 years
consequently, often low level recall only

Students need the opportunity to practice and demonstrate writing skills ---> if there is an
over-reliance on objective testing, students will not have this opportunity.
o Before provincial exams I had a university student who didn't know that a
sentence started with a capital and ended a period. When I asked how he got
through high school, he said that he only had to write four papers in three years;
he failed those, but was brilliant on everything else...now we don't get that
problem because teachers have to use essays and therefore the problem fixed
before it gets to me.
o Objective items should therefore be balanced with other forms of evaluation
--> Why there is written response on Diploma Exams
multiple-choice good enough to predict student ability, but need to
include to enforce writing on teachers

Objective items often test skills other than those that the teacher intended
e.g. --> cause and effect relationships, when teacher thought was straight recall
Can become a straight reading test If time is limited, while student spends little time
writing, spends a lot of time reading and thinking.
May be inappropriate for Grades 1-3; grade 4 is iffy
o (except True/False)
o requires too much reading comprehension and abstract thinking
"Piaget says 'No'"
o matching, multiple choice definitely beyond them; maybe short answer
o MAYBE true/false if careful about reading level

-->Ultimately, it may be better to use non-test instruments for these grades

Wording is crucial because only one specific and predetermined answer is acceptable.
--> Easy to become "guess what I am thinking" question
VERY time consuming to construct
Reusing tests may be problematic but since time consuming to write, you'll want to reuse
them --> item banking is ok
o Suggests that you're not doing diagnostic teaching because you should be
custom making exam to current group...
o means that a "leak" can be devastating
o case at U of A where student wrote to her old campus to get copy of old exams
to "study"

Promotes guessing (gets 25% for choosing all 'a's; 50% on T/F)
Test-wiseness often a problem means a student can get the right answer without really
knowing.
Not suitable for language skill testing
Regardless of whatever test salesmen try to tell you
Does not reveal thinking process
o in math, for example, the teacher wants to see WHERE child went wrong as well
as if he or she got the right answer
o if students can't display higher thinking, teachers can't observe it, so limited
diagnostic information available

Often fail to provide a learning experience for students because only deal with recall
Over-reliance on objective item tests will encourage rote memorization
Again, there is a danger of sabotaging the neat things you are doing in your
classroom if students learn that none of that higher thinking class discussion
counts and all you are testing for is names and dates
"What year did Picasso begin his Blue period"

Test Advantages and Disadvantages of Essay type of Test
Advantages
1. No guessing
o have to generate answers rather than select them
o students can demonstrate their knowledge within broad limits
2. Allows divergent thinkers to demonstrate originality, creativity
o in contrast to multiple-choice which are terrible for divergent thinkers who always
chose the answer that isn't in the list of alternatives provided
o of course, leaves open danger of diverging a bit too far... divergent thinking can
be problem for marking in same way as for short answer
Example: famous court case of physics student and barometer.
3. Reduced lead time required to produce
o less stuff to type, run-off if caught short
o write 'em on the blackboard if you have to
4. Less work to administer for smaller number of students
o easier to mark ten essays than to build a multiple-choice test for ten students;
advantage decreases with class size
o But also have to consider long-term: teaching this course again next year?
5. Can be rich in diagnostic information
o one topic in real depth


Disadvantages
1. Impossible to mark objectively
o I personally have seen scores go from 0 to 100 per cent on the same paper
defensibly so --> creative, original response vs. off topic
o that's rare, but not uncommon to vary 35-40%
o tons of studies on this area --> fascinating stuff
o some markers emphasize content, some style
o same essays, but introduced spelling mistakes, made huge difference, even
when markers specifically told to disregard spelling
o same with handwriting-
- Not valid measure of social or English if handwriting gets in the way
o teacher expectations more important than actual performance
eg., guy who sent lesser known stories by Dostoevsky, Hemingway, etc.
into CBC short story contest...
o so "good class" or "poor class"
o halo effects: student's earlier work prejudices expectations
Two students give same wrong answer:
-"Johnny probably meant this when he wrote that"
-"Susan got that wrong, just like I knew she would"
o another kind of halo effect is where in the stack your paper appears:
if read an ok paper after five fantastic standout ones, it looks terrible by
comparison; but if you just read 5 really ghastly papers, it looks pretty
good
2. even different times of day make a difference
-late afternoon papers scored lower because tired; but bluffers may do better
because too tired to catch 'em out
-
one study said might make a pass/fail difference to about 10% of the papers
depending on time of day marked

3. --> first paper to be read often sets standard
--> Or if you decide later to lighten up, the first paper's in trouble
teachers who know content are easier graders than those who don't
same teacher asked to mark the same paper the next day will often give
completely different scores
not just English either --> studies suggest math is just as bad
o Can sample only limited range of course CONTENT
what you gain in depth you lose in breadth
it is possible to come up with questions that get the student to combine
different content BUT make sure that it's YOU doing the combining here,
not simply loading it on the student
o Time consuming for student to write
several students have talked to me about having an essay on their exam
and allowing 25 minutes --> that's long enough for one or two paragraphs,
depending on grade level, not full-fledged essay
unlike m-c, students not only have to interpret the question, they have to:
recall everything they know, sort out what might be relevant
they have to plan their response
they have to think up a catchy opening sentence --> getting
started writing is VERY time consuming in itself --certain inertia here
--BLANK PAGE SYNDROME
--some teachers taught their students to restate the
question in first paragraph, just to get them started, but is
now a bad idea because turns markers off
then they have to devote time to the actual mechanics of
physically writing
they have to stop to think about spelling, or look up words in
dictionary
o Time consuming (and mind numbing) to mark
after reading 42nd essay on Hamlet, your brain will explode
to do a proper job, often end up reading essays twice
o Contrary to expectation, most essay questions do not get at higher thinking skills
mostly just ask "tell me everything you've memorized"
too often marked for factual content plus style, not displays of higher
thinking
o Encourages first draft approach to writing
good writing requires time to think, organize, write and revise
often, the need to 'beat the clock' discourages reflection, thoughtful
answers
--> Certainly kills revision
o Puts a heavy emphasis on student's writing skills
much of student's time devoted to writing not thinking
poor writers are slaughtered, no matter how well they know content
writing ability has more to do with class/social background than with
mental ability
sociologists note it is mostly style of middle & upper class that gets
rewarded, even though "ain't never" communicates equally clearly
Mike Hall, editor of Ft McMurray magazine wrote Writing Competency Test
for university, fails because in answering paper on question "Should there
be a death penalty" used the personal pronoun "I"
Ultimately, have to evaluate writing or students won't learn to do it,
but term paper may be better approach to this
BUT term paper unlimited work, exam setting limits to two hours
and then its over
have to use essay format for objectives that specify write or supply
information
if you take Bloom literally, can't really test "knowledge" because by
definition their ability to incorporate into an essay takes you up a couple
of steps to application -- though I suppose it would be possible to count
objectives like "naming" as "knowledge" but relatively few opportunities in
essay
Only use essays when essays are called for