English Language Testing and Evaluation Özlem KÖLE – TESTING- 3.

vize
English Language Testing and Evulation
Overview Unit 13: Assessing Speaking - 1
This chapter focuses on one of the most important aspects of language assessment: assessing the ability to
speak. It starts with the challenges involved in speaking assessment. Then, five different types of speaking
and a list of speaking macro- and microskills, from which we will select one or several to become the
objective of our assessment, are presented. The chapter also covers assessment tasks for imitative,
intensive and responsive oral production tasks.

By the end of the unit the students will be able to

• recognize challenges involved in oral production assessment,

• categorize and distinguish between basic types of speaking,
• discriminate between micro- and macroskills of oral production,
• identify the assessment tasks designed for imitative, intensive and responsive speaking,
• identify the strengths and weaknesses involved in assessment techniques designed for each basic types
of speaking.

Read the introduction to the chapter on page 140 to list the three unique challenges of testing
speaking.

• What are the challenges of testing speaking?

ANSWER:
1- The interaction of speaking and listening
2- Elicitation techniques
3- Scoring

Basic Types of Speaking

on pages 141 and 142

Categories Purpose
Imitative Testing the ability to imitate a
word / phrase / sentence
Intensive Producing short stretches of oral
language
Responsive Briefly interacting with the
interlocutor
Interactive Involving long stretches of
interactive discourse
Extensive Testing the ability to produce
longer speeches delivery
Skills to be tested
Pronunciation
Narrow band of grammatical,
phrasal, lexical, phonological
relationships (stress / rhythm /
intonation)
Interaction & test comprehension
limited to short conversations
Interaction in the form of either
transactional language or
interpersonal exchanges
Oral production of planned or
casual speeches
Examples
Word / phrase / sentence repetition
Directed response task / reading aloud /
sentence completion / limited picture cued
tasks and sentence level translation
Very short conversations, standard
greeting, small talk, simple request
Interviews, role plays, games, discussions
Speeches oral presentations story telling

Micro- and Macroskills of Speaking

The microskills of speaking refer to producing small chunks of language such as phonemes,
morphemes, words and phrasal units. The macroskills include the speakers' focus on the larger
elements such as fluency, discourse, function, style cohesion, nonverbal communication and
strategic options.

Read the relevant section on pages 142 and 143 to do the following.

1- Look at the list of micro and macroskills of speaking on pages 142 and 143. Brainstorm some tasks that
assess those skills and discuss your results in Web discussions.
2- Answer the following questions according to the information provided on pages 143 and 144.
• Is it possible to assess the skill of oral production in isolation?

ANSWER:
No, interaction between speaking and listening or reading is unavoidable.

• What are the basic issues to keep in mind while designing speaking tasks?

ANSWER:
a. Interaction effect: impossibility of testing speaking in isolation
b. Elicitation techniques: to elicit the specific criterion we expect from test takers.
c. Scoring: to achieve reliability

Designing Assessment Tasks: Imitative Speaking

Answer the following questions according to the information provided on pages 144 and 145.

• What is the significance of simple phonological imitations for assessing speaking?

ANSWER:
It provides accuracy in speech to help learners to be more comprehensible.

• What are the points to keep in mind for repetition tasks to assess imitative speaking?

ANSWER:
a. don't allow them to be dominant in your assessment of speaking
b. try to avoid negative washback of non-meaningful and fruitless imitations
c. have clear scoring specifications to avoid reliability breakdowns.

Phonepass® Tests
Read the relevant section to decide whether the following sentences are True or False.

• Phonepass® Test only tests test takers' phonological ability.

ANSWER:
FALSE. It also tests discourse and overall production

• Test takers read aloud, repeat sentences, say words, answer questions over a telephone in the
Phonepass® Test.

ANSWER:
TRUE

• The scoring procedure in the Phonepass® Test is not reliable.

ANSWER:
FALSE. Scoring using speech recognition technology has been validated to be reliable and practical.

• The test scores are quickly reported back to the test takers in Phonepass® Test.

ANSWER:
TRUE

• Pronunciation, reading fluency, repeat accuracy and fluency, listening vocabulary are the sub-skills
scored in Phonepass® Test.

ANSWER:
TRUE
Designing Assessment Tasks: Intensive Speaking

Complete the following blanks considering the characteristics of task types designed to assess
intensive speaking.

Type of the Task Purpose Practical Possible Scoring

Advantages Drawbacks
Directed- To produce short Require minimal Mechanical and not -
Response Tasks stretches of processing of communicative at
discourse to show meaning to all
linguistic ability at produce accurate
a certain level output
Read-Aloud Tasks To improve Predictable output, Inauthentic not real Easy to score /
pronunciation and practical, reliable in life control production
fluency scoring
Sentence / To produce omitted Moderate control of -Inauthentic -
dialogue lines, words in a the output / the -Reliance on
completion tasks dialogue written form of literacay
& Oral appropriately questions avoiding
questionnaires ambiguities
Picture- Cued To elicit oral -Visual stimulus Reliability May be problematic
tasks production by -Steps closer to depending on the
using pictures authenticity expected
performance
Translation To translate from -Control of the - Must be specicfied
target language to output
native langa-uage -Well-proven
communcation
strategy

English Language Testing and Evulation

Designing Assessment Tasks: Responsive Speaking Unit 13: Assessing Speaking - 1

Read the relevant parts on pages 159-162 to do the following. Complete the following blanks
considering the characteristics of task types designed to assess responsive speaking

Type of Task Purpose Practical Advantages Be careful about

Question and answer To elicit either the
predetermined correct
response or appropriate
questions
Giving instructions and To assess the ability to
directions give instructions and
directions
Paraphrasing To assess the ability of
paraphrasing what is
read / heard in 2-3
sentences
In referential questions, -
test takers are given
more opportunities for
meaningful language
Test takers engage in Memorization of set
extended stretch of sentences / topic choice
discourse / the length of speech
Practicing being precise Skill integration
and concise

Test of Spoken English (TSE®)

Test of Spoken English

Time allocated 20 minutes
Objective To elicit oral production in various discourse categories rather than in phonological,
grammatical or lexical targets
Objective Test takers are given a holistic score incorporating task performance, function,
appropriateness, coherence and form focused factors.

Designing Assessment Tasks: Interactive Speaking Interview

Procedure A test administrator and a test taker in a direct face to face exchange / proceed through
a protocol of questions and directives
Duration May vary depending on the purpose and context from five to sixty minutes
Recording Audio or video recorded for re-scoring
Stages Involved Tasks
1- Warm Up Mutual introductions
2- Level Check Pre-planned questions to
make learners use expected
forms and functions
3- Probe Questions and prompts
challenging test takers to
extend beyond the
expectations
4- Wind-down Final face of the interview
to put test takers at ease.
Aim Scoring
To put students at ease No scoring
To give clues about test Linguistic target criteria
takers' readiness to speak are scored
and level
To discover the limitations Either scored or ignored
and ceiling of test takers'
proficiency.
To make test takers relaxed Not scored
with some easy questions

Answer the following questions considering the information provided in oral interviews.

• What does the success of an oral interview depend on?

ANSWER:
It depends on 4 things: Clearly specified administration, focusing the probes on the purpose of assessment,
elicitation factors and consistent scoring.

• Why is oral interview difficult to score?

ANSWER:
Since the tasks are open ended and they involve a significant level of interaction, the reliability in scoring is a
problem in assessing speaking.

• How can you provide accurate assessments of oral production?

ANSWER:
Through experience, training and careful attention to the criteria

• What are the categories considered in oral proficiency scoring?

ANSWER:
Grammar, vocabulary, comprehension, fluency, pronunciation and task achievement.

• What are the benefits and drawbacks of taking two test takers at a time?

ANSWER:
Benefits:
- appropriate for student-student interaction
- increased authenticity
- scheduling two candidates at once eases administration

Drawbacks:
- one pair dominating the other
- scoring two people simultaneously
- interaction effect of unequal comprehension and production abilities
Role-play / Discussions and Conversations / Games

Read the relevant pages 174-176 to fill in the missing parts in the grid below.

Possible Advantage(s) Keep in mind

Role-Play • Creativity in the output • Scoring of unpredictable responses
• Determining the assessment
• Lower anxiety because of objective of the role-play
pretending to be someone else

Discussions and Conversations • High level of authenticity and • More difficult to specify and score
spontaneity • Carefully designed checklist to suit
the objectives of the assessment
• Informal assessment technique

Games • Directly involve language • Specify a set of criteria and reliable

production scoring method
• Informal assessment technique

English Language Testing and Evulation

Oral Proficiency Interview (OPI) Unit 13: Assessing Speaking - 2

Read the relevant part on pages 176-179 to fill in the blanks below.

ANSWERS:
1. Objective: To elicit pronunciation, fluency, integrative ability, sociolinguistic and cultural knowledge and linguistic
ability
2. Administered by: Only certified examiners
3. The scoring method: ACTFL Proficiency guideline is used
4. The characteristics of the ACTFL Proficiency guideline:
1. Holistic evaluation is emphasized instead of separate scoring
2. Focuses more on the overall task and discourse ability
3. Not appropriate to offer positive washback. Thus less useful for classroom adaptation

English Language Testing and Evulation

Designing Assessment Tasks: Extensive Speaking Unit 13: Assessing Speaking - 2

Extensive speaking tasks involve complex and long stretches of discourse. Read the relevant parts and fill
in the blanks given in the table.

Task Type Procedure Keep in mind

Oral presentations Presenting a report, plan, method or Using practical and reliable checklist
product
Picture-cued Using a picture or series of pictures as Clear criteria for scoring
Story-telling stimulus for longer stories or discriptions
Re-telling a story, news Re-telling a story or news that is read or Scoring should meet the intended
event heard to produce a number of oral criteria
discourse features and fluency
Translation Translating longer text from mother Highly specialized skill
tongue to English
conclusion Remember the rules for effective assessment
Specify the criterion
Set appropriate tasks
Elicit the optimal output
Establish practical and reliable scoring procedure