TSL 3105 Teaching of Listening and Speaking Skills

in the Primary ESL classroom

lgp/wsl/kj 2012



TOPIC 4

ASSESSING LISTENING AND SPEAKING SKILLS


4.1 SYNOPSIS
Topic 4 introduces you to the Assessment of Listening and Speaking Skills. It
provides an overview of the important principles that a teacher has to bear in mind
when assesing listening and speaking skills. You will also be introduced to the
differences between teaching and testing, and develop the ability to distinguish the
differences between accuracy and fluency based tests of listening and speaking. It
also aims to help you develop a better and clearer understanding of the assessment
strategies related to the testing of listening and speaking skills that we most often pay
little heed to. The conventions of spoken language and the factors that affect listening
and speaking skills are also included in this section. You will also be introduced to the
importance of giving effective feedback and support to improve overall performance
of the students.

4.2 LEARNING OUTCOMES
By the end of Topic 4, you will be able to:

Identify and deliberate on issues related to assessing listening and speaking
Differentiate between teaching and testing
Distinguish the difference between accuracy and fluency based tests of
listening and speaking
Develop a clearer and better understanding of assessment strategies related
to the testing of listening and speaking skill most notably the communicative
testing listening and speaking
Develop a better understanding on giving effective and construtive feedback
and support to improve performance





TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012



4.3 ISSUES IN ASSESSING LISTENING AND SPEAKING
Comment on the following statement:

The best teacher is the teacher who devises classroom methods and techniques
that derive from a comprehensive knowledge of the total process of language
learning, of what is happening within the learner and within the teacher and in the
interaction between the two. All of this knowledge, however, remains somewhat
abstract in the mind of the teacher unless it can be empirically tested in the real
world.
H.D. Brown (1987:218)

a. Answer True or False:
Tests involve students and teachers only.
Tests are carried out on a weekly, monthly and yearly basis.
A test is a form of measurement.
All tests are formal.
Questions asked by the teacher about what has been learnt in a lesson can
be considered a test.

Revision:
Four basic criteria to devise, use or adapt tests of listening and speaking
Validity
Reliability
Practicality
Washback Effect/ Instructional Impact

Validity
We must make sure that we are testing what we are teaching and what the
students want to be learning. Only then is the test fair and appropriate
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


A test that measures what it is intended to measure is called a valid test
We need to determine in advance what we want to measure (test)
We must then design items, tasks, or prompts for the test that measure what
we want to measure
For example: A student who has not been taught prepositions is tested on
prepositions --> the test can be viewed as lacking in validity because it tests
what the student has not learnt

Reliability
We have to be sure that a test or an assessment procedure is reliable.
Reliability is concerned with consistency
For example: if you tape record your students speaking in English and ask
another teacher to evaluate the students speech using a ten-point scale, your
evaluation and the other teachers evaluation should not differ greatly
Rater reliability Intra (the rater himself) and Inter-rater (between raters)
Can you think of some factors that affect intra-rater and inter-rater reliability?
What can you do to minimise the variability (differences) that exist?

Practicality
Practicality refers to the fact that a test or other assessment procedure can
only be useful if it does not make unreasonable demands on resources,
including time, money and personnel
For example: Interviewing each student for thirty minutes (very thorough way
to assess but what if you have 100 students?)

Washback Effect/ Instructional Impact
The effect a test has on teaching and learning
Does the test encourage people to prepare for speaking tasks, or does it
cause them to study grammar rules or obscure vocabulary items?
Washback can be positive or negative
Positive promotes the development of the skills or knowledge to be learned
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Negative hinders the development of the skills or knowledge to be learnt
For example: Listening and speaking skills are neglected in schools mainly
because the two skills are not tested at the national level and more
importantly, the grades or marks for the testing of listening and speaking skills
are not included in the overall results. Teachers therefore view the teaching of
listening and speaking as an unnecessary thing in schools. Hence, the over
emphasis on the reading and writing skills and a neglect of the listening and
speaking skills in schools (Negative washback effect).
Can you think of some other instances of positive and negative washback effects
of tests on the teaching and learning process in schools?

b. What is a test?
A yardstick a teacher uses to measure the performance of a student
(Nesamalar et. al., 2005)
A method of measuring a persons ability or knowledge in a given area
(H.D. Brown, 1987)
We test every day in virtually every cognitive effort we make when we read
a book, listen to the news, or prepare a meal, we are testing hypotheses and
making judgments
We will now deconstruct the definition of a test by H.D. Brown, 1987 to develop a
better understanding of what a test really is:

A method of measuring a persons ability or knowledge in a given area

A Test is a method
There is a set of techniques, procedures, test items
The method generally requires some performance or activity on the part of
either the testee or the tester, or both
The method may be intuitive and informal or explicit and structured


A Test is a method of measuring
A test has the purpose of measuring
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Formal and informal tests nature of the quantification of data
Informal tests intuitive and difficult to quantify; judgment rendered in global
terms (good, poor, fair etc.)
Formal tests quantification is important: using carefully planned techniques
of assessment; for comparison within individual or across individuals

A Testmeasuring a persons ability or knowledge
Need to understand who the testees are?
Their previous experience
Their entry behaviour
Is the test appropriate for the testee?
How are scores to be interpreted for individuals?

A Test measuring a persons ability or knowledge
Competence
A test samples performance but infers certain competence
A driving test for a drivers licence a sample of performance to infer general
competence to drive a car
A language test samples language behaviour to infer general ability in a
language

A Testability or knowledge in a given area
Proficiency test actual performance involves only a sampling of skills, that
area is overall proficiency in a language
A pronunciation test might test only a particular sound/ a phonemic minimal
pair

4.4 TEACHING VERSUS TESTING
Tests serve many of the needs of teaching
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


To monitor progress as we teach
Topic tests used to assess how well a student has mastered what has
been taught
To find out specifically what areas pose difficulties for students diagnostic
tests
Testing and teaching are very closely interrelated but do not have the
same focus
Tests assess the products of learning (students previous learning)
Teaching materials, methodology and the classroom enable students to
succeed in the process of learning (prepares students for current and future
learning)
Primary function of teacher to ensure learning takes place; tests are one of
the tools to help him do this job well;
A test is only a sample of what the student is supposed to know a test is
supposed to pick out the most important aspects of the skill(s) that have been
taught
A test is also often used as a guide as to what would be the most important
things to teach a test influences what is taught (backwash effect)
A test often leaves out certain important skills because of practical constraints
UPSR & PMR no Listening and Speaking component; therefore teachers
do not pay sufficient attention to these important skills
What is taught should be decided by reference to how important the skills are
to a childs present or future life
Teachers should not just teach students only those things that will help them
pass exams
Some teachers use the exam formats for teaching purposes this is
WRONG!!
A test usually seeks the most economical way of finding out what the student
already knows; leaves out many things that are important
Using MCQs for speaking tests (not suitable)
Teaching should keep in mind the real reasons for learning and the real
circumstances in which the language or skill would be used
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Discrete Point vs Integrative Testing
Discrete point tests based on the assumption that language can be broken
down into component parts and tested adequately (listening, speaking,
reading, writing, phonology/ graphology, morphology, lexicon, syntax)
Discrete point approach met with criticism in the integrative sociolinguistic
era where the emphasis is on communication, authenticity, and context.
John Oller (1976, 1979) argued that language competence is a unified set of
interacting abilities which cannot be separated apart and tested adequately
Communicative competence is so global and requires integration that it
cannot be captured in additive tests of grammar and reading and vocabulary
and other discrete points of language
If discrete items take language skill apart, integrative tests put it back
together. Whereas discrete items attempt to test knowledge of language one
bit at a time, integrative tests attempt to assess a learners capacity to use
many bits all at the same time (Oller, 1979:37)

Example of a Discrete Point Test
A typical proficiency test with MCQs divided into grammar, vocabulary,
reading etc.

Examples of Integrative Tests
Cloze tests Oller (1976, 1979) good measure of overall proficiency
requires a number of abilities that lie at the very heart of language
competence
Knowledge of vocabulary, grammatical structure, discourse structure,
reading skills and strategies and an internalized expectancy grammar
Dictation a potentially appropriate integrative test
Taps into certain grammatical and discourse competencies
Requires careful listening, reproduction in writing of what is heard,
efficient short-term memory, and some expectancy rules


TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Discrete Feature Listening Tests
Auditory Discrimination
distinguishing specific sounds from a background of different sounds
Minimal pairs sound discrimination test
Pen or Pan; Pill or Peel; Van or Ban
Identifying words that rhyme
Identifying intonation patterns
Identifying stressed and unstressed syllables

Discrete Feature Speaking Tests
Minimal pair tests
/pen/ & /pin/; /bell/ & /bill/
Intonation tests
He is here.
He is here!
He is here?
Here he is!
Reading aloud
A test of pronunciation, stress and intonation
Tests of language functions and their linguistic realizations
Writing down what is missing in a conversation

Task-based Listening Tests
Listening tasks that students normally engage in authentic tasks
Listen to a text/ description and answer questions/ identify places on a route
map


TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Task-based Speaking Tests
Conversational exchanges different degrees of control (controlled, partly
controlled, one-sided dialogue, incomplete dialogue with prompts)
Using pictures single picture/ object, pictures for comparison, a series of
pictures to make up a story
Oral interviews common oral interaction test in which one or two testers
interview a candidate regarding a set of pre-determined topics

Assessing Listening and Speaking Skills
Do we use discrete point tests or integrative tests to assess L&S skills?
Or do we use a Communicative Test?

Communicative Test
To test the communicative production of language
A communicative test has to meet some stringent criteria. It has to test for:
Grammatical competence
Discourse competence
Sociolinguistic competence
Strategic competence
Illocutionary competence
(Wesche 1983, Swain 1984)
Has to be pragmatic
Learners have to use language naturally for genuine communication
Use authentic language within a context
Should be direct
Should test learners in a variety of language functions
DIFFICULT TO MEET ALL THESE CRITERIA!!!


TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Communicative Tests - Merrill Swain (1984)
Four primary criteria
1. Start from somewhere tests should build on existing knowledge &
principles
2. Concentrate on Content have interesting motivating and substantive
content; integrated and interactive
3. Bias for best need to do everything possible to elicit the very best from
students
4. Work for washback we should not teach toward a test but we can use
tests as teaching tools; feedback devices

4.5 ACCURACY VERSUS FLUENCY


Activities which focus on accuracy try to get students to say something
correctly (correct grammar, correct word form).
Activities which focus on fluency try to get students to communicate
successfully, even if they make some mistakes.
When using communicative tests, the teacher constantly faces the dilemma of
deciding which is the focus of the assessment: accuracy or fluency or both
Both are equally important for language learning and language use.
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Accuracy certainly helps students communicate effectively and efficiently, and
more importantly, they may need a high level of accuracy to pass exams.
Fluency activities are important because they allow students to express their
ideas and communicate in a meaningful and enjoyable context.

REFLECTION
As a teacher, which would you consider more important when teaching: accuracy
or fluency?
As a teacher, which would you consider more important when testing: accuracy
or fluency?
Can you think of an activity that focuses on accuracy which can be used to test
L&S skills

o What will the focus of the activity be?
o Grammatical accuracy?

Can you think of an activity that focuses on fluency which can be used to test
L&S skills?
o What will the focus of the activity be?
o Ability to communicate? What about errors made?

TUTORIAL TASK
What is validity and how does it differ from reliability?
Why is practicality an important issue in learners listening and speaking skills?
What are positive and negative washback? Provide a few examples of positive
and negative washback effects of testing on teaching.
Think about an important listening and speaking test that you have taken. How
did you prepare for that test? What did your teacher do to help you prepare for
the test? Did that test have a positive or negative impact on you? Describe the
test that you took.
If you have not taken such a test before, think of possible reasons why it was not
carried out.



TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


4.6 ASSESSMENT STRATEGIES FOR LISTENING AND SPEAKING SKILLS
Types of activities that can be used for testing L&S communicatively
Reaching a consensus (Groups of 3/4)
Deciding on ten out of twenty items that they would take on a 2-week
trip to New York; need to provide justifications
Moral Dilemmas (Groups of 3/4)
Invigilating an important exam
See a student cheating with notes
4 possible action: ignore, warn, ask student to leave, report to
authorities
Discussion
Using controversial topics (smokers should be banned, forced to quit
habit, jailed, allowed to smoke in designated areas)
Debates (two sides argument: points for or against); Balloon debate
who to throw out of a balloon which is losing air!
Relaying instructions
Giving each other instructions
Making models
Describe and draw
Communication Games
Find the differences between two pictures
Describe and arrange
Story reconstruction
Problem Solving
Desert Dilemma
Simulation (reality of function, a simulated environment, structure) and role
play (pretending to be someone they are not)
Creating the pretence of real-life situations in the classroom;
simulating the real world
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


To provide practice in real world use of English

Devising Marking Bands - Descriptors
When assessing L&S skills communicatively, what are the sub-skills or areas
you will be looking at?
Topics to be covered?
Activity type? Individual? Pair? Group?
Duration?
Holistic or analytical marking?
Accuracy? Grammatical?
Fluency?
Pronunciation?
Appropriacy?
Message conveyed?
Others?
Depending on what your assessment objectives are, you may choose to
incorporate a number of sub-skills or areas listed above in your marking band.
The marking band is formulated to address issues related to reliability inter-
rater variability (the inconsistencies between markers) so that a more
objective score can be obtained by all raters.

4.7 THE SKILLS OF PROVIDING CONSTRUCTIVE FEEDBACK
What is feedback?
Why feedback?
How to provide constructive feedback?
Exercise?
Innovations in methods of feedback?
Take home message?

TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


What is a feedback?


Why Feedback?
Provision of important information to the pupil.
A stimulus for further learning and training.
Show pupil the level of their performance.
Address the weaknesses and deficiencies of the pupil.
Decide the progress of the pupil.
An effective method to evaluate the whole program.


TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Characteristics of Good Feedback








Constructive Feedback
Constructive feedback can help the trainee to improve in their knowledge,
skills and attitudes
It can help them to rate their learning in a realistic way
It can help them to be more self-regulated about their learning
It should be focused on behaviour rather than the person, and on
observations rather than inferences or judgments

Appropriate Feedback
Feedback should be completed as soon as possible after the event, before
they forget details of the events
Confidentiality and privacy should always be respected
Check if the feedback has been understood
More than one approach is preferred (e.g. visual and verbal in the same time)
4.9 HOW TO PROVIDE AN APPROPRIATE CONSTRUCTIVE FEEDBACK

A FEEDBACK SANDWICH
Good Feedback
CONSTRUCTIVE APPROPRIATE
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012



A feedback which starts with a positive statement,
followed by a negative statement and finished
with a positive statement
POSITIVE -----> NEGATIVE -----> POSITIVE

Recommended feedback techniques in giving feedback
Create a respectful, friendly, open-minded unthreatening climate
Elicit thoughts and feelings before giving feedback
Be non-judgmental
Focus on behaviour and specific observed facts
Give right amount of feedback
Suggest ideas for improvement
Base feedback on well-defined, negotiated goals
A constructive & appropriate feedback is an essential tool to improve
performance

Some Questions To Think About
Look at an existing language test (listening and speaking), evaluate it in terms
of the various criteria for a communicative test:
Is the test communicative? Why do you say so?
Does it focus on both the listening and speaking skills?
What is the focus of the test --> accuracy or fluency or both?
TSL 3105 Teaching of Listening and Speaking Skills
in the Primary ESL classroom
lgp/wsl/kj 2012


Is the activity suggested appropriate?
Does it truly measure the pupils ability to communicate?
Is there any way to resolve the dilemma of giving large-scale communicative
tests and still maintaining a sense of practicality (the feasibility of scoring
thousands of tests relatively quickly and cheaply)?
Computers are revolutionizing the testing industry. Can computerized tests
meet communicative test criteria? In what way can the addition of an
interactive video component to a computerized test help to meet those
communicative criteria?

References:
Brown, G., & Yule, G. (1983). Teaching the Spoken Language. Cambridge:
Cambridge University Press.

Brown, H.D. (1994). Teaching by Principles: An Interactive Approach to Language
Pedagogy. Englewood Cliffs, NJ: Prentice Hall Regents.

Brown, H.D. (2004). Language assessment Principles and Classroom Practices.
Longman.

Celce-Murcia (Ed.), Teaching English as a Second or Foreign Language (2nd ed.)
(pp. 81-106). Boston: Heinle and Heinle.

Hughes, A. (2003). Testing for Language Teachers. Cambridge: Cambridge
University Press.
Nunan, D., & Miller, L. (Eds.). (1995). New Ways in Teaching Listening. Alexandria,
VA:

Penny Ur. (1996) Teaching Listening Comprehension. Cambridge: Cambridge
University Press.

Weir, C.J. (1990). Communicative Language Testing. Prentice Hall.