Jul. 2005, Volume 3, No.7 (Serial No.

22)

US-China Foreign Language, ISSN1539-8080, USA

How to Develop a Test on English Listening Skills:

Insights from an Operational Definition of English Listening Ability
Meng Jiang* Sichuan International Studies University
AbstractAn operational definition of the theoretical construct listening ability is both a premise and a
priority for any listening test development. This paper, by way of examining the authentic listening comprehension
process and inquiring the source of authentic listening hindrance, first proposes an operational definition of
English listening ability---a checklist of English aural microskills, and then derives therefrom a host of guidelines
and principles for either overall English listening test design or for more specific English test items writing.
Key wordslistening test development

operational definition of English listening ability

aural microskills

1. Introduction
Any test development should begin with the conceiving of the theoretical constructs underlying the test, for
they are what the (construct) validity builds on. As Davidson et al. (1985: 203) wisely states, Tests are, in a
manner of speaking, operational definitions of theoretical constructs, in that they operationalize the entity that is
being measured.To develop an English listening test, needless to say, an operational definition of the theoretical
construct English listening abilityis both a premise and a priority. Undoubtedly, the theoretical construct
English listening abilityis also the most crucial point that sets English listening test far apart from other sorts of
English tests, say, English reading comprehension test. Therefore this paper chooses to make an exclusive study of
English listening ability, in an attempt to approach the HOW(to develop a test on English listening skills)
from the perspective of an operational definition of this theoretical construct, though such a decision is taken at
the cost of leaving aside many other considerations that are no less vital to English listening test development.
In further consideration of the widely-held belief that listening test serves to infer the test-taker
s ability to
function in real-life listening situations by the sampling of his performance on listening test-items (e.g., Carroll,
1968; Bachman, 2000; Xiaoju Li, 2001), it is presumed that an inquiry into such authentic listening questions as
what are listeners doing when they listen?What makes listening difficult?What specific skills are called for
in listening comprehension?will be both necessary and enlightening.

2. What Authentic Listeners Do in Listening

Listening is not a one-way street, confined to the psychomotor process of receiving sound waves through the
ear and transmitting nerve impulses to the brain. It is an interactive process that involves both the process of the
receiving of sound waves (audible symbols) and the process of the brain
s acting on the nerve impulses,
*

Meng Jiang, lecturer of Department of English Language and Culture, Sichuan International Studies University, PhD candidate of
Shanghai Jiaotong University; Research fields: cognitive linguistics, second language acquisition, second language teaching; Address:
A0214091#, Minhang Campus, Shanghai Jiaotong University, Shanghai, P.R.China; Postcode: 200240; Tel: 021-54741258; E-mail:
johnbarley@sjtu.edu.cn.
32

How to Develop a Test on English Listening Skills: Insights from an Operational Definition of English Listening Ability

employing the operation of various complicated cognitive and affective mechanisms. Richards(1983) interactive
model of listening comprehension which elaborates on eight mental operations involved may well supply an
answer to the question what are listeners doing when they listen.
1) The hearer processes what well call raw speechand holds an imageof it in short-term memory. This
image consists of the constituents (phrases, clauses, cohesive markers, intonation and stress patterns) of a stream
of speech.
2) The hearer determines the type of speech event that is being processed. The hearer must, for example,
ascertain whether this is a conversation, a speech, a radio broadcast, etc., and then appropriately color the
interpretation of the perceived message.
3) The hearer infers the objectives of the speaker through consideration of the type of speech, the context,
and content. So, for example, one determines whether the speaker wishes to persuade, to request, to exchange
pleasantries, to affirm, to deny, to inform, and so forth. Thus the function of the message is inferred.
4) The hearer recalls background information (or schemata) relevant to the particular context and subject
matter. A lifetime of experiences and knowledge are used to perform cognitive associations in order to bring a
plausible interpretation to the message.
5) The hearer assigns a literal meaning to the utterance. This process involves a set of interpretations of the
surface strings that the ear has perceived.
6) The hearer assigns an intended meaning to the utterance (in many instances, perceived and intended
meanings match).
7) The hearer determines whether information should be retained in short-term or long-term memory.
Short-term memory---a matter of a few seconds---is appropriate, for example, in contexts that simply call for a
quick oral response from the hearer. Long-term memory is more common when, say, you are processing
information in a lecture. There are, of course, many points, in between.
8) The hearer deletes the form in which the message was originally received. The information is retained
conceptually.
With the exception of the initial and final processes, no sequence is implied here; they all occur, if not
simultaneously, then in extremely rapid succession. Neurological time must be viewed in terms of microseconds.

3. What Makes Listening Difficult

In listening, there are a host of factors that highly influence the processing of speech and that can often block
comprehension, which are therefore what makes the listening comprehension process difficult. The sources of difficulty
include (based on what H.D. Brown (2001: 238-241) described as special characteristics of spoken language):
1) Over-clustering or under-clustering. In written language, we are conditioned to attend to the sentence as
the basic unit of organization. In spoken language, due to memory limitation and our predisposition for
chunking, or clustering, we break down speech into smaller groups of words. To comprehend the utterance, the
listener needs to pick out manageable clusters of word (phrases, word groups, or clauses). An attempt to retain
overly long constituents (a whole sentence or several sentences), or an attempt to attend to every word in an
utterance will both fail in comprehension.
2) Distraction from redundancy. The speaker normally says a good deal more than is strictly necessary for
the conveying of the message. Redundancy includes such things as repetition, paraphrase, glossing with utterances
in parenthesis, self-correction, the use of fillerssuch as I mean, well, er. A learner listener is unable to profit
from such redundancy by becoming aware that not every new sentence or phrase contains new information and
33

How to Develop a Test on English Listening Skills: Insights from an Operational Definition of English Listening Ability

that there is extra time available for comprehending. Instead, they will feel interfered and distracted.
3) Illiteracy with colloquialisms. In spoken discourse, utterances are not neatly divided into sentences; a
grammatical structure may change in mid-utterance; unfinished clauses are common. Besides, colloquial
vocabulary, idioms, slang, reduced forms, idiosyncratic pronunciation, slurred articulation, ungrammatical forms
and shared cultural knowledge are all manifested at some point in spoken discourse. They pose a barrier to
learners, especially those who have been confined to classroom learning environment.
4) Interference from performance variables. Spoken discourse (except speeches, lectures, etc.) is featured by
hesitations, false starts, pauses, and corrections. While native listeners are conditioned from very young ages to weed
outsuch performance variables, they can interfere with comprehension in second language (learner) listeners.
5) Rate of delivery. Virtually every language learner initially thinks that native speakers speak too fast. He
often feels overloaded with incoming information. Actually, the number and length of pauses used by a speaker is
more crucial to comprehension than sheer speed. A capable listener, nevertheless, needs to be able to comprehend
language delivered at varying rates of speed and, at times, delivered with few pauses.
6) Prosodic features. As a stress-timed language, English speech can be a terror for some learners as
mouthfuls of syllables come spilling out between stress points. Stress and rhythm are very important for
comprehension. Also, intonation patterns are very significant not just for interpreting such straightforward
elements as questions and statements and emphasis but more subtle messages like sarcasm, endearment, insult,
solicitation, praises, etc. However, stress, rhythm, and intonation constitute no small hindrance to learner listeners.
7) Noise. In listening, for any number of reasons, there are bits of the discourse that are unintelligible to the
hearer, and therefore, as far as he or she is concerned, they are meaningless noise. Since we usually comprehend
somewhat less than 100 percent of what is said to us, we make up for the deficit by guessing the missing items or
simply ignoring them and gathering what we can from the rest. But with these incomprehensible noise bits, the
learner listeners often feel they are failing and get worried and stressed.
8) Single exposure. In authentic listening, the discourse will not be repeated verbatim; normally it is heard
only once, though this may be compensated for by the redundancy of the discourse, and by the possibility of
requesting repetition or explanation. To learner listeners, single exposure poses another difficulty: I need to hear
things more than once in order to understand.
9) Lexical and syntactical difficulty. Difficulty of language fails the learner listeners too.
10) Too much information compacted in per stretch of utterance. The learner listeners often feel that they can
catch the actual sounds of the utterance but comprehension doesnt follow.

4. What Specific Skills Are Called for in Listening

From the above examination of the authentic listening comprehension process and the inquiry into the
sources of listening hindrance, we can derive a checklist of aural skills, which are what an authentic, smooth
listener must employ in the process of listening comprehension. So they represent the specific skills called for in
smooth listening comprehension.
1) Retain chunks of language of different lengths in short-term memory.
2) Discriminate among the distinctive sounds of English. Recognize English stress patterns, words in stressed
and unstressed positions, rhythmic structures, intonational contours and their role in signaling information.
3) Recognize and use reduced forms of words or sentences to clarify the meaning of an utterance.
4) Discriminate between registers of speech and tones of voices.
5) Use context to build and confirm listening expectation.
34

How to Develop a Test on English Listening Skills: Insights from an Operational Definition of English Listening Ability

6) Chop the utterance in varied sizes of chunks that fit comprehension needs.
7) Feel comfortable with and profit from pauses, errors, corrections and other performance variables.
8) Use incomplete sensory data and world knowledge to construct a more complete understanding.
9) Recognize cohesive devices in spoken discourse.
10) Recognize the communicative functions of utterances, according to situations, participants, goals, etc.
11) Infer situations, participants or goals using real world knowledge.
12) Distinguish between literal and implied meanings.
13) Extract information from a single hearing.
14) Listen selectively for specific information and globally for general gist.
15) Use features of stress and volume to identify important information.
16) From events, ideas, etc., described, predict outcomes, infer links and connections between events, deduce
causes and effects, and detect such relations as main idea, supporting idea, new information, given information,
generalization and exemplification.

5. Define English Listening Ability as a Checklist of English Aural Microskills

So far, we have derived a range of aural microskills (these skills are so minute that we call them microskills
from now on) from the inquiry into the three authentic listening-related questions. Since they are a necessity in
authentic smooth listening comprehension, they are skills that a learner listener is supposed to acquire. The
corollary for test is that if the performance of a leaner listener on listening test demonstrates that he has possessed
all these skills, he should be able to function fairly well in authentic listening situations. These microskills are a
touchstone of the learner
s listening competence. Therefore, a valid listening test should be able to probe the
test-taker
s possession of these microskills. To put it another way, listening test should focus on the testing of
these microskills. What this taxonomy provides is a blueprint for what to test. Concerning the theoretical
construct listening ability, we may assume that all these microskills are just a breakdown of it. They are
combined to compose the entity. In the same vein of thought, this checklist of aural microskills, when applied to
the English language, can likewise be taken to have provided an operational definition of the theoretical construct
of English listening ability. This definition will shed great light on English listening test development: firstly,
we may derive a host of guidelines for its overall design; secondly, the checklist of aural microskills will provide
soul and essencefor the writing of more specific test items.

6. Derive Guidelines for English Listening Test Design

As is stated previously, the definition of English listening ability as a checklist of English aural microskills
will enable us to derive a range of guidelines for English listening test design. These guidelines include:
1) Select natural and authentic rather than concocted, artificial spoken materials.
2) Use sources of difficulty to control the difficulty of listening materials.
3) Select materials involving common-sense background knowledge (both cultural and topic-specific).
4) Select materials with appropriate information loading.
5) Tape record listening materials with natural speech delivery rate and performance variables.
6) Try to avoid the need to present listening materials more than once in test administration.
7) Involve no less use of world knowledge in test than is necessary for real-life listening comprehension.
8) Involve no less demanding retention in test than is necessary for real-life listening comprehension.

35

How to Develop a Test on English Listening Skills: Insights from an Operational Definition of English Listening Ability

9) Involve no less reasoning in test than is necessary for real-life listening comprehension.
10) Test the ability to process spoken utterance top-down, bottom-up and interactively.
11) Drop hints in the test or test item instructions for the wise test-taker to activate the common-sense
knowledge structure for comprehension.
12) The length of the test should not be tiring to test-takers.
13) Base test items on message, not on language code.

7. Derive Principles for English Listening Test Items Writing

As said above, the various aural microskills provide soul and essencefor English test items writing. They
can be incorporated into different types of English test items.
1) Design test items that test comprehension of slurredlypronounced utterance stretches.
2) Design test items that test comprehension of attitude, tone, mood, irony, personal preference, etc.
implied in specially intoned or accentuated utterance stretches.
3) Design test items the response of which demands detection and intelligent use of contextual cues scattered
in the spoken discourse.
4) Design test items that test point-comprehension: people
s names, dates, certain fact, event, or participant.
5) Design test items that test global comprehension: tenor, mood, attitude, intention, conclusion, goal,
situation, location, etc.
6) Design test items the response of which must be based on both the text and the activation of
common-sense knowledge structure.
7) Design test items the response of which demands reasonable retention of what is comprehended.
8) Design test items that test comprehension of the illocutionary meaningof an utterance stretch.
9) Design test items the response of which demands reasoning or inference on what is heard.
10) Design test items that test semantic relationsimplied by major cohesive markers in spoken discourse.

8. Conclusion
This paper makes a focused study on an operational definition of the theoretical construct English listening
ability. The handful of guidelines and principles derived thereof are supposed to point toward the HOWof the
development of an English listening test. Needless to say, they represent no more than a tiny portion of the total
considerations entailed by the practical undertaking of English listening test design and writing. But the
significance lies in the implication that an examination of a working definition of the major theoretical construct
English listening abilitywill offer useful insights into the development of a test on English listening skills.
References:
1. Bachman, L.F. Fundamental Considerations in Language Testing. Shanghai: Shanghai Foreign Language Education Press.
2000.
2. Brown, H.D. Teaching by Principles: An Interactive Approach to Language Pedagogy. Beijing: Foreign Language Teaching
and Research Press. 2001.
3. Carrol, J.B. The Psychology of Language Testing. In Davies (ed.) Language Testing Symposium: A Psycholinguistic
Perspective. London: Oxford University Press. 1968.
4. Davidson, E., T. Hudson, B. Lynch. Language Testing Operationalization in Classroom Measurement and L2 Research in M.
Celce-Murcia (ed.): Beyond Basics: Issues and Research in TESOL. Rowley, Mass.: Newbury House. 1985: 137-52.
5. Richards, Jack C.. Listening Comprehension: Approach, Design, Procedure. TESOL Quarterly. 1983(16): 153-168.
6. Xiaoju Li. The Science and Art of Language Testing. Changsha: Hunan Education Press. 2001.

(Edited by Linqiong Yan, Yanhong Zuo, Hua Zhou and Thelma)

36