You are on page 1of 3

Book review

Alderson, J. C. 1 988: Innovation in language testing: can the


microcomputer he l p Lancaster: University of Lancaster. iv +
44 pp. (Special Report No. 1 : Language Testing Update)
In the area of computers and language testing, much attention has been
focused of late on delivery systems (notably computerized adaptive testing),
as well as on related research and theoretical issues (Bailey, Dale and
Clifford, 1987; Stansfield, 1986; see also Weiss, 1983). By contrast, the
Alderson volume sets out to describe innovative item types that are
facilitated by the computer. Not at all erring on the 'grand side' (Pollitt,
1987), the volume under review has an unusually restricted aim: it examines
the potential of CBELT (computer-based English language testing) exams
that are delivered as well as scored by computer. The focus is on computer-
assisted rather than on adaptive testing, on the microcomputer rather than
the main-frame, on testing technique rather than test analysis or delivery.
But the restricted scope of this computer testing survey seems unneces-
sarily limiting. Throughout the text, one has the impression of a tension
between the self-imposed CBELT constraint of totally machine-scored pro-
cedures on the one hand, and tantalizing options involving human inter-
action and human scoring on the other. And well there might be. Another
area in which computers have been utilized in the broad field of language is
that of translation. Early efforts to manage the entire process by machine
have largely been abandoned in favour of computer-assisted translation.
This speeds up translation and provides for much improved accuracy and
consistency, by allowing for considerable human interaction and initiative.
It does not seem unreasonable to encourage this same latitude in computer
applications to language testing.
Despite this limitation, the text is very useful indeed. It is based on
'research' (I would prefer 'exploration') sponsored by the British Council
and carried out at the University of Lancaster by a 10-member team, includ-
ing such luminaries as Dick Allwright, Geoffrey Leech, and the writer of the
report, Charles Alderson. Chapters 1 and 2 are introductory. The initial
chapter outlines the objectives of the team's investigation, looking at limita-
tions t o date in CBELT t est i ~g and summarizing interesting options for the
future. The second chapter defines CBELT, including its limitations, and
surveys technological options ranging from computerized spelling checkers
to video-disc sequences.
Chapters 3 and 4 help place in perspective the specific techniques discussed
later in the report, but they also go beyond this by indicating specific
120 Book review
technological procedures and techniques available on the computer. Chapter
3 discusses and illustrates possible advantages of the computer in language
testing, including diagnosis and tailoring (through branching). Chapter 4
explores possible CBELT applications in listening, reading and writing.
Chapters 5, 6 and 7 provide systematic presentations of CBELT tech-
niques. Chapter 5 looks at CBELT options which could be employed in con-
junction with CALL software presently available from the British Council
and Cambridge University Press. Chapter 6, the most detailed and lengthy in
the report, turns from CBELT and test content to CBELT and test method
or technique. Here Heaton's paradigm of objective-test items is followed,
illustrating how the computer might produce each of his seven item types.
The short Chapter 7 looks beyond item types to innovations involving
matters ranging from graphics to student feedback. The concluding section,
Chapter 8, examines seven characteristics of computers which provide the
potential for innovation in language testing.
There is much to recommend in the report. While upbeat about the poten-
tial of microcomputer applications in English language testing, the investiga-
tors probe some of the limitations of the new technology. Not inclined to be
prescriptive, the research team raise a number of interesting questions: for
example, might the massive data that can be collected on a given student or
item make profiles almost uninterpretable? Won't the constraints of
machine scoring severely limit the nature of the evaluation process, perhaps
causing us to take a step backward to exclusively multiple-choice techniques?
Might 'help' options on the computer reduce a desired level of risk taking? Is
everything that we can do on a computer worth doing? Moreover,
limitations of CBELT in the testing of writing and speaking are clearly
acknowledged.
While the 'main concern' of the investigation was 'the possibility for
innovation in test types through CBELT', the published report explores a
wide range of computer options available to test makers. It is these special
computer options - far more than any innovative test types - that are of the
most value in the report, for they hold out the promise of improving the
effectiveness of conventional as well as of innovative items. For example,
Alderson and his team demonstrate how the computer can remind examinees
(as often as necessary) of test instructions, so that the exam might better
reflect the candidates' actual skills. They point out how the computer can
readily show the consequences of one's selecting a given multiple-choice
option; this procedure alone holds great promise for improving the effective-
ness of objective items.
Interesting adjuncts that are discussed include a built-in speller, thesaurus,
and 'help' menu. It is shown how computers can restrict what the sudent is
exposed to, utilize video sequences to simulate real-life situations, and
provide great flexibility in how and what language is presented. In addition,
various nonmultiple-choice options are presented, such as dictation, trans-
lation, C-Tests, and intrusive word (e.g. cloze elide or editing); while these
are not innovative test types, they do represent a reasonably broad scope of
options available on the computer. In addition, the report demonstrates
ways in which the computer can facilitate examinee initiatives while taking
Book review 121
a test, an important ingredient in the development of future communicative
language exams.
The text will also be appreciated for its rich array of examples and its
dozens of video-screen displays which exemplify the test items under discus-
sion. Commendable, too, are the organization and delimitation of the inves-
tigation and the clarity and practicality of this useful 43-page report.
Although potentially helpful to those planning to develop computerized
language tests, this report could be improved in the following areas. First of
all, some recommended testing techniques and processes may be debatable to
employ. For instance, the word-scramble approach to sentence construction
can be inefficient, and ineffective for those who might conceptualize a
perfectly good sentence that does not include all the words in the test writer's
model. Moreover, while the provision of immediate feedback on perfor-
mance could easily be built into a computer test, this would probably not be
advisable while the examination is in progress, due to the anxiety that might
well be aroused in many examinees.
In addition, the categories to which various techniques or question types
are assigned need some modifying, particularly if the report is to be used by
relatively unsophisticated test developers. For example, I feel that dictation
would be more appropriately classified under general proficiency than under
listening, and intrusive word also under general proficiency, rather than
under reading. Also, one would want to qualify the 'writing test' categoriza-
tion of the partial parser.
Since so much effort seems to have gone into the investigation reported in
this text, a few more details regarding the feasibility of certain options seems
logical. For instance, the cost of suggested video-disc sequences and graphics
- per minute, say - could help one to determine whether or not these are a
possibility for the near future. Fascinating as these are, the cost may be pro-
hibitive (the cost of merely pressing a single disc, not t o mention the very high
cost of filming and producing a video-disk, is in the range of $1,800 or
1 loo).
Caution is likewise called for in relation to implementing diagnostic
testing. While the computer is obviously capable of reporting individual per-
formance on matters such as prepositions and phrasal verbs (as mentioned in
the report), the test would have to be rather lengthy indeed to achieve
adequate statistical reliability for a satisfactory variety of grammatical
structures. And even though the report is relatively short, an index would
have been useful.
A more important area of concern is the book's overview on computers
and language testing. Applications of item response theory are numerous
and highly significant, yet are scarcely mentioned in this report. IRT-based
test evaluation including bias identification and appropriateness measure-
ment deserves mention. IRT-based computerized adaptive testing likewise
begs attention, particularly since it qualifies as CBELT measurement: an
exam that is 'delivered by computer snd also scored by computer'. Its
potential for increased precision, its invariant scaling and its efficiency make
the adaptive or tailored test one of the truly significant testing achievements
of the decade. Although the emphasis in this text is not on test delivery,
122 Book review
computerized-adaptive testing merits more attention than it is receives.
But an even more significant concern is the nature of the 'innovative' test
types presented - the principal focus of the book. In the introduction, the
author refers to the 'very conservative' techniques used to date in computer
tests. He adds: 'Unless we are careful, there is every danger that the increased
popularity of CBELT might represent . . . a step backwards in language test-
ing'. Yet the techniques that receive the focus in this report are almost exclu-
sively conservative sentence-level, multiple-choice formats, adaptations of
Heaton's seven objective-test item types. Discourse-level material is essen-
tially absent, as is any truly innovative item type.
Nevertheless, the investigation was certainly worth doing, and the result-
ing report deserves very much to be read and used. Even though the item
types described are not particularly innovative, the host of testing procedures
described in the book are innovative. And they hold the promise of comple-
menting CALL, assisting our research, easing the anxiety of the testing
situation, providing increased flexibility in the measurement process, and
spurring student initiative.
References
Bailey, K., Dale, T. and Clifford, R.T. 1987: Language testing research.
Monterey, CA: Defense Language Institute.
Pollitt, A. 1987: Review of Henning, G., A guide to language testing:
development-evaluation-research. Language Testing 4 , 233-36.
Stansfield, C. W. 1986: Technology and language testing. Washington, DC:
TESOL.
Weiss, D.J., editor, 1983: New horizons in testing latent trait test theory and
computerized adaptive testing. New York: Academic Press.
Brigham Young University Harold S. Madsen

You might also like