Professional Documents
Culture Documents
Previous studies of washback (the influence of a test on teaching and learning) have
provided insights into the complexity of educational systems and test use, especially
in relation to the role of the teacher, but have given insufficient attention to the rela-
tionship between observed practices and test design features. In this article a wash-
back model is proposed that incorporates both test design and participant characteris-
tics. The model is used to predict behaviour on preparation courses directed toward
the Academic Writing component of the International English Language Testing
System (IELTS) test. 197 learners and 20 teachers were observed over 51 classroom
hours. These encompassed 22 IELTS preparation classes and, for comparison, 13
classes of English for academic purposes (EAP). Evidence was found for substantial
areas of common practice between IELTS and other forms of EAP but also for some
narrowing of focus in IELTS preparation classes that could be traced to test design
features.
System, 2005). The rapid expansion of the test has brought with it increased de-
mand for test preparation books and courses.
IELTS is used as a means of determining whether candidates should be ac-
cepted into English-medium courses and whether they will require further lan-
guage support. However, concern has been expressed that preparation for tests like
IELTS, because of the limitations on what can realistically and equitably be tested
in a few hours, may not develop the full range of skills required for successful uni-
versity study, particularly in the area of academic writing (Deakin, 1997; Read &
Hirsh, 2005). J. Turner (2004), for example, argues that “what the IELTS test or the
TOEFL test delivers underspecifies the complexity of language issues in the aca-
demic context” (p. 98). Her concern is that education in academic literacy is being
supplanted by training in test taking. But what influence does the IELTS writing
test really have on teaching, and how different are writing classes in IELTS prepa-
ration courses from other forms of EAP?
To investigate these questions, this article compares the practices observed in
writing classes of two types: IELTS preparation classes directed at success on the
test and, as a suitable point of comparison, presessional EAP writing classes pro-
vided by universities to prepare learners for academic study.
LITERATURE REVIEW
Washback studies, investigating the effects of tests on the teaching and learning di-
rected toward them, have often involved direct observation of the behaviour of
teachers and learners in the classroom. The inclusion of an observational element
in such studies has been recommended as a means of contextualising, corroborat-
ing, or correcting data from surveys and interviews (Alderson & Wall, 1993; C.
Turner, 2001; Wall, 1996; Watanabe, 2004).
Table 1 summarises the methods and findings of recent case study investiga-
tions of washback in language education that have included an observational ele-
ment. These studies covered a wide range of educational contexts, with observa-
tion either focussing on a small number of participants observed intensively over a
sustained period (Alderson & Hamp-Lyons, 1996; Read & Hayes, 2003) or on a
lighter sampling of classes to allow for observation of larger numbers of teachers
and a broader perspective (Hawkey, 2006; Wall, 2005).
With the exception of Burrows (1998, 2004) and Hawkey (2006), all included
comparisons between different types of class. Wall (1996, 2005) and Cheng (2005)
focused on changes over time as a new test is introduced. This approach also in-
formed work relating to the recent update of the Test of English as a Foreign Lan-
guage (TOEFL; Hamp-Lyons, 2005; Wall & Horák, 2004). Alderson and
Hamp-Lyons (1996) and Watanabe (1996, 2004) compared the practices of teach-
TABLE 1
Studies of Washback in Language Education That Include Observation and Interview Data
Alderson and Test of English as Specialised Purpose-designed 16 classes: 8 3 focus group TOEFL washback may be
Hamp-Lyons a Foreign language instrument, general English, interviews with generated more by
(1996) Language institute in 2 teachers 8 TOEFL 3–12 students; teachers, materials, writers,
(TOEFL) United States preparation over 9 teachers in and administrators than by
1 week group and test. “Amount” and “type”
individual of washback vary
sessions according to test status,
relation of test to nontest
practice, degree to which
teachers and material
writers think about
appropriate methods and
their willingness to
innovate.
Brown (1998) International University-based Records of All classes Not reported IELTS preparation instruction
English language instructional designated for closely focussed on the test
Language institute focus and writing over 10 with regular timed practice
Testing System materials, weeks; 70 hours of IELTS tasks. IELTS
(IELTS) 2 teachers in IELTS and 30 preparation was more
hours in successful than EAP
non-IELTS EAP course in improving IELTS
course scores from entry to exit,
but small samples (9
IELTS, 5 EAP) limit
interpretation.
335
(continued)
TABLE 1 (Continued)
336
Burrows (1998; Certificates in Adult English COLT, 4 teachers Two 4-hour lessons 30 teacher Teachers vary in responses to
2004) Spoken and Migrant English per teacher interviews— innovations in assessment
Written English Programme including those in line with individual
(CSWE) (AMEP) in observed differences and experience.
Australia (conducted With Cheng (2003) and
prior to the Wall (2005), links
observation washback to theories of
phase) change.
Cheng (2005) Hong Kong 3 HK secondary Adaptation of 31 70-min classes Follow-up Following innovation,
Certificate schools (main COLT, c. 6 months interviews with teachers engaged in
Examinations study) 3 teachers before exam over observed exam-like activities e.g.,
in English 2 years teachers role-plays, and used
(HKCEE) (conservative) exam prep
materials in class, but
beliefs and attitudes to
learning did not change.
Differences in how
teachers cope with change.
Extensive support required
in implementation of
innovations.
Hawkey (2006) IELTS 10 language Cambridge IIS 10 IELTS 120 students, 21 Teacher preference for
schools in UK, instrument, 10 preparation teachers, and 15 task-based, often
Japan, teachers classes receiving interrelated macroskills
Cambodia institution activities, involving
administrators microskills relevant to
in focus groups IELTS. Focus on IELTS
but willingness to use a
range of materials and
methods.
Read and Hayes IELTS 2 language COLT and 22 hours of 32-hour 23 teacher Differential teacher practices
(2003); Hayes schools in New Cambridge IIS course and 28 interviews; at least partially linked to
and Read Zealand instrument, hours of 320- weekly institutional context.
(2004) 2 teachers hour (8-month) interviews with Greater pressure to “teach
course the teachers to the test” in private
observed language schools.
Wall (2005) Sri Lankan ‘O’ c. 50 secondary Purpose-designed 5 lessons observed 64 teachers in Teachers used textbook
level schools in 11 observation over 2 years focus groups; content but failed to follow
areas of Sri checklist (39–64 different follow-up suggested methods, e.g.,
Lanka classes observed interviews with reading for gist.
per round) at each class Proportion of classes was
varying periods observation dedicated to exam
ahead of exam preparation. Neglect of
speaking skills traceable to
exam content.
Watanabe (1996; Various Japanese 3 high schools in Adaptation of 964 min of exam Teacher follow-up Teachers vary in their
2004) university Japan COLT, 5 preparation and interviews with approaches to exam
entrance teachers 833 min of each preparation—informed by
examinations regular classes observation attitudes towards the exam.
over 6 months School culture an
important factor. Material
designed for exam
preparation may
sometimes be used for
other purposes.
337
338 GREEN
ers in test preparation and non-test-preparation classes, while Read and Hayes
(2003; Hayes & Read, 2004) compared two approaches to test preparation.
Three of the observational washback studies listed in Table 1 investigated
IELTS preparation classrooms. Brown (1998) compared practices in two courses
provided by the same institution: IELTS preparation and non-IELTS EAP. Read
and Hayes (2003) compared two IELTS preparation courses at different institu-
tions: one an intensive preparation course, the other combining IELTS preparation
with other forms of EAP. The ongoing IELTS Impact Study (IIS; Hawkey, 2006)
also includes observational data of IELTS preparation classes.
These studies all found IELTS to affect behaviour. Brown (1998) found that stu-
dents in the 10-week IELTS preparation course used IELTS preparation textbooks,
completed one Task 1 and one Task 2 essay each week (and no other writing), per-
formed three timed practice examinations, were informed about IELTS scoring
criteria, received feedback on the accuracy of their written work, and were in-
structed in strategies for writing under timed conditions. In contrast, students in the
EAP course worked on a 1,000-word project, did no timed writing, were instructed
in strategies for writing in academic contexts, and were encouraged to develop re-
search skills. Read and Hayes (2003) combined a broad survey with targeted class-
room observation of two teachers to provide mutual corroboration of findings. The
briefer and more intensive of the two IELTS classes they observed was more
narrowly concerned with the test and included more test practice under timed
conditions.
In common with other washback studies, Hawkey (2006) found variation be-
tween the 10 teachers he observed in how they conducted their classes, notably in
the number of opportunities they provided for learners to communicate together in
English. Teachers were willing to employ a variety of teaching methods and to use
material both within and beyond the textbook. However, both the institutions pro-
viding the courses and the students, who were motivated to succeed on the test, ap-
peared to constrain teachers to focus their instruction on IELTS. In class, teachers
showed a preference for task-based activities, targeting microskills they believed
to be relevant to the test.
Data from observational studies have informed insights into the complexity of
educational systems and test use, especially in relation to the role of the teacher.
However, a shortcoming identified by Bachman (2005) is the lack of a coherent ev-
idential link between test design characteristics and the practices observed or re-
ported in the classroom. In studies that involve predicting the effects of a test on in-
struction or learning, appeal is more often made to the views of educational
authorities (Cheng, 2005; Ferman, 2004; Qi, 2004), teachers (Banerjee, 1996), or
widely held public perceptions (Watanabe, 1996) than directly to the design of the
test instrument (as evidenced by available test materials or test specifications).
How might the design of the IELTS Academic Writing component be expected to
influence instruction?
WATCHING FOR WASHBACK 339
Watanabe (2004) suggested two sources of evidence that may be used to relate ob-
served practices to the influence of a test. One is evidence that test design features are
reflected in teaching or learning. The other is the absence of such features in teaching
or learning not directed toward the test (or directed toward an alternative test). Of the
three IELTS-related studies, only Brown (1998), in common with Alderson and
Hamp-Lyons’s (1996) study of TOEFL preparation, incorporated a comparison
with courses that were not directed toward the test. The inclusion of just two classes
in each of the Australian studies limits their generalisability, while the lack of a
nontest comparison in Hawkey (2006) makes it difficult to disentangle test influence
from teacher variables. This study involves observation of a larger number of teach-
ers and learners than Brown (1998) and Read and Hayes (2003) but includes the
comparison with nontest EAP classes missing from Hawkey (2006).
In considering the mechanisms of washback, a growing body of theory relates
test design, test use, and classroom behaviours, although as Wall (2005) argued,
too little is sufficiently informed by empirical evidence. Most of this work takes
the form of recommendations to test developers. Chapman and Snyder (2000) pro-
vided a framework for relating tests to educational practices, and Brown (2000)
cited Hughes (1989), Heyneman and Ransom (1990), Kellaghan and Greaney
(1992), Bailey (1996), and Wall (1996) in identifying features of a test that may be
manipulated in efforts to improve instruction. These embrace both contexts for test
use and technical qualities of the test instrument.
Drawing together these two elements in washback theory, Green (2003) pro-
posed the predictive model of test washback set out in Figure 1. The model starts
from test design characteristics and related validity issues of construct representa-
tion identified with washback by Messick (1996) and encapsulated in Resnick and
Resnick’s (1992) formulation of overlap, or the extent of congruence between test
design and skills developed by a curriculum or required in a target language use
domain. Test design issues are most closely identified with the direction of
washback—whether effects are likely to be judged beneficial or damaging to
teaching and learning.
The model relates design issues to contexts of test use, including the extent to
which participants (including material writers, teachers, learners, and course pro-
viders) are aware of and are equipped to address the demands of the test and are
willing to embrace beliefs about learning embodied therein. These features are
most closely related to washback variability (differences between participants in
how they are affected by a test) and washback intensity. Washback will be most in-
tense—have the most powerful effects on teaching and learning behaviours—
where participants see the test as challenging and the results as important (perhaps
because they are associated with high stakes decisions, such as university entrance;
Bailey, 1999; Cheng, 2005; Hughes, 1993; Watanabe, 2001).
A survey of IELTS preparation courses conducted in tandem with this study
(Green, 2003) indicated that the test was regarded both as important and challeng-
340 GREEN
FIGURE 1 A model of washback direction, variability, and intensity adapted from Green
(2003).
ing by a majority of learners in the IELTS preparation courses involved (70% rated
success on the test as “very important”). It also indicated that almost all the learn-
ers were taking the test because they intended to enter higher education in the
United Kingdom. Most demonstrated at least a basic knowledge of the test format
and viewed the direct test of writing as a positive feature. In short, the conditions
for intense washback to a majority of participants would seem to be in place.
Beyond a general prediction that the direct testing of writing in IELTS would
encourage the teaching of writing in test preparation programs, the washback
model suggests that features of task design will impact on the nature of that in-
struction. The IELTS Academic Writing component (AWC) is one of four
skill-based subtests in the IELTS battery intended to assess “the language ability of
candidates who intend to study or work where English is used as the language of
communication” (International English Language Testing System, 2005). On the
AWC, candidates are required to compose two short essays in 1 hour. The first
WATCHING FOR WASHBACK 341
• abstract and impersonal topics, but these are not targeted at learners’ chosen
academic subjects
• a limited range of text types (descriptions of iconic data and five-paragraph
persuasive essays)
• composition of texts based on personal opinions about how the world should
be.
In short, IELTS may imply an approach to instruction that passes over features
of canonical EAP, such as the integration of source material in learners’ writing,
learning of subject- specific vocabulary and text types, and strategies for coping
with the length of university-level written assignments. Preparing for IELTS may
involve learning how to shape texts to meet the expectations of examiners (as ex-
pressed through the scoring criteria) rather than those of university staff. The scor-
ing criteria may imply a focus in the classroom on grammar, vocabulary, and or-
ganisation of text, with limited attention given to quality of content. The timed
conditions may lead to practice in composing short texts under timed conditions,
perhaps in response to past (or imitation) examination papers.
342 GREEN
It should be noted that the IELTS AWC has undergone continual modification
and revision since its inception in 1989. The test has changed in certain respects
since this research was conducted. Where these changes are of relevance to the
study, this is indicated in the text. Regular updates on IELTS developments can be
found on the IELTS Web site (http://www.ielts.org).
METHODS
Methodology
The literature review has pointed to specific features that might indicate the influ-
ence of IELTS AWC test design on instruction. To investigate whether these pre-
dictions about the likely impact of IELTS would be borne out in this context and to
explore how instruction varied between IELTS preparation and EAP classes in ac-
ademic writing, a series of classroom observations were carried out at selected
U.K. centres. The observations provided evidence of how preparation for the
IELTS AWC was conducted in practice and how this compared with other forms of
EAP writing provision. The courses included in the study involved IELTS prepara-
tion in combination with varying proportions of EAP and general English classes.
However, comparisons were made at the narrower level of the writing class: be-
tween classes directed toward the IELTS AWC and EAP classes directed at prepar-
ing learners for writing in English at U.K. universities.
Instrumentation
One observation instrument that has been widely used in washback studies (Bur-
rows, 1998; Cheng, 2005; Read & Hayes, 2003; Watanabe, 1996) is the Communi-
cative Orientation to Language Teaching (COLT) observation schedule (Spada &
Fröhlich, 1995). The scheme is designed to be used flexibly and in real time, de-
scribing classroom events at the level of activities and their constituent episodes.
These are described qualitatively and recorded quantitatively under a series of
headings to build a picture of the balance of patterns of classroom organisation,
content, and student modality. Spada and Fröhlich (1995) explain the terms thus:
“Separate activities include such things as a drill, a translation task, a discussion or
a game. Three episodes of one activity would be: teacher introduces dialogue,
teacher reads dialogue aloud, individual students read parts of dialogue aloud”
(p. 14).
As Read and Hayes (2003) discovered, the COLT schedule could not, without
adaptation, identify features of direct test preparation, such as learning test-taking
strategies. Read and Hayes (2003) chose to supplement the COLT with the draft
IIS observation schedule, an instrument developed by the IELTS partners in col-
laboration with researchers from the University of Lancaster at the inception of the
IIS (Saville, 2000). This schedule includes lists of text types and activities antici-
WATCHING FOR WASHBACK 343
earlier survey of U.K. course providers; they had indicated willingness to partic-
ipate in further research and were conveniently located. Three of these (two uni-
versities and one private language school) declined the invitation. The remaining
institutions—six universities, three colleges of further education, and four pri-
vate language schools—represented a variety of courses for students intending
to enter higher education in the United Kingdom, both in IELTS preparation
and, for purposes of comparison, presessional courses in EAP offered by univer-
sities to prepare international students for the language demands of academic
study.
The intention was to observe a minimum of one writing-focused class for each
month of a course, although this did not always prove possible. As summarised in
Tables 2 and 3, some 36 scheduled classes were observed covering over 51 hours,
involving a total of 197 learners and given by 20 different teachers. The classes in-
cluded eight different EAP groups. Each of these was observed either once or
twice over periods ranging from 8 to 12 weeks to give a total of 13 classes. There
were 12 IELTS preparation groups. Each of these was observed on between one
and three occasions over periods of between 4 and 12 weeks to give a total of 22
preparation classes.
Following each observation, teachers were briefly interviewed about the class.
The interviews focused on five issues: the aims of the class, the extent to which the
aims had been met, the place of the focal class in a teaching sequence, the extent to
which the class could be described as typical of writing classes on the course, and
the influence of the IELTS AWC. Because of other commitments, teachers did not
always have the time to take part in these interviews. Nonetheless, 22 of the classes
were accompanied by interview data (8 EAP and 14 IELTS).
Analysis
Comparisons were made between writing-focused IELTS preparation classes and
EAP classes across courses (Table 4). The length of time spent on each activity
provided an index of how much time and what proportion of class time (calculated
as a percentage) was given to each form of participant organisation, content focus,
content control, and student modality. Mentions of tests or of test-taking strategies
were treated as simple frequency data. As the data were not normally distributed,
differences between classes were evaluated for significance (p < .05) through non-
parametric Mann–Whitney U tests.
The activities observed were also reviewed qualitatively in the light of the
washback predictions to explore (a) how far the predictions were borne out in
IELTS preparation and (b) how far they differentiated between writing-focused
IELTS preparation and EAP classes. Follow-up interviews provided an opportu-
nity to probe aspects of teachers’ behaviour.
TABLE 2
Summary of IELTS Preparation Classes Observed
Course
No. of Hours/ Hours/ Obs No. of
Centre Course Type Class Title Teacher Week Weeks Week Week Activities Min Ss
345
346
TABLE 3
Summary of EAP Classes Observed
Course
No. of Hours/ Hours/ Obs No. of
Centre Course Type Class Title Teacher Week Weeks Week Week Activities Min Ss
Episodes
Observed Min per per
Course Type Centre time Activities Episodes Activity Activity
347
348 GREEN
The observed IELTS and academic writing classes were of similar length. The ra-
tio of episodes to activities was also very similar across class types; there were ap-
proximately 10 episodes to every 3 activities, with each activity taking up just over
18 min on average.
Table 5 compares participant organisation by course type. The results of the
Mann–Whitney test displayed in the final row of Table 5 indicate that there were
no significant (p < .05) differences between class types in participant organisation.
The predominant form across classes was Teacher–Students/Class. Calculating
this as a percentage of total class time for each course type gives 56% of
presessional EAP class time and 54% of IELTS preparation class time (see Table
5). This did not generally involve lectures, but whole-class interactions centred on
the teacher. The differences found by Read and Hayes (2003) between teachers in
how they organised their classes were not repeated across the broader sample here,
nor did students spend as much of their time in class on individual test practice.
However, the proportion of class time spent on individual work on the same exer-
cise did increase as IELTS classes progressed: an average of 20% of class time in
the first half of these courses compared with 33% in the second half.
Reflecting the focused atmosphere of the classes observed, discipline was not
recorded as the main focus of any activity, although occasionally, a teacher re-
minded students to speak English rather than their first language. Procedural issues
took up 9% of time in IELTS and 12% of time in presessional EAP courses. Ob-
served variation in the amount of time spent on procedures also appeared to be a
matter of teacher style, rather than of test focus.
The results displayed in Table 6 reveal some evidence of differences between
class types in content. Forty-six percent of IELTS class time involved a major fo-
cus on language form (grammar and vocabulary) as compared with 22% of
presessional EAP class time. However, this did not prove to be significant in the
analysis (p = .07). Across classes observed, topics were mostly broad: relating to
issues of general rather than personal interest. Fifty-eight percent of time in IELTS
preparation classes and 48% of time in presessional EAP classes was spent work-
ing with topics in this category (p = .41). Little time (4% of IELTS and 1% of
presessional EAP classes) was spent on immediate personal topics (and these typi-
cally occurred only during brief introductory episodes). These proportions are
similar to those observed by Read and Hayes (2003). Academic topics—those top-
ics that became the focus of the class in their own right—occurred only in two
presessional EAP classes, making up 2% of the total presessional EAP class time
observed. Again, the differences between courses were not significant (p = .19).
Across the classes observed, topics were generally limited to one or two activi-
ties. Switches of topic would occur with each change of task. However, there was
evidence that topics were pursued in more depth in at least some EAP classes. Four
TABLE 5
A Comparison of Participant Organization in IELTS- and EAP-Focused Classes (Minutes)
349
350
TABLE 5 (continued)
Average IELTS preparation 87.64 46.73 0.41 14.50 0.00 23.55 0.77
EAP College G 61 31 0 3 0 26 0
College G 58 41 0 8 0 18 0
College A 123 64 14 49 0 0 0
College A 126 34 0 12 0 20 0
University D 89 28 0 13 0 43 0
University D 52 64 0 17 0 45 0
University E 97 62 0 16 0 7 0
University E 95 50 2 0 0 0 0
University E 93 78 0 16 0 6 0
University E 98 53 0 35 0 6 0
University F 73 52 24 6 0 20 0
University F 127 36 0 17 5 40 0
University F 79 20 23 0 22 0 3
Average EAP 90.08 47.15 4.85 14.77 2.08 17.77 0.23
Significance 0.68 0.90 0.09 0.68 0.06 0.69 0.73
(Mann–Whitney U test)
TABLE 6
A Comparison of Content in IELTS- and EAP-Focused Classes (Minutes)
Observed
Course Type Centre Time Procedural Discipline Form Function Discourse Sociolinguistic Immediate Broad Academic
351
352
TABLE 6 (Continued)
Observed
Course Type Centre Time Procedural Discipline Form Function Discourse Sociolinguistic Immediate Broad Academic
Average IELTS 87.64 7.77 0.00 39.18 16.09 13.27 0.18 3.73 49.55 0.00
preparation
EAP College G 61 8 0 0 0 54 0 0 0 0
College G 58 7 0 25 3 37 3 0 29 0
College A 123 18 0 0 0 90 0 0 58 0
College A 126 3 0 24 16 45 0 0 73 0
University D 89 2 0 33 0 6 29 2 99 0
University D 52 16 0 47 45 0 0 0 35 0
University E 97 7 0 22 74 0 0 0 0 0
University E 95 5 0 0 0 47 0 0 0 23
University E 93 28 0 48 0 3 0 5 9 0
University E 98 7 0 0 0 92 0 0 0 0
University F 73 3 0 0 0 89 0 0 89 0
University F 127 21 0 0 0 5 0 0 71 0
University F 79 10 0 38 0 0 0 0 60 0
Average EAP 90.08 10.38 0.00 18.23 10.62 36.00 2.46 0.54 40.23 1.77
Significance 0.68 0.36 1.00 0.07 0.22 0.08 0.28 0.72 0.41 0.19
(Man–Whitney
U test)
WATCHING FOR WASHBACK 353
354
TABLE 8
A Comparison of Student Modality in IELTS- and EAP-Focused Classes (Minutes)
Observed
Course Type Centre time Listening Speaking Reading Writing S&R S&W L&R R&W Other
355
356
TABLE 8 (Continued)
Observed
Course Type Centre time Listening Speaking Reading Writing S&R S&W L&R R&W Other
Average IELTS 87.64 44.64 9.64 12.41 7.50 2.32 0.82 0.14 8.77 3.05
Preparation
EAP College G 61 31 3 26 0 3 0 0 0 0
College G 58 46 8 2 0 0 0 0 0 3
College A 123 50 12 1 12 0 0 13 39 0
College A 126 15 0 1 20 27 0 16 0 0
University D 89 48 0 13 62 0 0 0 0 0
University D 52 47 8 17 25 17 10 0 0 0
University E 97 62 0 4 11 5 0 0 7 0
University E 95 52 0 0 0 0 0 0 0 0
University E 93 36 6 0 0 6 3 0 0 0
University E 98 53 0 7 0 12 0 5 22 0
University F 73 56 7 0 20 3 0 7 0 0
University F 127 36 17 41 0 0 0 0 4 0
University F 79 22 29 8 0 7 0 0 7 0
Average EAP 90.08 42.62 6.92 9.23 11.54 6.15 1.00 3.15 6.08 0.23
Significance 0.68 0.93 0.35 0.46 0.81 0.61 0.22 0.16 0.96 0.34
(Mann–Whitney
U test)
WATCHING FOR WASHBACK 357
toward the test, rather than collections of practice test material. In presessional
EAP courses observed, none of the titles included the acronym IELTS, and none
was intended primarily as a test preparation text.
The frequency data were indicative of broad similarities between courses in teach-
ing method. There was little variation associated with the type of course in partici-
pant organisation, content control, or student modality. In contrast, the qualitative
observations captured in the activities and episodes sections of the schedule high-
lighted differences in the content that were not well captured in the frequency data.
These qualitative observations are presented here in relation to the task description
categories developed by Weigle (2002).
However, IELTS preparation and EAP courses also had much in common. EAP
classes incorporated concern with the discursive essay genre; the rhetoric of de-
scription and evaluation; and cause and effect, comparison, and problem and solu-
tion patterns of exposition.
Cognitive Demands
As observed by Brown (1998), Hayes and Read (2004), and Hawkey (2006),
IELTS classes were explicitly directed toward test success. In preparation classes,
there were frequent mentions of IELTS and of strategies for dealing with the tasks.
Most of the materials used were either taken from IELTS textbooks or chosen by
the teacher to reflect the test. Frequent essay writing practice involved test practice
under timed conditions and completing tasks closely modelled on IELTS (often as-
signed as homework). Learners were encouraged to become thoroughly familiar
with the test tasks.
Instruction in EAP courses included stages that did not feature in IELTS prepa-
ration. As these courses progressed, learners were often encouraged to go beyond
the classroom to locate material and to integrate this into their writing. Learners
were never observed to leave an IELTS preparation classroom to carry out other
activities, but EAP classes observed included one group visit to a library, and in an-
other, individuals left the classroom to carry out research work elsewhere. Atten-
tion was given in EAP classes to issues of plagiarism, the appropriate use of
sources, and the compilation of bibliographies.
Dimension Observations
Subject matter Lists of typical IELTS topics were prepared for students on IELTS
preparation courses. IELTS topics were broad and were not explored
beyond task demands. Topics were sometimes sustained for longer in
EAP courses, and these included some attention to students’ academic
disciplines.
Stimulus IELTS-like task stimuli heavily used in IELTS courses. Focus on topics of
general interest. Students were encouraged to read newspapers or serious
magazines outside class to learn about relevant topics. This was not the
case in EAP courses.
Genre IELTS classes limited to writing only Task 1–and Task 2–type essays. EAP
courses involved wider range.
Rhetorical task Rhetorical tasks limited to those required for the test: predominantly
description for Task 1 and evaluation or hortation for Task 2. No
integration of academic sources (unlike EAP, where this was a focus).
Pattern of exposition IELTS classes limited to Task 1 and Task 2. EAP involved greater variety,
for example, classes on writing definitions
Cognitive demands IELTS classes involved reproducing information: from graph or from
personal knowledge. EAP classes included more writing from source
material.
Specification
Audience Some mention in IELTS classes of university lecturers (the specified
audience), but more attention given to examiner expectations, including
teaching of “ways to impress the examiner.”
Role IELTS classes did not focus on the role of the university student. This was
an occasional focus in EAP classes.
Tone, style IELTS preparation taught formal tone, but not including features of an
academic style such as hedging, which were included in EAP classes.
Length Essays for the IELTS classes were short (100–300 words). Writing
assignments were generally longer in EAP classes. IELTS students were
encouraged to count words. Teaching focus in some IELTS preparation
classes on how to use more words—sentence expansion.
Time allowed Timed essay practice activities were more frequent in IELTS classes.
However, redrafting of work and error correction were common practice
in both EAP and IELTS classes.
Prompt wording IELTS preparation involved question analysis based on the generic IELTS
task prompts.
Choice of prompts Students were sometimes given a choice of topic for IELTS practice essays,
but this was always limited to IELTS-like tasks.
Transcription mode Little word processing observed in IELTS writing (and none in class). Use
of IT was observed in EAP classes.
Scoring criteria Feedback on essays was mostly in the form of band scores in IELTS
preparation classes.
IELTS preparation involved
(a) Teaching of organisational templates for coherence, argument
structure
(b) Teaching grammar points relevant to test—error analysis/useful
structures
(c) Encouraging use of more formal and varied vocabulary
360
WATCHING FOR WASHBACK 361
Postobservation interviews with teachers did not cast light on this as teachers did
not generally seem to be aware of an emphasis on grammar in IELTS classes.
Although the analysis of the frequency data revealed few differences between
classes in organisation, modality, or content, interviews with teachers indicated
that they approached the two class types very differently. Thirteen of the 14
IELTS classes for which interview data were available included mention of the
IELTS writing test as an aim of the class. The one IELTS preparation class said
not to be influenced by the test came in the first week of a 6-week course. It
emerged from the postobservation interview that the central activity (finding
supporting examples for popular proverbs from students’ countries) was in-
tended as a means of preparing students for the demands of Task 2 without di-
rectly introducing test-like questions. Thus, although the teacher maintained that
the test did not directly influence the class, the demands of Task 2 were the ulti-
mate goal.
Class aims in IELTS preparation courses included both practice in performing di-
rectly test-derived tasks (Task 1 or Task 2 writing practice) and a wide variety of
other activities intended to build test-relevant skills. These aims were categorised as
follows:
Some of the aims for presessional EAP classes were similar to those reported
for IELTS:
Both IELTS and presessional EAP classes were said to follow a similar cycle,
with input from the teacher, practice writing tasks, and diagnostic feedback. From
the interview data it appeared to be the content of this cycle, not the process, that
differentiated presessional EAP from IELTS preparation classes. This reported
emphasis was borne out in the content of the activities and episodes observed. As
Brown (1998) found in the courses he observed, the IELTS preparation cycle was
closely tied to test content and practice of test tasks. Teachers said that the fre-
quency of test practice intensified as the courses progressed, and this was again
consistent with the observational data reported previously. In the presessional EAP
classes, teachers built toward longer writing tasks, with learners being given
greater independence (for research activities and library work) as the courses
neared completion. Where courses involved a final test (as with University F),
there might be some attention given to this, with students having opportunities for
test practice. However, in contrast to the IELTS preparation classes, which focused
on the test throughout, the teacher of the test familiarisation class observed at Uni-
versity F reported that this would be the one session (in week seven of the
eight-week course) to concentrate on the test.
In one combination course the teacher reported that as the test date approached,
IELTS preparation leaked across the curriculum, exerting an influence on the con-
tent of classes beyond the identified IELTS component, with students requesting
practice in test tasks during non- IELTS lessons. This was consistent with the ob-
served mention of IELTS in non-IELTS classes at College A and University E,
where learners requested more information about the AWC in nontest classes.
Teachers of IELTS courses claimed that writing class content was entirely dic-
tated by the AWC. Conversely, those of EAP courses either dismissed the idea that
IELTS had any influence on their classes or suggested that it served as a useful
baseline for their teaching; they could assume that learners arriving in their courses
with an IELTS score would have some knowledge of how to write a basic
five-paragraph essay.
Student Work
It was plain that the work collected from IELTS and EAP classes differed in the va-
riety of tasks completed by learners. Reflecting the focus reported by teachers and
seen in the classes observed, all written work collected from IELTS classes con-
WATCHING FOR WASHBACK 363
sisted of responses to practice IELTS tasks (with varying degrees of guidance from
supporting materials). EAP tasks ranged from timed writing exercises on broad
topics of general interest based on personal knowledge or experience (similar to
IELTS Task 2) to extended projects on topics relating to students’ academic disci-
plines and including tables of contents, references, and bibliographies.
The task responses also differed in their presentation. All but two of the practice
IELTS tasks were handwritten, while the work collected from presessional EAP
courses, with the exception of work done in class under time constraints, was all
word processed. As anticipated, there was a much greater range in the length of
EAP task responses than of IELTS responses. IELTS tasks collected ranged in
length from 98 to 445 words (compared with the IELTS requirements of 150 and
250 words), while the presessional EAP essays ranged from 128 to 3,495 words. It
will be interesting to see whether the recent introduction of a computer-based
IELTS, which offers test takers the option of word processing their responses, will
lead to greater use of computers in IELTS preparation courses.
Of the six IELTS teachers, five marked student work using IELTS band scores,
with one providing a breakdown of the score by the criteria used on the test. The
five teachers giving scores often added a comment to the awarded score, such as
“good 6” or “5.0+.” Two of the three EAP teachers provided scores (one as marks
out of 20, the other as percentages), while the third made written comments but did
not give a score. One of the teachers giving scores used an analytical style of re-
porting that might have been influenced by IELTS. This used the criteria “content
and task achievement,” “organisation and coherence,” “range and accuracy of lan-
guage,” and (clearly beyond the scope of IELTS) “improvement between drafts.”
CONCLUSIONS
Although the primary concern of this article has been differences between IELTS
and EAP classes, it should be emphasised that, as the washback model would pre-
dict, there were found to be considerable areas of overlap. Organisation, content
control, and student modality were all very similar across classes. Differences in
language content were nonsignificant. Excluding the references to the test, many
of the activities in the IELTS classes observed might not have been out of place in
the EAP classes. Both class types involved brainstorming and planning, with fre-
quent practice in extensive writing. Although there was some variation, both en-
couraged a formal, objective style of writing; offered instruction in discourse-level
organisation; were concerned with the requirements implicit in task instructions;
and involved work (often in the form of remediation) on grammar and vocabulary.
There was also evidence that teacher variables (such as common beliefs about
the value of editing and redrafting) may encourage practices that cannot be pre-
dicted from test design. Given the lack of differences between class types in the
frequency data, it appears that many of the differences observed between classes
364 GREEN
REFERENCES
Alderson, J. C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A study of washback. Lan-
guage Testing, 13, 280–297.
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115–129.
Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2,
1–34.
Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing.
Language Testing, 13, 257–279.
Bailey, K. M. (1999). Washback in language testing. Princeton, NJ: ETS.
Banerjee, J. (1996). The design of the classroom observation instruments (Internal Report). Cambridge,
UK: University of Cambridge Local Examinations Syndicate.
Banerjee, J. (2000). Using English language screening tests in your institution. Paper presented at the
BALEAP Professional Issues Meeting, Trent University, Nottingham, UK.
Bridges, G., & Shaw, S. (2004). IELTS writing: Revising assessment criteria and scales. Research
Notes, 18, 8–12.
Brown, J. D. H. (1998). An investigation into approaches to IELTS preparation, with particular focus on
the Academic Writing component of the test. In S. Wood (Ed.), IELTS research reports (Vol. 1, pp.
20–37). Sydney: ELICOS/IELTS.
Brown, J. D. (2000). University entrance examinations: Strategies for creating positive washback on Eng-
lish language teaching in Japan. Shiken JALT Testing and Evaluation SIG Newsletter, 3(2), 4–8.
Burrows, C. (1998). Searching for washback: An investigation of the impact on teachers of the imple-
mentation into the adult migrant English program of the assessment of the certificates in spoken and
written English. Unpublished doctoral dissertation, Macquarie University, Sydney.
Burrows, C. (2004). Washback in classroom-based assessment: A study of the washback effect in the
Australian Adult Migrant English Program. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback
in language testing: Research contexts and methods (pp. 113–128). Mahwah, NJ: Lawrence
Erlbaum Associates, Inc.
Chalhoub-Deville, M., & Turner, C. E. (2000). What to look for in ESL admission tests: Cambridge cer-
tificate exams, IELTS, and TOEFL. System, 28, 523–539.
Chapman, D. W., & Snyder, C. W. (2000). Can high stakes testing improve instruction: Re-examining
conventional wisdom. International Journal of Educational Development, 20, 457–474.
Cheng, L. (2005). Studies in language testing: Vol. 21. Changing language teaching through language
testing: A washback study. Cambridge, UK: Cambridge University Press.
Coffin, C. (2004). Arguing about how the world is or how the world should be: The role of argument in
IELTS tests. Journal of English for Academic Purposes, 3, 229–246.
Deakin, G. (1997). IELTS in context: Issues in EAP for overseas students. EA Journal, 15, 7–28.
Douglas, D. (2000). Assessing languages for specific purposes. Cambridge, UK: Cambridge University
Press.
Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and learning. In
L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Research contexts and
methods (pp. 191–210). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Green, A. (2003). Test impact and English for academic purposes: A comparative study in backwash
between IELTS preparation and university presessional courses. Unpublished doctoral dissertation,
University of Surrey, Roehampton, UK.
Hale, G., Taylor, C., Bridgeman, B., Carson, J., Kroll, B., & Kantor, R. (1996). A study of writing tasks
assigned in academic degree programs. Princeton, NJ: ETS.
Hamp-Lyons, L. (2005). The effect of the newly developed TOEFL format on EFL/ESL teaching and
learning in four countries. ILTA Online Newsletter, 1. Retrieved November 4, 2005, from http://
www.iltaonline.com
366 GREEN
Hawkey, R. (2006). Studies in language testing: Vol. 24. Impact theory and practice: Studies of the
IELTS test and Progetto Lingue 2000. Cambridge, UK: Cambridge University Press.
Hayes, B., & Read, J. (2004). IELTS test preparation in New Zealand: Preparing students for the IELTS
Academic Module. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Re-
search contexts and methods (pp. 97–112). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Heyneman, S. P., & Ransom, A. W. (1990). Using examinations and testing to improve educational
quality. Educational Policy, 4, 177–192.
Hughes, A. (1989). Testing for language teachers. Cambridge, UK: Cambridge University Press.
Hughes, A. (1993). Backwash and TOEFL 2000. Unpublished manuscript, University of Reading,
Reading, UK.
International English Language Testing System. (2005). The IELTS handbook. Cambridge, UK: Uni-
versity of Cambridge Local Examinations Syndicate.
Kellaghan, T., & Greaney, V. (1992). Using examinations to improve education: A study of fourteen Af-
rican countries. Washington, DC: The World Bank.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13, 241–256.
Mickan, P., & Slater, S. (2003). Text analysis and the assessment of academic writing. In R. Tulloh
(Ed.), IELTS research reports (Vol. 5, pp. 59–88). Canberra: IELTS Australia.
Moore, T., & Morton, J. (1999). Authenticity in the IELTS Academic Module Writing Test: A compara-
tive study of Task 2 items and university assignments. In R. Tulloh (Ed.), IELTS research reports
(Vol. 2, pp. 64–106). Canberra: IELTS Australia.
Qi, L. (2004). Has a high-stakes test produced the intended changes? In L. Cheng, Y. Watanabe, & A.
Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 171–190).
Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Read, J., & Hayes, B. (2003). The impact of the IELTS test on preparation for academic study in New
Zealand. In R. Tulloh (Ed.), IELTS research reports (Vol. 5, pp. 153–206). Canberra: IELTS Austra-
lia.
Read, J., & Hirsh, D. (2005). English language levels in tertiary institutions (Export Levy Programme
Project E4). Wellington: Education New Zealand.
Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educational
reform. In B. G. Gifford & M. C. O’Conner (Eds.), Changing assessments: Alternative views of apti-
tude, achievement and instruction (pp. 37–75). Boston: Kluwer Academic.
Saville, N. (2000). Investigating the impact of international language examinations. Research Notes, 2,
4–7.
Spada, N., & Fröhlich, M. (1995). COLT observation scheme: Coding conventions and applications.
Sydney: National Centre for English Language Teaching and Research, Macquarie University.
Thorp, D., & Kennedy, C. (2003). What makes a “good” IELTS answer in academic writing? (IELTS
Research Report). London: British Council.
Turner, C. (2001). The need for impact studies of L2 performance testing and rating: Identifying areas
of potential consequences at all levels of the testing cycle. In C. Elder, A. Brown, A. K. Hill,
N. Iwashita, T. Lumley, T. McNamara, & K. O’Loughlin (Eds.), Experimenting with uncertainty: Es-
says in honour of Alan Davies (pp. 138–149). Cambridge, UK: Cambridge University Press.
Turner, J. (2004). Language as academic purpose. Journal of English for Academic Purposes, 3,
95–109.
Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education and
from innovation theory. Language Testing, 13, 334–354.
Wall, D. (2005). Studies in language testing: Vol. 22. The impact of high-stakes testing on classroom
teaching: A case study using insights from testing and innovation theory. Cambridge, UK: Cam-
bridge University Press.
Wall, D., & Horák, T. (2004). TOEFL impact study: The influence of next generation TOEFL on prepa-
ration classes in Central and Eastern Europe. Language Testing Update, 36, 40–41.
WATCHING FOR WASHBACK 367
Watanabe, Y. (1996). Does grammar translation come from the entrance examination? Preliminary
findings from classroom-based research. Language Testing, 13, 318–333.
Watanabe, Y. (2001). Does the university entrance examination motivate learners? A case study of
learner interviews. In A. Murakami (Ed.), Trans-equator exchanges: A collection of academic pa-
pers in honour of Professor David Ingram (pp. 100–110). Akita, Japan: Akita University.
Watanabe, Y. (2004). Methodology in washback studies. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.),
Washback in language testing: Research contexts and methods (pp. 19–36). Mahwah, NJ: Lawrence
Erlbaum Associates, Inc.
Weigle, S. C. (2002). Assessing writing. Cambridge, UK: Cambridge University Press.
368