UNIT 1: Introduction to Language Testing

LEARNING OBJECTIVES
At the end of the unit, you should be able to:
1. introduce fundamental concepts of assessment
2. contrast classroom assessment with large-scale testing
Language testing plays a crucial role in many peoples lives, acting as gateways at important
transitional moments in education, in employment and internationally. Language tests are
devices for institutional control of individuals especially for those who are in the teaching
professional relying on information from tests to make decision on the placement of students
or particular course. At various stages of students learning, teachers may want to test their
students ability in the English language. For example, if the students arrive at a school and
need to be placed in a classroom at an appropriate level, they may do a placement test. The
placement test often takes the form of a number of discrete (indirect) items, together with an
oral interview and perhaps a longer piece of writing. The purpose of the test is to identify not
only what students know or what students dont know, but also what students have learned
after the completion of a lesson or unit. When the classroom tests are tied to effectively
written lesson objectives, the teacher can analyze the results to see where the majority of the
students are having problems with in their class. These tests are also important when
discussing student progress at parent-teacher conferences. Testing is a process, so a test is an
instrument which need to be developed to measure a sample of behaviour.
For many, language tests may conjure up an image of an examination room, a test paper with
questions, desperate scribbling against the clock. As teachers we must, try to avoid creating
negative reactions in the classroom testing we get involved in. We should always test for
best, for example, give students the chance to demonstrate their optimal ability. Classroom
testing should be separated from the teaching that precedes it. Tests provide many of the
needs of teaching. For example, when we teach our students, we have to monitor their
progress. Topic tests which assess how well a student has mastered what has been taught (e.g.
the present tense, getting the thesis statement) serve this purpose. At different stages in
teaching, a teacher has to come up specifically what areas pose difficulties for students.
Testing and teaching are very close interrelated. Tests focus on assessing the products of
learning. Teaching emphasizes on assisting students to succeed in the process of learning. It
seeks to organize the context of learning, the materials, the methodology, the classroom in
such a way that the students have the possible opportunity of learning what he/she is
attempting to learn.
Testing is a universal facet of social life. Throughout history people were tested to prove their
abilities. However, currently tests have proliferated rapidly (McNamara 2000: 3) resulting in
the appearance of many test types and procedures because of several educational issues. To
understand those test procedures, we should first conceptualize assessment and testing
especially of language. Let us begin with the timing of the test that technically distinguishes
three kinds of testing such as quizzes that are short, 5-10 minutes and includes just the current
and tests that last 30-60 minutes and covers one or more units; whereas, exams are two hours
or longer and contains at least half of the course content (Brumfit and Johnson, 1979).
Whatsoever the length of time, assessment is basically a measurement process and
measurement itself is used to determine the degree of something (Earl and Katz, 2006) by
obtaining numerical descriptions (Brumfit and Johnson, 1979). So, language testing is a tool

of measurement for determining test-takers language proficiency (Piggin, 2012). Simply, the
next step will be evaluation which uses the information collected by measurement. So, tests
measure the achievement of students but the grades assigned depending on test results are
evaluations of students achievements (Brumfit and Johnson, 1979).At the end, there are
three reference points for considering students achievements and performance; firstly,
students performance in relation to pre-determined criteria (criteria-referenced); secondly,
students performance compared to other studentsperformance (norm-referenced); and
thirdly, students performance in relation to his/her previous performance (self-referenced)
(Earl and Katz, 2006). In other words, testing demonstrates students proficiency in a set of
skills by comparing students performance to a criterion and it determines students ranks by
comparing the students performance to each other (Brumfit and Johnson, 1979) and then
each students progress over time.
Test
Carroll (1968) provides the following definition of a test:
a psychological or educational test is a procedure designed to elicit certain behavior
from which one can make inferences about certain characteristics of an individual.
(Carroll 1968: 46)
From this definition, it is understood that a test is a measurement instrument designed to elicit
a specific sample of an individuals behaviour. Tests may be formal or informal. Tests try to
assess a students previous learning. Teachers use tests as an instrument or tool to assess
students learning in the classroom. Tests are be used to help decide who will be accepted or
rejected by an institution. Tests are used to identify a particular performance level of the
student and to place him or her at an appropriate level of instruction. The test content may be
based on either a theory of language proficiency or on the learning objectives of the syllabus
to be taken. Tests are used to determine a persons strengths and weaknesses in order to
improve performance. We can use tests to provide information about the effectiveness of
programs of instruction. The aptitude test is conceived as a prognostic measure that indicates
whether a student is likely to learn a second language readily. It is generally given before the
student begins language study, and may be used to select students for a language course or to
place students in sections appropriate to their ability. In short, we test to monitor progress, to
obtain feedback, to diagnose problems, to screen and select, to place students into courses, to
evaluate programmes, to provide research criteria and also to promote confidence in a
system.
The testing process begins from the preparation stage, followed by the implementation stage
and finally with an examination stage. Through these three stages of testing process, the
teachers are able to find out whether the students have successfully mastered certain skills,
and acquired the knowledge which has been learned.

Measurement

Measurement in the social sciences is the process of quantifying the characteristics of persons
according to explicit procedures and rules (Bachman, 1990). It occurs when a quantitative
value is assigned to the behavioral sample collected by using a test. Measurement is the
process of obtaining a numerical description of the degree to which an individual possesses a
particular characteristic. It answers the questions how much. Hence, measurement includes
quantification of characteristics in ways other than test such as observations, rating scales, or
anything that follows us to obtain in a quantitative form.
Measurement is a process used to establish the quantitative value of a certain test in learning.
This quantitative value is usually shown in a fixed unit of number, in terms of percentage or
grade, by using a related measuring instrument. During the process of teaching and learning,
measurement covers all aspects of testing, including the use of ordinal scale or interval scale
to determine the position and achievement of students in the class or among classes. Ordinal
scale is a measuring instrument used to rank students grades or positions according to the
marks they obtain in a certain test. Interval scale which contains unit of mark with the same
weightage in its value, is not only used to determine the students grades in the same class,
but it is also used to determine the students performance and achievement. Interval scale,
which covers marks and grades, is also used to compare the level of achievement among the
students in different classes.
Assessment
Assessment is a process performed to gain an understanding of an individual learners
strengths and weaknesses in order to make appropriate educational decisions. It is an
ongoing process involves the systematic collection, analysis, and integration of information.
It describes the process of determining the extent to which specific objectives have been
attained. It is also an act of collecting and providing information to enable decision makers to
function more intelligently. Assessment starts with process of collecting data to gain an
understanding of strengths and weaknesses of learners, teachers or learning programs. It ends
with a description of frequently voiced concerns about assessment and subsequent decision
making.
Evaluation
Evaluation is the systematic process of collecting, analyzing, and interpreting information to
determine the extent to which learners are achieving instructional objectives. It determines
the worth or merit of an evaluation. Evaluation is the identification, clarification, and
application of defensible criteria to determine an objects value, quality, utility, effectiveness,
or significance in relation to those criteria. It uses inquiry and judgment methods including
determining standards for judging quality, collecting information, applying the standards to
determine value, and effectiveness. It includes a number of techniques that are indispensable
to the teacher. However, evaluation is not merely a collection of techniques; it is a process, a
continuous process that underlies all good teaching and learning. The evaluation procedure
contains the aspects of test, measurement, analysis and interpretation. It is implemented in the
following stages:
Test
Preparation

Test
Implementation

Marking and
Measurement

Analysis of
Results

Interpretation &
Follow-up action

The above explanation shows that test and measurement are two basic components of the
evaluation activities. The relationship between the test, measurement and evaluation is
illustrated in the following Figure 1:

Evaluation

Measurement
Test

Figure 1: Relationship between Test, Measurement and Evaluation

The why in Language Testing
The why is aligned to the critical perspective in language testing that may be linked to
Shohamys (1998) seminal interpretation of test power. This perspective involves a
significant paradigm shift in language testing. Shohamy also notes that critical language
testing is ideologically loaded in favour of the powerful. There is an unequal power
relationship between the test as an organization and the demands put on test takers. Test
takers are the true victims of tests, they are seldom represented.
.in the testing literature test takers are often kept silent; their personal
experiences are not heard or shared. It seems that the testing profession
.(is) not interested in such accounts..listening to the voices
of test takers can provide testers with a new and unique perspective and
a deep insight into tests and their meanings (Shohamy, 2001:7)
Four standards for professional practice among language testers (Shohamy, 2001) are:
a) utility
b) accucary
c) feasibility
d) fairness
Fairness
The ensuring of a test method that is conducted legally, ethically and with due regard to the
welfare of the tested individuals as those affected by test results (p.152). Test fairness
generally giving equal treatment to all individuals but it has been extended to encompass the
need to address the consequences of assessment.the intended and unintended effects
on the individuals being assessed (Lynch, 2001:232).
McNamara (2000) on the social responsibility of the language tester:

Language testers themselves meanwhile stand to benefit from a greater

awareness of language testing as a social practice. It may lead to a more
responsible exercise of the power of tests and a deeply questioning approach
to the question of test score meaning that lies at the heart of the validity of
language tests. (McNamara, 2000:77).
The power that tests have is discussed eloquently by Shohamy (2001a: 374). In her words,
Tests play a major role as their results have far-reaching implications for individuals and
educational systems; tests can create winners and losers, successes and failures, the rejected
and the accepted. Yet, it is often performance on a single test, on a single occasion, at one
point in time that leads to the irreversible and far-reaching high stakes decisions.
In understanding the power of tests, Bachman (2000) is one more testing theorist who agrees
that tests are not value-free, culturally neutral tools developed in psychometric test tube
environments. They are, in fact, powerful tools with far-reaching and sometimes detrimental
consequences for individuals and society at large. Test scores, for one thing, can be used for
admission to universities or to a profession, as well as for employment decisions and grants
of citizenship. Bourdieu (1991 cited in Shohamy, 2001a) highlights the symbolic power of
tests, arguing that they can be used as rites of passage, as a means of socializing the public,
and creating a dependency of test takers on tests as a main criterion of worth. While tests
represent cultural capital, the knowledge contained in tests, on the other hand, represents in
the minds of test takers and users what counts as legitimate, worthwhile knowledge, and,
thus, tests can provide the means for controlling knowledge.

People differ in their relative knowledge, skills, abilities, competencies, personality, interests,
and values. These characteristics are called constructs. Constructs are used to identify
personal characteristics and to distinguish between people in terms of how much they possess
of
such characteristics. Constructs cannot be seen or heard, but we can observe their effects on
other variables. Tests give us information about characteristics we may not otherwise observe
that, in turn, can be used to help individuals and organizations to develop their skill base or
competencies. Assessment strategies should be developed with a clear understanding of the
knowledge, skills, abilities, characteristics, or personal traits you want to measure. It is also
essential to have a clear idea of what each assessment tool you are considering using is
designed to measure.

Activity 1:
1. Is test practice relevant to test takers needs and interests?
2. Central to the issue of language testing is the question What are language tests
measuring? (Brown and Hidson, 2002).

This question can be answered in part by the test format adopted in the tests.
In traditional testing, objective tests are commonly used in language education.
In part the purpose of the test is determined by various stakeholders.

Stakeholders involved in assessment are as the followings:

a)
b)
c)
d)

Students
Teachers
Society
Educational institutions

Different purposes of assessment may overlap and sometimes are in tension with one another,
e.g. Teachers needs may override students needs.

Activity 2:

1. Test takers are the true victims of tests, they are seldom represented. Discuss.

Language testing involves the assessment of some or all aspects of the language ability of
individuals in some context (not necessarily that of a language class) and for some set of
purposes (not necessarily common to all parties). Testing is sometimes used almost
interchangeably with assessment and in this spirit is taken here as a broad cover term for
both formal and informal assessment procedures. Testing is used more narrowly to denote
only those formal modes of assessment that are officially schedules, with clearly delimited
time on task and strict limitations on available guidance. Language teaching is not the only
context in which language use is evaluated or language abilities are tested, but it is the
context or set of contexts for which language testing and language program evaluation are
sometimes brought together. It is convenient to follow the best practice of Nunan (1992) in
distinguishing program evaluation from student assessment, and to take assessment (or
testing) data as one source of information among others for program evaluators to consider.
The key theme that links testing and evaluation in the context of language teaching is that of
decision-making. Information about language learner performance, obtainable through
various kinds of testing, is clearly relevant in principle to decisions about what to teach or re-

teach as a course takes place, or about what to emphasise more or less in future versions of
course. Decision-making is also critical in accounts of language program evaluation, which is
distinguishable from other studies of language teaching precisely because it is oriented
towards decisions about current or future practice.

Activity 3:

1. What are your own main learning goals for this course at this stage?

2. What reservations, if any, do you have about language testing and examining, and about
evaluations of language teaching, as these are carried out in teaching-learning situations
known to you?

3. What reservations, if any, have you heard other people express in these areas?

4. Assuming that you were to meet an educator from a very different and unfamiliar
education system, what could you most usefully tell this person about language testing and
evaluation issues within the local context known to you? Also, what questions would you ask
about language testing and evaluation in the unfamiliar context.

Activity 4:
1. How do you define testing, test, evaluation, measurement and assessment?

2. What phrases come to your mind that include the words testing, test, tests,
evaluation, measurement and assessment?

*There are no generally right answers, of course, but your responses should afford an
additional basis for comparison when you read on.

Activity 5:
1. How important are tests and examinations in the education system best known to you?

2. What is your position on this situation? Are tests / examinations important enough? Are
they too important?

3. Are tests / examinations in use appropriate for the educational goals that teachers value?
What are these goals?

4. Are language teaching programs evaluated in a systematic way? Are they evaluated at all?
If so, who carries out the evaluation? Do teachers have a role, and is this role suitable one? Is
evaluation also linked to staff appraisal, and should this be the case? Why / why not?

Activity 6:

Critique Article: English Language Test-taking Experience in the Malaysian Context by Chan
Swee Heng and Ain Nadzimah Abdullah

An overview of the article is given below. Critique the article in your own words and discuss.
Submit your work to the class after the discussion ends.

The Malaysian English Language Test-Taking Experience

An exploratory study was undertaken among a group of tertiary students who are enrolled in
a English language course to provide an understanding of the perspectives that the test-takers
have about their test experience. The experience could have been described as illustrating
culturalised test characteristics that have been built up and perpetuated over the years as
participants in the Malaysian school system. The sample size was 252 respondents who
represented a cross-sectional population of tertiary students at University Putra Malaysia.
Findings:
1. Test effects
* Test-taking experience was threatening and frightening (44%).
* Test results usually have detrimental consequences (53%).
* Tests are accurate indicators of achievement (56%).
* Feeling of failure is demoralizing (44%).
* Teachers teach to test (61%).
2. Test expectations
* My school tests are detached from reality (37%).
* School tests are bias (34%).
* Teachers should be better at arguing our achievement than a standardized test (64%).
* When I attend an English language course, my first concern is how I am tested (55%).
* I always do well in an English language test (45%).
* My teacher always tests what is taught in my class (60%).
* I would like to have more say in deciding what to test in schools (55%).
* Results in standardized tests should be complemented by grades in school based tests
(57%).
* My school teachers are skillful in test making (58%).
* Most of the time commercial tests from work books are used for assessment (57%).
* My teachers take a long time to mark my tests (37%).
* Public exams should be abolished (29%).
* Tests are best taken in formal settings (42%).
* My teachers pay more attention to students who do well in tests (53%).
* Public exams are bias (29%).
* Ask teachers for help (59%).
* When I prepare for tests, I prefer to do it in groups (32%).
* Grouping students into different levels and classes based on tests is a fair practice (44%).
3. Test-taking Skills
* I do not know enough of test-taking skills (42%).
* Performance in test is governed by luck (39%).
* I prefer objective tests to subjective tests (63%).
Gerhard and Oprandy (1999) in recommending that teachers should be explorers of learners
characteristics listed some beliefs and assumptions that guide language teaching awareness.

* Taking responsibility for our own teaching (and testing).

* The need for others.
* Description over prescriptions.
* A nonjudgmental stance.
* Attention to language and behaviour.
* Avenues to awareness through exploration.
* Personal connections to teaching.
* Attention to process.
* A beginners mind.
Conclusion
Testing approaches attempt to answer the how, and the what and currently places emphasis
on the why. Tests have political nuances. Decisions related to testing policies should be made
consciously and purposefully (to adopt, adapt or develop). Practical issues such as fairness,
cost, ease of construction, administration and scoring were discussed. The tension between
reliability and validity were discussed too.

Activity 7:
1. How do you make judgments about the learning success of students?
2. How much does the role of tests play in judging learning success?