LECTURE 3: TESTING AND EVALUATION

PRINCIPLES OF LANGUAGE TESTING AND EVALUATION

A test is used to measure the performance of a student. Some tests are given under examination conditions, with invigilators and special conditions- formal test. Continuous assessment- often made by teachers all the time in which they are making judgements about the pupils performance .
- listening to pupils - observing pupils working individually, in pairs or in groups - marking their written work

 Tests serve many of the needs of teaching,

as we teach we need to monitor progress. Testing and teaching are very closely interrelated but they do not have the same focus. Tests focus on assessing the products of learning and teaching concentrates on enabling students to succeed in the process of learning.

 Tests try to assess a students previous learning. A teacher is to ensure that learning takes place and tests are one of the tools to help a teacher to do so. A test is only a sample of what the student is suppose to know as it picks out the most important aspects of the skills that have been

taught. The performance of the given tests is a way of measuring a students mastery of the language areas he is tested on.

 A test is often used as a guide on what would

be the most important things to teach and it influences what is taught (backwash effect of a test). In practice, a test is not always the best guide as it often leaves out important skills because of practical constraints. For example, the UPSR & PMR examination do not have the Listening and Speaking component. Therefore, the teachers do not pay sufficient attention to these important skills.

 Some teachers use the formats used in testing

for teaching purposes so as to help students to pass examination. This is wrong as what is taught should be decided by reference to how important the skill is to a students present and future life. A test usually seeks the most economical way of determining what the student already knows. Therefore leaving out many important content which are used for communication and normal language used.

Purpose of Testing
A teacher needs a test in order to:

- put students into groups according to their abilities. - to find out whether a student has learnt what he has been taught. - to discover if students seem to be able to cope with the curriculum and to evaluate the effectiveness of the methodology - to compare the performance of one student against the performance of their peers

Types of Test
Tests fall into 2 categories:

- Summative Tests - Formative Tests

Formative Tests

Formative Tests are used during a course to monitor the progress of the students. The most common formative tests are informal. Formal test are also sometimes designed. Type of Formative Test 1. Topic/ Progress Tests - to test how well pupils have learnt what has been taught - to test at the end of each topic to be covered. For example: at the end of each unit or module to find out if students have mastered what was taught.

.....Types of Test: Formative

Purpose of Topic/Progress Tests

- To evaluate effectiveness of teaching and teaching material. - To obtain information about individual pupils level of mastery of skills taught. 2. Diagnostic Tests - attempts to diagnose areas of difficulty - help teachers in deciding for whom and what areas of remedial work needs to be given.

.....Types of Test: Formative

Purpose of Diagnostic Tests

- to obtain information about pupils strengths and weakness in a specific area. - This information would be an important input for decisions on material design, provision of remedial classes, etc.

.....Types of Test: Summative

Are those that test all that has been taught

in a course. It usually comes at the end of the course or at the end of each year of the course. End-of-year examination is a summative test. Type of Summative Test 1. Achievement tests - which tests what has been taught throughout the course/year.

.....Types of Test: Summative

Purpose of Achievement Test

- to evaluate effectiveness of teaching and teaching materials and methods.

2. Placement Tests - are used to help group the students according to overall entry-level proficiency of students. - to assess all that a student knows at the point he begins a new course of study

.Types of Test: Summative

Purpose of Placement Test - to help place pupils into groups according to ability

3. Proficiency Tests - it is important but not often used in schools. - it is not pegged to any syllabus or course. - it attempts to assess a students future performance in a specific area of learning or work. - it is intended for students from all kind of background

Types of Tests
Norm-Referenced Tests

- if the scores are used to compare a students performance with peers. - a student is considered average when his results are compared to results of excellent students, and appear to be excellent when compared to another group that is less proficient. - good and poor are relative concepts in this kind of evaluation.

Types of Tests
Criterion-Referenced Test

- when a student is required to achieve a pre-determined level of proficiency in doing a specific set of tasks, his performance is measured not against the performance of his peers but in terms of whether he is able or not to achieve the stipulated, fixed standard

Types of Tests
Discrete Point Tests

- relate to how this test is constructed - it aims at testing one and only one point of grammar, phonology, vocabulary, or etc. at a time. Integrative Tests - seek to test global proficiency - may isolate discrete features like pronunciation, fluency and so on. - dictation, cloze and all tests based on communicative tasks

Types of Tests
Communicative Tests

- are based on activities which a person would recognize as relevant to the use of the target language - giving directions in English is a communicative test ( would have to do outside a classroom)

What makes a good test?

Valid Reliable

Practical

Criteria of A Good Test

1.

A good test must be valid, reliable and practical. Validity

- a test is said to have validity when it tests what it claims to be testing. - is concerned with the issue of whether a test measures what the tester wants to know about the performance of the target students.

- there are 5 types of validity:

i. ii. iii. iv. v. Face Content Predictive Concurrent Construct

Validity summarized by Davies (1968)

Type of Validity Test

Face

Looks like a good one to learner /layman

Content

Accurately reflects the syllabus it is based on Predictive Accurately predicts future performance Concurre Gives similar results to already nt validated tests or other immediate external criteria (eg: teachers subjective assessment) Construct Reflects closely a valid theory of foreign language learning that it takes as its model

........Criteria of A Good Test

2. Reliability - is concerned with the precision with which the test measures what the tester wants to know. - a reliable test must yield the same results on the same sample of students whenever the test is administered and whoever administers and scores it. - when several markers are required to mark candidates performance, reliability is often low which is caused by inter-variability

..Criteria of A Good Test

....Reliability - Unreliability is minimized by developing

performance bands which describe as accurately as possible, the language behaviour at each performance band. - unreliability is also sometimes discovered in the marks awarded by the same examiner which is called intrarater variability. - it is often caused by factors such as fatique after long hours of marking, long intervals between periods of marking, personality factors such as mood of examiner at different periods in the marking, etc

........Criteria of A Good Test

3. Practicality - some test may be ideal in theory, become difficult is practice, eg mass oral testing. - testing oral proficiency is very important at all stages in the learning process. - it is very expensive to conduct, very timeconsuming and its reliability is usually rather low because of inter-rater variability.

Activity

Comment on the validity, reliabilty and practicality of the following tests. 1. A group of waiters being assessed while they are working in a restaurant after they have completed a course on English for Waiters. 2. A national level essay test. 3. An oral component in monthly tests at school. 4. Reading aloud as a test of oral proficiency 5. National level testing of oral proficiency using language laboratories for candidates to record their oral responses to taped questions

Test Specifications

What to test? Why you need to test? How you want to test? You need to spell out the details of your test: - what kind of sub-tests and how many - what formats you want to use, etc - what skills will be tested (eg: reading, writing/ listening) - what sub-skills will be focused on

Test Specifications
Sectio n Skill

Sub-skills/ Elements
Guessing meanings of words using contextual clues; Distinguishing main point from supporting ideas; inference

Reading

C D

Grammar Conjunction Past Tense Formal and Informal Expressions Vocabulary Words related to animals, occupations and family Writing Descriptions of people

Writing Test Specifications

Tasks - need to be carefully chosen to give a good indication of the skills you want to test. - For example, to test on speaking spontaneously, you cannot give a reading aloud test. You need to prepare a test : oral interview/ discussion situation - Typical tasks answering comprehension questions, writing guided compositions, filling in blanks, etc

Writing Test Specifications

2. Types of Text - The KBSR and KBSM spell out a number of types of texts: letters, stories, advertisements, labels, poems,etc. - In specifying texts, you would need to specify length, eg read a story of 250 words; write a letter of 100 words. - Specify level of difficulty- it is difficult to specify level of difficulty. One way that testers use is to specify a known level, for eg the same level of texts as found in the KBSR school textbook. - Number of texts also need to be specified, for example, one text of 200 words/ 2 texts, each about 300-350 words.

Writing Test Specifications

3. Topics - select the topics for your test from syllabus specifications; = monthly test : achievement/progress test - you can also choose any topic within the maturational level of students. For example, you can choose a text on People for a Year 4 students but not on life insurance

Writing Test Specifications

4. Format - Same skill can usually be tested in more than one way. - For example: reading proficiency can be tested using MCQ questions, T/F questions, cloze passage, open-ended questions, etc. - The format that you would be using in testing should be specified. - The number of items in each sub-test need to be spelt out. - The format should be familiar to the students. If not, use and introduce in the teaching first.

Writing Test Specifications

Component Reading Format Rational Cloze Open-ended Cloze Error Identification No. of Items 15 blanks 5 questions 10 blanks 15 items

Grammar

Writing Test Specifications

5. Weightage - All the skills in a test are equally important, but not often. - So, you need to allocate different marks to different sections of your test/ different items in the same section. - Allocate marks according to the importance of each section/item in relation to the test as a whole and what you want to know from the test results.. - The weightage of each item/section need to be specified.

Writing Test Specifications

6. Time Allocation - How much time will be given for each sub-test will also need to be specified. - 2 things to bear in mind: (i) how important the skill is (generally, the more important skill should be given more time); (ii) how long the task would take (eg: writing a free composition would take a longer time than a guided composition/ filling in blanks in already written composition)

Types of Test Items

1.

Close-ended and open ended test items - it is called according to how much freedom the testee has in deciding on the scope and quality of the answer. - a close-ended item has a predetermined answer or set of answers which are usually short. - open-ended items- an essay test, an oral interview, a reading or listening comprehension question that expects students to answer using their own words

Types of Test Items

Close-ended items are of 2 types: 1. objective 2. structured Objective items - option to choose the correct answer from the alternatives provided (MCQ) - pure cloze only one answer is accepted Structured Items - the structure of the answer is determined by the examiner. For example: guided composition

Activity 2
State whether the items on the test are objective, structured or open-ended. 1. Fill in the blanks in each of the following sentences. Ali jumped .......... The river. 2. Write an composition on the topic , My Mother. 3. Combine the following sentences using the word in brackets. 4. Read the following text and answer the comprehension questions that follow: a. The cat was weak and thin. (True or False) b. Do you think the cat in the story is well-cared for? Give reasons for your answer. c. We know the cat was hungry from the fact that it ...

Types of Test Items

Subjective and Objective Tests - All tests are subjective as it involves sampling. - Sampling means selection - Selection is usually based on personal preference and

personal judgement. - The method of scoring distinguishes objective from subjective test.

Types of Test Items

Subjective Test
- Each examiner uses his own judgments in evaluating performance and awarding marks.

- Inter-rater as well as intra-rater variability are probable.

Objective Tests

- All items have a predetermined answer or set of answers.

- Whoever marks this section, same person or different, the

result will be same.

Types of Test Items.... Subjective & Objective Tests Objective Test Strengths - Easy to mark. - Can be used for mass testing because computer marking is possible. - High reliability - Item banking for recycling is possible Weaknesses - Difficult to construct. - Low validity - Sometimes skills and areas tested because they are testable not because they are important

Types of Test Items.... Subjective & Objective Tests

Subjective Test

Strengths - Easy to set - High validity - Can assess affective and interpretive aspects of language skills Weaknesses - Marking is time-consuming - Reliability is low

 Please refer to Nesamalar Chitravelus ELT

Methodology Principle & Practice TOPIC : Testing and Evaluation Page 330 349 for your further reference and preparation for your Tutorial & Practical Presentation.

TUTORIAL
In pairs, analyse test items in sample UPSR

exam papers.

Practical Teaching and Assessing Listening and Speaking Skills

Workshop: Testing and Evaluation Using UPSR exam papers as samples, prepare test specifications. Design the following to assess listening and speaking skills: - Multiple Choice Questions - Blank Filling - Text Completion - Structured Questions Present the workshop materials on testing and evaluation.

THANK YOU