EDU 549 UNIT 1 TESTING, MEASUREMENT AND EVALUATION By the end of these two sessions you should be able

to: explain meaning of test, measurement, evaluation and assessment describe the stages of test preparation describe the basic principles of the construction of objective and subjective items/questions

MEANING OF TEST, MEASUREMENT, EVALUATION AND ASSESSMENT Let us first of all clarify the meaning and differences between Test, Measurement, Evaluation and Assessment. Many of us are still confused with these terms as they are sometime used interchangeably in education. This means that some individuals argued that these terms are all the same, whereas they are not. Thus, it is very important for teachers to know the fundamental meaning applied for these terms. However, to make better understanding let us go through them by looking at the meaning of these terms in the opposite order. What is Evaluation? Evaluation is the procedure used to determine and make judgment based on the results of assessment and test. It is the overall procedure where decision can be made after getting information or data from the whole process of assessment and test. Decisions are made in the evaluation procedure by saying whether it is good, poor, appropriate, excellent, and the like. However, we need to measure first before we can obtain data or information in order to make judgment and then a decision. What is Measurement? Measurement refers to quantifying or assigning a number to express the degree to which characteristic or phenomenon is presented. We measure the weight of things by the kilograms, grams, milligrams, etc. Therefore, by assigning numbers to express the degree to which it is presented, we can see the different weight between an object to another. For instance a kilogram of iron is the same weight of a kilogram of cotton regardless of the mass it may appear. We need an instrument or a tool to make the measurement. In the above example we need to use a weighing machine to measure the weights of iron and cotton. What about assessment or test? How do we measure them? What instrument or tool do we need to measure them? In education, we need to do an assessment before we can measure students learning performance.

What is a Test? Test is commonly used by educators (teachers, lectures, trainer, instructors and etc.) who are involved in teaching and learning. It is to determine whether learning has taken place or not. In other words, it is used to measure academic performance in education. Also, test is used to measure performance at workplace for promotion and selection of competent workers. Professionals are required to take tests for the purposes mentioned above, and sometimes it is compulsory for them to sit for test in order to obtain license before they can practice the trade. So, what is a test? In simple explanation, it is a type of assessment which is an instrument or a tool used for measurement purposes. Thus, the word testing describes the procedure of getting information about an individuals performance. According to the American Psychological Association (APA), a test may be thought of as a set of tasks or questions intended to elicit particular types of behaviours when presented under standardized conditions and to yield score that have desirable psychometric properties (1974). A test can be in the form of paper and pencil test in which candidates are required to be in a special room where they are given questions to respond in writing within the stipulated time. An example is the monthly test or quiz. Test can also be conducted orally like the language oral test or it can be in the form of action or performance like the driving test. What is Assessment? Assessment is sometimes used interchangeably with evaluation. Meaning, many would say that they are the same. In actual fact, assessment is a general term that includes the full range of processes used to gain information about students learning and the formation of value judgments. Having had the information from the result of assessment, only then we can execute evaluation. In general, assessment is an instrument or a tool to collect information for measurement purposes. In order to measure, we need to carry out the assessment first. Another important fact about assessment is that it may appear in different forms such as test, project, seminar, portfolio, performance, interview, observation, experiment, role-play, practicum and etc. For this reason, we can conclude that teachers are extensively involved in the assessment of the students, where educational assessment is concerned. Why do we assess students? Basically we assess students for two reasons: (i) To assist learning Results of assessment provide information to students about their performance or achievement. They will know to what extent the progress they have made towards the instructional goals. This will reflect their strengths and weaknesses in their learning strategy.

Assessment can also provide communication to parents, knowing how well their children are doing in school. Thus, parents too can assist their children in their learning problem. Nowadays, parents are very concerned with their childrens progress in school. Apart from that, assessment results can also be used for streaming purposes. Students will be placed in appropriate groups as according to their level of achievement as well as of their interest. Therefore, diagnosis for remedial and enrichment actions is made easy to be conducted. Another purpose is for the school administrator to keep records of the students performance. Further actions such as counseling can also be provided to nonperforming students. Also, the school administrator can make comparison to other schools in terms of the overall school academic performance. Last but not least, assessment results will determine certification. Marks are converted into letter grades representing students performance in various subjects and then the grades are used as a basis for comparing overall students performance in any major examinations such as the Sijil Pelajaran Malaysia. The certificate awarded is then used to decide on the selection of students application for further study or employment.

(ii)

To improve teaching Assessment results are also beneficial to teachers in evaluating their teaching strategies. Teachers can check on their instructional objectives to determine whether the learning outcomes are achieved or not. On top of that, teachers can also verify the methods and techniques used in delivering the lesson. Under normal circumstances, most of the students in Malaysia are placed in classes which are heterogeneous in nature. Thus, teachers must always decide to use various methods and techniques appropriate to the level of the students. Next, teachers may consider effective teaching aids or materials to support the methods and techniques used. It is important for teachers to use varieties of resources in order make learning easy. Another benefit that teachers gain from the assessment results is, knowing the point of entry. Teacher may consider where to begin the lesson, especially a new topics by taking into account the students previous knowledge.
You may add other relevant points on why we need to assess students.

Types of Assessment This part discusses on types of assessment by looking at these two concepts: (i) Formative vs. Summative Assessments Formative assessment is continuous in nature and it is on-going throughout the course of study. In this context, the course of study refers to the school year. Nevertheless, it can also refer to semester or programme. Formative assessment can be carried out at the beginning or during the course of study. Examples of formative assessments are test, quiz, projects, or any other form of assignments. The purpose is to improve quality of learning from time to time with regards to topics or any particular subject area to determine whether the instructional objectives have been achieved. Teachers may change their teaching strategy if needs be and students too, may change their learning strategy in order to meet the objectives. No evaluative measure is taken because it does not involve grading. However, individual students performance is compared to another but from the same class so as to confirm quality of teaching and learning. Summative assessment is carried out at the of the school year, study or programme. Examples of summative assessment are the Sijil Pelajaran Malaysia (after a period of two years) and Penilaian Menengah Rendah (after a period of three years). Another example in a smaller context is the final or end of year examination for form one students which is carried out at the end of the school year..

The purpose is to evaluate the effectiveness of the instructional programme and also to make judgment of students competency. Apart from that, certificates will be awarded if it is for the national examinations like Sijil Pelajaran Malaysia and Penilaian Menengah Rendah. (ii) Norm-reference vs. Criterion-referenced Tests Norm-reference test emphasizes on students achievement in learning certain subject. Marks will be compared among students of the same group. In doing so, these types of tests focus on students performance based on a continuum of high and low achievers. It measures students general ability which covers a wide range of content area. Examples of this type of test are the national examinations and year end examinations. Thus, grades are awarded. Criterion-referenced test Unlike norm-referenced test, criterion referenced test emphasizes on determining what students can do or cannot do. It measures students mastery of basic skills. Individual marks are not compared against the norm but it is compared against a criterion which is already predetermined. It covers specific aspects of the content area. Examples of this type of tests are the classroom test, exercises and assignments. Another clear example is the driving test where the passing mark is like 90% as the criterion to determine either pass or fail. No grades are awarded.

PLANNING STAGE Normally, all teachers teaching the same subject are involved in test preparation. They have to meet and plan for the test. Involvement of all teachers is necessary to determine and identify the content areas to be tested. Before constructing a test paper it is wise to have a proper planning in the form of a blueprint which is commonly known in education as test specifications. It describe clearly what you intend to test in terms of content areas, objectives and levels of thinking skills (instructional objectives - normally we refer to Blooms Taxonomy), number of items or questions, allocation of marks, distribution of items according to the levels, etc. These criteria are then tabulated in matrix form which is called the Table of Specifications. Table of Specifications 1. 2. 3. 4. It is a plan like a blueprint It is a two-way chart It clarifies the content validation in preparing classroom assessment It is a guide to teachers constructing the test paper

Some considerations before developing a table of specifications: 1. 2. 3. 4. 5. 6. 7. 8. Prepare a list of instructional objectives Outline the course content to be tested Prepare a two-way chart Decide on the format Allocate sections if necessary Determine the types of items Scoring etc.

Developing a table of specifications can be as simple as in Example 1 and can be slightly comprehensive as in Example 2, or it can be more comprehensive as in major national examinations.

Example 1 Table of Specifications Monthly Test Mathematics - Form 1 (March 2005)

Instructional Objectives Content Area Knowledg e Simple fraction Mixed number Decimals Decimal-fraction relationship No of Items 3 3 2 2 10 Comprehensi on 3 4 4 4 15 Applicatio n 2 1 1 1 5 8 8 7 7 30 26.7 26.7 23.3 23.3 100% Marks %

Example 2 Table of Specifications - EDU XXX Semester January/June 2007

Sections A Contents Definition and meaning of basic terms Item analysis: Index of difficulty Index of discrimination Basic Statistics: Mean, mode , median, standard deviation, zscore and t-score Portfolio Type of Items/questions Short answers No of Items 2 (With 6 sub questions) 1 (With 3 sub questions) Instructional Objectives Knowledge Comprehension Application Analysis Evaluation Marks 30

Short answer Calculation

Calculation

1 (With 3 sub questions)

Analysis Application

12

Essay

Scoring essay question

Essay

Application Analysis Evaluation Application Analysis Evaluation

25

25

Total Marks

100

CONSTRUCTING STAGE After you have decided and planned what to test, you may proceed by constructing the items or questions. Based on the Table of Specifications, teachers then divide the work among themselves in constructing the items. However you have to be clear of the two types of items namely the objective and subjective types.

Objective Type of Items Objectives items are classified into two, namely the supply type and the selection type. 1. Supply Type (short answer and completion) Test takers are required to provide the answers. They are basically the same. Short Answer uses direct question while Completion item uses incomplete statement. Example: List 5 characteristics of a good test. (Short answer) The boy was _____. (Completion)

2. Selection Type (true-false, matching, and MCQ) Instead of supplying the answers, test takers are required to select the correct answer from a list of options. True/False The most common use is to measure the ability to identify the correctness of statements of facts, definition, principles, etc. Example: Instruction: Read the following statements. If the statement is true circle letter T and if the statement is false circle letter F. No. 1. 2. Statement Answer F F

Tun Dr Mahathir was the third prime minister of T Malaysia. The capital of Malaysia is Ipoh. T

Matching Matching questions are used to measure students ability to identify the correct relationship between two lists.

Example: Instruction: Choose the correct answers from Column B to match the statements in Column A.

Column A ___ 1. ___ 2. ___ 3.

Column B

Formerly know as Burma A. Indonesia The capital of Thailand B. Mynmar The most populated country in South East C. Bangkok Asia

Multiple Choice Questions (MCQ) Multiple Choice Questions are widely used because it can measure the lower level of the thinking skill as well as to some extent the higher level.

Below is an example of the MCQ item:

1.

Who was the first Prime Minister of Malaysia? A. B. C. D. Tun Abd Razak Tengku Abd Rahman Tun Mahathir Tun Sambanthan
Options or alternatives

Stem

Distracters

Correct option or answer

Stem 1. 2. 3. 4. Can be in the form of statement or question Clearly and concisely stated Avoid grammar error, ambiguity, being verbiage, etc Ask for one correct or best answer only

Options 1. Usually four options are provided, but sometimes some exam setters may provide five options

2. Distracters should be truly incorrect but plausible 3. Grammar should be congruent to the stem
Think of other possible consideration in stating the stem and the options.

Essay Questions Provide the freedom of response and originality that is needed to adequately assess the ability of students to formulate problems; conceptualize, construct, organize, integrate, relate, and evaluate ideas and information; and apply knowledge and skills It introduces scoring difficulties that make essays inefficient as a measure of factual knowledge. However, complex learning outcomes measured are of such great educational significance that the expenditure of energy in the difficult and time-consuming task of evaluating the answers can be easily justified. It may also be used to focus teacher and student attention on the writing process itself through the use of various prewriting activities (e.g., discussion, listing and organizing ideas, constructing outlines, and clarification of audience) as well as the initial drafting and revision of essays.

There are two types of essay questions: 1. Restricted-response essays The ability to supply rather than merely identify interpretations and applications of data Extended-Response Essay Questions The ability to organize, integrate, and express ideas in a general attack on a problem Usually limits both the content and the response. The content is usually restricted by the scope of the topic to be discussed. It is most useful for measuring learning outcomes requiring the interpretation and application of data in a specific area. Limitations of the form of response are generally indicated in the question. Example: State the two main differences between formative and summative evaluation. Elaborate with an example for each of the difference discussed.

2. Extended-response essays Allows students to select any factual information that they think is pertinent, to organize the answer in accordance with their best judgment, and to integrate and evaluate ideas as they deem appropriate. Enables students to demonstrate their ability to analyze problems, organize their ideas, describe in their own words, and/or develop a coherent argument. Measure a students global attack on a problem. Example:

Evaluate the usability of the taxonomy of Bloom in the classroom testing and measurement. Cite examples when possible.

Advantages of essay type of questions It measures complex learning outcomes that cannot be measured by other means. Its emphasis on the integration and application of thinking and problem-solving skills. It enables the direct evaluation of writing skills. Its contribution to student learning. Ease of construction.

Limitations of essay type of questions Scoring tends to be unreliable Scoring is time consuming Only a limited sampling of achievement is obtained Essay questions especially ones requiring extended responses, should be limited to assessing those outcomes that cannot be measured well by objective items.

Now, you should be able to provide the advantages and disadvantages objective items by studying the essay types of questions.

Suggestions for Constructing Essay Questions Restrict the use of essay questions to those learning outcomes that cannot be measured satisfactorily by objective items. Construct questions that will call forth the skills specified in the learning standards. Phrase the question so that the students task is clearly indicated. Indicate an approximate time limit for each question. Avoid the use of optional questions.

ASSEMBLING STAGE Once the items have been constructed, teachers meet again and compile the items. Some editing work has to be done looking at the suitability of the level according to the instructional objectives, check language in terms of grammar and spelling errors, arrange as according to the intended format, prepare the cover page, provide instructions and etc.

Another point to remember is to check for the answer key and then ddetermine how students record the answers. Finally is to print the test paper. Always make sure that the number is sufficient for the students as well as for record keeping.

ADMINISTERING STAGE Below are some important guidelines before administering the test paper.

Scheduling the test by making announcement or notice prior to the test Make sure that the venue is conducive. Make sure that the sitting arrangement is at a proper distance Instruction should be clear and brief Avoid students from cheating .Avoid threatening the candidates/students. Avoid distractions. Keep interruptions to a minimum Acknowledge or remind students of the time.

Can you elaborate on the points mentioned above? Or, through your experience as teachers you may want to add more points to the list.