DEVELOPING AND IMPROVING TEST ITEM

WHAT IS AN ITEM ?
Brown (1996: 49) defines an item is the basic unit of language testing. The item is sometimes difficult to define. Some types of items, like multiple-choice or true-false items, are relatively easy to identify as the individual test questions that anyone can recognize as discrete units.

Item Format Analysis

Item Facility Analysis

Developing NormReferenced Language Test

Distractor Efficiency Analysis

Item Discrimination Analysis

General Guidelines Item Format Analysis Receptive Response Items Productive Response Items

Has at least one other colleague looked over the item?

Is the item format correctly matched to the purpose and content of the item?

Is there only one correct answer?

Have race, gender, and nationally bias been avoided?

Is the item written at the students level of proficiency?

General Guidelines
Have ambiguous terms and statements been avoided?

Is only relevant information presented?

Are all parts of the item on the same page?

Does the item avoid giving clues that could be used in answering other items?

Have negatives and double negatives been avoided?

TrueFalse Items

Receptive Response Items

Matching Items

MultipleChoice Items

True-False Items

Is the statement worded carefully enough that it can be judged without ambiguity?

Have absoluteness clues been avoided?

Have all unintentional clues been avoided?

Have distractors such as none of the above and a and b only been avoided?

Are all of the distractors plausible?

MultipleChoice Items

Has the ordering of the options been carefully considered? Or are the correct answers randomly assigned?

Has needless redundancy been avoided in the options

Are there more options than premises?

Matching Items
Are the option and premise lists related to one central theme?

Are options shorter than premises to reduce reading?

Fill-in Items

Productive Response Items

Shortresponse Items
Task Items

Fill-in Items

Is the required response concise?

Is there sufficient context to convey the intent of the question to the students?

Are the blanks of standard length?

Does the main body of the question precede the blank?

Has a list of acceptable responses been developed?

Short-response Items

Is the item formatted so that only one relatively concise answer is possible?

Is the item framed as a clear and direct question?

Task Items
Have scoring procedures been worked out in advance with regard to the approach that will be used? Have scoring procedures been worked out in advance with regard to the categories of language that will be rated? Have scoring procedures been clearly defined in terms of what each score within each category means?

Is the students task clearly defined?

Is the task sufficiently narrow (and/or broad) for the time available?

Is scoring to be as anonymous as possible?

ITEM FACILITY ANALYSIS

Brown (1996: 64) states that item facility (IF), also called item difficulty or item easiness, is a statistical index used to examine the percentage of students who correctly answer a given item.

ITEM FACILITY ANALYSIS

Where Ncorrect = number of students answering correctly Ntotal = number of students taking the test

ITEM DISCRIMINATION ANALYSIS

Item discrimination (ID) indicates the degree to which an item separates the students who performed well from those who performed poorly. These two groups are sometimes referred to as the high and low scorers or upper and lower-proficiency students. The reason for identifying these two groups is that ID allows teachers to contrast the performance of the upper group students on the test with that of the lower-group students (Brown, 1996: 66-67).

ITEM DISCRIMINATION ANALYSIS

=

where
ID = item discrimination for an individual item whole test IFlower = item facility for the lower group on the whole test IFupper = item facility for the upper group on the

DISTRACTOR EFFICIENCY ANALYSIS

Brown (1996: 71) states the primary goal of distractor efficiency analysis is to examine the degree to which the distractors are attracting students who do not know the correct answer. To do this for an item, the percentages of students who chose each option are analyzed.

NRT Development and Improvement Projects

Pilot a relatively large number of test items on a group of students similar to the group that will ultimately be assessed with the test. Analyze the items using format analysis and statistical techniques. Select the best items to make up a shorter, more effective revised version of the test.

Item Quality Analysis

Difference Index Developing CriterionReferenced Language Test The B-index

CRT Item Selection

CRT DEVELOPMENT AND IMPROVEMENT PROJECTS

Brown (1996: 79) states that a central purpose of a CRT is to assess how much of an objective or set of objectives has been learned by each student, CRT assessment has to occur before and after instruction in the concepts or skills being taught in order to determine whether there was any gain in scores.

CONCLUSION
In testing, one way to improve and develop a test is to examine the individual items which are called item analysis. This is usually done for purposes of selecting the best items that will remain on a revised and improved version of the test.