1

2
 TEST BIAS
• Explain concept of test bias
• Discuss several categories of test bias
• Discuss several types of bias TEST ACCOMMODATIONS
• Explain the effects of bias
• Identify the bias • Explain the concept of test accommodations
• Explain types of cheating (Prior to test, During the test) • State the types of student's disabilities eligible for
• Discuss stealing test questions (piracy) test accommodations
• Discuss categories of test accommodations
• List out the examples of test accommodations (in
wide areas)
TEST FAIRNESS • Explain helpful and harmful aspects of test
• Explain the concepts of test fairness accommodations
• Discuss the fairness framework

EXAMPLES OF ISSUES IN TESTING PRACTICES

3
4
This presentation will....
• explain concept of test bias
• discuss several categories of test bias
• discuss several types of bias
• explain the effects of bias
• identify the bias
• explain type of cheating (prior to test, during the test)
• discuss stealing test questions (piracy)

5
Bias is defined as the presence of some characteristic
of an item that results in differential performance for
two individuals of the same ability but from different
ethnic, sex, cultural or religious groups.

Jorgensen (2005) 6
7
• it's refer to the degree to which a test fails to capture important aspects of
the construct.
• Too few test questions results in failure to insufficient sample the learning
content in the achievement domain desired. Consequently, the reliability of
the examination suffers.

• For example, a test of reading comprehension intended to measure

children’s ability to read and interpret stories might not contain a sufficient
variety of reading passages or might ignore a common type of reading
material. 8
• it's refer to the degree to which test scores are affected by
processes that are extraneous to the test's intended purpose.

Example:
• In the case mathematics test, it might over reliance on
reading comprehension skills that english language learner
may be lacking.

9
Refers to whether a test
exactly measures what it
was designed to measure.

For Example:
On an intelligence test, for
example, students who are
learning English will likely
encounter words they 10
 It occurs when the content of a test is comparatively more difficult for one
group of students than for others.
 It can occur when members of a student from a subgroup, such as various
minority groups, when:
 they have not been given the same opportunity to learn the material being tested
 scoring is unfair to a group
 questions are worded in ways that are unfamiliar to certain students because of
linguistic or cultural differences.

11
 Gender Bias
 Cultural Bias
 Regional Bias
 Ethnic/Racial Bias
 Language Bias
 Socio Economic Status (SES) Bias
 Special Needs Bias
12
Gender bias exists
• prejudice for or against one gender, especially in
a way considered unfair.
• it's occurs because of personal values,
perceptions and outdated, traditional views
about men adn women.

• when males and females with the same skills and

knowledge obtain different scores on a test.1

13
1 . Axman (1990)
• A test is culturally biased when it makes
assumptions about what is “common
knowledge.1”
• Examples: cultural norms and conventions,
literary knowledge, conventions of
language2,etc.
1. McGinley, S. (2002)
2. Del Rosario B. (1998)

14
nA test has regional bias when
people are penalized because of the
use of regional names or dialects.
n Examples: sundrop, pop, bubbler,
pies for pizzas
(pizza pie) in NY, etc.

15
Ethnic Bias or Racial Bias occurs
 - when members of a group are portrayed in a
stereotypical manner.
 - when members of such groups obtain different scores
than members of another group–with the same skills and
knowledge1.

Popham (2006)

16
n Socio-economic bias occurs in tests when students
are penalized based on their socio-economic status
(SES)1:
¨ The isolated, rural environment, the restrictive poverty of
many families, and cultural ties with a tribe or other
group deny many students
important knowledge of the
outside world2.

1. Popham (2006)
2. Brescia & Fortune (1988)
17
n Language bias in tests occurs when
second language learners (L2) are
penalized because of their lack of
knowledge of the English Language.

n For example, by their inability to read the

questions accurately, or to give proper
verbal responses.

Brescia & Fortune (1988)

18
n According to Popham (2006), a test is biased if
it unfairly penalizes a group of students so
that their performance is less than that of
another group of students with the same
achievement level with respect to the
knowledge or skill being tested.

Popham (2006)

19
• Emotional effects
• Students struggle with testing and its results, and they may
question their own capabilities to learn.
• They lose confidence and the ability to take risks and become
embarrassed and feel the need to “save face”.

• For children who haven’t understand English,

testing in the language becomes a painful and
shame experience.
20
• We are creating more achievement
gaps between students, rather than
helping them because we are testing
groups unfairly.
• Disabled students not given enough time to test.
• ELL/ESL students have limited reading skills and
can’t even read the test and get no assistance.
• Many students with special needs don’t have
fair achievement measures.

21
• Content Bias
-fair for all examinees
• Language Bias
-Is the item free of group specific language, vocabulary, or
reference pronouns
• Item Structure and Format Bias
-Are there any flaws in the items to which members of DSI
(designated subgroup of interest) are differentially sensitive?
• Stereotyping of Minorities
-Avoid material that is controversial, inflammatory, demeaning
or offensive to members of DSI.
22
• Is the test item fair for all examinees and
free of annoying stereotypes?
• Are both sex groups recognized fairly?
• Will all examinees have equal opportunity
to respond?

23
24
q Purchasing test items, item pools, or tests from web sites.
q Using a chat room.
q Bulletin board.
q Discussion forum to solicit.
q Collaborating with individuals who take the test during an
earlier administration and then provide test items to individuals
who take the test at a later time.

25
Strategies using various forms of soft technology to interact with others
who may also be taking the test or who are outside the testing environment.

qFind answers using “cheat” sheets.

qNot stopping when the time limit is reached.
qReviewing sections of the test before receiving
directions.
qUsing calculators that are not permitted by the
testing program.
qCopying other’s examinee test responses.

26
§ Item resellers to help other examinees to
get high scores.
§ Buyer’s (i.e., the examinee) only objective
is to memorize the specific content and
response options for a set of items.
§ Put the items on the Internet for resale to
future test takers.
§ Tell their friends or classmates about the
test.

27
1. What are the categories of test bias?

2. How many types of cheating?

28
29
This presentation will....
• explain the concepts of test fairness
• discuss the fairness framework

30
CONCEPT OF TEST FAIRNESS
• Test Fairness of a test relates to fair use.
• The analytical methods that can help bring about fair tests and testing practice
have not been clearly articulated.
• Willingham and Cole (1997) and Willingham (1999) view test fairness as an
important aspect of validity, arguing that fairness should be viewed as
“comparability in assessment; more specifically, comparable validity for all
individuals and groups
• Fairness in testing is defined many ways (AERA, APA, & NCME, 1999) but at its
core is the idea of comparable validity. A fair test is one that yields comparably
valid inferences from person to person and group to group.
CONCEPT OF TEST FAIRNESS (cont…)
“[A testing] principle that every test taker should be assessed in an equitable way.”
(AERA, et al., 1999, p. 175)
THE CODE OF FAIR TESTING PRACTICES IN
EDUCATION (1988)
• The Code of Fair Testing Practices in Education (1988) presents standards for
educational:
1) test developers (thinker, researchers, raters, writers)
Test developers are trained to examine all aspects of a test for its fairness
2) test users (population of peoples)
Test users are peoples and agencies that select test for its fairness

• The Code (1988) presents standards for educational test developers and users
in four areas:
• developing and selecting tests
• interpreting scores
• striving for fairness and
https://www.apa.org/science/programs/testing/fair-testing.pdf
• informing test takers
THE FAIRNESS FRAMEWORK (Kunnan, 2004)
1) VALIDITY
2) ABSENCE OF BIAS
3) ACCESS
4) ADMINISTRATION
5) SOCIAL CONSEQUENCES
THE FAIRNESS FRAMEWORK (Kunnan, 2004)
1) VALIDITY:

Content representativeness • (described as content validity) refers to the adequacy with which the
or coverage evidence: test items, tasks, topics and language dialect represent the test domain.

• (described as construct validity) refers to the adequacy with which the

Construct or theory-based
test items, tasks, topics and language dialect represent the construct or
validity evidence: theory or underlying trait that is measured in a test.

• (described as criterion validity) refers to whether the test scores under

Criterion-related validity
consideration meet criterion variables such as school or college grades
evidence: and on the job-ratings, or some other relevant variable.
THE FAIRNESS FRAMEWORK (Kunnan, 2004)
1) VALIDITY:

• refers to the reliability or consistency of test scores in terms of

consistency of scores on different testing occasions (described as
stability evidence),
Reliability • between two or more different forms of a test (alternate form evidence),
• between two or more raters (inter-rater evidence), and
• in the way test items measuring a construct functions (internal
consistency evidence).
 THE FAIRNESS FRAMEWORK (Kunnan, 2004)
2) ABSENCE OF BIAS
• content that is offensive to test takers from different backgrounds, such as stereotypes of group
members and overt or implied slurs or insults (based on gender, race and ethnicity, religion, age, native
Content or language: language, national origin).

• refers to content that may cause unfair penalisation because of a test taker’s group membership (such as
Unfair penalisation that based on gender, race and ethnicity, religion, age, native language, national origin).
based on test taker’s
background:

• differing performances and resulting outcomes by test takers from different group memberships. Such
group differences are defined by salient test-taker characteristics such as gender, race and ethnicity,
Disparate impact
and standard religion, age, native language, national origin
setting:
 THE FAIRNESS FRAMEWORK (Kunnan, 2004)
3) ACCESS
Educational access Geographical access
refers to whether or not a test is accessible to refers to whether a test site is accessible
test takers in terms of opportunity to learn the in terms of distance to test takers.
content and to become familiar with the types of
task and cognitive demands.

Personal access Conditions or equipment access

refers to whether a test provides certified refers to whether test takers are familiar with the
test takers who have physical and/or test taking equipment (such as computers),
learning disabilities with appropriate test procedures (such as reading a map), and
accommodations. conditions (such as using planning time).
 THE FAIRNESS FRAMEWORK (Kunnan, 2004)
4) ADMINISTRATION

Physical conditions
• Refers to appropriate conditions for test administration, such as optimum light and
temperature levels and facilities considered relevant for administering tests.

Uniformity or consistency
• refers to uniformity in test administration exactly as required so that there is
uniformity and consistency across test sites and in equivalent forms, and that test
manuals or instructions specify such requirements.
 THE FAIRNESS FRAMEWORK (Kunnan, 2004)
5) SOCIAL CONSEQUENCES

Wash back:
• refers to the effect of a test on instructional practices, such as teaching,
materials, learning, test-taking strategies, etc.

Remedies:
• refers to remedies offered to test takers to reverse the detrimental
consequences of a test, such as re-scoring and re-evaluation of test responses,
and legal remedies for high-stakes tests.
 Recap....
• What is test fairness?
• List out 5 components of test fairness framework by
Kunnan (2004)
 TEST
ACCOMMODATIONS

42
This presentation will....
üexplain the concept of test accommodations
üstate the types of student's disabilities eligible for test
accommodations
üdiscuss categories of test accommodations
ülist out the examples of test accommodations (in wide areas)
üexplain helpful and harmful aspects of test accommodations

43
Concepts of Test Accomodations

Any modifications made to tests or testing

conditions that allow students with
physical disabilities, learning disabilities, or
limited English-language ability to
demonstrate their knowledge and skills in
a testing situation.

44
 Concepts (continued…)
§ The accommodations may apply to both
standardized tests administered to large
populations of students, including high-stakes tests
used to make important decisions about students,
and to the assessments that teachers use to
evaluate what students have learned in a particular
course.
§ The general goal of providing testing
accommodations is to create a level playing field for
students whose disabilities or language abilities that
may adversely affect their ability to show on a test
what they have learned. 45
 Types of Students Disabilities Eligible for
Test Accommodations
§ Blindness/Vision Impairment
§ AD/HD (formerly known as A.D.D./A.D.H.D.)
§ Learning Disabilities
§ Deafness/Hearing Impairment
§ Certain Medical Conditions
§ Certain Physical Disabilities
§ Certain Psychiatric Conditions
46
Categories of Testing Accommodations

Presentation Response

Time Location

47
Presentation Accomodations

48
Response Accomodations

49
Time Accomodations

50
Location Accomodations

51
 Examples of Testing Accommodations
(in wide areas)
§Braille or large-print exam booklets
§Screen reading technology
§Scribes to transfer answers to Scantron bubble sheets
or record dictated notes and essays
§Extended time
§Wheelchair-accessible testing stations
§Distraction-free rooms
52
 Helpful Aspects of Test Accommodations
§ When carefully applied, accommodations enable teachers to get a
more accurate assessment of student's knowledge and skills.
§ Accommodations can help students feel more comfortable with the
testing process, knowing that his/her disability will not prevent
him/her from succeeding.
§ Accommodations can potentially help students receive better grades
and possibly more academic recognition.
§ Accommodations may result in slightly higher scores on college
entrance exams
53
 Harmful Aspects of Test Accommodations
§ Testing accommodations may be misused and misapplied.
§ Students may not get extra help they need because they appears to
have higher skill levels than they really have.
§ In the worst cases, teachers may inadvertently focus less on students,
believing that the accommodations are all the support they needs.
§ When accommodations are over-used, students with disabilities have
inflated scores.

54
 Recap....
1) List out 4 categories of testing accommodations.

2) List out 3 examples of testing accommodations (in

wide areas).

3) How to relate certain accommodations for certain

bias? Take an examples : Special needs bias
55
Examples of Issues in
Testing Practices

56
MedSI Test

57
STPM 2018

58
UPSR 2014

59
PT3

60
PBS

61
Recap
• Form into a group of 5 members. Each group can think of
one example of test item that has one or more type(s) of
bias. Put the examples on padlet.

Link :
https://padlet.com/n_bukhar/SGDE4013_A181_TestBiasFai
rnessAccommodations
 TEST BIAS
• Explain concept of test bias
• Discuss several categories of test bias
• Discuss several types of bias TEST ACCOMMODATIONS
• Explain the effects of bias
• Identify the bias • Explain the concept of test accommodations
• Explain types of cheating (Prior to test, During the test) • State the types of student's disabilities eligible for
• Discuss stealing test questions (piracy) test accommodations
• Discuss categories of test accommodations
• List out the examples of test accommodations (in
wide areas)
TEST FAIRNESS • Explain helpful and harmful aspects of test
• Explain the concepts of test fairness accommodations
• Discuss the fairness framework

EXAMPLES OF ISSUES IN TESTING PRACTICES

63
64