Professional Documents
Culture Documents
System
Professor Barry OSullivan
Senior Advisor, English Language Assessment
Overview
Developing theories
Localisation
Computerisation
Conclusions
DEVELOPING THEORIES
System Design
Social Factors
Target
Language
Task Language
Personal
Characteristics
PERFORMANCE
Cognitive
Characteristics
Consistency
Answer Key or
Rating Scale
Machine or
Human
Interpretation
Validation Theory
Decisions, Decisions
Why are we
testing?
Interlocutor [who;
number]
Relative Age, gender
etc.
Familiarity & Status
Etc.
Personal
Characteristics
Physical
Psychological
Experiential
Target
Language
Cognitive
Characteristics
Vocabulary &
Grammar
Function, Mode etc.
Maturity
Ability
Etc.
Task Language
Vocabulary &
Grammar
Genre, Mode, etc.
TEST
SCORE
Scoring System
Raters & Rating
Scales & Keys
Analysis & Grading
Etc.
Decisions
Delivery
Assessment
room design
materials
teachers
judgemental
developmental
FOCUS ON READING
How it Works
C
Inter Text
[full text level]
Inter Text
[paragraph level]
Text [external]
Text [within]
Paragraph
Word/Sentence
Introducing
CORE
[Grammar & Vocabulary]
Writing
Reading
Speaking
Listening
LOCALISATION
Test
Taker
Appropriate
Topic
Appropriate
Task
Test
Task
Appropriate
Language
Appropriate
Scoring
LOCALISATION
Scoring
System
Defining Localisation
The process of ensuring that all construct-irrelevant
variance related to the test taking population is
controlled for in the design and delivery of a test
Delivery
Assessment
Localisation
Language Reviewed
[inc. instructions]
Reading Topics
Listening Topics
Writing Topics/Tasks
Speaking Topics
Speaking Images
COMPUTERISATION
Test Scoring
Rater Training
Test Break Up
Scoring
Analysing/Awarding
Score Reporting
Test Taking
Knowledge
Receptive Skills
Productive Skills
System Management
Back Office
Reporting to Clients
Item Development
Specification
Writing
Trialling
Banking
Archiving
Test Delivery
Computer
Tablet
Other
CONCLUSIONS
The Message?
Every decision impacts on ALL stakeholders, though
especially on the test taker
Without a clearly defined ABILITY MODEL the test is
suspect
Without a clearly defined VALIDATION MODEL the test
is suspect
It is close to impossible to deliver any test without
technology