Secondary data

indicators of country-
level education quality

Robin Horn
Emilio Porta
key issues in the measurement of
education quality and actions needed to
close data-gaps.
 Outline

■ Measuring learning outcomes and

selecting proxy indicators for
education quality
■ Data availability
■ Strengths and weaknesses of using
“secondary” data sources
 How is student learning
achieved?

Education
Policy
Students Policy actions
Schools
Endowments and Specific
programs Endowments and
Behaviors of Behaviors of
Systemic
Students Teachers
reforms
Parents Schools
Families Authorities

Indicators The Education

System
? Its organization and
institutionality

Learning

Vegas & Petrow, 2008

Note: to see this slide please activate slide show and click the right bottom of your
mouse then select play.
In the search of indicators

PISA 2006
 In search of indicators
Variable associated with performance Fuller & Clarke, Vélez et al, 1993 Unesco-LLECE, 2001
and expected sign 1994
Studies % Studies % Studies %

Spending per student + 11 55% 8 13%

Student teacher ratio - 48 23% 21 10% 14 14%
Students with individual texts + 39 67% 17 76%
School library + 22 86%
Teacher's years of schooling + 26 50% 68 46% 14 14%
Teacher training + 17 65% 8 0 14 7%
Teacher’s years of Experience + 35 40% 62 40% 14 14%
Time for teachers to prepare lessons + 10 60% 1 0
Time available for teaching-learning + 33 82%

Frequency of "homework" + 13 85% 28 43%

Multigrade school - 8 75% 12 8%
Student's self-esteem + 7 100%

Schiefelbein, 2008
 In search of indicators
Participation in Selected International Student Assessments by Country
(PISA, TIMSS or PIRLS)

map
 In search of indicators
■ Need more research to determine the
right indicators
■ Consensus building
■ Promote learning assessments in
developing countries
■ Promote impact evaluation
■ Country benefits:
◆ Improve policy dialogue among policy
makers
◆ School accountability
“The LORD spoke to Moses in the Tent of Meeting in the Desert
of Sinai on the first day of the second month of the second year
after the Israelites came out of Egypt. He said: "Take a census
of the whole Israelite community by their clans and families,
listing every man by name, one by one. You and Aaron are to
number by their divisions all the men in Israel twenty years old
or more who are able to serve in the army. One man from each
tribe, each the head of his family, is to help you. These are the
names of the men who are to assist you: from Reuben, Elizur son
of Shedeur;”

Numbers, The Census

Data availability: a challenge
we must face
 Data availability: a challenge
we must face

Mapping of data availability for 2000-2006

EdStats Newsletter
Data availability: a challenge
we must face
Data availability: a challenge
we must face

EdStats Newsletter
Bridging existing information
gaps
■ Household
Surveys
■ Learning
Plausible “secondary” Assessments
data sources Questionnaires
■ Existing
reports and
research
 Using Household Surveys
Strengths
•Most developing countries •Currently they don’t focus
have HHS’s
•DHS is comparable
•The cost of adding new
questions is low (U$50 per
household in the case of
Vietnam)
•Could include youth
outside school and adult
learning
Weaknesses
on education quality
indicators
•They aren’t just for
education.
•Generally they are not
conducted annually
•Not representative at
school level
Using Learning Assessments
Questionnaires
Strengths
•They have rich information
about plausible quality
indicators
•They can be comparable
•The cost of extracting data
from existing databases is
low (U$ ½ M)
Weaknesses
•Most developing countries
do not participate in
international assessments
• Background questions
change over time
•They are not conducted
annually
Existing reports and research

Strengths Weaknesses

•Most developing countries

have a variety of research •Comparability
papers with rich
•Not conducted on regular
information.
basis
•They can be serial (EAS,
PER, PAD, EFA-FTI
proposals, USAID sector
assessments)
•The cost for extracting data
is low
 Recommendations for
discussion
Defining quality indicators Collecting information
■ Support ongoing ■ Renew countries’
research on commitment on
determining learning reporting their data to
outcomes UIS
■ Reach consensus on ■ Create a task team to
selecting education conduct a global
quality indicators assessment of
■ UNESCO Task Team countries’ EMIS (U$
2M)
■ Create a trust fund to
support countries’ data
plans
■ Strengthen countries’
efforts on improving
their analytical capacity