Bangkok Office

Research Framework
The Culture of Testing:
Socio-cultural drivers and the effects on youth in the Asia-Pacific region

Context
In the Asia-Pacific region, a growing culture of testing has seen many countries striving to increase
levels of student learning achievement as a core focus of their education policies and as a perceived
measure of their education systems performance. A variety of assessments and exams exist
throughout the region, both low-stakes and high-stakes, that education systems aim to utilize in
order to measure learning outcomes. While the existence of these exams may be rationalized as
efforts to enhance educational quality, this focus purely on high scores may also be undermining
other fundamental aspects of learning that are often not captured in tests and examinations, at least
in the way that they are traditionally conceived. To examine the relationship between these socio-
cultural factors and education systems, particularly examinations, UNESCO Bangkok has launched a
regional study on The Culture of Testing.

Objectives
This study proposes to better understand the socio-cultural drivers behind the culture of testing
and their effects on the relevance of education and aspirations to youth. This focus on testing more
specifically refers to the reliance and focus on examinations utilized for transition and selection
purposes, so called high-stakes exams; however, the impact of low-stakes assessments and exams
will also be explored. The study will critically analyse what the manifestations are of this culture of
testing and the relationship between socio-cultural factors and education outcomes. It will also
highlight how countries may be responding to this culture of testing, specifically reporting any best
practices that may exist which aim to reduce the focus on high-stakes testing. The study will
examine the perceptions that shape, and are shaped by, education policies and systems as a result
of the culture of testing, examining implications for policy making, particularly in light of the SDG 4
targets.

Expected Outputs
1. Comprehensive analysis of the socio-cultural drivers of the culture of testing in the Asia-
Pacific region, based on primary data collection and regional and national desk studies. The
analysis will highlight the perspectives of: learners (youth), parents, teachers, as well as
reflections from other education stakeholders and experts, with regard to the relevance of
testing and examinations, educational provision and student learning in the selected
countries of focus.
2. An analysis of major educational policies on testing, the content of national assessments and
examinations at the upper secondary level, and university admissions requirements in
selected countries.
 3. A set of recommendations that include, for example: how education systems can improve
learning and aspirations for youth, how education systems can lessen the negative impact of
the culture of testing, what implications there are for policymakers, education providers, as
well as for school-level community members, and any further study needed.

Proposed Research Questions

The following are the proposed research questions for the study. There are three overarching
research questions, while various sub-questions are included to assist in the conceptualization. The
research study is based on the three main questions. The sub-questions are included for guidance,
and can be modified/revised/removed based on the country context. Further, researchers need not
address all sub-questions

1. What are the social and cultural factors that create a culture of testing?
a. What does a culture of testing mean in your country context? What role do social
institutions play in creating a culture of testing? What are the features of these
institutions and how do they impact the education system? How do schools and
education systems create, reinforce or intensify this culture?
b. What is the importance of the examination(s)? How do stakeholders interpret the
purpose and role of these high-stakes exams?
c. Does the culture of testing affect equity of education? Are some learners affected by
this culture more/less than others? How?

2. What is the relationship between the culture of testing and education policies, education
reform, curriculum, and/or teacher pedagogy?
a. What effect does this culture of testing have on education policies? Are education
policies a reflection of this culture? Has the curriculum been changed as a response to
these exams? Or vice versa?
b. Does the culture of testing affect the teachers role in the classroom? How? Is there
any relation between this culture and teacher training and pedagogy?
c. How does this culture affect the teaching of the curriculum? Do teachers have enough
time to teach curriculum and prepare learners for exams? How does the culture of
testing affect the schools or teachers ability to implement and include a variety of
learning activities? (i.e. transversal competencies/21st century skills; School-Based
Assessments, extra-curricular activities; experiential learning; happiness and well-
being in education, etc.)
d. How does this culture affect the role of public education and private education? How
prevalent are private tutoring/exam preparation schools/classes (i.e. shadow
education)?

3. How does this culture of testing affect learners and learning outcomes?
a. How do socio-cultural pressures affect learning? How do these pressures influence the
learners perception of education and learning?
b. What are the learners/parents/teachers perceptions of these high-stakes exams?
How does each group think these exams impact learning?
c. How does shadow education affect learning?

2
 d. What are the social and cultural drivers on youth learning and learning outcomes?
What are the effects of the culture of testing on learning outcomes and learner
wellbeing? How does this culture of testing impact the quality of education?

Proposed Case Study Methodology

In order to answer these research questions, researchers are expected to carry out a case study for
their country/region. The case study should be based upon primary data collection as well as
supplemented with a desk study and literature review. For example:

1. Primary Data Collection

A. To gather as much information as possible, researchers should undertake a
survey/questionnaire. The following are proposed selection criteria guidelines:
a. respondents: learners, parents, teachers, school leaders
b. suggested size: i.e. the number of schools (9-10); number of learners (min 100); the
number of teachers (min 30); parents (min 50)
c. suggested criteria: i.e. gender balance; urban/rural; private/public school; grade
level etc.
d. modality: surveys can be mailed, completed online, or in-person
B. To add context to the information gathered from the surveys, researchers should aim to
conduct Focus Group Discussions and/or Interviews. The proposed selection criteria
guidelines are:
a. respondents: learners, parents, teachers, school leaders, education policy makers,
culture and education experts
b. suggested size: i.e. the number of focus groups (min 5), the number of participants
(teachers, parents, learners, etc.) in each group (min 5)
c. suggested criteria: i.e. gender balance; urban/rural; varying socio-economic status,
educational backgrounds, etc.
d. suggested size: i.e. the number of interviews (5-10)
e. modality: semi-structured

Each participating researcher is expected to select a broad range of schools, learners, teachers, etc.
in order to gather robust and representative data. Country researchers are encouraged to make the
necessary logistical arrangements with schools, teachers, Ministry of Education, School Boards, etc.
as soon as possible. While conducting the research during exam period may be difficult, this would
be the ideal time to gather the opinions and reactions of learners, teachers, and parents about the
culture of testing.

As part of the primary data collection, researchers are welcome to undertake a piloting of the
questionnaire in order to modify and test the questions. However, this is not required.

Suggested/Possible Stages for Piloting Questionnaire

Stage 1: Pilot test - Conduct 3 focus group discussions (each sub-group: learners, teachers, parents;
ex: 6-10 participants/group) distribute and complete the questionnaires, and then discuss.
Stage 2: Modify the questionnaires based on feedback. Share feedback and modified questionnaire
with UNESCO/other researchers.
Stage 3: Continue with the focus group discussions and interviews to collect valuable primary data.
Stage 4: Issue revised questionnaires to learners, teachers, parents.

3
 2. Desk Study
C. The case study should provide an overview of high-stakes examinations in the country (i.e.
entry-examinations of secondary school and/or university). This includes:
a. Provide background information on the exam(s) and the culture of testing
regarding these exam(s)
b. Review current education policies, practices particularly in terms of these
examinations and assessments
c. Review the role of private tutoring, tuition, cram schools as a result of these exams
D. The case study should present a literature review of socio-cultural influences on education
a. Background of predominant cultural, social, religious and family norms, customs and
practices, as these relate to education and learning
b. Review previously published studies related to impact of society and culture on
education

Proposed Case Study Outline

Total draft (20-40 pages)
I. Executive Summary (2-4 pages)
II. Introduction
III. Methodology (2-4 pages)
IV. Background and Literature Review (3-5 pages) example:
a. Analysis of major educational policies on testing, national assessments, and entrance
examinations (university/secondary, etc.)
b. Analysis of socio-cultural impact on education and examinations
V. Key Findings (10-20 pages) example:
a. Analysis of the culture of testing
b. Implications for education policy and practice
c. Analysis of effects on learners
d. Implications for quality of education
VI. Achievements and Challenges (2-4 pages)
VII. Recommendations and Way Forward (2-4 pages)
VIII. References
IX. Appendix (data tables, results, etc.)

Timeline
Dates Activity
November 22-23, 2016 Orientation Meeting finalize research questions,
methodology, tools
January-May 2017 Country research
March 31, 2017 1st Draft Country Case Study
May 31, 2017 Final Country Case Study
May-July 2017 Regional Synthesis Report
July 2017 Peer review
August 2017 Publication