Bilag 13 GB 28 Progress On Questionnaire Development

For Official Use
EDU/PISA/GB(2012)28
Organisation de Coopration et de Dveloppement conomiques

Organisation for Economic Co-operation and Development
12-Oct-2012
___________________________________________________________________________________________
_____________
English - Or. English
DIRECTORATE FOR EDUCATION
PROGRAMME FOR INTERNATIONAL STUDENT ASSESSMENT
EDU/PISA/GB(2012)28
For Official Use
Governing Board
PROGRESS REPORT ON QUESTIONNAIRE DEVELOPMENT AND DESIGN

34th meeting of the PISA Governing Board
29-31 October 2012

Madrid, Spain
Andreas Schleicher; E-mail: andreas.SCHLEICHER@oecd.org; Tel: +33.1.45.24.93.66
English - Or. English
JT03328578
Complete document available on OLIS in its original format
This document and any map included herein are without prejudice to the status of or sovereignty over any territory, to the delimitation of
international frontiers and boundaries and to the name of any territory, city or area.
EDU/PISA/GB(2012)28
TABLE OF CONTENTS
1.
Conceptual background ........................................................................................................................3

1.1 The PISA approach to large scale assessment ....................................................................................4
1.2 Overarching conceptual framework for PISA 2015 ...........................................................................5
1.3 Determining priorities for context assessment in PISA 2015.............................................................8
2. Questionnaire Development from April to September, 2012 .............................................................10
Module 1: Teacher qualification and professional knowledge, and the new Teacher Questionnaire ....11
Module 2: Science Teaching Practices ...................................................................................................12
Module 3: School-level learning environments for science ...................................................................12
Module 4: Science-related outcomes .....................................................................................................13
Module 7: Student SES, family and home background .........................................................................13
Module 9: Educational pathways in early childhood, Module 14: Parental involvement, and the Parent
Questionnaire option ..............................................................................................................................13
Module 10: Domain-general student behaviour and attitudes ................................................................14
Module 12: Learning Time and Curriculum, and the Educational Career optional questionnaire .........15
Module 15: School leadership and management ....................................................................................16
Module 19: Assessment, evaluation, and accountability.......................................................................16
3. Questionnaire design and rotation in the Main Study StQ .................................................................17
3.1 Designing the PISA 2015 Field Trial ...............................................................................................17
3.2 Time restrictions and design implications for the Main Study.........................................................18
REFERENCES ..............................................................................................................................................20
APPENDIX: DRAFT OVERVIEW OF FIELD TRIAL CONTENT ..........................................................22
Tables
Table 1.
Table 2.
Questionnaire content used in..................................................................................................5

Distribution of content .............................................................................................................9
Figures
Figure 1.
Modular structure of the PISA 2015 context assessment design .............................................7
EDU/PISA/GB(2012)28
PROGRESS REPORT ON QUESTIONNAIRE DEVELOPMENT AND DESIGN
Introduction
1.
This document provides an update on work on the PISA 2015 context questionnaire framework
and questionnaire content, following feedback from the previous two PGB meetings and from further
discussions in the Questionnaire Expert Group. The document outlines the content for the 2015 core and
optional contextual questionnaires and discusses issues relating to design and sampling. In particular, the
document:
Describes the policy-driven overarching conceptual framework that drives questionnaire

development in PISA 2015 and outlines criteria for choosing the questions to be included in the
field trial and main study context questionnaires.
Outlines progress in questionnaire development for each of the policy modules and discusses
design implications (e.g. mode or sampling) for each of the core and optional questionnaires.
Closes with a discussion of the implications of design issues (e.g. timing and rotation of the
student questionnaire in the main study).
Lists (in an appendix) all of the constructs under consideration, by module and questionnaire
instrument.
2.
The PGB is invited to:
REVIEW the draft frameworks and content for the PISA 2015 context questionnaires;
PROVIDE DIRECTIONS for the further development of the PISA 2015 core and optional
contextual questionnaires.
3.
1.
In particular, the PGB is asked to advise on:
The criteria set out in paragraph 24 for content reduction.
The recommendation in paragraph 73 to have one version of the Student Questionnaire in the
Main Survey, with no rotated content.
Conceptual background
4.
The questionnaire design for PISA 2015 builds on the frameworks set up in earlier waves and
intends to serve policy needs by providing two types of knowledge: (1) indicators used to evaluate and
monitor educational systems, and (2) multi-level (student, school, and system) models that analyse
relationships between different components of the educational system, especially between context factors
and student outcomes.
5.
The main innovation in the development of the PISA 2015 Questionnaire Framework and the
questionnaire instruments is a much closer relationship with policy interests. This was achieved by
focusing the development of new material on a series of modules addressing major policy areas, after
consultation with the PGB to determine their priorities in the development of this new material. As a result,
EDU/PISA/GB(2012)28
questions on teaching and learning, including learning time allocation and assessment practices, will
receive more attention, as well as school policies (evaluation, leadership, school management, and school
curriculum), and non-cognitive outcomes (both general and science-related). At the same time, it was
proposed that all policy areas should receive as much coverage as possible. Also, comparing new PISA
results against previous cycles becomes increasingly important, as PISA is maturing as a data base for
long-term educational monitoring and trend analysis.
6.
As a result, PISA 2015 will address a very broad range of topics - a mixture of new and well
established content -, some of them being presented in more detail than others. To cope with the amount of
material, the Main Survey design for the Student Questionnaire will need to be revisited, possibly adjusting
the administration time, once exact time estimates are available from the Field Trial.
7.
Computer-based delivery will allow for new, interactive item formats and questions being
tailored to the respondent. These innovations will support enhanced monitoring of policy-relevant
indicators, and more sophisticated analyses of patterns and relationships both within and between
countries. Computer-based delivery will also enable research on the processes of filling out the
questionnaire, helping to enhance the validity of the PISA context assessment.
1.1 The PISA approach to large scale assessment
8.
In the first assessment cycle, PISA mainly focused on the installation of procedures and the stepby-step implementation of the three major domains of assessment: Reading in 2000, Mathematics in 2003,
and Science in 2006. Assessment frameworks had to be developed, negotiated with stakeholders, and
implemented. PISA has also been recognised for its detailed, theory-based assessment of family
background, socio-economic status, and immigration background, allowing for sophisticated analyses of
equity issues within and across countries. A lot of effort went into the definition and operationalisation of
individual student background indicators, finally leading to the establishment of a powerful, integrated
index of economic, social, and cultural status for each student (ESCS).
9.
Furthermore, PISA has covered specific aspects of school environments and out-of-school
learning opportunities. With regard to science education which will be the major domain of student
assessment in PISA 2015 - PISA 2006 covered science-related learning opportunities, teaching practices,
and school-level context. In addition to these measures of educational processes, PISA 2006 captured
various kinds of non-cognitive learning outcomes such as student motivation, student engagement, beliefs,
and aspirations. Thus, with PISA 2006, the so-called Background Questionnaires had become a mixture
of (a) general student background variables and (b) scales covering a variety of domain-specific processes
and outcomes. At the same time, analytical techniques and reporting improved substantially. The initial
report on PISA 2006, Science Competencies for Tomorrow's World (OECD, 2007), was a breakthrough
in that it established multi-level modelling estimates of school effects explaining student outcomes and the
degree of equity, both within and across countries.
10.
The second wave of PISA cycles, starting with PISA 2009, established a coherent overarching
framework for context questionnaire development. The PISA 2009 Questionnaire Framework (OECD,
2009) was built on a strong and broad knowledge base grounded in educational effectiveness research. The
PISA 2009 report, organised into six volumes, was based on even more diverse and more sophisticated
analyses (see Table 1). In addition to student achievement, non-cognitive outcomes such as student
engagement were studied in detail, together with cognitive and meta-cognitive strategies. Moreover, the
impact of background variables, individual support factors, school level input, processes and policies, as
well as system level factors was reported all assessed by student and school questionnaires. Enhanced
reporting on trends both for student achievement and for context measures constituted another important
improvement in PISA 2009.
EDU/PISA/GB(2012)28
Table 1. Questionnaire content used in "PISA 2009 Results: 'What Students Know and Can Do'". (OECD 2010)
Volume I: Student Performance in Reading, Mathematics and Science:

Student background: Gender
Volume II: Overcoming Social Background: Equity in Learning Opportunities and Outcomes.
Student background: Socio-economic background, gender, immigration status (language spoken at home, age of
arrival, country of origin)
Individual support: Parental support (at beginning of primary education/at age 15), pre-primary education
(attendance, quality)
Volume III: Learning to Learn: Student engagement, strategies and practice
Student background: Socio-economic background, gender, immigration status, language spoken at home
Outcomes: Enjoyment of reading, time and material used for reading, metacognition (awareness of strategies),
self-reported use of reading strategies (memorisation, elaboration, control)
Volume IV: What makes a school successful? Resources, policies and practices
Student background: Socio-economic background, age of school entry, grade repetition
Student-reported processes: learning time (previous education, learning time at school, enrichment/remedial
education, after-school lessons), teacher-student relations, disciplinary climate, teachers stimulation of reading
engagement
School input, policies and processes (reported by the principal): type of school (public/private), number of
programmes, class size, educational resources (e.g., ICT, library), school responsibility for assessment and
curriculum/for resource allocation, extra-curricular activities provided, school admittance/grouping/transfer
policies, assessment practices/purposes, use of achievement data, school accountability, methods for monitoring
teachers, teacher and student behaviour, parent involvement and expectations, leadership.
11.
The Questionnaire Expert Group (QEG) for PISA 2012 set out to further develop and clarify the
goals, the policy needs, the opportunities as well as the challenges of PISA. The PISA 2012 Questionnaire
Framework (EDU/PISA/GB(2010)23) acknowledged that PISA, as a policy-oriented programme, has to
serve the needs of policy makers in participating countries.
1.2 Overarching conceptual framework for PISA 2015
12.
The Core 6 proposal for Questionnaire Development in PISA 2015 identified four broad areas of
interest: i) Outcomes, ii) Student background, iii) Teaching and learning and, iv) School policies and
governance.
Outcomes
13.
Educating a person means fostering his or her individual development as a unique, selfdetermined, knowledgeable person who gradually gains in ability to participate in society. As each PISA
assessment is a cross-sectional study, PISA cannot capture developmental processes, but PISA serves as a
snapshot of development at the age of 15 years. This snapshot of course includes an assessment of literacy
and life skills, but in addition to these cognitive outcomes, factors such as motivation, self-regulation,
attitudes, and beliefs are important educational outcomes. Recent scientific research on education has
highlighted that non-cognitive factors present important skills for individual and social development
(Heckman, Stixrud and Urzua, 2006; Almlund, Duckworth, Heckman and Kautz, 2011). Therefore, PISA
questionnaires also address non-cognitive outcomes. In PISA 2015, with collaborative problem solving
being one of the cognitive domains, dispositions that are relevant for collaboration should be part of the
design.
EDU/PISA/GB(2012)28
Student background
14.
In addition to the snapshot of cognitive and non-cognitive outcomes, PISA gathers retrospective
and prospective information about educational pathways and careers across the lifespan. In recent years,
researchers as well as public debates in many countries have stressed the importance of early childhood
education (Barnett, 2008; Blau and Curie, 2006; Cunha, Heckman, Lochner and Masterov, 2006; Sammons
et al. 2011). Therefore, PISA aims at collecting retrospective information on early childhood education.
However, validity of such data gathered from 15-year old students is limited; therefore, the optional Parent
Questionnaire will be used for this purpose. In order to understand educational careers, family background
such as the socio-economic status and immigrant background should be taken into account. The
distribution of educational opportunities and outcomes can help describe the extent to which countries
succeed in granting equal opportunities to their students.
Teaching and Learning
15.
School-based instruction is the core process of formal, systematic education. Therefore, policy
makers need information on teaching, learning, and organisation of schools. To increase explanatory power
of the study, assessment of teaching and learning tends to focus on the major domain of assessment, i.e.
science in PISA 2015. The knowledge base of school effectiveness research (Scheerens and Bosker, 1997;
Creemers and Kyriakides, 2008) allows for the identification of core factors, i.e. teachers qualifications
and professional activities, teaching practices, and generally learning opportunities provided both within
and outside of school. It is a challenge to address teacher and teaching-related factors in PISA, but student
and school questionnaires may provide at least some proximal indicators, and the optional Teacher
Questionnaire will significantly enhance the scope of the design. Both science-related and general
information on teaching are indispensable if PISA aims at covering educational factors that are strongly
related to student achievement.
School policies and governance
16.
As policy makers have a limited direct influence on teaching and learning processes, information
on school-level factors which help improve schools, and thus indirectly improve student learning, shall
have high priority. As to teacher and teaching variables, school effectiveness research has built a strong
knowledge base showing that certain essential supports (Bryk, Sebring, Allensworth, Luppescu and
Easton, 2010) drive school improvement. These essentials are professional capacity, a well-organised
curriculum, parental and community involvement, school climate and leadership. All of these will be
addressed within the PISA 2015 questionnaire design. In addition, resources (budget, personnel) and their
distribution between schools are important in educational policy making.
17.
To meet policy needs directly, PISA also needs to address issues related to governance on the
system level (Hanushek and Woemann 2011; Woemann, Ldemann, Schtz and West, 2007). Locus of
decision making measures and accountability practices describe core aspects of governance, namely the
distribution of power and control between central and local institutions. Allocation, selection, choice, as
well as assessment and evaluation are the basic processes deployed by policy makers and/or school
administrators to steer school quality and school improvement.
18.
These four areas were differentiated into 19 more fine-grained policy issues, which were
accepted by the PISA Governing Board (at its meeting in October 2011) as the building blocks of the PISA
2015 design for context assessment. Figure 1 provides a schematic overview of this modular structure. It
also shows (circles) how PISA covers all levels of activities related to education: the individual and
classroom level (teaching and learning), the school level, and the system (governance) level.
EDU/PISA/GB(2012)28
Figure 1. Modular structure of the PISA 2015 context assessment design
19.
Because PISA intends to support educational policy making, the framework can be compared
with one of the most comprehensive pieces of recent international research on education policy. Sykes,
Schneider and Plank (2009) provide 76 chapters with state-of-the-art reviews of educational policy
research. The section on policy implications of educational resources, management, and organisation
mentions eight policy issues, e.g. class size, equity policies, school choice, and resources. All of these are
covered by the modules provided by this framework. The section on teaching and learning policy covers
10 issues, such as opportunity-to-learn and teacher quality, which are also covered by the modules with
the exception of demography. The sections on educational access and differentiation as well as politics
and the policy process each include 4 or 5 issues covered in our modules, such as early childhood
education, shadow (out-of-school) education, assessment policies and ethnicity. Still, a few differences can
be found between the context assessment framework and the Handbook of Educational Policy Research.
PISA does not relate to research on policy processes and specific actors/stakeholders (e.g., the role of
teacher unions). The assumption followed in this framework is that this is so because PISA is not meant to
provide data on decision-making processes. On the other hand, the Handbook of Educational Policy
Research does not include any chapter on non-cognitive outcomes, which nevertheless are important
reporting categories in PISA.
20.
To sum up, the modular approach to context assessment in PISA 2015 allows for broad coverage
of policy issues and related research questions. However, there are strict limits to length of questionnaires
7
EDU/PISA/GB(2012)28
and the time needed to fill them out. Therefore, in addition to breadth, PISA needs to focus on the most
important issues (modules, or constructs within modules).
1.3 Determining priorities for context assessment in PISA 2015
21.
The PISA Governing Board was asked to indicate the top six priority modules for further
developmental work, based on policy relevance and the need for improvement from previous cycles. In one
way or another, all topics will be covered in the PISA 2015 questionnaires, but more emphasis will be
devoted to those priority areas identified by the PGB.
22.
The following six areas received the most votes for high policy relevance and need of further
development work for PISA 2015 in order of popularity:
19. Assessment, evaluation and accountability
02. Science teaching practices
04. Science-related outcomes: motivation, attitudes, beliefs, strategies
12. Learning time and curriculum
10. Domain-general student behaviour and attitudes
15. Leadership and school management
In addition to these priorities, four modules are assigned medium priority:
01. Teacher qualification and professional knowledge
09. Educational pathways in early childhood
03. School-level learning environments for science
07. Student SES, family and home background
The following modules were rated of lowest priority. While these will be included to some extent in
the questionnaires, there has been less development in these areas:
13. School climate: Interpersonal relations, trust, expectations
08. Ethnicity and migration
11. Student dispositions related to collaborative problem solving
05. Out-of-school science experience
06. Science career
16. Resources
14. Parental involvement
18. Allocation, Selection and choice
17. Locus of decision making within the school system
23.
Independently, the PISA 2015 Questionnaire Expert Group (QEG) had delivered a similar rating.
Taking into account the results of the PGB rating exercise as well as suggestions from the QEG,
EDU/PISA/GB(2012)28
questionnaire development focused on the high priority and (to a lesser extent) the medium priority
modules.
24.
Besides strengthening the alignment with policy needs, a clear focus on high and medium priority
modules helps limit the length of the questionnaires. Another method of keeping the size of material
manageable is distributing it across different questionnaires, such as Student and School Questionnaire,
optional Parent and Teacher Questionnaires, and short, optional add-ons to the Student Questionnaires
(Educational career, ICT familiarity), as shown in table 2. As a rule, information will be gathered from
those stakeholders who can provide the most valid perspective.
Table 2. Distribution of content
Module
StQ
Teacher education and professional knowledge
Teaching practices
School environment for science learning
Science-related non-cognitive outcomes
Out-of-school science learning experiences
Science career
Student SES, family and home background
Ethnicity and migration
Educational pathways in early childhood education
ScQ
PQ TQ ICT
EC
10 General non-cognitive outcomes

11 Dispositions to CPS
12 Learning time and curriculum
13 School climate
14 Parental involvement
15 Leadership and school management
16 Resources
17 Locus of decision making
18 Admission, selection and choice
19 Assessment, evaluation, and accountability
25.
As a third step in content reduction, constructs are prioritised according to the following
principles:
EDU/PISA/GB(2012)28
In as far as possible, questions and items with proven validity and measurement quality
(established measures) will be retained from previous PISA cycles, especially from PISA 2006,
to allow for trend reporting.
Questions that serve the needs of multiple modules are preferred for reasons of efficiency.
Short questions will be preferred to longer questions.
Questions that can be expected to work across cultures, and/or may help to describe
country/culture differences, will receive higher priority.
If quality is otherwise equal, constructs will be retained that are more strongly related to student
outcomes.
If there are two or more similar constructs, all but one will be dropped.
26.
A reduced, but still relatively large set of questions will be submitted to the PISA 2015 Field
Trial which will take place in 2014. After the Field Trial, a final reduction will be conducted (again in
accordance with above described procedures).
2.
Questionnaire Development from April to September, 2012
27.
Two meetings of the Questionnaire Expert Group (QEG) took place in 2012, one in May 2012
and one in August 2012.
28.
The May meeting focused on the research and policy background of high and medium priority
modules, discussing both conceptual and methodological innovations. In preparation for that meeting, The
PISA consortium (Core 6) staff contracted questionnaire developers and provided background papers for
10 modules and four international options. These papers reviewed relevant research and experience from
previous PISA cycles and laid out conceptual foundations for further development.
29.
Based on recommendations from the May QEG meeting, PISA consortium (Core 6) staff and
contracted developers started to revise existing PISA measures and develop new questionnaire material,
once again with a clear focus on high priority modules. At this stage of work, no limitations regarding
breadth and depth were applied, because the goal was to procure a wide set of measures sometimes even
including several alternative approaches to cover a specific construct. The QEG was thus to receive a wide
array of possible high-quality questions and items, which could then be discussed, reviewed, prioritised,
and where needed - selected by the QEG. 16 working papers were submitted to the QEG meeting in
August, 2012, comprising some 368 measures across all 19 modules and the international questionnaire
options. At their August meeting, the QEG made a first selection. Following their advice and comments
received from the Secretariat, PISA consortium (Core 6) staff are currently engaged in a second wave of
questionnaire development, revising the proposed material and in some cases also developing new material
to meet requests from the QEG and/or the Secretariat. Results from this second wave of questionnaire
development will be discussed by the QEG in December, 2012 during a series of webinars. In January
2013, PISA consortium (Core 6) staff will open a web-based forum for NPMs, asking for overall
comments and discussion of specific issues. This forum will provide an opportunity for countries to
provide comments on cultural appropriateness and national feasibility of new measures. PGB members and
the Secretariat will receive the final instruments in March 2013 for discussion at the April, 2013, meeting
of the PGB, and for implementation, translation, and adaptation for the Field Trial.
10
EDU/PISA/GB(2012)28
30.
To provide a more detailed insight into this on-going process, the following section describes
conceptual foundations and questionnaire development for the high and medium priority modules. Also,
the innovations planned within related international options will be explained in this context.
31.
A full overview of material currently discussed within the PISA consortium is presented in the
appendix.
Module 1: Teacher qualification and professional knowledge, and the new Teacher Questionnaire
32.
Teachers are at the core of school instruction. Many studies have demonstrated a clear influence
of teaching practices and other teacher-related factors on student learning and outcomes. In recent years,
this relation has received growing interest. Many stakeholders are valuing the central role teachers play in
education, also regarding their role as school leaders and key sources of innovation. The PISA 2015
Questionnaire Framework particularly pursues this issue in two Modules: While Module 2, Teaching
practices, deals with professional behaviour within the classroom, Module 1, Teacher qualification and
professional knowledge, deals with background variables, individual qualification and competence, as
well as professional practices such as collaboration and professional development - on the school level.
33.
This layout of teacher-related factors in the PISA 2015 Questionnaire Framework is in line with
OECDs Teaching and Learning International Survey (TALIS), which distinguishes between the teachers
tasks in terms of classroom practices on the one hand and teachers professional activities on the other
hand.
34.
In PISA, teacher-related information so far has been gathered from school administrators and
students only. The School Questionnaire asked about initial education of recruited staff and participation in
professional development. These measures will be addressed in PISA 2015 as well, and information on
policies regarding both issues will be added. Also, a scale on students perception of teacher engagement
has been adapted from the research literature. Thus, there will be some coverage of Module 1 issues in the
standard questionnaires. In addition, PISA 2015 will include an optional Teacher Questionnaire (TQ) to
tackle this high priority policy module in more detail.
35.
The implementation of Module 1 constructs will become much more feasible with the
introduction of this International Option. Teachers themselves will provide much more valid information
on their background, initial education, professional career and professional development activities than
school administrators could do. It will thus be possible to address policy issues of high relevance:
recruitment, selection, and allocation of teachers to schools; career changers; teachers teaching science
classes without any formal training for science education; fluctuation among the teaching staff.
36.
An important feature of the Teacher Questionnaire (TQ) in PISA will be the split of the sample:
Up to 10 science teachers will be sampled (TQ-Science), as well as 10 or more teachers teaching 15-year
old students in other subjects (TQ-General). Both groups will receive specific questionnaires, with an
overlap for background variables (TQ-Core). Thus, science teachers can be asked about science-specific
aspects, such as the relative coverage of content knowledge, pedagogical content knowledge (i.e.
knowledge about student learning, typical misconceptions etc.) and general pedagogical knowledge (e.g.,
classroom management techniques) information that will be gathered through an interactive question
which allows respondents manipulate a pie chart to provide this information. Also, questions about initial
education and professional development will be tailored to science teachers in one case, and otherwise
address more general issues.
37.
As discussed at the PGB when the TQ option was submitted in April 2012, and at the QEG, it
will not be possible to measure teachers professional knowledge in PISA. However, related beliefs,
11
EDU/PISA/GB(2012)28
especially self-efficacy beliefs, and non-cognitive factors, such as teacher enthusiasm and job satisfaction,
will be addressed. To check the validity of teachers self-reports of enthusiasm, (a) anchoring vignettes will
be provided, and (b) teacher engagement/enthusiasm will also be covered from the student perspective.
38.
Finally, teachers will provide information on other policy issues, such as leadership, school
climate, parental involvement, and assessment practices.
Module 2: Science Teaching Practices
39.
In line with the PISA approach of measuring scientific literacy as the capacity of students to
identify scientific questions, to explain phenomena scientifically, to use scientific evidence, and to value
scientific arguments, most science education experts nowadays recommend so-called inquiry-based
teaching as the via regia for fostering science literacy. Accordingly, PISA 2006 focused on various aspects
of inquiry-based teaching when asking students about their classroom learning experience. Four scales
were established: (1) interactive science teaching, (2) hands-on activities, (3) student investigations, and
(4) real-life applications. However, re-analyses showed that these scales were highly correlated. Also, the
QEG recommended extending the coverage of science teaching practices to capture learning environments
that are not aligned with the inquiry paradigm.
40.
As a first step, PISA consortium (Core 6) experts revised the scales, shortening them and
aligning them with more recent research that identifies four aspects of inquiry-based teaching: procedural
aspects (e.g., students designing their own experiments), social aspects (e.g., class debating alternative
hypotheses), epistemic aspects (e.g., drawing conclusions from observations), and real life applications.
41.
Secondly, two scales were added that ask about other kinds of science teaching practices. One
scale covers direct instruction (e.g., teacher providing lectures and designing well-structured practice
sessions) which is very popular, and has been shown to have positive effects especially on low-achieving
students. Another scale, named opportunity to learn, covers various aspects of media use, instructional
methods (such as teacher vs. student presentations) and interactive settings (e.g., small group vs. whole
class discussion).
42.
All the measures discussed above are rather close to observable behaviour, mostly teacher
practices. Research on the quality of teaching, however, suggests that latent dimensions of teaching
quality are most predictive for student outcomes. Following experience with mathematics teaching quality
in PISA 2012, measures of teacher support, disciplinary climate, and cognitive activation (i.e. challenging
tasks) in science classes have been developed. Classroom discipline and teacher support have been
measured in most PISA cycles except for 2006, and have been shown to be highly correlated to student
achievement and student motivation, respectively. Both will be adapted to science education in the PISA
2015 Field Trial.
43.
Student perceptions proved to be valid indicators of teaching practices and teaching quality in the
research literature. However, with an optional Teacher Questionnaire being introduced, there will also be a
scale reporting on classroom practices from teachers point of view. Teachers will be asked how they
perceive inquiry-based science education, and to what extent they implement its components. Teachers
perceptions will be addressed by a set of vignettes, which can also serve as anchoring vignettes for
teachers self-reports, allowing to correct for response bias.
Module 3: School-level learning environments for science
44.
Conceptually, this module has quite some overlap with other modules dealing with school-level
factors, such as Module 12, Learning Time and Curriculum, Module 15, School Leadership and
Management, or Module 19, Assessment, Evaluation, and Accountability (see below for brief reports on
12
EDU/PISA/GB(2012)28
those). In addition to those, the QEG requested some questions in the School Questionnaire that would
directly focus on the status of science education in the school, and available resources. Accordingly, a
question has been developed to investigate the overall value of science within the school (Is science
education a shared priority for stakeholders?), along with questions on resources available for science
education: size of teaching staff per science area; physical resources such as laboratories and equipment for
student experiments; and cooperation with science museums, companies and other external partners.
Module 4: Science-related outcomes
45.
Positive attitudes towards science and technology, but also motivation, value beliefs and
strategies that motivate students to engage in scientific issues are important educational and policy goals in
many countries. Moreover, such attitudes affect students learning and performance as well as their choice
of courses and future careers. However, the relationship between attitudes, motivation, value beliefs and
strategies on one hand and science performance on the other hand is reciprocal. They can cause higher
science performance as well as being a consequence of higher science performance. Therefore, they are
important outcomes of education themselves.
46.
PISA 2015 will not use the embedded measures of content or task-specific interest that were
used in some PISA 2006 cognitive assessment units, because of limited validity and worries about negative
impact on achievement. However, classical questionnaire scales covering student interest (investigating
two aspects: science in general, and science as experienced in school) and instrumental motivation will be
developed. To ensure cross-cultural validity, and control for response bias, anchoring vignettes will be
provided.
47.
While PISA 2006 implemented several highly correlated scales for student beliefs on
environmental issues, only one scale for environmental awareness will be kept in PISA 2015. Instead,
PISA 2015 will address more general student views on science, especially as explicated in the PISA 2015
Science Framework Valuing scientific approaches to enquiry. Another innovation in the PISA 2015
science framework is the treatment of technology as a substantial and integral aspect of science literacy,
rather than a matter of application. The Student Questionnaire reflects this change by asking about
frequency of, commitment to, beliefs about, and perceived competence in use of technology. Finally, the
2006 measure of science-related self-concept will be taken up, now complemented by self-efficacy a
construct that in PISA 2012 was shown to be strongly related to student performance.
Module 7: Student SES, family and home background
48.
In order to compare equity related to social and ethnic factors across PISA cycles, the PISA
consortium (Core 6) staff intend to keep measures of socio-economic status and other background
variables basically unchanged. However, the measures of home possessions will be updated, experience
with the new international coding system for occupations (ISCO) from PISA 2012 will be reviewed, and
the new international system of educational levels (ISCED) will be applied in PISA 2015. Furthermore,
alternative ways of asking about parental occupations in the Field Trial will be developed, because the
quality of these data may change when students need to type into a computer.
Module 9: Educational pathways in early childhood, Module 14: Parental involvement, and the Parent
Questionnaire option
49.
When children enter primary school, they already differ in their language, pre-reading, and early
numeracy skills and these differences are often maintained later in life. Promoting school readiness and
better adjustment to school is hypothesised to be an efficient means of raising the achievement levels of all
children, but especially of those children who experience a lack of parental support or who grow up in
13
EDU/PISA/GB(2012)28
disadvantaged circumstances. It has been argued that investing in early education programmes will have
large long-term monetary and non-monetary benefits. Early childhood education has been included in three
PISA cycles so far. The importance of pre-school quality has been acknowledged and analysed by OECD
reporting as well.
50.
There are good reasons to gather more information on students' early educational pathways in
PISA. However, research on biographical memory suggests that regarding the most important aspects, e.g.
type, quality, or dosage of early child care, no reliable data will be gleaned from 15-year-olds answering
retrospective questions about educational experiences they made a decade or more ago. So, while keeping
a short question on ISCED 0 attendance in the Student Questionnaire, PISA consortium (Core 6) staff
following advice from QEG and external experts intends to implement a series of questions in the parent
questionnaire, expecting parents to be the more reliable source of information. Those countries applying
the optional parent questionnaire will gain information on basic characteristics of the early childhood
education and care arrangements of PISA participants and reasons for attending or not attending early
childhood education and care.
51.
Over the past years, the involvement of parents in educational processes has gained importance in
the educational debate, and to some extent it has also become relevant for educational policy. Parents are
not only an important audience, but powerful stakeholders in education, thus information on parents
opinions and engagement is highly valuable also for large scale assessments like PISA. PISA already
covers important aspects of parental involvement, mostly addressed in the international questionnaire
option of the Parent Questionnaire. In countries participating in this option, the parents of the students
sampled for PISA provide information on their engagement that focuses on the interaction with the
institutional learning setting, i.e. teachers and schools, and their engagement that focuses on the interaction
with the student, i.e. learning support. Aspects that are covered in this questionnaire include home to
school collaboration, the learning support parents provide to their child, the availability of home resources
for learning as well as parents attitudes and beliefs. In PISA 2015, the aspects of home to school
collaboration and parental learning support shall be covered in more depth than in previous cycles by
adding new constructs to the parent questionnaires.
52.
Independently from the parent questionnaire option, aspects of parental involvement are
addressed in other questionnaires: Teachers will be asked about their collaboration with parents, for
example through scheduled parent-teacher meetings. More importantly because this will allow the topic
to be covered in all participating countries -, a question from PISA 2012 has been adapted to address
parental influence on students views of science, and new items on parental support have been developed.
Following approaches that have been successfully implemented in research, several aspects of parental
support can be discriminated: stimulation/scaffolding, emotional warmth, guidance and boundaries, and
stability. Finally, school administrators will be asked about school policies on home to school collaboration
implemented in their schools. This includes the aspect of informing parents about their childs progress,
but also different forms of enhancing parents engagement in school activities.
Module 10: Domain-general student behaviour and attitudes
53.
Domain-general student behaviour and attitudes refer to the set of student factors or constructs
that cut across curricular topics, or that are independent of curricula, but that are nevertheless important for
and reflective of success in education. Alternative labels that have been used in the literature are noncognitive skills, non-intellectual correlates of GPA, or 21st Century Skills. Recently economic
models have included this information to better understand labour market outcomes. Meta-analyses show
that these characteristics predict educational success to a degree that is comparable to cognitive ability
measures.
14
EDU/PISA/GB(2012)28
54.
So far, ICT-related skills and attitudes, and general attitudes towards the school (including sense
of belonging, a measure of well-being in school) have been the only types of domain-general outcomes
addressed in PISA. The former will be taken up in the optional ICT Questionnaire, while the latter will be
included in the Student Questionnaire, providing trend information relating to previous PISA assessments.
PISA 2012 added an assessment of dropout rates a kind of negative outcome by school administrators,
and an assessment of some basic student preferences related to problem solving, called industriousness
and openness in the psychological literature. Also, academic self-efficacy was assessed as a very general
indicator of self-related beliefs. PISA consortium (Core 6) intends to broaden the array of student
preferences and self-related beliefs, and provide anchoring vignettes to establish cross-cultural validity. For
example, grade-related expectations are added as a new aspect of self-related beliefs, and a link to Module
19 (student assessment), which is also a high priority area in PISA 2015.
55.
All non-cognitive outcomes mentioned so far are conceptually related to student achievement,
learning and schooling. However, with support from PGB and QEG, the PISA consortium (Core 6)
intends to move into broader areas of non-cognitive outcomes, namely general well-being, health, and time
use. While student behaviour and attitudes have generally been rated very high in policy priority, there
may be some doubts as to the feasibility and relevance of these specific constructs. The QEG, at its recent
meeting, acknowledged these cautions, but advised to apply these questions in the Field Trial to see how
students react, and how the respective data relates to other student, family, and school related factors. Are
well-being and health related to student achievement? Is this relationship shaped by family background
and/or school policies and school climate? Addressing such questions for analysis and reporting should be
highly relevant for educational policy making. The time use data are important for Module 12 (learning
time) as well. This question has been simplified, shortened, and focused on learning activities; it is now a
very innovative computer-based approach to the assessment of learning activities.
Module 12: Learning Time and Curriculum, and the Educational Career optional questionnaire
56.
Students educational achievement is influenced by a wide array of opportunities to learn. On a
very broad level of abstraction, students opportunities to learn can roughly be described by the quantity
and the quality of learning instances. As outlined above, Module 2 will predominantly introduce concepts
of quality of education and teaching in PISA 2015 and will thus cover the quality of learning opportunities.
Module 12 and the Educational Career optional questionnaire (ECQ) then complement Module 2 by
focussing on the current quantity of learning experiences. While these constructs have been part of most
previous PISA cycles, PISA 2015 will deliver critical innovations.
57.
The learning time and curriculum experienced by students in their educational career are closely
related to student outcome. On a global level, learning time and curriculum are system characteristics, yet
the intended learning opportunities need to be realised on school and on classroom level, and finally
students must avail themselves of learning time and curriculum in order to learn. A number of factors
influence this implementation in schools and in the classroom, e.g. resources such as rooms, media
devices, and textbooks as well as staff. Also all levels involved in providing and using learning
opportunities hold the possibility to lose a share of the intended learning time and learning opportunities.
This happens when schools close unscheduled (e.g. due to bad weather or safety issues) or need to
reschedule due to poor resources or problematic teacher and student behaviour; on the individual students
part there remain problems of truancy and inattentiveness. Taking into account the different perspectives of
intended learning opportunities on the one hand (i.e. learning time provided and intended curriculum) and
loss of opportunities due to difficulties in realisation and implementation on system, school, classroom, and
individual levels on the other hand, PISA 2015 will provide more detailed and more valid information on
learning time than previous cycles. This approach to measuring learning time as it is offered, implemented,
used or not used was highly appreciated by the QEG; its feasibility will be checked in the Field Trial.
15
EDU/PISA/GB(2012)28
58.
Besides allowing for a detailed comparison of school systems on country level, the proposed
questionnaire components provide valuable indicators to compare educational opportunities across
different subgroups of students. Social or ethnic disparities in access to learning opportunities can thus be
analysed. The approach used in Module 12 will enable differentiation between disparities in provision and
use of educational opportunities.
59.
Students learning does not only relate to education in schools alone. Achievement level at the
age of fifteen is determined by all synchronic and diachronic learning experiences. In recent years, more
and more non-school learning instances have gained importance worldwide. Besides looking only at
opportunities to learn in school, PISA needs to ask for time spent in other kinds of educational settings. In
previous cycles, a certain problem has always been to disentangle regular (mandatory) and additional
lessons, private supplementary lessons and private additional study, in- and out-of-school learning
experiences. Therefore, Module 12 will expand the assessment of learning time, including loss of intended
time, to some out-of-school environments. However, detailed coverage of such context goes beyond the
scope of the standard Student Questionnaire. Therefore, with strong support from the QEG, the PISA
consortium (Core 6) has added a focus on supplementary learning opportunities to the Educational Career
optional questionnaire (EC Option). Module 12 and the EC Option seamlessly connect to each other
providing a basic understanding of intended and used learning opportunities in school and supplementary
to school, in curricular and non-curricular domains. PISA 2015 will therefore provide insight in a large
array of learning instances tapping the most relevant sources of education in the students lives.
Module 15: School leadership and management
60.
The PISA measure of educational leadership has seen a major revision in the 2012 study. Core 6
will carefully analyse the results and revise the measure if needed. An additional question focusing on
instructional leadership has been recommended by the QEG. The new optional Teacher Questionnaire will
be used to gather information on instructional and transformational leadership from teachers as well,
because research has shown that teachers perspective on leadership can differ from the positions held by
school administrators.
61.
The PISA 2015 Field Trial will also address specific school policies in various areas of school
management and school improvement. Several new measures are spread across a number of Modules,
assessing policies on learning time and the school curriculum (Module 12), teacher recruitment and
professional development (Module 1), internal as well as external evaluation and teacher accountability
(Module 19), parental involvement (Module 14), and multicultural education (Module 8). Altogether, these
measures allow to draw a multi-faceted picture of key management activities.
Module 19: Assessment, evaluation, and accountability
62.
Research has shown that assessment-related policies and practices are among the most influential
processes on the classroom, the school, and the system level. Educational reforms in many countries are
centred on assessment, evaluation and accountability. Instruments like performance standards, standardbased assessment, centralised exams, internal and external evaluation, annual reports on student progress,
and school inspectorates have been promoted and implemented across continents. Analysing their
relationships with other characteristics of schools can add to our understanding of school policies and
student outcomes. The PISA questionnaires can help to describe assessment practices, to compare them
between countries, and to study their relationship with school level variables.
63.
This module has received the highest rating in priority from the PGB members probably
because the issue is very prominent in OECD reports on PISA, but still weakly represented in the
questionnaire design itself. In the most recent School Questionnaire (PISA 2012), student assessment
16
EDU/PISA/GB(2012)28
practices and their purposes were covered, as well as school evaluation, teacher evaluation, and
accountability policies and practices. The very broad notion of standardised tests is used here whether
these tests are high stakes or low stakes, local or national, etc.
64.
Module 19 has been designed to draw a more sophisticated picture. Therefore, the module
systematically discriminates student assessment, which mostly takes place in classrooms, from evaluation,
which takes place at the school level. For both levels, information is gathered on (a) frequency of using
different assessment practices, (b) purposes and consequences of the assessment. While the questions on
student assessment focus on formative assessment and feedback (also called Assessment for learning),
the questions on evaluation mainly deal with internal rather than external evaluation, because research has
shown formative assessment and internal evaluation to have larger impact on student learning and school
development than summative assessment and external evaluation, respectively. Information from different
perspectives (student, teachers, administrators) will be combined. Results will also be important for
Modules 2 (teaching) and 15 (school leadership and management).
3.
Questionnaire design and rotation in the Main Study StQ
3.1 Designing the PISA 2015 Field Trial

65.
The full material which is proposed for the Field Trial is documented in the Appendix. Some
further changes to this material can be expected based on:
cognitive labs and focus group studies which are currently underway,
feedback from QEG (at its virtual meetings in December, 2012), from NPMs (through the webbased forum which will open in January, 2013), from the Secretariat, and of course from the
PGB,
and further revisions made by the PISA consortium (Core 6) staff, questionnaire developers and
contracted experts.
66.
A number of single items or questions may be changed or dropped as a result of these
deliberations. However, the overarching modular design of the questionnaires as explained in section 1 of
this paper - and the framework of policy issues and constructs developed over the past year as elaborated
in section 2 will remain unchanged. Development of the constructs listed in the appendix has been based
on policy priorities defined by the participating countries, results from previous PISA studies, and an
extensive review of relevant research literature.
67.
The material documented in the appendix altogether covers an estimated assessment time of 97
minutes for the FT Student Questionnaire, 55 minutes for the FT School Questionnaire, 27 minutes for the
FT Parent Questionnaire, 60 minutes for the FT Teacher Questionnaire, and about 18 minutes each for the
FT Educational Career Questionnaire and 14 minutes for the FT ICT Familiarity Questionnaire. In all
cases, the amount of material adheres to the time estimates for Field Trial instruments outlined in the
Integrated Design (see Document submitted by Core 3 and Core 6). This is possible because the Field Trial
design in line with practices established in all previous PISA studies will allow for different booklets
randomly being allocated to subgroups of students (for Student Questionnaire, Educational Career
Questionnaire option, and ICT Questionnaire option) and Teachers (for the Teacher Questionnaire option).
No person will have to sit on the questionnaire above time limits.
68.
met:
By field trialling this material, the following goals of questionnaire development in PISA can be
17
EDU/PISA/GB(2012)28
allowing for broad coverage of topics in all 19 modules;
assessing high priority modules with more material to provide deeper insights;
establishing close fit to the PISA 2015 major domain of assessment, namely science;
reviewing and - if feasible - reusing or revising any relevant material from earlier PISA cycles,
especially from PISA 2006, (a) to make efficient use of earlier translation and adaptation efforts,
and (b) to establish trend in reporting on context variables and non-cognitive outcomes;
trying out new, innovative questions and question formats including interactive formats,
context-dependent questions, and formats intended to secure cross-cultural comparability, such as
anchoring vignettes;
ensuring feasibility of computer-based delivery for all previously established material.
69.
The breadth of Field Trial material will allow for selecting the most appropriate content for the
Main Study. Introducing new, innovative content and format always bears the risk of not working on a
large scale, even if these innovations are based on sound research experience. This especially holds true
when a new mode of delivery (here: computer-based assessment) is being introduced at the same time.
From a policy point of view, using larger amounts of material in the Field Trial provides more degrees of
freedom for policy makers (PGB) when selecting Main Study content based on the Field Trial data.
3.2 Time restrictions and design implications for the Main Study
70.
The Main Study will see much stronger time restrictions than the Field Trial (see Integrated
Design): approximately 30 minutes for the Student Questionnaire, 45 minutes for the School
Questionnaire, 30 minutes each for the Teacher and the Parent Questionnaire, and 10 minutes each for the
Educational Career and the ICT options. Thus, severe cuts will be made from the Field Trial to the Main
Study, especially regarding the two standard instruments, School and Student Questionnaire.
71.
In preparation for this reduction exercise, the amount of material that might be taken up from
previous cycles has been estimated (cf. footnote in the Appendix). This hypothetical selection was based
on the PGBs priorities, discussion at the QEG and a review of existing PISA reports (as in Table 1 above).
This selection of well established questions amounts to about 30 minutes for the Student Questionnaire,
and about 22 minutes in the School Questionnaire. Regarding the School Questionnaire, there is ample
space left for new, innovative content. Regarding the Student Questionnaire, however, these estimates
underpin the importance of finding ways of optimising testing time. Otherwise, the PISA 2015 Student
Questionnaire would be left without any innovation and improvement; neither in content nor in measures.
72.
PISA 2012 introduced a rotated booklet design for the student questionnaire, with an eight
minutes core set of questions covering student and family background (in our terminology: Modules 7
and 8), while all other content was given to 2/3 of the students only. An estimated 33 minutes of non-core
content were formed into three clusters (A, B, C), and each student was randomly assigned to a
combination of two of these clusters (AB, BC, or CA). Random assignment of students to booklets ensured
that parameters reported for questionnaire constructs like means, standard deviations, and percentiles
would be unbiased. Altogether, the rotation scheme allowed for coverage of approximately 40 minutes of
estimated assessment time in the Student Questionnaire.
73.
The test design has become more complex in PISA 2015. The computer-based tests will assess
four domains (science, reading, mathematics, and collaborative problem solving), while most individual
18
EDU/PISA/GB(2012)28
students will be working on two of these domains only. These settings require more student scores (namely
50 per cent of all student scores) to be estimated (imputed) from achievement in other domains and
background information, thus putting high burden on the background information produced from
questionnaires. The suitability of a rotated design as used in PISA 2012 has been called into question
because of this increase in complexity for the assessment in PISA 2015.
74.
Therefore the recommendation from the PISA 2015 Consortium and the QEG is that there should
be only one version of the student questionnaire in the PISA 2015 main study and no rotation, if enough
innovative material can be included. The results of the field trial would be used - taking advantage of the
more accurate information on timing which will be obtained from the computer-based administration - to
determine how much time will be needed to cover both established and innovative content in the main
study.. Furthermore, the time available for the administration of the student questionnaire in the main study
for 2015 might need to be revised based on the results of the field trial. The analysis of PISA 2012 data,
especially a careful evaluation of the rotation design applied, will also be considered when determining the
design of the background questionnaires in the PISA 2015 main study.
19
EDU/PISA/GB(2012)28
REFERENCES
Almlund M., A.L. Duckworth, J.J. Heckman and T. Kautz (2011), "Personality Psychology and
Economics," IZA Discussion Papers 5500, Institute for the Study of Labour (IZA), Bonn.
Barnett, W.S. (2008), Preschool Education and its Lasting Effects: Research and Policy Implications,
Education and the Public Interest Center & Education Policy Research Unit, Boulder and Tempe,
http://epicpolicy.org/publication/preschooleducation, retrieved 27 September 2012.
Blau D. and J. Curie (2006), "Pre-School, Day Care, and After-School Care: Who's Minding the Kids?" in
Handbook of the Economics of Education, Elsevier, North Holland, pp. 1164-1278.
Bryk, A.S., P. Sebring, E. Allensworth, S. Luppescu and J. Easton (2010), Organizing School for
Improvement, Lessons from Chicago, The University of Chicago.
Creemers, B.P.M. and L. Kyriakides (2008), The Dynamics of Educational Effectiveness: A Contribution
to Policy, Practice and Theory in Contemporary Schools, Routledge, London.
Cunha, F., J.J. Heckman, L.J. Lochner and D.V. Masterov (2006), "Interpreting the Evidence on Life Cycle
Skill Formation,", in E. Hanushek and F. Welch (eds.), Handbook of the Economics of Education,
Elsevier, North Holland, pp. 697-812.
Hanushek, E.A. and L. Wmann (2011), The Economics of International Differences in Educational
Achievement, in E.A. Hanushek, S. Machin and L. Wmann (eds.), Handbook of the Economics
of Education, Vol. 3, North Holland, Amsterdam, pp. 89200.
Heckman J.J., J. Stixrud and S.Urzua (2006), "The Effects of Cognitive and Noncognitive Abilities on
Labor Market Outcomes and Social Behavior", Journal of Labor Economics, University of Chicago
Press, Vol. 24, 3, pp. 411-482.
OECD (2007), Science Competencies for Tomorrow's World, Executive Summary, OECD Publishing,
Paris.
OECD (2009), PISA 2009 Questionnaire Framework, in PISA 2009 Assessment Framework: Key
Competencies in Reading, Mathematics and Science, OECD Publishing, Paris, pp. 149-169.
OECD (2010), PISA 2009 Results: What Students can do? Student Performance in Reading, Mathematics
and Science, Vol. 1, OECD Publication, Paris.
Sammons, P., K. Sylva, E. Melhuish, I. Siraj-Blatchford, B. Taggart, K. Toth, D. Draghici. and R. Smees
(2011), Effective Pre-School, Primary and Secondary Education Project (EPPSE 3-14): Influences
on Students Attainment and Progress in Key Stage 3: Academic Outcomes in English, Maths and
Science in Year 9, Institute of Education, University of London / Department for Education, London.
20
EDU/PISA/GB(2012)28
Scheerens, J. and R.J. Bosker (1997). The Foundations of Educational Effectiveness, Pergamon Press,
Oxford.
Sykes G., B. Schneider and D.N. Plank (eds.) (2009), Handbook of Educational Policy Research,
Routledge, New York.
Wmann, L., E. Ldemann, G. Schtz and M.R. West (2007), School Accountability, Autonomy, Choice,
and the Level of Student Achievement: International Evidence from PISA 2003, OECD Education
Working Paper No. 13.
21
EDU/PISA/GB(2012)28
APPENDIX: DRAFT OVERVIEW OF FIELD TRIAL CONTENT
Module Policy issue and construct area

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Specific construct
measured
to
be
Teacher qualification and professional knowledge

Teacher background
Gender
Teacher background
Age
Initial education
Level of education
Teacher education or training
Initial education
programme: completion, mode of
qualification, and duration
Subjects
studied,
subjects
Initial education
teaching
Teacher background
Workplace selection
Teacher background
Employment status
Job experience number of
Teacher background
schools
Teacher background
Job experience number of years
Teacher background
Teaching modal grade
Professional
development
Professional development
activities
Obligation to participate in
professional
development
activities
Support
for
professional
development activities
Content distribution for initial
Education
and
professional
development Activities
22
Status of the measure
Estimated
Time
Instrument
(sec)
new to PISA, from TALIS 2013

new to PISA
10
10
15
TQ Core
TQ Core
TQ Core
new to PISA
60
TQ Core
100
TQ Core
new to PISA
120
10
TQ Core
TQ Core
new to PISA
10
TQ Core

new to PISA
20
10
TQ Core
TQ Core
new to PISA
110
TQ Core
new to PISA
10
TQ Core
new to PISA
20
TQ Core
new to PISA
140
TQ General
new to PISA
EDU/PISA/GB(2012)28
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
Teacher cooperation - general
Professional knowledge and teacher Self-efficacy
in
classroom
beliefs
management
Professional knowledge and teacher Self-efficacy in
maintaining
beliefs
positive relations with students
Professional knowledge and teacher
Performance vs. needs orientation
beliefs
Non-cognitive factors
Job satisfaction
Content distribution for initial
Initial education
education
Science
specific
teacher
collaboration
Content
distribution
for
professional
development
activities
Professional knowledge and teacher Self-efficacy related to science
beliefs
content
Professional knowledge and teacher Self-efficacy related to teaching
beliefs
science content
Teacher enthusiasm (vignettes)
Teacher enthusiasm (scales)
Initial education of all teaching
Initial education
staff
Requirement
for
teacher
Initial education
education or training programme
Initial education
Teacher recruitment policies
Professional development policy
Participation in professional
development
Perceived teacher engagement
Science teaching practices
Science teaching: Inquiry-based
(4
dimensions:
procedural,
Teaching practices
epistemic, social, and real life
applications)
Science
teaching:
direct
Teaching practices
instruction
Teaching practices
Science teaching: Opportunities
23
new to PISA
80
TQ General
new to PISA
40
TQ General
new to PISA
40
TQ General
new to PISA
60
TQ General
new to PISA
100
TQ General
new to PISA
50
TQ Science
new to PISA
220
TQ Science
new to PISA
50
TQ Science
new to PISA
80
TQ Science
new to PISA
80
TQ Science
new to PISA
new to PISA
previous (2000)
(2009) (2012)
80
190
TQ Science
TQ Science
120*
ScQ
new to PISA
20
ScQ
new to PISA
new to PISA
20
150
ScQ
ScQ
2000, (2012)
20
ScQ
new to PISA
70
StQ
previous (2006) and extended
130*
StQ
changed (2012)
70
StQ
new to PISA
120
StQ
(2003)
(2006)
EDU/PISA/GB(2012)28
2
2
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
4
4
4
to learn
Teacher support
Perceived support for competence
Perceived support of autonomy
Disciplinary climate
Cognitive activation
Concepts of inquiry based science
Teacher perception
teaching
Teacher perception
Science teaching practices
School
learning
environment
for
science
(For other aspects of the school environment, see modules 8, 12, 13,
15, and-19)
Number of teachers per science
Science teaching staff
area
Policy and actions taken to improve Value of science education in
teaching/
school community
Policy and actions taken to improve
Science-related collaborations
teaching/
Policy and actions taken to improve
Science-specific resources
teaching/
Science-related outcomes: attitudes, motivation, value beliefs,
strategies
Motivation
Interest in school science subjects
Motivation
Interest in broad science topics
Anchoring Vignette Interest in
Motivation
Science
Motivation
Instrumental motivation
General and personal value of
Beliefs about science
science
Occupational prestige
Dealing with uncertainty and
ambiguity
Valuing scientific approaches to
enquiry
Students' view on science
Self-related cognitions
Self-Concept
Self-related cognitions
Self-Efficacy
Use of technology
Use of technology: frequency
Teaching quality
Teaching quality
Teaching quality
Teaching quality
Teaching quality
24
previous (2000, 2003, 2012)

new to PISA
new to PISA
previous (2000, 2003, 2009, 2012)
changed (2012)
50*
40
40
50*
70
StQ
StQ
StQ
StQ
StQ
new to PISA
160
TQ Science
new to PISA
120
TQ Science
new to PISA
120
ScQ
new to PISA
80
ScQ
new to PISA
40
ScQ
new to PISA
60
ScQ
changed (2006)
changed (2006)
40*
160*
StQ
StQ
60
StQ
changed (2006), previous (2012)
40
StQ
previous (2006)
100
StQ
new to PISA
80
StQ
new to PISA
100
StQ
new to PISA
80
StQ
new to PISA
previous (2006)
new to PISA
new to PISA
40
50
80
90
StQ
StQ
StQ
StQ
EDU/PISA/GB(2012)28
4
Use of technology
4
4
Use of technology
Use of technology
Environmental awareness
Out-of-school science experience

(see also module 12)
5
6
6

Science career
Career aspiration
6
6
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
Use of technology: perceived

competence
Use of technology: commitment
Use of technology: beliefs
Awareness of environmental
issues
new to PISA
50
StQ
new to PISA
new to PISA
100
100
StQ
StQ
previous (2006)
70*
StQ
Leisure time science activities

changed (2006)
media related
Extra-curricular science activities
previous (2006)
in school
Science activities at age 10
previous (2006)
60*
StQ
110
ScQ
50
PQ
previous (2006)
40*
StQ
previous (2006)
20
StQ
previous (2006)
40
StQ
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
PISA 2012
10*
10*
10*
10*
40*
40*
20*
20*
20*
40*
20*
20*
20*
30*
10*
30*
120*
50*
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
StQ
Broad science aspiration

Specific career aspiration at age
Career aspiration
30
Student information on science
Students own preparation
careers
Student SES, family and home background
Student background
Grade
Student background
Educational programme
Student background
Age
Student background
Sex
Family background
Household members
Parental background
Mother's profession
Parental background
Mother's school education
Parental background
Mother's tertiary education
Parental background
Mother's employment status
Parental background
Father's profession
Parental background
Father's school education
Parental background
Father's tertiary education
Parental background
Father's employment status
Migration background
Age at immigration
Language spoken at home
Family background
Home possessions - existence
Family background
Home possessions - number
25
EDU/PISA/GB(2012)28
7
7
7
7
7
7
7
7
7
7
Family background
Parental background
Parental background
Parental background
Parental background
Parents' background
Parents' background
Parents' background
ICT
ICT
ICT
ICT
8
8
Ethnicity and Migration

Ethnicity and Migration
Multicultural
school
culture/Concrete
policies
and
practices
Multicultural
school
culture/Concrete
policies
and
practices
Multicultural school culture
Perceived discrimination in society
8
8
8
8
8
8
8
8
8
8
9
9
9
9
9
Number of books
Number of children in household
Age of parents
Parents' qualification (ISCED)
Parental occupation
Spending on educational services
Household income
Home resources
Availability of ICT at home
General use of ICT
Use of ICT outside of school (for
leisure activities)
Use of ICT outside of school (for
school work activities)
PISA 2012
previous (2009)
previous (2006, 2012)
previous (2006, 2009, 2012)
previous (2006, 2012)
previous (2006, 2009, 2012)
previous (2006, 2009, 2012)
previous (2009)
previous (2009, 2012)
previous (2012)
20*
7
10
20
60
40
15
20
60
50
StQ
PQ
PQ
PQ
PQ
PQ
PQ
PQ
ICT
ICT
previous (2012)
60
ICT
previous (2012)
60
ICT
Cultural diversity of school
new to PISA
30
TQ General
Multicultural education
new to PISA
50
TQ General
Multicultural education
new to PISA
50
ScQ
new to PISA
new to PISA
new to PISA
new to PISA
80
80
60
20
ScQ
TQ General
StQ
StQ
new to PISA
30
StQ
previous (2012)
40
StQ
new to PISA
previous (2012)
previous (2012)
60
60
20
PQ
PQ
PQ
previous (2003), (2009), (2012)

previous (2003), (2009), (2012)
previous (2003), (2009), (2012)
new to PISA
10*
10*
30*
30
StQ
StQ
StQ
StQ
Basic assumptions on diversity

Basic assumptions on diversity
School intercultural climate
Membership in a minority group
Perceived
reasons
for
Perceived discrimination in society
discrimination
Language
preferences
for
communication
Parental involvement
Barriers to parental involvement
Parents migration background
Country of birth
Parents migration background
Citizenship
Educational pathways in early childhood (and thereafter)
Prior to school entry
ISCED 0 attendance
Education biography
School enrolment
After school entry
Repeating grades
After school entry
Skipping grades
26
EDU/PISA/GB(2012)28
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
10
10
10
10
10
10
10
10
10
10
10
10
10
10
School enrolment relative to

School enrolment
mandatory school starting age
School entry
Age at school entry
Prior to school entry
Educational biography
Education biography
ECEC attendance prior to ISCED
1
Education biography
Age at first ECEC attendance
Education biography
Type of predominant ECEC
arrangement
Education biography
ECEC dosage
Education biography
ECEC type of care arrangement
Education biography
ECEC reasons for attendance
Education biography
ECEC reasons for not attending
Education biography
Reasons for ECEC setting choice
Education biography
Change of school in ISCED 1
Education biography
Education biography
Change of study program in
Education biography
ISCED 2
Change of study program in
Education biography
ISCED 3
Domain-general student behaviour and attitudes
Student preferences and study skills Perseverance
Student preferences and study skills Industriousness
Student preferences and study skills Organization
Student preferences and study skills Procrastination
Student preferences and study skills Openness (for Problem Solving)
Anchoring Vignettes for student
Student preferences and study skills
preferences
Self-related beliefs
Test Anxiety
Strategic Approach to Learning
Academic Self Efficacy
Performance Self Efficacy
Anchoring vignettes for selfSelf-related beliefs
related scales
Grade Goal and current grades
Attitudes towards School Attitudes towards school
Learning Activities
27
new to PISA
30
PQ
new to PISA
new to PISA
10
70
PQ
PQ
new to PISA
10
PQ
new to PISA
10
PQ
new to PISA
30
PQ
new to PISA
new to PISA
new to PISA
new to PISA
new to PISA
previous (2003, 2009)
previous (2003, 2009)
previous (2003, 2009)
10
20
40
80
100
15
20
20
PQ
PQ
PQ
PQ
PQ
EC
EC
EC
previous (2003)
20
EC
previous (2003)
20
EC
previous (2012)
new to PISA
new to PISA
new to PISA
previous (2012)
30*
30
60
60
50*
StQ
StQ
StQ
StQ
StQ
new to PISA
90
StQ
new to PISA
new to PISA
previous (2012)
new to PISA
60
50
30*
30
StQ
StQ
StQ
StQ
new to PISA
120
StQ
new to PISA
15
StQ
previous (2012)
40
StQ
EDU/PISA/GB(2012)28
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
11
11
11
11
12
12
12
12
12
12
Attitudes towards School Learning Outcomes

well-being
General Life Evaluation
well-being
Life Satisfaction
well-being
Anchoring vignettes
Well-being at school/Sense of
well-being
belonging
Health
Over-/Underweight
Health
Physical Activities
Health
Nutrition
Health
Health-related school practices
time use
Sleeping Time
time use
Time spent at school
shortened day reconstruction time
time use
protocol
focused
on
study/learning-related activities
Idea generation
Consequences
School dropout
Number of dropouts
ICT Engagement
ICT Interest
ICT Engagement
Perceived ICT Competence
Perceived Autonomy related to
ICT Engagement
ICT Use
ICT as a topic in Social
ICT Engagement
Interaction
Student dispositions related to collaborative problem solving
Collaboration
and
teamwork Teamwork
dispositions:
dispositions
Cooperate, guide and negotiate
Experience related to collaborative In-school
experience
in
problem solving
collaboration-related activities
Experience related to collaborative Out-of-school
experience
in
problem solving
collaboration-related activities
Learning time and curriculum
Allocated learning time
Instructional weeks per year
Allocated learning time
Instructional days per year
Loss of learning time
Loss on national/regional level
Loss on school level
Additional
learning
time,
Provided additional learning time
availability
Attitudes towards school
28
previous (2012)
30*
StQ
new to PISA
new to PISA
new to PISA
60
50
60
StQ
StQ
StQ
previous (2012)
90*
StQ
new to PISA
new to PISA
new to PISA
new to PISA
new to PISA
new to PISA
30
40
60
120
20
20
StQ
StQ
StQ
StQ
StQ
StQ
new to PISA
200
StQ
new to PISA
previous (2012)
new to PISA
new to PISA
180
20*
120
120
StQ
ScQ
ICT
ICT
new to PISA
120
ICT
new to PISA
120
ICT
new to PISA
100
StQ
new to PISA
55
StQ
new to PISA
25
StQ
new to PISA
new to PISA
new to PISA
new to PISA
20
20
40
60
ScQ
ScQ
ScQ
ScQ
previous (2012)
10*
ScQ
EDU/PISA/GB(2012)28
12
12

12
School policies
12
School policies
12
School policies
12
Students learning time
12
12
12
12

12
12
12
Students additional learning time
12
Students additional learning time
12
Intended Science Curriculum
12
Intended Science Curriculum
12
12
12
12
12
12
12
12
Curriculum coherence
Textbook use
Realized curriculum
Curriculum choice
Additional learning time
Additional learning time, purpose

Additional other study support
School policies on additional
lessons
School policies on additional
tutoring lessons content
Handling of additional tutoring in
the school/school district
Number of school lessons in
maths, science, and test language
Learning time per science subject
All lessons per week
Minutes per lesson
Loss on classroom level
Loss on individual level in
regular school lessons: within
lessons, whole lessons, full
school days
Reasons for loss on individual
level
Amount of additional lessons and
study time
Loss on individual level in
regular school lessons: within
lessons, whole lessons, full
school days
Content: Components described
in the science curriculum
Content:
Approaches
and
processes
Transfer across classes and grades
Focus of curriculum content
Science courses taken
Choice in science courses
Number of hours
Content of additional lessons
Type of additional lessons
Location of additional lessons
29
previous (2012)
previous (2000)
30
20
ScQ
ScQ
new to PISA
10
ScQ
new to PISA
30
ScQ
new to PISA
80
ScQ
previous (2000-2012)
30*
StQ
new to PISA
previous (2012)
previous (2012)
new to PISA
100
10*
30*
10
StQ
StQ
StQ
StQ
new to PISA
60
StQ
new to PISA
10
StQ
previous (2000-2012)
180*
StQ
new to PISA
50
StQ
new to PISA
30
TQ Science
new to PISA
80
TQ Science
new to PISA
new to PISA
new to PISA
new to PISA
previous (2006)
new to PISA
new to PISA
new to PISA
30
20
50
20
180
10 each
60
50
TQ Science
TQ Science
StQ
StQ
EC
EC
EC
EC
EDU/PISA/GB(2012)28
12
Teacher in additional lessons

and Cognitive stimulation, adaptivity
of teaching
and Structuring content and classroom
management
and
Teacher-student relationship
new to PISA
40
EC
new to PISA
150
EC
new to PISA
100
EC
new to PISA
50
EC
Reasons for additional lessons

Reasons for no additional lessons
History of private supplementary
Education biography
tutoring
School climate: Interpersonal relations, trust, expectations
(see also modules 2 for classroom climate and 14 for parental
involvement)
Teachers and students
Student teacher relation at school
Achievement pressure by teachers
Students
Bullying
Teacher and students
Fairness
Staff
Teacher morale
Teacher and student behaviour
Staff/Students
affecting school climate
new to PISA
new to PISA
80
100
EC
EC
new to PISA
30
EC
previous (2000 ff.) with new items

previous (2000)
new to PISA
new to PISA
previous (2000, 2003, 2012)
115*
40
70
60
40*
StQ
StQ
StQ
StQ
ScQ
previous (2012)
190*
ScQ
13
Student-teacher relation at school
adapted from previous StQ
90
TQ
General
14
14
Parental involvement
Parental background
previous (2006, 2009, 2012)
PQ
14
Parents' view on science
previous (2006, 2012)
35
PQ
14
Parents' view on science
changed (2012)
60
StQ
14
School choice
previous (2006, 2009, 2012)
45
PQ
14
Home-to-school-collaboration
previous (2009, 2012) and new to

80
PISA
PQ
14
new to PISA
60
PQ
14
new to PISA
25
PQ
14
Respondent
Parents' view on science skills in
the job market
Parent and peer influence in
student's view of science
Perceived school quality
Parental engagement that focuses
on
interaction
with
the
institutional learning setting
Topics
of
parent-teacher
communication
Modes
of
parent-teacher
communication
Parents' satisfaction with school
previous (2009)
120
PQ
12
12
12
12
12
12
13
13
13
13
13
13
13

Comparing regular school
additional lessons
additional lessons
additional lessons
30
EDU/PISA/GB(2012)28
14
14
14
14
14
14
14
14
Parental support
14
Parental support
14
Parental support
14
Parental support
14
Parental support
14
Parental support
15
15
15
15
Parents'motivation
for
participation
Parents' social relationship and
involvement
Parental engagement that focuses
on
interaction
with
the
institutional learning setting
Parental expectations towards
school
School policies for parental
involvement
Topics
of
parent-teacher
communication
Modes
of
parent-teacher
communication
Emotional warmth, guidance and
boundaries,
stimulation/scaffolding
and
stability
Parental support for learning at
home
Child's past science activities and
parental support for learning at
home
Emotional warmth, guidance and
boundaries,
stimulation/scaffolding
and
stability
Help
with
homework
or
additional study at home
Parents view on science careers
new to PISA
45
PQ
new to PISA
25
PQ
previous (2012)
80
ScQ
previous (2006, 2009, 2012)
30
ScQ
new to PISA
70
ScQ
new to PISA
60
TQ General
new to PISA
25
TQ General
new to PISA
140
PQ
previous (2009, 2012) focussing on

60
reading and mathematics
PQ
previous (2006)
35
PQ
new to PISA
110
StQ
new to PISA
80
EC
35
EC
50
135*
30
ScQ
ScQ
ScQ
previous
(2006,
mathematics)
Leadership
and
school
management
(For specific aspects of school management, see also modules 3, 8,
12, and 16-19)
Instructional leadership
Leadership tasks
new to PISA
School management
Educational leadership scale
2009(14), 2012(21)
School policies and practices
Curriculum
2012(3)
31
2012
for
EDU/PISA/GB(2012)28
15
15
15
16
16
16
16
16
Use of ICT Resources

ICT usage
Transformational leadership
Teachers' view on leadership
Instructional leadership
Teachers' view on leadership
Resources
(see also module 3 for science-specific resources)
Structure and organisation of the
Public or private school
school
Structure and organisation of the Type
of
school
location
school
community
Number of students
school
Source of resources
school
Teaching staff
Vacant positions
Class size
Average size of classes
ICT
ICT-Equipment
Resources
Problems due to little resources
ICT
Availability of ICT at school
16
ICT
17
Locus of decision making within the school system

Decision making within the school
Locus of decision making
system
Admission, selection, and choice
Grade retention
Policies on grade retention
Grouping
Grouping policies
Choice
Competition between schools
Choice
Parental school choice
Choice
Availability of schools
Admission
Student admission policies
Assessment, evaluation, and accountability
Student assessment/Instruments
General assessment practice
Classroom
assessment
instruments
Teachers' grading practice
Student assessment/Use
Adaption of instruction
Adaption on instruction
Perceived feedback
16
16
16
16
16
17
18
18
18
18
18
18
19
19
19
19
19
19
19
19
Use of ICT at school
32
previous (2012)
new to PISA
new to PISA
25*
70
110
2000(1), 2003(1), 2006(1), 2009(1),

10*
2012(1)
2000(1), 2003(1), 2006(1), 2009(1),
15*
2012(1)
2000(2), 2003(2), 2006 (2), 2009
15*
(2), 2012(2
ScQ
TQ General
TQ General
ScQ
ScQ
ScQ
40*
ScQ
previous (2006)
30
2006(1), 2012(1)
11*
previous (2012)
30*
previous (2012)
130
previous (2009, 2012)
60
previous (2009, 2012) and new to
90
PISA
ScQ
ScQ
ScQ
ScQ
ICT
previous (2006, 2009, 2012)
120*
ScQ
new to PISA
previous (2006)
previous (2012)
previous (2009), previous (2012)
previous (2009, 2012)
previous (2000- 2012)
100
20
30*
100
25
60*
ScQ
ScQ
ScQ
PQ
PQ
ScQ
previous (2000, 2003, 2009, 2012)
60*
ScQ
new to PISA
80
TQ General
new to PISA
new to PISA
new to PISA
new to PISA
170
30
30
50
TQ General
StQ
TQ Science
StQ
previous (2006), previous (2012)
ICT
EDU/PISA/GB(2012)28
19
19
19
19
19
19
19
19
19
19
19
School evaluation/Norms
School evaluation/Norms
School evaluation/Instruments
School evaluation/Use
19
19
19
Accountability
19
19
*)
Accountability
Source of feedback
Use of feedback to guide learning
Purpose of assessment results
Evaluation policies
Policies on teacher accountability
Foci of internal evaluation
Foci of internal evaluation
Teacher evaluation
Teacher evaluation
Processes on internal evaluation
Processes on internal evaluation
Consequences
of
internal
evaluation
Consequences
of
internal
evaluation
Use of achievement data for
accountability
Teacher incentives
Processes on external evaluation
Hypothetical selection of material that might be taken up from previous cycles
33
new to PISA
new to PISA
previous (2000, 2003, 2009, 2012)
previous (2012)
new to PISA
new to PISA
new to PISA
previous (2000, 2003, 2009, 2012)
new to PISA
new to PISA
new to PISA
90
60
100*
100*
30
130
130
40*
40
70
70
StQ
StQ
ScQ
ScQ
ScQ
ScQ
TQ General
ScQ
TQ General
ScQ
TQ General
new to PISA
130
ScQ
new to PISA
130
TQ General
previous (2006, 2009, 2012)
30*
ScQ
previous (2012)
new to PISA
70
50
TQ General
ScQ

Bilag 13 GB 28 Progress On Questionnaire Development

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bilag 13 GB 28 Progress On Questionnaire Development

Uploaded by

Copyright:

Available Formats

For Official Use

Organisation de Coopration et de Dveloppement conomiques

PROGRAMME FOR INTERNATIONAL STUDENT ASSESSMENT

PROGRESS REPORT ON QUESTIONNAIRE DEVELOPMENT AND DESIGN

29-31 October 2012

Andreas Schleicher; E-mail: andreas.SCHLEICHER@oecd.org; Tel: +33.1.45.24.93.66

English - Or. English

Conceptual background ........................................................................................................................3

Questionnaire content used in..................................................................................................5

Modular structure of the PISA 2015 context assessment design .............................................7

Describes the policy-driven overarching conceptual framework that drives questionnaire

The PGB is invited to:

In particular, the PGB is asked to advise on:

The criteria set out in paragraph 24 for content reduction.

Volume I: Student Performance in Reading, Mathematics and Science:

Teacher education and professional knowledge

School environment for science learning

Science-related non-cognitive outcomes

Out-of-school science learning experiences

Student SES, family and home background

Ethnicity and migration

Educational pathways in early childhood education

10 General non-cognitive outcomes

Short questions will be preferred to longer questions.

Questionnaire Development from April to September, 2012

Questionnaire design and rotation in the Main Study StQ

3.1 Designing the PISA 2015 Field Trial

allowing for broad coverage of topics in all 19 modules;

ensuring feasibility of computer-based delivery for all previously established material.

APPENDIX: DRAFT OVERVIEW OF FIELD TRIAL CONTENT

Module Policy issue and construct area

Teacher qualification and professional knowledge

Status of the measure

new to PISA, from TALIS 2013

new to PISA, from TALIS 2013

previous (2006) and extended

previous (2000, 2003, 2012)

changed (2006), previous (2012)

Out-of-school science experience

Out-of-school science experience

Out-of-school science experience

Out-of-school science experience

Use of technology: perceived

Leisure time science activities

Broad science aspiration

Ethnicity and Migration

Cultural diversity of school

previous (2003), (2009), (2012)

Basic assumptions on diversity

School enrolment relative to

Attitudes towards School Learning Outcomes

Provided additional learning time

Students learning time

Students learning time

Loss of learning time

Loss of learning time

Students additional learning time

Students additional learning time

Intended Science Curriculum

Intended Science Curriculum

Additional learning time, purpose

Teacher in additional lessons