Executive Summary

October 2010

Innovative Teaching and Learning Research

ITL Research: Executive Summary of Pilot Year Findings

Linda Shear, Gabriel Novais, and Savitha Moorthy, SRI International
Project Sponsor: James Bernard, Director, Partners in Learning Program Director: Maria Langworthy, Langworthy Research CONTRIBUTING RESEARCH PARTNERS: Agora Center, University of Jyvskyl, Finland Centre for Strategic and International Studies, Indonesia Institute of New Technologies, Russia Association of Teachers and Researchers of ICT in Education and Training, Senegal SRI RESEARCH TEAM: Barbara Means Kea Anderson Gucci Estrella Larry Gallagher Amy Hafter Corinne Singleton Yukie Toyama

Around the world, there is growing consensus among education leaders, researchers and educators
that teaching and learning must change to help students develop the skills they will need to succeed in the 21st century (Ananiadou & Claro, 2009; Partnership for 21st Century Skills, 2004; Scheuermann and Pedr, 2009).
While specic goals for change vary, common themes
include developing problem-solving and teamwork skills, and using technology to support more powerful learning. Although there are inspiring examples of innovative teaching promoting this kind of learning1, research continues to show that in most places classroom practice lags behind goals (e.g., OECD, 2009; Law, Pelgrum, & Plomp, 2010). The sources of this gap between the rhetoric of change and the realities of classrooms range from lack of access to resources and training to lack of clear expectations in systems that are still organized and incented toward traditional measures of achievement. Most students still experience instruction that is largely lecture-based, and extensive national education investments in technology have not yet resulted in widespread transformation of learning opportunities. about the distance between teaching and learning visions and practiceand what to do about it. ITL Research, sponsored by Microsofts Partners in Learning, investigates innovative teaching practices, the conditions that enable teachers to teach in new ways, and the resulting connection with students 21st century skills. As the program progresses, these methods will be adapted into tools and processes that educators can use to examine, discuss, and ultimately improve the educational opportunities they provide to students.

This report summarizes results from the pilot year of

ITL Research (2009-10), with data from four participating countries: Finland, Indonesia, Russia, and Senegal. The goal of a program pilot is to test and tune instruments and methods, so the ndings reported here should be considered preliminary and in need of conrmation through further research in subsequent years of this program. However, these results do raise issues and suggest important considerations regarding the conditions that support innovative teaching and learning.

In a growing number of countries, the Innovative Teaching

and Learning (ITL) Research program is fueling inquiry and discussion among policymakers, educators, and researchers

In a classroom in Russia students are conducting a 2-month investigation into climate factors such as temperature and humidity in their classroom, and devising plans to use live plants to improve the air quality in their school.
1 See for example http://www.microsoft.com/education/pil/IT_home.aspx

About ITL Research

ITL Research seeks to build an understanding of the conditions that support innovative teaching practice supported by effective use of ICT.2 The research design looks across the complex ecosystem of inuences that shape teaching and learning within national, school, and classroom contexts. While the conceptual framework below does not claim to be comprehensive, it does emphasize a range of factors that extensive prior research has shown to be linked to teaching practices and deep student learning.3 Methods and measures in ITL Research build on and are guided by these and many other leading studies and frameworks, and are described in greater detail in Shear, Means, Gallagher, House, & Langworthy (2009). In this model, innovative teaching practices encompass both pedagogy and technology. While ICT is an important focus of this research, its use is not seen as an end in itself. Rather, appropriate use of ICT is considered an important enabler of a student-centered learning environment that helps students build both deep subject matter knowledge and skills such as collaboration, communication, problem-solving, and self-regulation that they will need to succeed in the 21st century. See Shear et al., 2009 for more detailed denitions of the constructs central to ITL Research (available at www.itlresearch.com).

ITL RESEARCH MODEL

CONTEXT & INPUTS National School & Teacher School Culture and Supports ICT Access and Supports Educator Attitudes

PRACTICES Classroom

OUTCOMES Student

Education Policy Program Professional Supports Development

Innovative Teaching Practices

Students 21st Century Skills

ITL Research uses a distributed design to carry out research that is at once global and local in scope. SRI International is the global research organization responsible for design, coordination, and results synthesis, ensuring that overall design parameters and instruments are developed centrally and implemented consistently across countries. A research partner in each country manages local design, data collection, analysis and the sharing of ndings with local and regional stakeholders. ITL Research is grounded in partnerships with government and key organizations that inuence education policy in each country, including their nancial sponsorship of the research in many countries. Finally, ITL Research is guided by an advisory board of international experts (see http://www. itlresearch.com/ for a full list of all partners).

Student-Centered Pedagogy Extending learning beyond the classroom ICT used for teaching and learning
2 In this research, ICT (Information and Communication Technologies) is dened broadly to include not only computers and the Internet but also smart phones, electronic whiteboards, and other hardware and software tools. 3 See for example Law, Pelgrum, & Plomp, 2010; Bransford, Brown, & Cocking, 1999; DarlingHammond et al., 2008.

How do we measure innovative teaching?

ITL Research is a mixed-methods study that uses several distinct lenses into classroom practice, all based on a common framework. The major constructs that dene innovative teaching on this projectstudent-centered pedagogies, extension of learning beyond the classroom, and ICT integration into teaching and learningare translated into measurable indicators for data collection via survey instruments, classroom observation protocols, and rubrics to characterize teaching and learning. Although these data sources differ in focus, each allows us to construct a comparable, aggregate measure for innovative teaching practices.

ITL RESEARCH SAMPLE

METHOD ACTUAL 4-COUNTRY SAMPLE Teacher and school leader surveys

Survey Schools Teacher survey School leader survey Site Visit/Artifact Collection Schools Teacher interviews Teacher observations School leader interviews Learning activities Student work

91 schools 2,406 teacher surveys 82 respondents 24 schools 96 interviewees 96 classroom observed 24 interviewees 650 samples 3,647 samples

provide self-reported data across a wide range of participants. In the pilot year, we surveyed approximately 600 teachers and their school leaders in each country, representing a total sample of 2406 teachers and 82 school leaders. Teachers responded to questions about their teaching practices, the resources that were available to them, and their professional development experiences. By combining their responses to questions on the frequency with which they engage in a variety of teaching practices, we get a measure of what teachers say that they and their students do over the course of a school year. In the results that follow, this measure is reported as the innovative teaching practices index.

A more specic and objective lens on classroom practice comes from the analysis of learning activities and student work,
as illustrated in the following sample [page 4] from a classroom in Russia. Samples of learning activities and student work were collected from 48 teachers per country. Learning activities are the assignments that teachers ask students to complete, and can include work done in or out of class, activities that last one class period or two months, basic worksheets or complicated projects. Researchers also collected samples of the work that students did in response to these learning activities, which include essays, worksheets, presentations, or other student products. By recruiting and training experienced educators to code samples of learning activities and student work on common rubrics that describe specic dimensions of 21st century skills, we get a measure of the extent to which students have the opportunity to acquire these skills and the extent to which they are demonstrating those skills in the work they do. The denitions and rubrics that operationalize the concepts of 21st century teaching and learning were developed by SRI, building on prior research (Bryk, Nagaoka, & Newmann, 2000; Matsumura & Pascal, 2003; Mitchell, Shkolnik, Song, Uekawa, Murphy, Garet, & Means, 2005; Shear, Means, Gorges, Toyama, Gallagher, Estrella, & Lundh, 2009).

A third measure of teaching practice comes from classroom observations, which were conducted by
researchers during an on-site visit with a subset of the teachers who submitted examples of learning activities and student work. This data source, along with accompanying interviews, provides a more thorough understanding of teaching and learning in selected settings within each country. In 2010-11, case studies of selected schools will deepen this local perspective.

All ITL Research tools are available to the public at http:// www.itlresearch.com. These tools are updated annually for the three-year duration of this project. The project team is in the process of adapting these methods to support use by educators as well as researchers, with the goal of offering tools and processes that educators can use to analyze and discuss classroom practice as well as to plan learning activities that offer improved opportunities for students to build 21st century skills.

LEARNING ACTIVITY & STUDENT WORK SAMPLES CODING A LEARNING ACTIVITY FOR EVIDENCE OF COLLABORATION:

LEARNING ACTIVITY AND STUDENT WORK ORK CODING DIMENSIONS:

Learning Activity Dimensions
Knowledge building Problem-solving and innovation ICT integration into teaching and learning Collaboration Self-regulation and assessment f

Student Work Dimensions

Knowledge building Problem-solving and innovation ICT integration into teaching and learning Skilled communication

EXAMPLE LEARNING ACTIVITY AND STUDENT WORK RESPONSE FROM RUSSIA:

Essay: Why should we remember World War II?
1) 2) Remember what we have learned about writing an essay. Ask your relatives to tell you about the beginning of the war. Where were they during the War? How did they learn about its end? Transcribe the interview on a computer. Use the internet to gather materials about the places your relatives lived during the War, inclusing photos of that period.

3)

4) Write an essay for the school newspaper devoted to World War II.

Student B, Age 14

Preliminary ndings from the pilot year

This section reports preliminary ndings from analysis of surveys, learning activities and student work, interviews and classroom observations. It is important to recognize that these ndings only show associations between measures; they do not demonstrate that a given variable causes an outcome, and may not capture important explanatory factors.
Preliminary ndings: Innovative teaching and student learning of 21C skills4
The quality of a teachers assignment strongly predicts the quality of the work that a student does in response. Over 90% of the variance in student work scores on 21st century skills was due not to differences in the students but differences in the tasks they were asked to do. The data suggest a ceiling effect imposed by teacher assignments: while it is possible for students to build and exhibit a greater level of 21st century skills than their learning activities call for, they rarely do so. While innovative teaching practice was typically a goal at these schools, learning activity analysis suggests that most actual classroom instruction does not yet reect these goals.

In a classroom in Senegal students are able to see dynamic visualizations of cell division because their teacher, whose classroom is not equipped with technology, has brought in his own laptop computer so that he can enrich his teaching.
4Findings in this section are based on analysis of learning activities and student work.

 The quality of a teachers assignment strongly predicts the quality of the work that a student does in response.
On dimensions captured by both learning activities and student work, over two-thirds of student work products received the same score as their corresponding learning activity.5 Across all dimensions, mean scores for a given learning activity strongly predicted aggregate scores for associated student work. The results corroborate other ndings (see for example, Bryk, Nagaoka, & Newmann, 2000; Mitchell et al., 2005) that highlight the importance of giving students challenging learning activities if we want them to do high-level work.

LEARNING ACTIVITY SCORES AND STUDENT WORK SCORES

Over 90% of the variance in student work scores on 21st century skills was due not to differences in the students but differences in the tasks they were asked to do.
More often than not (55%), the six different student work products rated for a given learning activity all received identical scores on every dimension of 21st century skills. For any given dimension, over 70% of student work sets had no variation in scores among students.

The data suggest a ceiling effect imposed by teacher assignments: while it is possible for students to build and exhibit a greater level of 21st century skills than their learning activities call for, they rarely do so.
Very few pieces of student work scored higher than their associated learning activity on any given dimension. These data further suggest the importance of setting the bar high when designing learning activities to elicit 21st skill development from students.

While innovative teaching practice was typically a goal at these schools, learning activity analysis suggests that most actual classroom instruction does not yet reect these goals.
Overall, on every dimension, over half of all assignments received the lowest possible score. On average, scores were slightly higher on learning activities for younger students, which may reect the fact that teachers of older students may be more focused on preparing students with factual knowledge for exams.

5In reality, the rubrics for learning activities and student work are not exactly parallel, but a 2 on knowledge building for learning activities generally represents the same degree of complexity as a 2 on knowledge building for student work.

Preliminary ndings: Drivers of innovative teaching6

In this research, variables that associated signicantly with teacher-reported innovative teaching practices include: > Teacher participation in high-touch, ongoing types of professional development, such as research and networks of educators > Teacher collaboration and peer support > Incentives and recognition for new teaching practices, as reported by school leaders > Subject matter taught Professional development courses which emphasize ICT integration into instruction associate more strongly with teacher-reported ICT use than those which emphasize technical skills. By contrast, teachers report that the majority of the professional development they are offered focuses on ICT skills rather than ICT integration. While research suggests that teacher beliefs are an important driver of teaching practices, surveyed teachers often reported both constructivist beliefs and beliefs about the value of direct transmission.

PROFESSIONAL DEVELOPMENT ACTIVITIES & INNOVATIVE TEACHING PRACTICES

0.70 0.60 0.50 0.40 0.30 0.20 0.10 0.00
Research Teacher network for PD Formal mentoring Qualification Conference, Classroom program seminar observations Informal dialogue Class or workshop

Mean difference, innovative teaching practices*

0.59 0.50 0.43 0.42 0.39 0.39 0.30

0.26

In this study, teachers who participated in professional development had higher scores on the innovative teaching practices index than those who did not.
More specically, professional development activities with the strongest relationships to innovative teaching practices included individual or collaborative research and participation in a teacher network for professional development. Classes/workshops and informal dialogue showed weaker relationships to innovative teaching practices overall.
6Findings in this section are based on analysis of teacher and school leader surveys.

Peers can be an important source of support for teachers. Teachers who reported collaborating frequently with other teachers also reported more innovative teaching practices, on average. For example,
55% of teachers who scored in the upper third on the innovative teaching practices index discussed student work with other teachers at least once a week. By contrast, only 24% of teachers in the lower third on innovative teaching practices did so. In interviews, teachers commonly credited their colleagues with offering new ideas, teaching advice, and technological support, inspiring newfound energy from collaboration among peers.

INCENTIVES AND RECOGNITION FOR INNOVATIVE TEACHING

When I meet with other colleagues, we exchange experiences. For exampleif my teaching method for a certain teaching material is not appropriate, I might use my colleagues ways. Teacher in Indonesia I have autonomy but try to seek advice from more experienced teachers. Teacher in Russia My school welcomes our creative thinking and helps me teach as I feel best for my students. Teacher in Russia

In schools where school leaders report making efforts to incentivize innovative teaching, teachers scored higher on the innovative teaching practices index than those in schools with less supportive leadership. Among the school-level practices reported on by school leaders, the incentives and recognition scale associated most signicantly and most consistently across countries with teacher-reported innovative teaching practices.

Teachers of reading, writing, and social studies classes typically scored higher than teachers of other subjects on measures of innovative teaching. Math teachers had the lowest scores, echoing previous research (e.g., OECD, 2009; AIR/
SRI, 2006). Subject matter differences are likely due, at least in part, to varying levels of curriculum rigidity across subjects.

Professional development courses that emphasize ICT integration into instruction associate more strongly with teacher-reported ICT use than those that emphasize technical skills. On average, teachers who received training
focused on ICT integration scored signicantly above the mean on ICT use, while teachers who received ICT skills training scored only slightly above the mean. 7

By contrast, teachers report that the majority of the professional development they are offered focuses on ICT skills rather than ICT integration. In all four countries, more surveyed teachers reported receiving mainly technicallyfocused ICT training as opposed to training on ICT integration. Interviewed teachers also reported that ICT-related professional development is overly focused on technical skills.

ICT-FOCUSED PROFESSIONAL DEVELOPMENT AND ICT USE

While research suggests that teacher beliefs are an important driver of teaching practices, surveyed teachers often reported both constructivist beliefs and beliefs about the value of direct transmission. In 3 of 4 countries,
teachers constructivist beliefs coexist with direct transmission beliefs. In these countries, over 85% of teachers agreed that instruction should be built around problems with clear, correct answers. But over 80% also agreed with the seemingly contradictory statement that thinking and reasoning processes are more important than specic curriculum content. Differences in national culture, pedagogical traditions, and student examination systems might lie behind differences in patterns across countries. These ndings are consistent with OECDs TALIS study (OECD, 2009).

The best way for me to serve my students is to cover the curriculum by the end of the school year so that no student or parent could accuse me of not tackling a chapter or topic that has been chosen to be tested in one of the national exams. Teacher in Senegal
7 Analysis methods adjusted for the fact that many teachers participated in both forms of professional development.

Preliminary ndings: ICT Use8

Both teacher and student ICT use are signicantly associated with other innovative teaching practices (student centered pedagogy and extending learning beyond the classroom). In classroom observations, higher-level uses of ICT (i.e., uses that represent stronger integration with deep teaching and learning) associate more strongly with innovative teaching than do more basic uses of ICT. In observed classes and among surveyed teachers, basic uses of ICT remain the most frequent. ICT access in the classroom is a stronger predictor of teachers ICT use than is ICT access elsewhere in the school. Access to ICT is a necessary, but not sufcient, condition for ICT use. The most common teacher-reported barriers to ICT use are lack of computers for student and teacher use, and lack of technical and pedagogical support.

With the use of the Internet, I can see that the students are now even smarter than their teacher. The information they got was very rich. Teacher in Indonesia [When they began using ICT] teachers became aware that memorizing and recalling information are losing their previous value because information is easily accessible through ICT. Therefore, it is now more useful not just to memorize facts, but to work on development of such skills as analysis, systematization, storing, and applying of this very information. Education leader in Russia

BASIC AND HIGH-LEVEL ICT USE

EXAMPLES OF BASIC STUDENT ICT USE Examples of basic student ICT use
Take quizzes or tests in a multiple choice or short answer format Find information on the Internet or other electronic resources Write or edit stories / reports / essays using word processing Practice skills or procedures Turn in homework assignments or other class work

EXAMPLES OF HIGH-LEVEL use Examples of high-level student ICTSTUDENT

Take quizzes or tests in an innovative format Analyze or synthesize data or information Create presentations or other multimedia products

ICT

Use simulations or animations to explore a system or abstract concept Communicate with people from outside the school about school topics

Both teacher and student ICT use are signicantly associated with other innovative teaching practices. ICT integration into teaching and learning
correlates signicantly with student-centered pedagogies and extension of learning beyond the classroomthe studys other elements of innovative teaching. Over the course of a year, high-level and low-level ICT use tended to co-occur.

In classroom observations, higher-level uses of ICT (i.e., uses that represent stronger integration with deep teaching and learning) associate more strongly with innovative teaching than do more basic uses of ICT. Classrooms featuring at least some higher-level student ICT
use scored higher on innovative teaching practices, on average, than either classrooms featuring no student ICT use or classrooms featuring basic use only. This result echoes ndings from other research (e.g., Shear et al., 2009).

8 Findings in this section are based on teacher surveys and classroom observations.

10

In observed classes and among surveyed teachers, basic uses of ICT remain the most frequent. Students used ICT
in primarily basic ways in nearly three-fourths (72%) of classes observed across countries. Likewise, over the course of a year, teachers most commonly use ICT primarily for planning and administrative functions, rather than for pedagogical purposes. While basic uses of ICT occur with different frequencies across country samples, higher-level uses are almost equally infrequent in all samples.

ICT access in the classroom is a stronger predictor of teachers ICT use than is ICT access elsewhere in the school.
The data suggest that having ICT in the school building is not sufcient to promote its use to support teaching and learning. 39% of teachers with above-average ICT integration scores had desktop computers in their classrooms, compared to only 20% of teachers with below-average integration scores. Both groups of teachers, however, had high levels of school ICT access74% of above average teachers and 73% of below average teachers reported access to computers elsewhere in the school. Easy access to computers within the classroom, rather than the disruption of scheduling and moving to a shared computer lab, reduces barriers to ICT integration in teaching and learning.

ICT ACCESS AND USE

Access to ICT is a necessary, but not sufcient, condition for ICT use. While on
average, teachers with more access to ICT also had higher levels of ICT integration, there were many examples of teachers who had classroom ICT access to computers but did not use it actively in their teaching. In other words, teachers with the same or similar levels of ICT access used ICT in their teaching to varying degrees.

The most common teacher-reported barriers to ICT use are lack of computers for student and teacher use, and lack of technical and pedagogical support. Although teachers report similar barriers in all countries, the magnitude
varies depending on the availability of resources and histories of ICT use in individual countries. In some countries, barriers related to security and hardware reliability are critical factors.

The amount of devices is not a problem. The problem is the quality of using them. School leader in Finland Although we have a well-equipped computer room, we can hardly use it because of too frequent power outages. Teacher in Senegal

11

Reections and Implications

This report has summarized the main results from the 2009-10 pilot year of ITL Research. Because this was a pilot year, it is important to consider these results suggestive rather than nal. However, a number of reections are appropriate:
While most countries have clear goals to prepare students for the 21st century, specic denitions and guidance to educators are often lacking.9 The denitions, examples and tools offered by ITL Research can help educators on a productive path to change. The characteristics of students learning environments and the activities they are asked to do in the classroom strongly predict the work they will do in response. Reform efforts should consider carefully how they are encouraging and supporting teachers to teach in new ways, and in particular to choose student learning activities that provide opportunities to develop and exercise 21st century skills. Access to ICT is necessary but not sufcient to enable its use. School administrators and policymakers should be aware that ICT access within classrooms associates more strongly with integration into instruction than does access in computer labs or other shared spaces. However, drivers of ICT use are more strongly associated with individual teachers than with the environment: some teachers in low-access environments nd creative ways to gain access to tools, while ICT in some high-access classrooms goes unused. In ITL Research, types of professional development that associate most strongly with teacher-reported innovative teaching practices include professional development related to pedagogy and ICT integration rather than ICT skills, and professional development that is deeper and more ongoing than a single workshop or training (for example, teacher engagement in research on teaching and learning or teacher engagement in collaborative networks with other teachers in their school). Policymakers should be cautious about drawing causal implications from these results; it could be that teachers who already think more innovatively about their practices are more likely to engage in these forms of professional development. However, it should be noted that more common forms of professional development, such as single workshops, have a weaker association with innovative teaching practices, and teachers regard ongoing interactions and follow-up as essential (and often lacking) supports for helping them integrate ideas into practice. Use of ICT by teachers and students is signicantly associated with innovative teaching practices more generally. This association is stronger when ICT is used in higher-level ways that promote deeper student engagement with content. More commonly among teachers in the ITL Research sample, ICT is used in more basic ways, to access content (for students) and as a presentation tool (for teachers and students). In some countries, these uses may represent important steps toward innovation. Nevertheless, models and tools for using ICT in ways that powerfully promote deep student learning can help prepare teachers for taking the next step. Among schools visited in the pilot year of ITL Research in all participating countries, researchers found examples of teachers that are experimenting with at least some elements of innovative teaching practice, including more studentcentered approaches to instruction, the use of ICT to support student learning, or a combination. Much less common were teachers who beneted from a coherent set of supports at the school and national levels, including clearly-dened goals for innovative teaching and learning, effective and ongoing professional development experiences that go beyond ICT skills to integration and pedagogy, and a policy context in which teacher incentives and accountability systems are aligned with goals for innovative instruction. For investments in ICT and other tools for 21st century learning to bear fruit, education leaders and policymakers must be attentive to the integrity of the ecosystem within which educators operate, and offer programs and incentives at multiple levels to provide a fertile environment for change.

This nding echoes survey results reported in Ananiadou and Claro, 2009.

12

Next Steps
ITL Research is a multi-year program that is continuing to grow beyond the pilot year. Four additional countries have joined the research for the following two years, with participation sponsored by local policy partners.10 As the program proceeds, it will focus progressively on using the research tools and results from analysis to create hands-on supports and tools for schools. It is the programs aim that these tools will enable educators in partner schools to better envision, investigate, and improve innovative learning opportunities for students. ITL Research is also offering research instruments, methods and data to the public domain, and convening researchers around the world who seek to improve the study of 21st century teaching and learning.

We invite you to join this expanding circle of research and practice in innovative teaching and learning. To learn more, please visit www.itlresearch.com

References
AIR/SRI. (2006). Evaluation of the Bill & Melinda Gates Foundations High School Grants Initiative: 2001-2005 nal report. Washington, D.C.: American Institutes for Research. Ananiadou, K. & Claro, M. (2009). 21st Century Skills and competences for New Millennium learners in OECD countries. Organization for Economic Cooperation and Development. EDUWorking paper no. 41. Retrieved from http://www.olis.oecd.org/olis/2009doc.nsf/linkto/edu-wkp(2009)20 Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How People Learn: Brain, Mind, and Experience. Washington, D.C.: National Academy Press. Bryk, A. S., Nagaoka, J. K., & Newmann, F. M. (2000). Chicago classroom demands for authentic intellectual work: Trends from 19971999. Chicago, IL: Consortium on Chicago School Research. Darling-Hammond, L., Barron, B., Pearson, P. D., Schoenfeld, A., Stage, E, Zimmerman, T, Cervetti, G, & Tilson, J. (2008). Powerful learning: What we know about teaching for understanding. San Francisco, CA: Jossey-Bass. Law, N., Pellgrum, W., & Plomp, T. (2010). Pedagogy and ICT use in schools around the world: Findings from the IEA SITES 2006 study. Hong Kong: Comparative Education Research Centre. Matsumura, L. C., & Pascal, J. (2003). Teachers assignments and student work: Opening a window on classroom practice. Los Angeles: CRESST/University of California. Mitchell, K., Shkolnik, J., Song, M., Uekawa, K., Murphy, R., Garet, M., & Means, B. (2005). Rigor, relevance, and results: The quality of teacher assignments and student work in new and conventional high schools. Washington, DC: American Institutes for Research and SRI International. OECD (2009). Creating effective teaching and learning environments: First results from TALIS. Paris: OECD. Partnership for 21st Century Skills (2004). Framework for 21st century learning. Retrieved July 15, 2009 from http://www.21stcenturyskills.org/. Scheuermann, F. and Pedr, F., Editors. (2009). Assessing the effects of ICT in Education: Indicators, criteria and benchmarks for international comparisons. JRC/European Commission and OECD. Shear, L., Means, B., Gorges, T., Toyama, Y., Gallagher, L., Estrella, G., & Lundh, P. (2009). The Microsoft Innovative Schools Program Year 1 evaluation report. Seattle: Microsoft. Shear, L., Means. B., Gallagher, L., House, A., & Langworthy, M. (2009). ITL Research Design. Menlo Park, CA: SRI.

Academies Trust; United States, sponsored by the Stupski Foundation and the Council of Chief State School Ofcers (Next Generation Learning Project); and Australia, sponsored by New South Wales Department of Education & Training.

10New countries joining ITL Research in the 2010-11 school year are: Mexico, sponsored by the National Ministry of Education; England, sponsored by the Specialist Schools and

13

Visit www.microsoft.com/education/partnersinlearning for more research and information