Preventing Shool Failure in Disadvantaged Communities

Evaluation of the New Approaches, New Solutions Intervention Strategy (NANS) 2002-2008 Overview
Michel Janosz, Ph.D. Groupe de recherche sur les Cliquez pour modifier le style des sous-titres du environnements / Research Group on masque School Enviornments (GRES) Universit de Montral

Janosz 2010

Janosz 2010

Evaluation Team
Under the leadership of Michel Janosz (UdeM)

Implementation

Effects

Jean Blanger UQAM Franois Bowen, UdeM Christian Dagenais UdeM

Phil Abrami Concordia Sylvie Cartier UdeM Roch Chouinard UdeM Nadia Desbiens UdeM Jean-Sbastien Fallu UdeM

Gilles Roy, Louis Turcotte ve Paquette, Clia Kremer Frdric Brire, Alexandre Chabot, Didier Dupont, Julie Dutil

co-chercheurs entre 2002-05

(2002-2004; U. Laval) 2004; C. Jonquire)

Christiane Bouthillier Sophie Pascal, Larysa Lysenko Martine Lacroix, Cline Dufresne, Isabelle Cagiotti, Marcela Quesada Marianne Dub, Annie Chartier Marc-Andr Deniger (2002-2005; UdeM) Richard Bertrand Michel Perron (20022

Plan of presentation
1.

What is the New Approaches, New Solution Strategy? Evaluation questions and methodology A glimpse of results and recommendations

1.

3.

Janosz 2010

Janosz 2010

3 3

What is the New Approaches, New Solution Strategy?

Janosz 2010

Janosz 2010

4 4

General Goals
To break the vicious circle of the Poverty-School failure relation. To improve school success amongst secondary school students from the Cliquez pour modifier le style communities (200 most disadvantagesdes sous-titres du masque middle and high schools)
Annual funding of 25M$ for 6 years distributed according to school size and level of poverty.
Janosz 2010
Janosz, 2010

Janosz 2010

Model of action
Background
Schools as learning communities Schools decide the interventions they want to implement locally according to their specific Cliquez pour modifier le style des sous-titres du needs, strength and weaknesses. masque Specific interventions are not prescribed, but schools have to follow a structured collective problem solving approach in order to select, implement and evaluate them.
Janosz 2010
Janosz, 2010

Janosz 2010

 Janosz 2010

Janosz 2010

Cliquez pour modifier le style des sous-titres du masque

Implementation national levelImplementation school level

Janosz 2010

EFFECTS Janosz 2010

Evaluation questions and methodology

Janosz 2010
Janosz, 2010

Janosz 2010

9 9

Questions and method Implementation

Did the Strategy implemented itself as expected at the Ministry, School board and school levels? What explain the gap between what was planned and what was realized ?

Annual self-reports questionnaires and face-to-face interviews (180 principals, 2 000 teachers) Janosz 2010 Janosz 2010
Janosz, 2010

10 10

Questions and method School-level outcomes

Did NANS have positive effects on the educational potential of schools with regard to school climate, school violence, educational practices, school-community and school-family collaboration?

Student-level outcomes

Janosz, 2010

Did NANS have positive effects on the success of students with regard to learning and motivation, social relationships with peers and teachers, psychosocial well-being11 Janosz 2010 Janosz 2010 and life-styles, school perseverance? 11

Questions and method

Longitudinal quasi-experimental design with non equivalent group Annual multiple surveys of 30 000 students, 4 000 school personnel in 81 schools (70 NANS sampled school, 11 comparison schools) Breakdown of the effects according to the quality of implementation (a longitudinal index)
Janosz 2010
Janosz, 2010

Janosz 2010

12 12

A glimpse of results and recommendations

Janosz 2010

Janosz 2010

Janosz, 2010

13 13

Results
The Strategy seemed to be effective at improving the educational environment but only in the peripheral aspects of the classroom (school climate, violence, school-family collaboration). Consequently, it was no surprise that improvement at the student level were confined to the sphere of socialization Janosz 2010 Janosz 2010 (misbehavior, drug use) and not in 14
14

Janosz, 2010

Results
When positive changes we observed, they were almost always stronger in schools that appeared to have better implemented the NANS model.

Janosz 2010

Janosz 2010

15 15

Results
Nevertheless, most schools had significant difficulties at engaging successfully into the NANS structured process and many school boards failed at supporting their schools to do so. The most significant changes in schools practices appeared where support and guidance were provided in quantity and quality.
Janosz 2010 16 16

Janosz 2010
Janosz, 2010

Implications
The evaluation of implementation revealed a series of conditions (barriers, facilitators) explaining why, at the end, schools tended to do more of the same instead of doing things differently. Not having taken the time to reflect on ones practice and perceptions and understandings of what it meant to teach to disadvantaged students was one of the multiple barrier highlighted.
Janosz 2010
Janosz, 2010

Janosz 2010

17 17

Implications
Interpretation of the results helped us to make 13 recommendations to the Ministry, according to
1.

What components of the Strategy to maintain What components to revise significantly conditions to gather and18 Janosz 2010
18

1.

1. What Janosz 2010 new

Janosz, 2010

MERCI BEAUCOUP DE VOTRE ATTENTION Reports can be downloaded at WWW.GRESUMONTREAL.CA

Janosz 2010
Janosz, 2010

Janosz 2010

19 19

 Janosz 2010
Janosz, 2010

Janosz 2010

Table 2. Distribution of sampled NANS schools and comparison schools based on size and sociogeographic location Sampled NANS Comparison Location School size NANS schools schools schools (n=197) (n=66) (n=11) n Rural Small school Medium school Large school Total Small school Medium school Large school Total Small school Medium school Large school Total All locations Small school 19 12 3 34 18 31 58 107 11 11 34 56 48 54 95 (%) (9,64) (6,09) (1,52) (17,26) (9,14) (15,74) (29,44) (54,31) (5,58) (5,58) (17,26) (28,43) (24,37) (27,41) n 9 4 1 14 2 11 24 37 1 14 15 11 (%) (13,64) (6,06) (1,52) (21,21) (3,03) (16,67) (36,36) (56,06) (1,52) (21,21) (22,73) (16,67) n 1 9 10 1 1 1 10 (%) (9,09) (81,82) (90,91) (9,09) (9,09) (9,09) (90,91)

Urban

Major City

Janosz 2010

Medium school Large school

16 (24,24) Janosz 2010 39 (59,09)

(48,22)

 Janosz 2010

Janosz 2010