National Mentoring Summit

January 2013
Agenda
Mentor Video
Program Description
Research Overview
Key Findings
Conclusions and Continual Improvement
Lessons Learned: Mentoring for Academic Gains
Action Planning
Video: Be a Mentor
http://www.youtube.com/watch?v=YyK_03DccSY

Program Model
Afterschool, Summer,
High School Placement
650 hours/year,
+ 1000 hours in school
Weekly academic mentoring
Accelerated lessons, college trips,
career exposure
Positive culture and youth
development approach

RCT Study Design
$4 million over 7 years
Four-year results
published in 2013
By MDRC
Two-year results
and Summer
Snapshot
published in 2011
by P/PV

Researchers test
& survey:
4 summers and 3
school years +
one 9
th
grade
follow up
50% accepted
through lottery

50% assigned to
control group

Over 3 years
(2006, 2007,
2008),
951 students &
families apply and
interview for 3
cohorts
Only 2% of
nonprofits have this
gold-standard
randomized research
Randomized Control Trial (RCT) Study: funded by Wallace, WT Grant, Smith Richardson, Spencer, Atlantic Philanthropies, Bank of America
Randomization in Brief
1. Hear about HA at
school,
neighborhood,
etc.

2. Interview
99% - enter
lottery.

3. Lottery
50% enter
program:
treatment
Treatment
Access to full
program may
never attend.
Control
Never able to
access program.
But given list of
other after-school &
summer options
Annual testing
($120
payment),
surveys,

Follow-up from
researchers
Researchers
Dr. Jean Grossman
Princeton University, P/PV, MDRC
Dr. Leigh Linden
Columbia University, University of
Texas- Austin
Dr. Carla Herrera
P/PV
Outcome Measures Collected
Hypothesis:
- With high dosage and structure, Higher Achievement would have a
measurable impact on academic outcomes and high school placement,
by first impacting attitudes and behaviors.
2 key outcomes:
Standardized Test Scores
Application, Acceptance to, and Matriculation at Competitive High Schools
Other outcomes of interest:
Behavior
Academic Attitudes
Perceptions of Peer and Adult Support
Participation in HA and Other OST Programs
Engagement in Academic Activities and High School-Related Activities

KEY FINDINGS
Higher Achievements intensive
year-round program had a
significant impact on youths
standardized reading and math test
scores.
The longstanding Higher
Achievement model is making a
difference in the academic lives of
motivated, at-risk students who
could easily fall through the
cracks.
Two-year follow-up study

No summer learning loss.

Also - no impact on test scores,
compared to control group,
over the course of one summer:
2010
High levels of attendance and
retention, during tricky middle
school years:
97% of summer participants also in
after-school
Summer 2010 Snapshot
Four-year follow up study
Academic Impacts
First-
Year FU
Second-
Year FU
Fourth-
Year FU
Math problem-solving 0.03 0.10* 0.11*
Reading
comprehension
0.02 0.08

0.04
* = Statistical significance.
Academic and Enrichment Activities
Activity
First-Year FU Second-Year FU Fourth-Year FU
Treatmt
(%)
T-C Diff
(%)
Treatmt
(%)
T-C Diff
(%)
Treatmt
(%)
T-C Diff
(%)
Community service 53 0 60 4 74 4
Presented ideas to a group
out of school
63 6

64 9* 69 11*
Visited a college 72 28** 73 28** 78 25**
Read books out of school 74 3 79 4 85 8

Writing out of school 73 7* 74 3 82 14**
Visited a business 56 4 61 8* 70 14**
Events with OST 77 10** 80 6

84 9*
Academic contests at OST 68 13** 68 11** 71 16**
Conclusions
Year-round, multi-year:
high dosage yields results
No effect after 1 year, only after 2
years:
Academic results take time
Reading gains level with control
group in 4-year follow-up:
Control and treatment made gains.
Reading can be more self-directed.
Math requires more instruction.

Continual Improvement
Common Core Standards Alignment:
Curricula and Technology
Explicitly teach writing skills
Staff training to improve inference
skills for reading comprehension
How Children Succeed &
Social/Emotional Skills
Piloted three assessments:
PEAR HSA, SAYO, Gallup Student Poll +
Grit Scale
Annual Operating
Plan and Individual
Work Plans,
tied to Strategic
Plan.
Monthly Dashboard and
Site Observation
Discussions:
National & Local Staff
School Partners
Quarterly
Dashboard, linked
to Financial Model
Mid-Year Retreats:
Org & City course
corrections
Update individual
work plans
Annual Retreat:
Outcomes and
RCT Results
Continuous
Improvement
Cycle
Maximizing Mentors
Orientation
Ongoing mini-trainings
Scripted lessons
Nightly session feedback
Monthly observations
One Mentors Perspective
Ms. Liesa
Her involvement, academic gains of scholars
Mentor training and ongoing support
Best practices in academic mentoring
Table Activity Think, Pair, Share!
What are the goals for your mentoring program?
What challenges do you have to meeting your goals?
What resources do you have to meet your goals? What
resources do you have to overcome your challenges?
What support and/or trainings do you have to put in
place to maximize your resources and meet the goals
of your mentoring program?


ACTION PLANNING
"Whatever affects one
directly, affects all
indirectly.
I can never be what I
ought to be until you are
what you ought to be.
- Dr. Martin Luther King