Components of a Learning

System
David M. Williams, Ph.D.
Improvement Advisor
Institute for Healthcare Improvement

The Early Years

Collaborative
Scotland will be the best place to grow up.

Draws upon existing community infrastructure across

the country
Focused on age-based workstreams:
15% reduction in the rates of still-births and infant
mortality by 2015.
85% of all children within each CPP will reached all
of the expected developmental milestones at the
time of the childs 27-30 month child health review,
by end-2016.
90% of all children within each CPP have reached all
of the expected developmental milestones at the
time the child starts primary school, by end-2017.

Components of a Learning
System
1.
2.
3.
4.
5.

System level measures

Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: Act for the individual learn
for the population
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning
system
From Tom Nolan PhD, IHI

Workstream 1 Aim
To ensure that women experience positive
pregnancies which result in the birth of more
healthy babies as evidenced by a reduction of
15% in the rates of stillbirths
(from 4.9 per 1,000 births in 2010 to 4.3 per
1,000 births in 2015)
and infant mortality
(from 3.7 per 1,000 live births in 2010 to 3.1
per 1,000 live births in 2015).

Workstream 2 Aim
To ensure that 85% of all children
within
each Community Planning Partnership
have
reached all of the expected
developmental
milestones at the time of the childs
2730
month child health review, by end
2016.

Workstream 3 Aim
To ensure that 90% of all children
within
each Community Planning Partnership
have
reached all of the expected
developmental
milestones at the time the child starts
primary school, by end2017.

Big Aims

We are here
Source: Brandon Bennett, IA

Overall Project Measures vs.

PDSA Cycle Measures
Achieving
Aim
Adapting
Changes
During
PDSA
Cycles

Data for Project Measures:

- Overall results related to the project aim
(outcome, process, and balancing measures) for
the life of the project

Data for PDSA Measures:

- Just enough data
- Quantitative data on the impact of a
particular change
- Qualitative data to help refine the change
- Subsets or stratification of project measures
for particular patients or providers
- Collect only during cycles

Run Chart - Data for Learning &

Improvement
6.00
5.75

Pounds of Red Bag Waste

Measure

5.50
5.25
5.00
4.75
Median=4.610
4.50
4.25
4.00
3.75
3.50
3.25
1

Time

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Point Number

PDSA
Measure
s Guide
Learning
about our
testing.

Process
Measures
Guide
Learning
about how
our testing
is improving
reliability of
the process.

Outcome
Measures
Guide
Learning
about how
the
reliability
of the
process is
achieving
our aim.

Components of a Learning
System
1.
2.
3.
4.
5.

System level measures

Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: Act for the individual learn
for the population
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning
system
From Tom Nolan PhD, IHI

Early Years Collaborative

P

Pre-work

P
D

Expert
meetings

D
S

LS1 2 day
Kickoff

2 day
LS

2 day
LS

2 day
LS

Etc

Supports

Cluster
meetings
Key Changes

Expert QI & Early Years faculty

Networking events

Improvement

Listserv

Measures

Phone conf

Site Visits
Assessments

Monthly Reports via web

Oct 2012

Jan 2013

May 2013

Oct 2013

TBC

TBC

WORKSTREAM 1 (conception to 1 year)

Theory of what actions will

reduce infant mortality

Theory of what drives infant mortality

Aim

1
Poverty

Quality Of Home Environment

Societal Issues

Access To Services
Employment

Detailed aim:

Attachment

Post-birth actions

Improved money
management
Improved rate of
breastfed babies
Quicker diagnoses of
Neonatal Abstinence
Syndrome
Improved leadership,
culture and planning
Improved family
centred response
Improved stability /
permanence for LAC

Health

Improved identification

Parenting skills

Improved joint working

Smoking / Alcohol &

Drug Misuse

stillbirths (from 4.9

per 1000 births in
2010 to 4.3 per 1000
births in 2015)
infant mortality
(from 3.7 per 1000
live births in 2010 to
3.1 per 1000 live
births in 2015)

Workforce Issues
Transport, Community
Capacity & Cultures

Reduce infant
mortality

To ensure that women

experience positive
pregnancies which
result in the birth of
more healthy babies as
evidenced by a
reduction of 15% in the
rates of:

Domestic Abuse &

Violence

Improved teamwork,
communication, skills
and collaboration

Nutrition

Improved management
and quality of care
Improved sharing of
information
Improved access

Pre-birth actions

Mental health &

wellbeing

Identification &
reasons for current
resilience

Version: 06/03/2013

WORKSTREAM 2 (1 year to 30 months)

Theory of what actions will

ensure developmental
milestones are reached

Theory of what drives developmental milestones

Aim

1
Poverty

Quality Of Home Environment

Societal Issues

Children have all

the developmental
skills and abilities
expected of a 2730 month old

85% of all children

within each CPP
have reached all of
the expected
developmental
milestones at the
time of the childs
27-30 month child
health review by
end-2016

Improved childs dental

health

Workforce Issues

Improving child nutrition

Access To Services
Employment

Detailed Aim:

Health

Improving brain
development and
physical play
Improved family centred
response
Improved stability /
permanence for LAC

Attachment
Early Learning & Play

Improved early
identification

Additional Support

Improved joint working

Level of education

Improved sharing of
information

Misuse of alcohol &

drugs

Carers physical &

mental health and
skills

Improved money
management

Domestic Abuse &

Violence

Transport, Community
Capacity & Cultures

Childs physical &

mental health and
emotional
development

Improved teamwork,
communication, skills
and collaboration

Nutrition

Improved management,
planning and quality of
services

Disabilities & Mental

health

Improved leadership,
culture and planning

Parenting skills &

knowledge

Identification & reasons

for current resilience

Version: 06/03/2013

WORKSTREAM 3 (30 months to start of primary school)

Theory of what actions will

ensure developmental
milestones are reached at
the start of primary school

Theory of what drives developmental milestones

Aim

1
Poverty

Quality Of Home Environment

Societal Issues

90% of all children

within each CPP
have reached all of
the expected
developmental
milestones at the
time the child starts
primary school, by
end-2017

Workforce Issues
Transport, Community
Capacity & Cultures

Children have all

the developmental
skills and abilities
expected at the
start of primary
school

Detailed Aim:

Domestic Abuse &

Violence

Access To Services
Employment
Early Learning & Play

Childs physical &

mental health and
emotional
development

Improved uptake of
benefits
Improved childs dental
health
Improving child nutrition
Improving brain
development and
physical play
Improved family centred
response

Health

Improved stability /
permanence for LAC

Attachment

Improved identification

Additional Support

Improved joint working

Level of education

Improved management,
planning and quality of
services

Misuse of alcohol &

drugs

Carers physical &

mental health and
skills

Improved teamwork,
communication and
collaboration

Nutrition

Improved sharing of
information

Disabilities & Mental

health

Improved leadership,
culture & planning`

Parenting skills &

knowledge

Identification & reasons

for current resilience

Version: 06/03/2013

WORKSTREAM 4 (Leadership)

Theory of what drives leadership support

Aim

Theory of what actions will

ensure leadership support

Build commitment with

partners to focus on delivery

Early Years
Collaborative is a
strategic priority &
underpins all
policy planning
and operational
activity
Provide the
Leadership System
to support quality
improvement
across the Early
Years
Collaborative

Early Years
Collaborative
values, culture and
behaviours are
modelled by all
leaders at all levels

Leaders illustrate how users

are included in design,
improvement, and delivery
of Early Years

Leaders facilitate change by

cultivating innovation from
intelligence, insights and
wisdom of people working
together
Leaders demonstrate their
ability to set direction and
engage and mobilise staff to
constantly improve quality
of service
Leaders can describe how
they personally maintain
early years focus within their
working environment
Early years executive and
operational leads are
identified

Detailed Aim:
Timely delivery of all
three workstream
stretch aims

CPPs communicate the EYC

with enthusiasm and
consistency

Infrastructure to
support delivery of
Early Years
Collaborative

Measurement plan and

priorities are established
and triangulation with other
key data
Spread plan is in place for
core and innovative work
Strategy for capturing,
celebrating and spreading
innovation

Establish an EYC
Implementation Committee
Ensure a feedback
mechanism for issues raised
in Walk-rounds
Ensure the development of a
measurement system used
to understand and drive
quality indicators
Assign a senior leader to
each improvement area
(Workstreams 1-3 and
measurement)
Establish Programme
Management and remove
barriers
Meet regularly with the
Implementation Committee
to track progress and
remove barriers
Display data that depicts
progress towards aim
Ensure that the senior team
participates in Walk-rounds
Place quality issues at the
top of senior leader meeting
agendas
Add Early Years
Collaborative and outcomes
to the CPP agenda

Version: 06/03/2013

Components of a Learning
System
1. System level measures
2.
3.
4.
5.

Explicit theory or rationale for system changes

Segmentation of the population
Learn by testing changes sequentially
Use informative cases: Act for the individual learn
for the population
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning
system
From Tom Nolan PhD, IHI

An approach to achieving the IHIs The Triple Aim for a given

population - from the perspective of a consumer health plan-less, FFS
based, Medicare participating, not for profit, hospital->health system
A population, for which
claims data exists and
achieving Triple Aim results
will not result in perverse
economic loss.

Cannot be defined by a
clinical condition
(Diabetes) or issue
(readmissions).

Note: The size of the rectangles is meant to be

indicative of
population size, not cost.

An approach to achieving the IHIs The Triple Aim for a given

population - from the perspective of a consumer health plan-less, FFS
based, Medicare participating, not for profit, hospital->health system
A population, for which
claims data exists and
achieving Triple Aim results
will not result in perverse
economic loss.

A sub-population,
high cost and or
high utilization
people from the
larger
population.

People who
have fallen
through the
cracks of
our rescuecare
system.

Cannot be defined by a
clinical condition
(Diabetes) or issue
(readmissions).

Note: The size of the rectangles is meant to be

indicative of
population size, not cost.

An approach to achieving the IHIs The Triple Aim for a given

population - from the perspective of a consumer health plan-less, FFS
based, Medicare participating, not for profit, hospital->health system
A population, for which
claims data exists and
achieving Triple Aim results
will not result in perverse
economic loss.

A sub-population,
high cost and or
high utilization
people from the
larger
population.

People who
have fallen
through the
cracks of
our rescuecare
system.

Cannot be defined by a
clinical condition
(Diabetes) or issue
(readmissions).

Sub-groups,
people from the
high cost high
utilization subpopulation that
can be stratified
based upon
relatively similar
needs.

Sub-groups
based more
on needs
and less on
conditions.

Note: The size of the rectangles is meant to be

indicative of
population size, not cost.

An approach to achieving the IHIs The Triple Aim for a given

population - from the perspective of a consumer health plan-less, FFS
based, Medicare participating, not for profit, hospital->health system
A population, for which
claims data exists and
achieving Triple Aim results
will not result in perverse
economic loss.

A sub-population,
high cost and or
high utilization
people from the
larger
population.

People who
have fallen
through the
cracks of
our rescuecare
system.

Cannot be defined by a
clinical condition
(Diabetes) or issue
(readmissions).

Sub-groups,
people from the
high cost high
utilization subpopulation that
can be stratified
based upon
relatively similar
needs.

Interventions intended to
address the needs of
high cost high utilization
sub-groups, Plan Do
Study Act cycles.

Sub-groups
based more
on needs
and less on
conditions.

Note: The size of the rectangles is meant to be

indicative of
population size, not cost.

Some interventions will

work and some will
not. All should result in
learning and start on
the smallest practical
scale.

Components of a Learning
System
1.
2.
3.
4.
5.

System level measures

Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: Act for the individual learn
for the population
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning
system
From Tom Nolan PhD, IHI

P
A

D
P

Scale Test

Hunche
s
S
D
Theorie
s Ideas Very Small

Changes That
Result in
Improvement

Spreading

Sustaining the
gains
Implementation of
Change
Wide-Scale Tests of
Change
Follow-up
Sequential building of
Tests
S

Mini-measure tracks
improvement cycles

A
T
DA

Repeated Use of the PDSA Cycle for Testing

knowledge under a wide range

of conditions
48

Scottish Borders - Run chart of run charts!

R

Scottish Borders - PDSAs completed

R

30

Test! Test! Test!

Burt Sandemans Story Post LS1:
The Challenge To Be Quick

Burt Sandemans Story Post LS2:

The F-Word

Looked after two-year olds:

My Prediction Was Wrong

Components of a Learning
System
1.
2.
3.
4.
5.

System level measures

Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: Act for the individual learn
for the population
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning
system
From Tom Nolan PhD, IHI

Act with the Individual

Dads Care
Asset Based Community Development
in N. Ayrshire

Components of a Learning
System
1.
2.
3.
4.
5.

System level measures

Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: Act for the individual
learn for the population
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning
system
From Tom Nolan PhD, IHI

Things to Consider to Scale-up

Determine full scale at project setup and milestones to reach full
scale
Different changes may require different scale-up strategies
Consider different dimensions of structure
Information technology
Physical (e.g. space, equipment, capacity)

Human resources (workforce organization and capabilities)

Financial
Learning system

Use 5x (5--25--125--625--3125---) thinking to predict/define the

structural issues and set a path forward for testing
(What is working when testing with x that probably wont work with 5x, ...?)

Standardize processes (e.g. training, referral)

Understand oversight requirements as the system grows

Components of a Learning
System
1.
2.
3.
4.
5.

System level measures

Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: Act for the individual
learn for the population
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning
system
From Tom Nolan PhD, IHI

Period Review
Avg. Patient Satisfaction - Clinic A

Avg. Patient Satisfaction - Clinic B

Av g. Patient Satisfaction - Clinic C

80

Avg % Satisfaction
80

Avg
70% Satisfaction

70

60

60
50

50

40

40

30

30

20

20

10

10

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Q1
Last 12 Quarters

Av g. Patient Satisfaction - Clinic D

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Last 12 Quarters

Av g. Patient Satisfaction - Clinic F

Av g. Patient Satisfaction - Clinic E

80

Avg
70% Satisfaction
60
50
40
30
20
10
0

Av g. Patient Satisfaction - Clinic G

Av g. Patient Satisfaction - Clinic H

Av g. Patient Satisfaction - Clinic I

80

Avg
70% Satisfaction
60
50
40
30
20
10
0
Q1

Q2

Q3

Q4

Q5

Q6

Q7

Last 12 Quarters

Q8

Q9

Q10

Q11

Q12

Keeping an eye on the

journey
The Work Remaining to
Do

You are here!

Aim

Components of a Learning
System
1.
2.
3.
4.
5.

System level measures

Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: Act for the individual learn
for the population
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning
system
From Tom Nolan PhD, IHI