.. TABLE 5.1 . EXAMPLES OF MEASURES.

The Improvement Guide Using the Model for Improvement 97

larger, more complex systems, or for efforts that require fundamen,tal design or
redesign, it is advantageous to answer the question in two parts. Firsi, give some
Type of Education: Increase Health Care: Redesign Inventory: Reduce
broad concepts for the system (for example, better match of capacity to demand),
Measure the Percentage of the Processes by Costs of Inventory
Students Passing the Which Patie nt Flow Is While Maintaining and second, offer more detail on the actual changes that will be made (for exam-
State-Mandated Test Managed Within Our o r Improving Service ple, reallocating nursing resources from the day to the evening shift). Chapter Six
Hospital Level to Customers presents methods for developing ch anges.

Outcome Percentage of students Diversions from the Cost of inventory

passing state-mandated . hospital; transfe~ time
tests between units The Plan-Do-Study-Act Cycle
Process Percentage of students Number of units Number of inventory
who took a revi,e w conducting bed huddles checks The PDSA Cycle completes the framework for the Model for Improvement. The .
course cycle can be used to tum ideas into action and connect action to learning. Using
Balancing Percentage of students Admissions; patient Stockouts; customer the cycle effectively takes some discipline and effort. Figure 5.2 furnishes some
who took the test satisfaction; cost per bed satisfaction
detail on what should be considered in each phase of the cycle.

Ryan and his team planned a cycle to evaluate the participation of suppliers
In answering the question " How will we know that a change is an improve- during the early concept design phase for a -new computer. This cycle was part
ment?" different levels of measurement are useful. We should consider three levels
of measurement to facilitate learning while improving a compl.ex system. The
levels are described below; Table 5.1 gives some examples relative to improve- FIGURE 5.2. THE PLAN·DO-STUDY-ACT CYClE.
ment efforts. .
Outconw nJeasures . Outcome measures are measures of the perfonnance of
the system under study. They relate direcdy to the aim of the project. Outcome mea-
sures offer evidence that changes are actually having an impact at the system level. Plan
Process -measures. Process measures are measures of whether an activity • Objective
Act • Questions and
has been accomplished. For example, process measures could be whether inven-
• What changes are predictions (why?)
tory checks were made or whether patients received evidence-based interventions. to be made? • Plan to carry out
Process measures are often used to determine if a PDSA Cycle ,was carried out • Ne~t cyde? the cycle (who, what,
as planned. . where, when)
Balancing -measures. T o achieve an improvement in some measures while • Plan lor data colle<tion

degrading performance in others is usually not acceptable. In making cha~ges to

improve outcome and p rocess measures, we want to be sure any related measures S!udy Do
afe maintained I')f improved. • Complete the • Carry out the plan
analysis of the data • Documen t problems
• Compare data to and une~pected
predictions observations
What Changes Can We Make That Will Result in
• Summarize what • Begin analysis
Impr!'vement? was learned of the data

Answering this question requires developing ch anges to test. For relatively simple:
systems, a list of changes may be developed and tested almost immediately. Fa!
 The Improvement Guide
9ftheir project to increase the quality and lower the cost of their computers
and associated products through better supplier relations. In theory, the idea
ofincluding supplien in the design phase was appealing. However, it did take
more work, it made the meeting arrangements more complex, and it required
additional time for technical people from the suppliers. The team wanted to
learn about the return on t;his invesunent.
The plan for the cycle was to invite suppliers to three upcoming design
meetings. Each meeting was poncem ed with a different product, so a fey.'
people would be at all th ree of the meetings. In some cases, more than one
supplier for the same component would be at a meeting and the team was
interested in seeing the effect of having competitors at the same meeting. Two
members of the team were anigned to attend all three meetings to observe and
record their observations. In particular they were to record (Il attendance from
the suppliers and their level of technical expertise, (2) ideas that came from the
suppliers that would be pursued in the design, and (3) the willingness of suppli-
ers to perfonn further investigative work after the meeting. After the meeting,
a survey would be sent to each person who a ttended to get general impressions
of the meeting and the usefulness of including suppliers at this phase in the
design of a new product.
The improvement team would study the data for the three meetings.
According to the results, they would take some action, such as changing the crite-
ria for who gets invited, structuring the meetings differently, changing the timing
for when to invite the suppliers, or continuing the meetings in the same fonnat.
More than likely, they would need to continue testing some of these ideas.
Contrast what R yan and his team did with a more familiar scenario: have
the meetings; after the meeting, hear some ane<:dotes about how the meetings
went during a weekly staff m eeting; from the most compelling anecdote, decide
whether to keep inviting.suppliers; and regardless of the decision, neglect to set
up a good process to carry out the de<:ision. Although in this scenario action could
occur, it should not be considered a cycle. T o be considered a PDSA Cycle, fou r
I predictions. We make predictions in the Plan ph ase of a PDSA Cycle to:
aspects of the activity should be easily identifiable:
I. Plan: the learning opport unity, test, or implementation was plann('..l .,,,11 ..
included:
Q uestions to be a nswered
• Predictions of the answers to the questions
• Plan for collection of the data to answer the questions _ develop, test, or implement changes
2. Do: the plan was attempted. Observations are made and recorded, including,
those things that were not part of the plan.
Using the Model for Improvement ...
3. Study: time was set aside to compare the dat~ with the predictions and study
the results.
4. Act : action was rationally based on what was learned.
The POSA Cycle is a vehicle for learning and action. The three most com-
mon ways for using the cycle as part of an improvement effort. are:
I. To build knowledge to help answer anyone of the three questions
2. To test a change
3. To implement a change
Not all improvements require PO SA Cycles; some just happen. However,
purposeful improvements in large or complex systems will usually require one or
more cycles. Filling out a worksheet for each cycle is a convenient way to docu-
ment the project. Appendix B contains worksheets proven useful fo r establishing
a degree of discipline and thoughtfulness in carrying out a cycle. Examples of
PDSA Cycles are in the case studies in Chapter Twel",'e.
In the improvement effort to reduce medication errors described earlier, the
team was unanimous in their prediction that the changes would reduce errors.
Sally then asked wI!J they believed the changes would be successful. To the
team's surprise, there were contrasting reasons for their unanimous prediction
of ''yes.'' A few thought it was because of the use of a protocol; others thought
it was where the medications would be mixed or how they would be stored.
The act of making a prediction and considering the basis for the prediction
resulted in a much deeper understanding of why the changes were being made.
Il also caused the data collection plan to change for certain PDSA Cycles. (For
example, data collection on where the medications were mixed was included.)
The team feit better prepared. to learn from their tests as a result.
POSA cycles facilitate learning through an ite ration of cycles spurred by
Make the why unde rlying the prediction clear
Surface diffe rences in assumptions among team members before collecting
data or ru nning a test
. j Prevent hindsight bias {''I knew it all along" }
Improve the Plan phase of the PDSA Cycle relative to how we collect data,
.
.- Enhance the learning in the Study phase of the cycle when results are compared
to nr,.t"!i .... ;"...
100 The Improvement Guide Using the Model fOf Improvement ,.,
The next sections clarify the use a Clhe cycle as an aid to answering the three FIGURE 5.3. PARETO CHART OF CAllS FROM CUSTOMERS.
questions. The use a Clhe cycle to test a change is discussed in Chapter Seven, and
the use of the cycle for implementing a change is discussed in Chapter Eight.

Using the Cycle to Build Knowledge

Although the primary use of the PDSA Cycle is to test changes, we can use the
cycle when current knowledge or readily available background information is not
sufficient to ~r one or more"or the three questions. The primary use of data
in answering the question ''What are we trying to accomplish?" is to determine
why a particular aim should be chosen from among many initiatives. The data
might be warranty claims, responses to a customer survey, an analysis of defeclS
or complaints, plots of measurements over time, or documented comments from
employees. An example illustrates the use of a cycle to set the aim for an improve-
. ment effort.
A manufacturer of garage door openers established an 800 phone line to assist
customers. A! inpu~ to their quality planning process, management planned a
cycle to collect and analyze data from the calls to the 800 number according to
this procedure:
For one week, a summary of each call was logged.
The calls were classified into six categories: installation, warranty, parts, new
model, billing, and cosmetics.
The data were collected for three. more weeks by using a form with me six
categories. Space was allocated for summaries of calls that did not fall into
any of me categories.
The analysis of the data was used as input to planning for improvement.
Figure 5.3 contains a Pareto chart (a graph that illustrates the relative fre-
quency of occurrence of an activity; see Appendix B for more on Pareto charts)
of the nwnber of caUS for assistance from customers received on the 800 number.
Two aims wen: adopted as a result of analyzing the data: (I) to improve the ease
of garage door installation and use of accompanyj.ng instructions, and (2) to make
warranty information more understandable.
A PDSA Cycle can be used to answer the question "H ow will we know that
a change is an improvement?" by assisting in determining a balanced set ofmea-J
sures for a process or system. Such a cycle might consist of a survey of customers'
or interviews with a sample of people who have a stake in the system. Fouroth
cycles that might be completed to help answer this question include:
J
~
o 50 100 150 200
Frequency
T esting the feasibility of using a new measurement device, such as a color-
measuring instrument
Testing the clarity of StlIvey questions
Assessing the ability of workers to routinely collect data
T esting the accuracy ofa lab result
The question "What changes can we make that will result in improvement?"
can be answered by using existing knowledge or some creative activity (see C hap-
ter SiX for more detail on developing a change). If ideas for specific changes are
not available, one or more cycles can be used to build the knowledge necessary
to deve10p the changes. An example illustrates the use of a PDSA Cycle to build
I.knowledge to support the development of a change.
flIe management of a rapid rail transportation system in a large metropolitan
U'Ca designed a cycle (0 help set priorities for improvement. The cycle included
oliciting suggestions from riders of the subway as to how the system could be
mproved. More than,eighteen hundred replies were received. The replies were
Lnalyud in a variety of ways (most frequent, most creative, relative merit). The
IUtput of me analysis was a list of changtll to be tested in subsequent cycles:
Pay telephones on station platfonns
Better lighting in the statiom
,.2 The Improvement Guide
More direCtional signs
A twenty-four-hour telephQne assistance line for riders
Usc of credit cards to buy fare cards
Transit system infonnation on display ncar the convention center
Sclling the familyltourist pass at ~ stations
Indicators showing the color a fthe next scheduled train (by track)
When using POSA Cycles to build knowledge to answer any of the three
questions, it is imponam that the knowledge gained be subsequently translated
into action. Do not forget the A ul "PDSA. . Cycle 3. Continue to refine the: role and
Using the PDSA Cycle Sequentially to Te st Changes
The Model for Improvement is based on an iterative, trial-and-leaming approach to
improvement. We advocate the testing of changes on a small Scale initially 10 reduce
risk and then, from the learning, use of subsequent cycles 10 scale up the changes.
Even trials that are not successful can add to the learning if they are thoughtfully
reviewed. Although the use of small-scale, sequential cycles for the design or rede-
sign oflargc or complex systems is counterintuitive, it is an effective approach. The
bigger the system, the more uncertainties there will be. In our experience, spending
time in a conference room trying to pene« a change and then trying to "install" it
is not an effective way to make improvements. .
One criticism of this iterative approach is that for large systems the small
cycles will not be coordinated and therefore will no t result in real change to the
system . This pote ntial weakness is overcome by u sing broad concepts for
the syst~m design (see Chapten Six and Eleven and Appendix B for more discus-
sion on driver diagrams) to sel the context for all the smaller cycles. It is important
to note that small-stale refers 10 the size of the lest. The change lested could be
quite innovative and a significant departure from currenl practice-in this sense,
a very large change. Figure 5.4 shows cycles to test the use of a scribe to offload
paperwork from the dentist and hygienist. The team used these cycles with the
aim to reduce waiting time in their dental office.
In large syslems, it is often necessary to test multiple changes. For example,
in the dental office they might also be using a series of cycles to test extending the
intelV3I between appointments for certain patients. We will address this issue in.
Chapter Eleven, " Improving Large or Complex Systems."
Using Oata in a CyCle
To suppon an iterative approach, we need to establish a "learning loop." This can
be thought of as consciously building on what we have learned from a previo~
Using the Model for Improvement ,..
FIGURE S.4. REPEATED USE OF PDSA CYCLE TO TEST USE
OF A SCRIBE.
~@
C~n9es
that result In
A P ImprOftment
S 0
o~t- I \ ::...L/'/Cycle -4 ' FOfTNIu:e the POSItion and
o S ./ hire two scnbes part·time to wOO< with
both the dentists and hygienbts
test on a busier day.
~'e 2: Revi~ the role of the scribe
@ S
p
0
/' ;~d lest on a historically slow day with a dentist
YCf<le 1: Test the U~ of a scribe (use a temporary worker)
Theories working with the hygienist for a ff!W patients
Ideas
cycle or lest. When we started riding a bicycle, we learned about our technique the
hard way: we fell. Most of us learned quickly from the clear and immediate signals
we received- two imponant·characteristics ofa good learning loop. Our experi-
ences learning to ride a bicycle indicate that learning does not have to be based
on data; few of us probably documented learning about our falls. But data can
be a good source of feedback in PDSA Cycles to understand ifwe are having an
impact on the system. We establish a learning loop in the Study phase of the cycle
by comparing data to predictions. Data can take many forms but need to produce
learning to serve as input to the next cycle. This learning u1timately leads 10 making
changes that result in improvement. The use of data in a cycle to facilitate learning
and action is an important aspect of the approach presented in this book.
Theresa was a midlevel manager in a large corporation. Her department was
responsible for processing hundreds of complex forms every day. To improve
the performance of her group, Theresa purchased' new high-speed computers
and new software. She was quite sure these changes had produced improve-
ment in the department's performance. But some of the clerk:! complained
the new programs were tOO hard to usc. Others ta.1ked about how fast the new
computers were "once you learned the new system." Theresa's boss was not
happy that errors were still being made, and the department's work was backed
up worse than before they "invested all that money."
Theresa's problem cou1d have been greatly reduced ifher p lan" for the change
(new computers and new software) had included the use of data collection and
'04 The Improvement Guide
analysis. Data collected on measur~s s.uch as erron made and lime to process
could be used to answer the fundamentaJ question "H ow will we know that a
change'is an improvement?" Instead, Theresa relied on observation, memory, and
feelings to j udge the success oCthe changes being implemented in her department.
The feedbaCk she received might have been immediate, but it was not d ear. In
Chapter Two, we defined data as documented observations, including those that
result from a measurement process. The onJy difference between observations and
data is that data are thoughtfully documented. Data offer a more balanced
and objective view of the behavior of the systems you are trying to improve.
Earlier we discussed the three levels of measurement: process, outcome, and
balancing. In a cycle or sequence of cycles, each should be considered. Process
measures can be used to learn wbether the cycle is being carried out as planned.
Theresa could collect data on whether the computers and software were being
used correctly. She (and her boss) might expect an impact on outcome measures
(that is, errors and time to p rocess the form s) a~ the deparlme nt level after a
number of cycles have been run. 'Outcome measures might be affected in initial
cycles if the cycles are focused on part of the population, such as a few of the
cler,ks. Theresa should consider the feeHnS' of staff as they learn to use the new
computers and software to be important balancing measures. Collecting data on
personal experience is discusseo;1 in Appendix B.
Some Suggestions for Proper Collection of Oata
Since data are usually collected as part of·the PDSA Cycle, here are some sug-
gestions for collecting data:
I. ExpliciJ!J sto.u tJu questions tb be f11tJUJDed by tJu Jato.. Data are collected to facili-
tate learning. The data collection will be mOSt efficient if the questions to be
answered by the data are stated in the planning phase of a cycle. Deciding
how the data will be analyzed (perhaps by sketching some "dummy" tables
a nd graphs) gives those planning the cycle a check on whether they are col- .
lecting the data necessary to answer the questions posed. To make predictions
more explicit, the questions " How much?" and "Why?" should be considered.
Questions that elicit only yes-orono allSwers should be avoided. Column t
in Table 5.2 contains some questions for PDSA cycles. Column 2 shows the '
questions reworded to help make predictions more useful.
See' the application of the tree diagram in Figure 12.9 for identifying questions
for a PDSA cycle.
2. U.se sampling /() collect daJa. To conserve resources, people can study a sample
rather than all available items. For example, rather than recording every call
Using the Model fM ImprO'lffilent 'os
TABLE 5.2. CRAFTING QUESTIONS FOR MORE USEFUL PDSA CYCLES.
First Question Pro p osed fo r PDSA More Useful Questions for Prediction
Will the new rotes of the care team How much (as measured by a survey) will the
members result in more eHicient and new roles of the care team members improve
effective encounters wi th patients? encoun ters With patients? Why?
Will the new physical set-up of the How much time and resources will tile new
manufacturing p lant allow easy physical set-up of the manufacturing plant
movement of materials? save in moving material? Why?
WIll use of the new form red uce errors . How much witt use of the new form reduce
in billing? errors in billing? Why?
to an 800 number, analyze every twentieth call in more detail. Much can be
learned from samples, especially if they are collected over time. Reducing the
burden of collecting a large amount of data also enables people to do a better
job of studying the items selected. Studying a samph therefore often,fumishes
better information than studying all of the items available.
3. ·Dtsign and wt aformfor colluting the OOto.. Using a fonn carefully designed and
tcsted to collect data will make coUection easier and reduce the opportunity
for error. The fonn should contain the necessary instructions for collecting
the data and definitions of terms.
4. T,ain those who will collect tJu dma and give them undcsto.ndaiJh instructions. In most
. cases the coUcction of data to aid improvement efforts requires some new
tasks for those collecting the data. Do not assume people will know how to
do these tasks. Conduct training that includes the reasons for collecting the
data and the importance of the data. The specific process of measurement or
recording should be reviewed and practiced.
5. Ruo,d whoJ wtrIt Wf01lg during tJu dil1lJ collution. As the data are collected during
the Do phase of the PDSA Cycle, some things may not go as planned. Be
sure to require, in the instructions to those collecting the data, recording the
unplanned occurrences. The people analyzing the data can then assess the
impact of these occurrences on the conclusions drawn from the data. ·
Note that the appropriate rigor and fonnality of the application of these five
suggestions for proper data colk:ction will vary. With more complex measures and
,changes, more formality is useful. In simpler situations, especially those of shorter
duration, tess fonna lity can be applied.
~ DI~lay and Analysis of Oata
Good data analysis begins with clarifying what questions are being asked of the
data. Are we trying to p redict future performance, determine the extent of a
.. The Improvement Guide
Using the Model for Improvement 107

problem, find out where we should focus our efforts, or determine a relationship?
The questions we want to answer guide u.s not only to the appropriate data but
also to what type of data display is appropriate. Of the many ways to analyze
data, visual displays arc a particularly useful form. There are five basic types of
data d isplay (more on these tools and considerations for constructing visual dis-
plays of data can be found in Appendix B):
I . Plots showing data over time (n;m chart and control chart)
2. Plots showing the distribution (shape and spread) of data (pareto chart,
histogram, stem-and-Ieaf plot. bel;( and stem plot, pie chart)
3. Plots showing the relationship between different ~haracteristics (scatter plot
and response plot)
4. Plots showing location of data (map and physical layout)
5. Plots showing results for multiple measures (radar chart and spider diagram)
During improvement efforts, multiple measures are almost always required
to balance competing interests and help ensure the system as a whole is
improved.
Data collection will be most efficient and effective if the questio ns to be
answered by the data and predictions are stated ahead of time.
The Model for Improvement is based on an iterative, triaJ-and-leaming
approach to improvement. This iterative approach is spurred by prediction
and feedback loops for learning. Even trials that are not successful can add to
the learning if they arc thoughtfully reviewed.

. Of all the displays, the plots showing data over time play the central role in
making improvements. Change is a prerequisite fo r improvement, and people
experience change as a time-related phenomenon. The -answer to the question
" How will we know that a change is an improvement?" is often that one or more
measures will increase or decrease over time. By plotting data over time, we can
d isc~ patterns that indicate improvement.

Guiding Teams and Projects Using the Model for Improvement

Good c harters will help individuals and teams ensure they are guided by the
Model for Improvement and assist in team selectio n, sponsorship, and project
management. Possible roles on an improvement team are team ~ember. team
leader, improvement advisor (sometimes referred to as a "black belt" Of internal
conswtant), subject matter experts, and the management sponsor.
Table 5.3 describes the roles o f team members with regard to various respon-
sibilities during a project using the Model for Improvement.

Key Points from Chapter Five

Numerical goals contained in aims for improvement efforts must be connected

to methods for achieving the goals, or else they can cause unintended.
consequences.
In answeringthe question."H owwill we know that a change is an iJnprovemen t ?'
assure the interests of the customer of the product, process, or service are]
strongly represented in the list of measures.
448 Appendix B
Appendix B . .9

Form B.3: PDSA Cycle Form with Checklist Format DO Art historical da1a armilahk ro answtr Ihm questions? 0 Yes 0 Nd
DO Does tht team agree on the predictions?
Date: Team: Cycle #:
o Yes for question(s): o No for question(s):

~~. Develop a plan to answer the questions (who, what, where, whe n):
Plan 1.
2.
3.
••
Objective of this cycle:
-0 Collect data
The plan considered these methods:
o Develop a change (or modify a change from a previous change) o Data collection forms
o Test a change o Pareto diagrams
o Implement a change o Control charts
Describe: o Frequency plots
o Planned experimentation
~ o Survey methods

- Mat additional knowkdgt is necessary to lflke action? o Simulation and modeling

o Scatter diagrams
o Run charts
o Engineering analysis
"" Did you assign responsibilities for collection and analysis of the data? 0 Yes 0 No
"'" Is training needed? 0 Yes 0 No
Questions to be answered from the da ta obtained in this cycle:
"'" Is the plan consistent with the charter? 0 Yes 0 No
1. "'" Can the plan be carried o ut on a small scale? 0 Yes 0 No
2. "'" Have you considered people outside the team who will be affected by this plan?
3. O Yes O No
••
Predictions:
1.
2. ,
'~
3. ·1
J
••
l l
 Appendi)( B 4"
.so Appendi)( B

Summarize the new knowledge gained in this cycle:

Do
Observations in carrying out the plan: ,

o Flowcharts reflect what was learned.

o Cause-~nd·effect diagrams reflect what was /earned.

j~---.-.---'
Things observed that were not part of the 'plan.

1: Things that went wrong during the data collection.

o Used a Cantrol Chart o Idtntijitd.sptcial causes as da1a were colltctd.

,
,
Act

~
What changes are to be made to the process?

.
Study list other organizations and people that will be affected by t he changes:

Analysis of data:
o The cause system is sufficiently understood.
Compare the analysis of the data to the current knowledge: o An appropriate action or change has been developed or selected.
o The changes have been tested on a small scale.
o Change responsibilities far implementation and evaluation completed.
, o Actions or changes will improve performance in the future.
o Completed an analysis of forces in the orgcmization that will help Of hinder the
- Do the results rfthe cycle agru with predictions made in tJu plmmingphau?

O YesONo
,
changes

ObJective of next cycle:

- Under what conditions couid tJu conclusionsftflm this fYcle be different?

o Collect data
o Develop a change (or modify a change from a previous change)
-
,
W'hat are liIe implications rf the unplanned observations and problems during tk collection 'I o Test a change
oJdafa? . ! o Implement a change
,

- Do tJu Mfa help answer the qutstions posed in the pwn? 0 Yes 0 No
,
. DeSCription:

fl