Is your FMEA

performing for you?

Measuring FMEA
Effectiveness
Kathleen Stillings CPM, CQE, CQA, MBB
Quality is not an act it is a habit (Aristotle)

Todays Goals

Introduction and Review

Understand why we want to assess the process
Cover the typical gaps in the FMEA process
Review maintaining the FMEA
Discuss the reason why the RPN alone is not
an effective measure of the process
Introduce a published practical FMEA
assessment tool

5/23/2011 Kathleen

What does the FMEA do for us?

Reduces the likelihood of Customer complaints

Reduces maintenance and warranty costs
Reduces the possibility of safety failures
Reduces the possibility of extended life or
reliability failures
Reduces the likelihood of product liability claims

5/23/2011 Kathleen

Why Assess the Effectiveness of

your FMEA process?
Stay Competitive!
Reassure Management the investment
really does pay off!

5/23/2011 Kathleen

What work goes into the FMEA?

The process can be time consuming that
is why we ask if YOU are working for the
process when you prefer the process work
for you.
Are you getting the expected benefits for
all your hard work?

5/23/2011 Kathleen

One Critical Factor of Your Risk

Assessment System

AN EFFECTIVE FMEA PROCESS HAS

THE FOLLOWING COMPONENTS
Implementation

of a Good System
Maintaining that System
Assessing Your Systems Effectiveness

IMA Im a Fool If I Dont

5/23/2011 Kathleen

FMEA Success Factors

Align your success factors
Quality starts and ends with training

(Kaoru Ishikawa)

Does your Management team support the

process?
Do you have adequate resources?
Does your team understand the basics of the
FMEA process?

5/23/2011 Kathleen

Close the Gaps in the System

Gap 1 stated previously ineffective evaluation of RPN leading to

wasted time and effort

Gap 2 - lack of process improvements not effectively implemented

because of ignorance of assessing the critical factors with high risks

Gap 3 failure to address ALL high risk failure modes: High

Severity, High Criticality, RPN with a value under the threshold
assigned by the process that could lower Customer satisfaction
(internal and external Customers)

5/23/2011 Kathleen

Close the Gaps in the System

Gap 4 neglect to include systemic interfaces as part of

the FMEA

Gap 5 the dangerous we dont know what we dont

know not addressing the unknown because we are
only as good as the team working on the FMEA we
may not effectively choose the correct resources within
the organization to be part of the FMEA team

Gap 6 not including critical voices in the process

Customer, Operator, Quality, Testing or Analysis

5/23/2011 Kathleen

Close the Gaps in the System

Gap 7 not implementing effective quality tools to

perform the FMEA assessment with in the team
environment.

Gap 8 missing links to internal and external quality

data

Gap 9 failure to identify critical characteristics

Gap 10 failure to effectively train people on the basics

of completing a FMEA process

5/23/2011 Kathleen

10

Close the Gaps in the System

Gap 11 lack of resources assigned to the process time,

people, research, and follow-up

Gap 12 not linking Mistake Proofing with failure modes.

Gap 13 assuming Detection controls are better than they

really are or are implemented when they are not.

Gap 14 neglecting to capture all the details

Gap 15 Failure to maintain the system

5/23/2011 Kathleen

11

Maintaining the System

We often overlook variables that could have an

impact on product, process, and quality.
o

Typical FMEA Process: Planning stage, Performing

FMEA stage, review stage, implementation stage.
The process is missing the Maintenance stage.
The FMEA must be mapped, analyzed, and
controlled

5/23/2011 Kathleen

12

5/23/2011 Kathleen

13

Maintenance of the System

Ensure you update process flow charts,

control plans, and FMEA data with the
following:
Data

internal and external

Changes systemic, process, product
Audit Results

5/23/2011 Kathleen

14

Assess the Systems Effectiveness

The fundamental value enhancing

applications of FMEA are process
management and process improvement
One

measure of process improvement is the

reduction in RPN (Risk Priority Number)
FMEA is Process Improvement
Process Improvement is the reduction in failures
A reduction in failures is a reduction in RPN

5/23/2011 Kathleen

15

Just Say No

Why is the RPN not and effective measure

of the FMEA System?
o

We should assess Severity, Severity x

Occurrence = Criticality, and Risk = Severity x
Occurrence x Detection = RPN separately.
There is not weighting factor used to evaluate
the risk

S=10 (Hazardous without warning), O=10 (Occurs 10% of the time), D=1 (certain to detect) RPN = 100
S=1 (No effect), O=10 (Occurs 10% of the time), D=10 (not detectable) RPN = 100

5/23/2011 Kathleen

16

Just Say No

S=10 (Hazardous without warning), O=10 (Occurs 10% of the time),

D=1 (certain to detect) RPN = 100
S=1 (No effect), O=10 (Occurs 10% of the time), D=10 (not
detectable) RPN = 100
S=10 (Hazardous without warning), O=10 (Occurs 10% of the time),
D=1 (certain to detect) Criticality = 100
S=1 (No effect), O=10 (Occurs 10% of the time), D=10 (not
detectable) Criticality = 10

5/23/2011 Kathleen

17

How are you looking at your FMEA

effectiveness?

Use the process control capability index

Compare performance to a target.

How well is the process performing to expectations

Use

Internal Nonconformance metrics

Use Warranty / Field Failure Metrics

5/23/2011 Kathleen

18

Measure FMEA Effectiveness

Use Internal nonconformance data

o

Internal Nonconformances could impact the

occurrence and/or detection ratings

S = 1 (no effect), O = 10 (high occurrence rate), D = 1 (always likely to detect) = RPN = 10

S = 1 (no effect), O = 10 (high occurrence rate), D = 8 (difficult to detect) = RPN = 80

How do we do this?
Simply

be reviewing the identified failure modes listed

in the FMEA as compared to the failure mode
identified on the NCR

5/23/2011 Kathleen

19

Measure FMEA Effectiveness

How does one use warranty / field failure data to

assess FMEA effectiveness

Warranty / Field failures impact the Customers perception.

The same basic principles apply as with internal NCR
methodology
Look at the number of warranty / field failures failure modes
reported versus the number of FMEA identified failure modes
Management team should establish a goal for this KPI the
metric would report against that goal

Number if identified failure modes for the product / number of

warranty failure modes = capability

5/23/2011 Kathleen

20

Measure FMEA Effectiveness

18 Internal NCR issued analysis shows
5 identified failure modes were not listed
on the FMEA. The FMEA had 12 failure
modes identified. 5/12 = 0.42
40 missed warranty failure modes of 46
identified FMEA failure modes.
40/46=87%

5/23/2011 Kathleen

21

Actions Speak louder than Words

Dont forget the most important function of

the FMEA.
Actions that are identified and taken are more
important than a metric

5/23/2011 Kathleen

22

References
1.
2.
3.
4.
5.

Steve Pollock, Create a Simple Framework to Validate FMEA

Performance, Six Sigma Forum Magazine, August 2005.
http://www.qualitytrainingportal.com/resources/fmea/fmea_pitfalls.
htm
http://www.reliasoft.com/rnewsletter/v6i2/fmea/factors.htm
http://www.reliasoft.com/rnewsletter/v6i1/fmea/factors.htm
http://elsmar.com/forums/showthread.php?t=25760&page=2

5/23/2011 Kathleen

23