SARANGANI ENERGY

CORPORATION

Good
Morning
6/4/15

RCFA TRAINING : DAY 1: 2210141412

SARANGANI ENERGY
CORPORATION

Root Cause Failure Analysis

You Have to Know What You're Looking for If You're Going to Find It

December 3, 2014

6/4/15

Root Cause Failure Analysis Course

SARANGANI ENERGY
CORPORATION

DAY 1
Using RCFA and Making it Successful
True Causes of Plant and Equipment Failure
Risk Analysis, Reliability Growth and Failure
Elimination
Applying RCA Failure Analysis Tools and Their Use

DAY 2
The RCFA Team and The Role of the Facilitator
Introducing and using RCFA
Practice the RCFA Methods
Propagating Improvements Company-Wide

6/4/15

Introduction to RCFA and Analysis Tools

SARANGANI ENERGY
CORPORATION

Sometimes it is
very hard to spot
the real cause

Sometimes it is not

6/4/15

Root Cause Failure Analysis (RCFA)

SARANGANI ENERGY
CORPORATION

The failure event is the end result of many failed processes.

What We See
Failure

Effect Cause

Cause 1

Cause

Cause 2

Effect 2 Cause 6

What Caused It

Effect 1 Cause 4

Cause 3

Acts alone
to produce
the effect

Act together
in
combination
to produce
the effect

Cause 5

Root Cause Failure Decision Tree

But the failure started long before the failure event happened.
6/4/15

Where is the Root Cause?

SARANGANI ENERGY
CORPORATION

Process 23

Packaging

Process 10

FAILURE
R4
Process 15

Process 2

Process 11
Process 22

FAILURE

FAILURE

Process 18

Process 14

Process 20

Process 21

Product

R5

Process 14

Process 12

Process 8

Process 26

Process 17

Assembly

R3
Process 7

Process 3

Process 4

Process 16

R2

Process 13

Manufactur
e

Process 12
Process 19

Preparation

R1

Process 25

Raw
Materials

Process 9

Process 6

Process 1

Process 5

Process 24

Hundreds of activities across dozens of processes what

chance have you got?
6/4/15

What Route did Failure Take in the Pump

Set?
1
Motor

103

Pump
Fails

1
Drive
Coupling

33
1
Mech
Seal
Wet End

SARANGANI ENERGY
CORPORATION

An Internet search by the

Author for causes of
centrifugal
pump-set
failures
found
228
separate ways for the
wet-end components to
fail, 189 ways for a
mechanical seal to fail,
33 ways for the shaft
drive coupling to fail and
103 ways for the electric
motor to fail. This totals
553 ways for one common
item of plant to fail.

2
189
228

6/4/15

Parts Fail, Machines Stop; the Part is the

Evidence

SARANGANI ENERGY
CORPORATION

What
2nd bearing
sleeve

1st bearing
sleeve

Casing wear ring

6/4/15

2nd bearing bush

1st bearing
bush

1) HUMAN
FACTORS,
2) BUSINESS or
WORK
PROCESSES,
3) PHYSICAL
PROCESSES
AFFECTING
EQUIPMENT
4) LATENCY
FACTORS
caused the failure?

RCFA Cause and Effect Diagram

SARANGANI ENERGY
CORPORATION

Getting the logic

right
Develop the Cause Effect Tree

6/4/15

We Identify All Possible Cause and Effect

SARANGANI ENERGY
CORPORATION

Latent
Causes

Scientific
Causes / Effects
Incide
Business
nt
Action System
Causes
s
Investigative tools

6/4/15

3W2H
Fault Tree
Fishbone
5/7 Why
Etc...

10

Start with Scientific Cause-Effect Sequence

Roof
Fell

Foundation
Failed

Stop

Roof
Material
Failed

Stop

Column
Material
Failed

Columns
Tumble

6/4/15

Scientific Event
Sequence

Stop

Column to
Ground
Connection
Fails

SARANGANI ENERGY
CORPORATION

Columns
Tilt

Roof Moves

Trailer Hits
Roof

11

Showing OR - AND Arrangements

SARANGANI ENERGY
CORPORATION

Shows an
OR
arrangemen
t
Latent
Causes

Scientific Causes
/ Effects

Shows an
AND
arrangement

6/4/15

Incid
ent
Actio
ns

Business
System
Causes

12

No need to solve the root cause to prevent

a problem

SARANGANI ENERGY
CORPORATION

Latent
Causes

Scientific
Causes / Effects
Incide
Business
nt
Action System
Causes
s

BUT... The root causes are still

there, and may find other ways to
produce a failure

6/4/15

13

Production Rate

How RCFA Contributes To Improvement

SARANGANI ENERGY
CORPORATION

Potential Maximum Performance

RCFA
RCFA
RCFA
Expected NORMAL Operation

RCFA

Most plants would regard

this as a problem
Time

and miss this one!

6/4/15

14

The Power in RCFA

SARANGANI ENERGY
CORPORATION

Propagation
Permanent
Improvemen
t
Implementat
ion
Corrective
Action
Analysis
Investigatio
n
Evidence
Failure

6/4/15

15

Successful Uses of RCFA in Maintenance

SARANGANI ENERGY
CORPORATION

A famous example DuPont Chemicals

A
RCF
ted
Star

ERV = Equipment Replacement Value

Sourced from: Benchmarking Performance in the Mining Industry - Reliability and Maintenance as Strategy Components by
Edwin K. Jones PE, and William Holmes

6/4/15

16

RCFA for Continuous Improvement Process

SARANGANI ENERGY
CORPORATION

Continuous Improvement Cycle

RESTORE

DESIGN
PROCURE

Share
Best
Practices

OPERATE

Networks

6
Leverage
Learning

Prevent
Future
Defects

6/4/15

Symptoms of a
process with
improvement
no
cycle

CONSTRUCT

Lossess
&
Gaps
s

Asset Productivity
Improvement
4
Project

Sustain Improvement
the Gain

Root Cause
Analyze &
Analysis Prioritize Information
100
90
80
70
60
50
40
30
20
10
0

. . ..
. .

1
Collect
Data
Losses

Uptime

17

Incident/Problem

6/4/15

SARANGANI ENERGY
CORPORATION

18

18

RCFA Method

SARANGANI ENERGY
CORPORATION

Work on
Chronic or
Sporadic?

Sporadic
Describe the
Sporadic Failure
Get Info About
the Failure

People,
Records,
Orientation,
Beliefs,
Equipment

Failure
Observations
(Facts)
Hypotheses

Chronic
Choose an Area &
Define What Failure
Is
Gather Data on
all types
of Failures

Interview or FMEA

Prioritize Data
(Pareto)
Why Tree
and/or
Fault Tree

Choose a Failure
From Pareto
Describe the
Chronic Failure

Verify
Physical Cause
Human Cause
System Roots
Choose Solutions
6/4/15

Implement Solutions

Audit / Follow-up
19

What Scale of RCFA to Apply?

EQUIPMENT IMPACT

BUSINESS IMPACT

Creative Disassembly

SARANGANI ENERGY
CORPORATION

Individual persons
working on-the-job
Low cost, little time Escalate
Preventive focus
stops the many small
causes that lead to large
failures
Misses multiple causes

COMMUNITY IMPACT

Root Cause and

Effect

Risk Analysis

Team of experts in
Escalate
several meetings
High cost and time
focus on big
problems and you
keep having big

problems
Identifies wider
EFFECT OF MODERN SAFETY
SAFETYINITIATIVES
ACCIDENTS perspectives

Serious
Serious
Injuries
Injuries

Minor
MinorInjuries

30

Property
Property
Damage
Damage

Incidents
Incidents

10

60
0

Injuries

The Heinrich Accident Pyramid

6/4/15

Team of experts with

detailed FTA, failure
investigation and
reliability analysis
Problem of
catastrophic size
No stone left
unturned; the truth
comes out

EQUIPMENT
FAILURES
Serious
Failure

10
losses

Minor
Failures

6500 repairs

Process
Losses

20,000 defects

Procedural
Incidents
Source: Winston

The Failure Pyramid

Ledet,
Manufacturing
Game
20

Words and Definitions

SARANGANI ENERGY
CORPORATION

Root Cause(s)
the factor(s) that arose to initiate an eventual failure
Failure
unsatisfactory performance or outcome
Chronic Factors
ingrain, habitual
Sporadic Factors
random, intermittent
Hidden Failure
failure that is not known until the failed the item is required
Latency
the expectations, values and beliefs that cause us to behave as we do

6/4/15

21

Root Cause Failure Analysis Process

SARANGANI ENERGY
CORPORATION

RCFA fundamentals
The RCFA process is fault-tree based
Develop and implement solutions that prevent the problem everywhere

Finding the Evidence and Proof

Operating and maintenance records and history analysis
Interviews with persons involved
Creative disassembly of failed item(s)
Important to keep accurate records and history of equipment

Using RCFA in the Workplace

Requires a cross-functional team brainstorming
The 5 Whys method is simpler for the workforce if Why Tree is proven by
evidence
Needs operator and maintainer buy-in for sustained usage

Applying RCFA Recommendations

Needs Change Management methods to get buy-in of proposals
Implementation is 80% of the effort
Prove the changes actually prevent the problem

6/4/15

22

Phases of an Investigation
Phases of an RCFA)

6/4/15

(Not the same as

SARANGANI ENERGY
CORPORATION

23

Document and Describe Your RCFA Process

SARANGANI ENERGY
CORPORATION
Steps
1. Preserve and collect evidence NO BLAME, NO BLAME, NO
2. Appoint experienced facilitator BLAME... else truth disappears
andoffear
will reign.
3. Assemble a cross-functional team
persons
involved in the
incident and others knowledgeable in plant/process design
and operation
4. Cost the total impact of the incident to give power to the
investigation
5. Interview persons involved
6. Investigate the evidence for its history
7. Team brainstorms, starting at the failure, to develop a causeand-effect fault tree of all sequential steps that may have led
to the failure
8. Use the evidence and interviews to confirm the failure path
Use Hierarchy of
9. Record the findings
Controls
10. Develop the fault tree
11. Identify what can be done at various stages along the path
to prevent the failure and how they should operate
12. Communicate the findings and necessary changes companywide
6/4/15
13. Appoint a person to put the changes in place, including

24

Final Team task is recommendations and

report

SARANGANI ENERGY
CORPORATION

The effort expended during an RCFA is justifiable only because it

leads to recommendations which, if implemented, will eliminate or
minimise repeat occurrences in the future.

In preparing the report there may be some unpalatable truths but

these must be appropriately stated.
adverse events are usually the result of human errors
these errors must be accepted as system flaws, not character flaws

6/4/15

The report will contain all the documents produced during the
investigation and a summary Executive Report at the front. This
will enable senior, and possibly non-technical management, to
read the recommendations and appreciate the value of the RCFA
procedures.

25

Tools to Use During Equipment Failure

RCFA

FA
C
R

es
c
o
Pr

SARANGANI ENERGY
CORPORATION

Project
Manageme
Implementation
nt
Evaluation Table
Corrective Action
Affinity Diagrams
Relationship Digraph
and

Analysis
Identification

Brainstorming
ls Brain Writing
o
To
Is-Is Not Table
A
Evidence and
F
Why Tree (Fault Tree Analysis)
RC
Proof
5/7 Whys (to test Why Tree)

Flowchart
Failure

Fishbone Diagram3W2H
Timeline Plots
Interviews
Distribution Histograms
Protect Equipment/Parts
Pareto Charts
Understand
Documents, Records, Diagrams
FMEA
interactions and the
Creative Disassembly of Parts
Investigation and
Understanding

Expert Investigation

6/4/15

human element

Understand the physics

science key factors
progression
26

Overview of Basic Data Collection Methods

6/4/15

SARANGANI ENERGY
CORPORATION

27

Machine Failure Data Collection and

Analysis

SARANGANI ENERGY
CORPORATION

Initial Data Collection involves:

Data Analysis usually involves:

Collecting
failed parts.

Metallurgical analysis of failed

parts.

and

photographing

Data
Colle
ction

Photographing and taking note of

the
physical
conditions
and
circumstances surrounding the
failure.

Data
Analy
sis

Question persons present before,

during and after the failure.
Obtaining
historical
vibration,
thickness and other pertinent data.
Obtaining assembly and
pertinent drawings.

other

Determining operating conditions,

such as RPM and pressure,
leading up to and at the time of
failure.

Source: Latino, C. J., The Plant Managers Guide

to the Reliability Approach, Allied Chemicals 1980

6/4/15

No

Enou
gh
Inform
ation?
Identi
fy the
Root
Caus
e

Theoretical calculations of loads

and
stresses
using
design
drawings and known operating
conditions, if practicable.
Analysis of historical condition
monitoring records and other
pertinent
data
for
possible
contribution to the root cause.
If necessary conduct on-site
monitoring, measurement and data
collection of the system and
components that failed.
Root Cause
Construct a decision tree of the
failure incident using factual
information to trace back to the
root cause.

28

Data Gathering Following an Incident

SARANGANI ENERGY
CORPORATION

Collect Evidence
Get to the scene ASAP, take a video/camera
Pay extra attention to the things that will change later
Colours, temperatures, smells, liquid spills, cloth, wire, rope, unstable objects
Dont move anything, including paperwork, until recorded
Position, orientation, condition
Collect paperwork and documents
Logs, instructions, data, records
Isolate the scene so you can return and look again later, if not get some else to independently
record the evidence, video tape the scene
Collect Information and Record it
Ask people early, before memories fade or are influenced. Watch for additions, omissions and
distortions caused by fear of blame, retribution and desire to protect others.
What Happened?
How did it happen?
Sequence of events
Who or what was involved? people, machines, information systems, sensors, structures, etc.
Where did it happen?
When did it happen?
What has changed?
6/4/15

29

Protecting and Collecting the Evidence

SARANGANI ENERGY
CORPORATION

1.

People evidence talk to people working at time of failure

to record all their thoughts and ideas as to the cause.

2.

Position of controls note position of switches, settings,

dials and instruments.

3.

Position of People find out where everyone was standing

as well as what they were doing at time of failure.

4.

Parts condition record position of failed parts, examine

the parts in detail.

5.

Paper trails gather all manuals, procedures, design

documents, operating and maintenance history.

6/4/15

30

RCFA Interviews

SARANGANI ENERGY
CORPORATION

Answer these
questions using a
structured form

6/4/15

31

Data Analysis Tools to Release Information

XXXX

SARANGANI ENERGY
CORPORATION

X X X X

Bunched

Regular
repeat
Date/Tim
e

Time Log

Is/ Is Not table to Compare Operation

Problem:
high fuel consumption

Our
car

Neighbour
s car

Relatives
car

Got the problem?

Yes

No

No

Age

10
yrs

1 yr

2 yr

Petrol smell

Yes

No

No

Driven by son

Yes

No

Yes

Colour

Blue

Red

Red

All sedans
Similarities
All Camry

Use Visual Analysis

Tools for Patterns and
Trends
6/4/15

Differences

32

Fault Tree Analysis

SARANGANI ENERGY
CORPORATION

HHL
HL

6/4/15

33

Fault Tree Analysis

6/4/15

SARANGANI ENERGY
CORPORATION

34

Start with a Flowchart to Explain the Correct Process

Describe the design intent of

the proper and correct operation
of the equipment/plant/work
instruction/etc
Make required outcomes for
ideal performance of each
step clear e.g. fully closed,
detects overfill

HHL
HL

Tank Level
Falls

Y Sensor
Detects
Level

Valve B
Fully
Opens

Tank Level
Refills to Set
Point

ep 1) Start with descriptive flowchart

Overfill

Valve B
Fully
Closes

Water
Flows to
Plant

Valve B
Fully
Closes

High High
Level
Alarm

X Sensor
Detects
Overfill
6/4/15

SARANGANI ENERGY
CORPORATION

35

Activity 1 Fault Tree Analysis (FTA) of a

failure event

6/4/15

SARANGANI ENERGY
CORPORATION

36

Analyzing with the Fishbone Diagram

Measurements

Materials

SARANGANI ENERGY
CORPORATION

People

premium

Octane
Rating

accuracy

Odometer
readability

winter

Traffic
Level
light

Windy

summer
smooth

rough

old

fast

Towing

Gas
Mileage

Engine
Air
Conditioning

Speed
slow

Open
Windows

Gas Filter

city

Road
Surface
dry

Environment

spring

fall

Age

Gas
Quality

Season

moderate

young
women

readability

Temperature

6/4/15

Gender

high

Gas Gage

Tire Pressure

heavy

regular

accuracy

Experience

men

high

Driving
Conditions

wet

Tire
Pressure

highway

Methods

low

Machines

37

Activity 2 Do Fishbone on the Valve

Failure

Measurement

Materials

SARANGANI ENERGY
CORPORATION

People

Fill
Valve
Failed
Open

Environment

6/4/15

Method

Machines

38

Is Is Not Table pinpoints problem area

6/4/15

SARANGANI ENERGY
CORPORATION

39

Why Tree for a Stalled Car

6/4/15

SARANGANI ENERGY
CORPORATION

40

Stalled Car Engine Is / Is Not Table

6/4/15

SARANGANI ENERGY
CORPORATION

41

Activity 3 Develop Is / Is Not Table

failure event you experienced)

6/4/15

(for

SARANGANI ENERGY
CORPORATION

42

Maintenance Data Analysis with Trend

Plots

SARANGANI ENERGY
CORPORATION

Time Lines
Proposition
The simple Time Line is the most powerful data analysis tool (which is
so simple it is generally overlooked).
It is difficult to describe many events, collect relevant plant information,
find inconsistencies in information sources, and make sound
judgements without such a representation of relevant events.
Review of history on a Time Line is a mandatory first step.
35

40

3
0

50

New
materia
l

5
0

40

5
0

200

Precision
Maintenance

250

Time between Failures Days

Time Series Plot Showing Increasing Time between Failures for a

Component
6/4/15

43

Separate Failures and Problems into Modes

Top line indicates

dates of all
incidents. Lower
lines show dates of
each fail mode

Timeline

f
All Failures

f
Mode 1 Failures
Mode 2 Failures

SARANGANI ENERGY
CORPORATION

Cluster of failures indicating a

second mode of failure is present.
This distribution contains multiple
failure causes.
MTTF is consistent

f f f

Random Failure

Infant Mortality

f
f

f f
Date

Time Series Plot for Tracking Changes in Frequency of

Failures

6/4/15

44

Timeline Identifies Changed Effects

SARANGANI ENERGY
CORPORATION

Hours required to de-scale process. The failure definition was descale outage

Anti-corrosive chemical changed on this date and less

scale produced

Looking at this, a fair person would accept that due to the change it is likely
that there will be less de-scaling downtime in future and (considering this
issue only) the Facility throughput would improve
The future will not be the same as the past. Without the timeline an analyst
using MTBF to predict outage would produce erroneous predictions of
Facility future performance and bad recommendations
6/4/15

45

Frequency of Failures and Problems

6/4/15

SARANGANI ENERGY
CORPORATION

46

Frequency Histograms

SARANGANI ENERGY
CORPORATION

Requires more data. X (time) axis broken into time or usage

Piling the Dead - Tyre Replacement/ Failureintervals
Failures
a series
of histograms
Changed normal wear
Changedstacked
side-wall damageas in
Changed
tire defective
Changed alignment accelerated wear
Failed side-wall damage
Changed rock damage

Failed due to other reasons

Changed unbalance accelerated wear

2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
10,000
11,000
12,000
13,000
14,000
15,000
16,000
17,000
18,000
19,000
20,000
21,000
22,000
23,000
24,000
25,000
26,000
27,000
28,000
29,000
30,000
31,000
32,000
33,000
34,000
35,000
36,000
37,000
38,000
39,000
40,000
41,000
42,000
43,000
44,000
45,000
46,000
47,000
48,000
49,000
50,000

Tyre Failures Frequency

Stack at Age of Death

0 to 1000Km

Km Range

Number of Failure Events in Period

Failued due to wear

Failed from puncture

Note X axis is km not Time The term time can be

replaced by another measure of exposure to degradation
processes.
Note
use of colours Failures have been coded according to
Cause of Failure Very important as not all causes are wear
related.
6/4/15

47

The Double Pareto Chart Method

SARANGANI ENERGY
CORPORATION

Used to Identify Equipment and Problem Priorities for RCFA

Pareto worst
equipment

First Pareto in
Order of
Problem
Equipment

6/4/15

Conduct RCFA on
the worst problems

Second Pareto
in Order of
Equipment
Problems

48

Check Assumptions and Extra Clues to

Causes

SARANGANI ENERGY
CORPORATION

Ask People with Knowledge and Experience

Experience of similar processes and/or products
Relevant specialist technical knowledge
Experience with solving similar problems in the past
Suppliers of equipment , materials or services
Video process in action and discuss with those who do the job. Review video of the process at different
times.
Ask customers (with care for the relationship ) of their experiences when did problems start, when
does it happen, when was it noticed, etc?

Look for the Hidden Assumptions This one is and easy trap to fall into, and we all do it

Is it being assumed that

Only one possible way for a thing to arise
One thing must follow another in a particular way
Two things are the same when they are actually different? spare parts, methods of work, process plants
The information is correct? meters are accurate, figures were written correctly, time was right
Things that are usually related/co-dependant were in step? pressure/temperature, time of day/rest break
Cause is in one half of the problem when it could be in the other half? mating parts, leakage and not excess
flow
Simulate the experience act out the problem as accurately as possible looking for assumptions being made

Expose Fixed Ideas

Use the three steps to prove the cause Ego and
ownership can make you stick to
step one. When this
1. It must explain the problem convincingly happens say, Yes, that sounds convincing, now lets find
2. It appears and disappears with the problem
the evidence coinciding with history that turns the
3. It can switch the problem on and off problem on and off. Invite a third party to listen to your story of what
you think happened.
6/4/15

49

Operating and Maintenance Records and

Analysis
Operator log books

SARANGANI ENERGY
CORPORATION

Work order history and

records
Design and Construction
records
Enter into spreadsheets
Give logical categories to the information

e.g.
breakdown, preventive maintenance, operator error, corrosion,
assembly error, etc

Look for repetition and frequency of

issues/problems
Look at the comments written about incidents
Look for preceding factors

6/4/15

50

Importance of Keeping Accurate Records

and History

SARANGANI ENERGY
CORPORATION

Records:

Provide evidence of changes

Indicate how things were in the past
Collect trends and variance
Prove things were done in certain ways

Keeping Useful Engineering Records:

Develop a record numbering/sub-numbering system

recognisable to all
Collect information by operating area, equipment,
assembly, part no., date
Decide and document the specific recorded details to be
kept, suggest:

6/4/15

Keep original equipment manufacturers data and as-built project

detail
Keep history of failures, investigations and modifications
Keep history of calibrations, tests, performance reports, alignments,
etc

51

Case Study 1 Equipment Failure RCFA

Steps

SARANGANI ENERGY
CORPORATION

Note: This example is used in Activity 4 Guided RCFA

1 Describe story of the actual failure.
2 - Define the physical/scientific phenomena of the failure.
3 - Organize details of the failure by using Cause and Effect Tree tools and testing
the evidence with '3W2H questions (with what, when, where, how, how much). Teamup and trace every possible cause of the failure using post-it notes on a wall.
4 - Work as a team and organise the cause event tree in logical order, respecting
and applying each others expertise and knowledge, rather than individually.
5 - Verify all logical causes and eliminate all illogical causes with 5/7 Why and
3W2H complete answers full of facts (with what, when, where, how, how much) tools.
6 - If determined that a cause was human error, separate that cause from the
physical causes (beware our natural tendency to bias and jumping-toconclusions).
7 - Look for and identify the latent causes and human factor frailties.
8 Develop preferred recommendations and the RCFA report.
6/4/15

52

The Scientific Causes of Equipment Failure

6/4/15

SARANGANI ENERGY
CORPORATION

53

Common Wrongs Humans Do To Machines

6/4/15

SARANGANI ENERGY
CORPORATION

54

Strength of Materials Limitations

6/4/15

SARANGANI ENERGY
CORPORATION

55

Cause of Aging Failures

6/4/15

SARANGANI ENERGY
CORPORATION

56

Know the Stress Limits of Your Parts

SARANGANI ENERGY
CORPORATION

Failure

Failure

10,000
cycles at
this stress
level

1,000,000
cycles at this
stress level

Limited
life at this
stress
level for
nonferrous

Infinite
cycles at
this stress
level for
steel

We must know what our equipment parts are made of and prevent
high stress in those with infinite life but replace those of finite life
before they fail.
6/4/15

57

Stress and Fatigue are Optional

Degradation Management)

6/4/15

(Use

SARANGANI ENERGY
CORPORATION

58

Why You Need Defect Elimination and

Failure Prevention

SARANGANI ENERGY
CORPORATION

Business-wide
Errors
Design Errors
Fabrication
Errors
Operating Errors

Installation
Errors
Management
Errors
Workmanship Errors

Your Operation is a Bucket for

Collecting Defects.

Defect Creation

6/4/15

59

Problems Waste Time, Money and

Resources

SARANGANI ENERGY
CORPORATION

Business-wide
Errors
Design Errors
Fabrication
Errors
Operating
Errors

Maintenance

Installation Errors
Management
Errors
Workmanship
Errors

Business
Systems

Business Systems and Maintenance Help You

Handle Failures, and the Problems They
Bring.

Defect Management

6/4/15

60

Eliminate Defects to Prevent Problems

SARANGANI ENERGY
CORPORATION

Business-wide
Errors
Design
Errors
Fabrication
Errors
Operating
Errors

Maintena
nce

Installation Errors
Management
Errors
Workmanship
Errors

Business
Systems

The Best Answer is to REDUCE the Numbers

of Defects and so REDUCE the Need for
Managing Failures and Problems!

Defect Elimination

6/4/15

61

Variability and Risk Across the Life Cycle

SARANGANI ENERGY
CORPORATION

Every process across the life cycle will create many defects if the 3Ts are not used.
ManagementEngineering

Specif
y

Desig
n

Supply

Bu
y

Defect
and
Failure
Cost
Surge

st
er
Di
sa
to
Pa
th

10
6500

Repairs

20,000

Defect Modes

The Failure Pyramid

6/4/15

Insta
ll

Operations

Startup

Plant Uptime and

Throughput
Serious
Failure
Losses

Stor
e

Contractor

Product
Higher Unit
Cost, Poor
Quality and
Delayed
Delivery

Maintenance

Operat
e

Maintai
n

Introduced
defects

Variability in each
process
causes
defects
which
sometimes
progress to failure.

62

The Odds are Against Doing it Right!

SARANGANI ENERGY
CORPORATION

Only one way

to
disassemble
40,000+
ways to
incorrectly
reassemble!

6/4/15

63

Human Factors the limitations of People

SARANGANI ENERGY
CORPORATION

Psychological
Physical
Experience
Size
Knowledge
Gender
Training
Age
Attitude
Strength
The five senses Emotional state
Physiological
Psychosocial
Health
Interpersonal
Nutrition
relations
Lifestyle
Ability to
Alertness/fatigue
communicate
Chemical dependency
Empathy

6/4/15

Leadership
64

12 Most Common Causes for Human Errors

SARANGANI ENERGY
CORPORATION

Eliminate these
causes and you have
conquered most
human errors
Talk about each of
these with your
people
What are the causes
in your company?
What are the
corrective actions?
6/4/15

65

Human Factors are A Prime Focus

6/4/15

SARANGANI ENERGY
CORPORATION

66

Reliability of Human Dependant Processes

6/4/15

SARANGANI ENERGY
CORPORATION

67

Fundamentally Finding the Cause Behind

the Cause

SARANGANI ENERGY
CORPORATION

"Every defect is a treasure, if the

company can uncover its cause
and work to prevent it across the
corporation."
- Kiichiro Toyoda, founder of Toyota
This is NOT the
real Cause, it is
an effect!

Perhaps not so much why did it

happen?
But, more

Why was it able to happen?

This information is publically available from C. Robert Nelms, Failsafe Network, Inc.
www.failsafe-network.com
6/4/15

68

Cause Behind the Cause - Latent Thoughts

SARANGANI ENERGY
CORPORATION

Psychology definition of LATENT:

Present and accessible in the
unconscious mind but not
consciously expressed.

What thoughts did people have at

the time that contributed to the
incident?

6/4/15

69

The Real Cause?... Latent Values

What caused people to think like
that?

SARANGANI ENERGY
CORPORATION

Focus on LATENCY
The Organizational attitudes,
beliefs, and
assumptions accepted by all.
What allowed people to think
like that?

Two very basic truths

1. Adverse industrial events tend to be the
result of human errors that queue up in a
particular sequence.
2. Error must be accepted as a system
flaw, not a character flaw. Ask, Why did
the system let me fail?
6/4/15

70

What is a High Potential Incident?

SARANGANI ENERGY
CORPORATION

high potential incident means an incident not causing

loss or damage but, under different circumstances would
result in an accident.

HAZARD

T
H
R
E
A
T
S

Event

THREAT
THREAT BARRIERS
BARRIERS

6/4/15

C
O
N
S
E
Q
U
E
N
C
E
S

ACTUAL
CONSEQUENCE
S

ESCALATION
ESCALATION BARRIERS
BARRIERS

71

Barrier Analysis and Identification

SARANGANI ENERGY
CORPORATION

In a well run organisation there should be many barriers against major problems.
These can be physical barriers or management systems.
What barriers
should be in
place?

Barrier: Independent
check on critical
equipment jobs

What is the
likely cause of
holes in the
barriers?

Hazard

Barrier: Skilled
tradesman
employed

Consequences

-answer two questions

6/4/15

Hole: The maintenance

fitter installed the
wrong valve
Why: Lack of
training?
Poor work
procedures?
72

How the Swiss Cheese Slices Lined Up for

the Titanic
Worst
iceberg
season in
years

Moonless,
still night,
no
reflection

Sailing at
high speed

Berg seen,
ship turns, but
slow response
from small
rudder

SARANGANI ENERGY
CORPORATION

Experienced
Captain not
on bridge

Engines
stopped,
reversed,
rudder
becomes
ineffective

Hazard

Consequences

What barriers should

be in place?

What is the likely cause of

holes in the barriers?

Answer the two questions?

6/4/15

73

The 5/7 Whys Method to Confirm the Path

SARANGANI ENERGY
CORPORATION

The 5/7 Whys contributes to the success of cause and

effects analysis by asking you to use real evidence to
prove what really happened.
6/4/15

74

Using 5/7 Whys to Confirm Failure Path

SARANGANI ENERGY
CORPORATION

Latent
Causes

Scientific
Causes / Effects
Incide
Business
nt
Action System
Causes
s

Incid
ent
Why
1

Why
2

Why
3
Why
4

ollow the evidence.

6/4/15

Why
5

Why
6

Why
7

Remember the Latency

Issues

75

Why Tree of a Failure Incident

SARANGANI ENERGY
CORPORATION

His late arrival at work caused an important clients

deliver to be delayed and the company suffered a
$25,000 penalty payment.
Penalty Payment
Late Despatch
Manufacturin
g Completed

Storeman
Packages
Items

Stop

Storeman
Late to Work

This employees after-hours

behaviours caused a business
process failure, which unless
addressed, it will repeat again in
future.
6/4/15

Car Stopped
at Side of
Road
Car Ran Out
of Fuel
No Fuel in
Fuel Tank
76

Why Tree of a Failure Incident - Latent

Causes

SARANGANI ENERGY
CORPORATION

No Fuel in
Fuel Tank
No Money to
Buy Fuel
Most RCA teams would stop
here, thinking they had found
the root cause

Lost the
Money in a
Card Game
I Bet All My
Money
I Want to Be
with Friends

Latent values and

attitudes

6/4/15

I Often Lose
at Cards

I Spent all I
Had in My
Wallet
Easy come
easy go
I Live from
Day to Day
I Have a
Limited
Income

77

The 5/7 Whys and 3W2H Form

SARANGANI ENERGY
CORPORATION

Why Tree Questionnaire Form

Team Members:

Date:

Problem Statement: On the way to work your car stopped in the middle
of the a
road.
Give
complete

answers full of facts

Estimated DAFT Cost: Taxi fare x 2 = $50, Lost 4 hours pay = $100. Contract Penalty Payment. Plus possible loss of clients.
Recommended Solution: Carry a credit card to access money when needed.
Latent Issues: Gambling away all the money shows a lack of personal control and responsibility of money.
(with what, when, where, how, and how much)

Evidence

1. Why did the car stop?

Because it ran out of gas in a back street

on the way to work

Car stopped and standing at

side of road

2. Why did gas run out?

Because I didn't put any gas into the car

on my way to work this morning.

Fuel gauge showed empty

Why Questions

3. Why didn't you buy gas this

morning?
4. Why didn't you have any
money?
5. Why did you lose your
money in last night's poker
game?

3W2H Answers

Because I didn't have any money on me

to buy petrol.
Because last night I lost it in a poker
game, I played with friends at my
buddys house.
Because I am not good at bluffing
when I don't have a good poker hand
and the other players jack-up the bets.

Solution

Wallet is empty of money

Keep a credit card in the

wallet

Poker game is held every

Tuesday night

Stop going to the game

Have lost money in many other Become better at

poker games
bluffing

6.
7.

6/4/15

78

Activity 4 Complete a 5-Why Table

SARANGANI ENERGY
CORPORATION

5 Why Questionnaire
Team Members:

Date:

Problem Statement: On your way home from work your car stopped in the middle of the road.
Estimated DAFT Cost:
Recommended Solution: Carry a credit card to access money when needed.
Latent Issues: Lack of personal control over money is concerning.
Why Questions

3W2H Answers
(with what, when, where, how, how much)

Evidence

1. Why did the car stop?

Because it ran out of gas

Car stopped at side of road

2. Why did gas run?

Because I didn't buy any gas on my

way to work.

Fuel gauge showed empty

3. Why didn't you buy gas this

morning?

Because I didn't have any money.

Wallet is empty of money

4. Why didn't you have any money?

Because I lost it in a poker game last

night.

Poker game is held every Tuesday

night

5. Why did you lose your money in

last night's poker game?

Because I'm not good at bluffing when

I don't have a good hand.

Have lost money in many other poker

games

Solution

Keep a credit card in the wallet

Stop going to the game

Become better at bluffing

6.

6/4/15

79

Failure Modes and Functional Loss

SARANGANI ENERGY
CORPORATION

A failure is any unwanted or disappointing behaviour of a product.

A failure mode is the effect by which a failure is observed. Failure

modes can be electrical (open or short circuit, stuck at high),
physical (loss of speed, excessive noise), or functional (loss of
power gain, communication loss, high error level).

Failure mechanism refers to the processes by which the failure

modes are induced. It includes physical, mechanical, electrical,
chemical, or other processes and their combinations. Knowledge
of failure mechanism provides insight into the conditions that
precipitate failures.

A failure site describes the physical location where the failure

mechanism is observed to occur, and is often the location of the
highest stresses and lowest strengths.

We can foretell what parts are going to cause trouble by doing experiments,
conducting tests and using past failure history of similar parts. If we can
predict what will go wrong, and the conditions that cause it to happen, we
can design maintenance and operational loading strategies to give
maximum part life.
6/4/15

80

Failure Modes and Evidence of Failure

SARANGANI ENERGY
CORPORATION

Failure

Failure Mode

Failure
Mechanism

Failure Site

Car does not start

Starter Motor
does not run

Corroded relay
contacts

Main contact of
starter relay

Toy has faded

colour

Colour changes
from red to pink

Accumulation of
high UV dose

Red plastic leg

Hard disk failure

Computer has no
access to hard
disk

Hard disk address

is 11 instead of 12

Line 87 in the hard

disk driver
software

Once this is known we put strategies and practices into place to; 1)
Design-out the failure, 2) prevent the failure, 3) monitor the failure
mode, 4) replace before failure and 5) prevent the conditions causing
failure.
6/4/15

81

Top-Down Block Diagram Analysis

Start at the top level &

draw the process as a
block diagram

Take each item and

draw its process block
diagram

Only go to a lower level

if there is too much
complexity to analyse
effectively

For the selected

analysis level rate the
criticality of the item

6/4/15

SARANGANI ENERGY
CORPORATION

82

ENERGY
Failure Modes: What You See/Hear When it FailsSARANGANI
CORPORATION

Example of an expanded list of failure modes

1

Cracked/fractures

11

Fails to stop

21

Binding/jamming

31

Burned

Distorted

12

Fails to start

22

Loose

32

Collapsed

Undersize

13

Corroded

23

Incorrect
adjustment

33

Overloaded

Oversize

14

Contaminated

24

Seized

34

Omitted

Fails to open

15

Intermittent
operation

25

Worn

35

Incorrect assembly

Fails to close

16

Open circuit

26

Sticking

36

Scored

Fails open

17

Short circuit

27

Overheated

37

Noisy

Fails Closed

18

Out of tolerance
(drifted)

28

False response

38

Arcing

Internal leakage

19

Fails to operate

29

Displaced

39

Unstable

10

External leakage

20

Operates
prematurely

30

Delayed
operation

40

Chafed

Source Table 2 BS 5760

6/4/15

83

Failure Mode and Causes Analysis

Mounting failure
Improper Lubrication
Physical Damage
Overload
Excessive Wear
Looseness
Excessive Movement of Cylinder

Cylinder barrel failure

Leaking Barrel Seal
Physical Damage
Environment heat/dirt
Excessive Wear

SARANGANI ENERGY
CORPORATION

Cylinder seal failure

Improper Lubrication
Physical Damage
Age/deterioration
Excessive Wear
Excessive Piston/rod movement

Piston rod failure

Piston failure
Excessive Wear
Age/deterioration
Excessive Load/impact
6/4/15

Over Extension
Fatigue/age
Connection failure
Excessive Lateral Movement
Loose Hydraulic Fittings
Damaged Hydraulic Fittings Physical Damage
Damaged or Fatigued Hose
84

Hidden Failures

SARANGANI ENERGY
CORPORATION

Failures of equipment that either dont get used

regularly or only get used when something goes
wrong
Standby equipment eg. a standby pump
Protection equipment eg. Fire extinguisher, 2nd safety limit
on a machine movement, pressure relief valves

You typically only know these items have already

failed when you really need them

Carry out Failure Finding tasks. These are simple

function checks e.g. changeover to standby pump

6/4/15

85

Failure Mode and Effects Analysis (FMEA)

SARANGANI ENERGY
CORPORATION

Water
In

FM

TG

PS

Heat Exch

Turbocharger Oil
Cooling System

Water
Out

RCM

Engine
Sump

FMEA
Maintainable Item

Maintenance Actions

Operating Unit

Turbocharger
Lube Oil Pump

Bearing Seizes

Total Stoppage

Impeller/Casing
Wear
Coupling Shears

No Immediate
Impact
Total Stoppage

Mech Seal Leaks

No Immediate
Impact

FAILURE MODE

6/4/15

FAILURE
EFFECT

Oil Analysis,
Vibration
Monitor Flow Rate
Look for Wear &
Lube
Look for leaks

CM
Watch
Keeping
PM
PM

OPS/MAINT
ACTIONS
86

Failure Mode Effects Analysis (FMEA)

Worksheet

6/4/15

SARANGANI ENERGY
CORPORATION

87

Activity 5 FMEA

SARANGANI ENERGY
CORPORATION

Do FMEA and identify failure modes and functional failures in a bearing

Overstress failure mechanisms when a
single stress excursion exceeds
strength
Mechanical
Fracture
Buckling
Yielding
Thermal
Melting

6/4/15

Electrical
Fused or shorted wires
Electrostatic discharge
Electrical overstress
Physical/Chemical
Crystal lattice attack

Wear-out failure mechanisms when

accumulated damage exceeds
endurance
Mechanical
Fatigue
Creep
Corrosion

Electrical
Leakage current
Threshold Voltage Shift
Metal migration

Thermal
Elasticity degrade

Physical/Chemical
Depolymerisation

88