Professional Documents
Culture Documents
Title
VI-404842-TM
Version
Author
Issue Date
1
Michael Shuff
05 Aug 2015
Page 1
Summary
The new version of the ISO 9001:2015 standard is scheduled for final publication on September
23rd 2015. One of the new requirements is to show evidence of risk-based thinking (RBT) in the
quality management system. How do you do that? How are auditors likely to respond to the new
challenges that ISO 9001:2015 brings? How do you produce documented evidence of risk-based
thinking?
Although ISO 9001:2015 does not call for formal methods of risk management, it is likely that
anyone trying to understand RBT may turn to ISO 31000 and the list of risk assessment techniques
in particular. However, this is not as easy as it sounds. There are many techniques to choose from
and many may not be applicable to the sectors that ISO 9001 serves.
This white paper has two major sections. The first part provides a primer on many of the ISO
31000 risk assessment techniques and considers their applicability to quality management. The
second part provides a six-step methodology that you can follow to deliver evidence of a risk
based approach to quality. It is a practical methodology that is specific on inputs / outputs, and
what you need to do in-between. Several example templates are provided that could form the
basis for your documented information.
Page 2
2.1.1
Checklists ........................................................................................................................................................ 11
2.1.2
2.2
2.2.1
2.2.2
What can we learn from ISO 31000 risk assessment processes? .................................................................... 14
2.2.3
2.3
2.3.1
2.3.2
2.3.3
2.4
2.4.1
2.4.2
2.4.3
2.4.4
2.4.5
2.4.6
2.4.7
2.4.8
2.5
2.5.1
2.5.2
Page 3
2.5.3
2.5.4
HACCP ............................................................................................................................................................. 26
2.6
2.6.1
2.6.2
2.7
2.7.1
2.7.2
2.7.3
Risk based thinking is the new 'preventive actions' for QMS ................................................................................ 32
3.1.1
3.1.2
What actions are required to plan for risks and opportunities? ..................................................................... 34
3.2
3.3
3.3.1
3.3.2
3.3.3
3.3.4
3.4
3.4.1
3.5
3.5.1
3.5.2
3.5.3
3.5.4
Summary: ........................................................................................................................................................ 46
3.6
Page 4
3.6.1
3.6.2
What is the value of the Semi-Quantitative approach in Step 4, following the Qualitative Assessment
conducted in Step 3? ...................................................................................................................................................... 48
3.7
3.7.1
3.8
4
4.2
Conclusion ............................................................................................................................................................. 57
Page 5
Page 6
What is acceptable?
What is unacceptable?
Then...
However, this list presupposes that you have identified risks and opportunities.
So if you have not done so yet, how do you approach risk identification in your context?
Read on...
Will ISO 31000:2009 help in taking a 'risk-based approach' to the quality management system,
component processes and activities?
Short answer: it can do, depending on your organization's context.
The ISO 9001 DIS says that ISO 31000 provides guidelines on formal risk management, which can be
appropriate in certain organizational contexts.
Those working for large, indeed global entities understand this. They have long since adopted risk
management methodologies and have risk managers on their team who are familiar with ISO 31000.
But what is ISO 31000 attempting to achieve, and is it relevant to the majority of organizations that
are trying to gain or transition to ISO 9001?
ISO 31000 describes an "overall approach to risk management, not just risk analysis or risk
assessment. It deals with the links between risk management process and both strategic direction
and day to day actions and treatments."1 This on the face of it sounds an ideal recipe for risk-based
thinking. However, pick up the Standard and read it and this thought is quickly dispelled because ISO
Project risk management guidelines: managing risk with ISO 31000 and IEC 62198, Dale F Cooper, et al, Wiley, 2014
Page 7
31000 takes a generic approach that has to be developed - in considerable detail - to be useful in a
given context.
Great for the Strategic aims of the senior management, but not of any great value to the 'poor
bloody infantry' of quality managers out there.
Perhaps the first (and frustrating) conclusion you will come to, having spent at least 120 ($180) on
your personal copy is that you also need to buy ISO.IEC 31010:2009 Risk management Risk
assessment techniques.
Therefore, your boss says, "OK, buy the one you actually need, but don't come back to me asking for
more. We've got by without 'risk-based thinking' in the past [insert number of years or decades];
surely we can do so this time?" You thank her or him for authorizing the purchase.
The PDF arrives on your computer. You open it. There are 92 pages, 6 of which in Annex A are a
comparison of risk assessment techniques (some useful tables here) before you arrive at Annex B,
consisting of 61 pages describing the 31 risk assessment techniques. These seem suited for the kind
of people who enjoyed Mathematics (and Statistics especially) at school, but who may not be that
interested in helping you to design effective quality processes.
Yes, there is a worthy (absorbing even?) preamble about risk assessment concepts and processes.
There also a Clause describing how to select techniques for risk assessment, this starts with the valid
advice:
Risk assessment may be undertaken in varying degrees of depth and detail and using one or
many methods ranging from simple to complex. The form of assessment and its output
should be consistent with the risk criteria developed as part of establishing the context.
[Clause 6.2]
There is no point in making life more complicated than it needs to be; thus:
In general, suitable techniques should exhibit the following characteristics:
Great!
By now, you are probably fired up with the possibility of finding a suitable risk assessment technique
that fits the context of your organization and its quality management system. You cannot wait to get
started on the job.
You turn to...
Annex A
(informative)
Comparison of risk assessment techniques
Page 8
You quickly realize there are more risk assessment techniques than you thought existed, and even a
cursory reading suggests that some are complex. Notably the ones that are strongly applicable to
each step of the full risk assessment process; specifically:
risk identification;
risk analysis consequence analysis;
risk analysis qualitative, semi-quantitative or quantitative probability estimation;
risk analysis assessing the effectiveness of any existing controls;
risk analysis estimation the level of risk;
risk evaluation.
Below is the list of the 31 tools. Depending on the industry you are working in, you will almost
certainly recognise at least some of them, even if you have not actually used any of the techniques
to assess risk.
Tools used for risk assessment
1. Brainstorming
2. Structured or semi-structured interviews
3. Delphi
4. Check-lists
5. Primary hazard analysis
6. Hazard and operability studies (HAZOP)
7. Hazard Analysis and Critical Control Points (HACCP)
8. Environmental risk assessment
9. Structure What if? (SWIFT)
10. Scenario analysis
11. Business impact analysis
12. Root cause analysis
13. Failure mode effect analysis
14. Fault tree analysis
15. Event tree analysis
16. Cause and consequence analysis
Page 9
Page 10
Page 11
Page 12
When planning for the quality management system, ISO 9001:2015 requires organizations to
consider the issues referred to in 4.1 [Understanding the organization and its context] and the
requirements referred to in 4.2 [Understanding the needs and expectations of interested parties]
and determine the risks and opportunities that need to be addressed, in order to:
a) give assurance that the quality management system can achieve its intended
result(s)
b) prevent, or reduce, undesired effects
c) achieve continual improvement.
We should integrate and implement the actions into the organization's quality management system
processes (see clause 4.4) and evaluate their effectiveness.
Brainstorming as a technique could be particularly useful when, for example, identifying risks of new
technology where there is no data or where novel solutions to problems are needed. To quote ISO
31010 "...it encourages imagination which helps identify new risks and novel solutions". However, it
is not applicable to risk analysis tasks of consequence, probability or level of risk. It therefore has its
limitations and along with the 'Look-Up Methods' of Checklists and Primary hazard analysis, and
most of the 'Supporting Methods' of structured interviews, Delphi technique, SWIFT (Structured
"what if") and, it does not provide any quantitative output - although this is not a requirement of ISO
9001.
[Note: in the section 'Supporting Methods', Human reliability analysis (HRA), which deals with the
impact of humans on system performance and can be used to evaluate human error influences on
the system, is able to provide quantitative output and is 'strongly applicable' to risk analysis and
'applicable' to risk evaluation - see Table A.1 in ISO 31010.]
However, before we get bogged down in too much detail with regard to the other Supporting
Methods, Scenario Analysis, Function Analysis, Controls Assessment and Statistical Methods, we
should ask what are we trying to achieve here, and how will any of these assessment tools help?
Let us take a step back.
If we were considering risks in relation to a quality management system and its associated
processes, we would be asking the following questions:
1. What are the risks associated with the organization's context and objectives - and why
does each risk occur? [identifying the risk and the reason for its occurrence].
2. What would be the likely negative consequences of process, product, service or system
nonconformities? [consequences if the risk occurs].
3. How likely is it that the organization will deliver nonconforming products and services in
relation to the risks we have identified? [probability of the risk occurring].
There are other possible questions worth considering at this stage - for example, 'How effective are
our existing controls?' - in order to identify factors that reduce the consequences or probability of
the risk; however, in terms of what we actually need to know, these will make a good start.
Page 13
2.2.2 What can we learn from ISO 31000 risk assessment processes?
ISO 31000 states that risk assessment attempts to answer the following fundamental questions:
Providing that you adhere to this basic structure, you are following the framework that is set out in
the International Standard ISO 31000:2009.
Rather than spending several days reading the Standard and having long meetings with colleagues to
see how it might be applicable, why not look for methods that would help you to meet the
requirements of ISO 9001?
For me, a good start would be:
Documenting the results of any 'consideration of risks and opportunities' exercise as evidence of
your management team's "risk-based thinking".
Even if it is clear from the design of your processes that you have taken account of Clause 6.1 and
determined the risks and opportunities that need to be addressed, having a record of your risk
assessment processes might prove useful, if only as a reminder to keep matters under review!
Then, evaluate the risk assessment tools (numbering 31 in total) in ISO 31010 to see if they are
applicable to your organizational context.
It's probably not the time to use them in anger yet (see below), but at least you will know they exist
and that some tools could help to identify risks and opportunities and be useful in carrying out risk
analysis (if you consider consequences, probability and level of risk) and risk evaluation?
Draft BS EN ISO 9001 Quality Management Systems - Requirements, Date: 14 May 2014, A.4 Risk-based approach
Project risk management guidelines: managing risk with ISO 31000 and IEC 62198, Dale F Cooper, et al, Wiley, 2014.
Page 15
are too complicated and "not part of your job", it is worth pondering this quote from the
Introduction to the ISO 31000:2009:
"The generic approach described in this International Standard provides the principles and
guidelines for managing any form of risk in a systematic, transparent and credible manner and
within any scope and context".5
Check-lists
Brainstorming
Structured or semi-structured interviews
Page 16
Despite the mention of probability above, Table A.1 Applicability of tools used for risk assessment,
the Delphi method is marked 'NA' [NA = Not Applicable] for Risk Analysis to assess Consequence,
Probability and Level of risk - although personally we would agree with the commentary on page 29
[Clause B.3.2 Use] which states:
"The Delphi technique can be applied at any stage of the risk management process or at any phase
of a system life cycle, wherever a consensus of views of experts is needed."7
A true consensus approach that avoids the bias of dominant members of the team can be the wakeup call that management needs to assess risk.
British Standard BS: IEC61882:2002 Hazard and operability studies (HAZOP studies)- Application Guide, published by BSI Group.
Page 17
Discussion is facilitated by creating a question using a what-if phrase and a prompt word or subject.
The what-if phrases to be used are what if, what would happen if, could someone or
something, has anyone or anything ever. The intent is to stimulate the study team into
exploring potential scenarios, their causes and consequences and impacts.10
The risks identified are summarized and the team considers the controls already in place - assuming
that there are any - before confirming the description of the risk, its causes, consequences and
expected controls.
This information is then recorded.
What we particularly like about the SWIFT concept approach is the inherent discipline which forces
the team members to consider the effectiveness of the controls. Assessing risk is one thing, but
treating it is another entirely. They have to agree a statement of risk control effectiveness, which, if
it proves to be less than satisfactory, triggers the task of further considering risk treatment tasks and
potential controls.
The application of this team-based model does not have to be complex. ISO 31010 simply rates the
Complexity of the technique as "Any".11
In 2009, the Health and Safety Laboratory compiled a report12 for the Health and Safety Executive
(HSE) outlining HRA methods for review.
10
Ibid.
11
12
Review of human reliability assessment methods, Prepared by the Health and Safety Laboratory for the Health and Safety Executive
2009, PR679 Research Report, Julie Bell & Justin Holroyd, Health and Safety Laboratory; First published 2009.
Page 18
They identified 35 tools that constituted true HRA techniques and that could be used effectively in
the context of health and safety management.
Obviously, it is well beyond the scope of this article to define the merits and demits of all these
methods. However, the HRA tools in the table below illustrates that there are a large number of risk
assessment techniques in the Health & Safety arena that could be applied elsewhere. It is also worth
reflecting that Risk Management is usually associated with the financial risk; however, risk
assessment techniques have other well-established uses including helping to maintain safe working
environments.
Without being specific at this time, we think that it is possible that some of these tools could be
adapted (if they haven't been?) to identify, analyse and evaluate risks and opportunities in the
design of quality processes. After all, corrective and preventive actions usually involve human
beings!
Acronym for Tool
ASEP
AIPA
APJ
ATHEANA
CAHR
CARA
CES
CESA
CM
CODA
COGENT
COSIMO
CREAM
DNE
DREAMS
FACE
HCR
HEART
HORAAM
HRMS
INTENT
JHEDI
MAPPS
MERMOS
Expanded name
Accident Sequence Evaluation Programme
Accident Initiation and Progression Analysis
Absolute Probability Judgement
A Technique for Human Error Analysis
Connectionism Assessment of Human Reliability
Controller Action Reliability Assessment
Cognitive Environmental Simulation
Commission Errors Search and Assessment
Confusion Matrix
Conclusions from occurrences by descriptions of actions
COGnitive EveNt Tree
Cognitive Simulation Model
Cognitive Reliability and Error Analysis Method
Direct Numerical Estimation
Dynamic Reliability Technique for Error Assessment in Manmachine Systems
Framework for Analysing Commission Errors
Human Cognitive Reliability
Human Error Assessment and Reduction Technique
Human and Organisational Reliability Analysis in Accident
Management
Human Reliability Management System
Not an acronym
Justified Human Error Data Information
Maintenance Personnel Performance Simulation
Method d'Evaluation de la Realisation des Missions Operateur pour
la Surete (Assessment method for the performance of safety
operation.)
Table 2: List of HRA tools
Page 19
As ISO 31010 points out in the section on the 'Limitations' of HRA, many activities of humans do not
have a simple pass/fail mode. HRA has difficulty dealing with partial failures or failure in quality or
poor decision-making.13
13
14
Root Cause Analysis, Tracing a Problem to its Root Origins, Mind Tools website:
http://www.mindtools.com/pages/article/newTMC_80.htm
15
16
Page 20
17
Ecological Risk Assessment: Technical Overview, Ecological Risk Assessment Process, U.S. Environmental Protection Agency website:
http://www.epa.gov/oppefed1/ecorisk_ders/index.htm#WITERAP
18
19
Elliot, D.; Swartz, E.; Herbane, B. (1999) Just waiting for the next big bang: business continuity planning in the UK finance sector. Journal
of Applied Management Studies, Vol. 8, No, pp. 4360. Here: p. 48
Page 21
item of documented information to show risk-based thinking because you will have assessed (by
means of the BIA) how key disruption risks could affect an organizations operations and
identified/quantified the capabilities that would be required to manage it.
If not, well ... you could consider conducting a BIA; although we would strongly recommend calling in
a qualified business continuity consultant.
20
Page 22
outcomes and overall system analysis. Using inductive reasoning, ETA translates probabilities of
different initiating events into possible outcomes. It is arguably less resource intensive than fault
tree analysis (see Table A.2 in ISO 31010).
ETA can be applied to a wide range of systems including: nuclear power plants, spacecraft, and
chemical plants.21
Once again, if you are managing the quality system of a small enterprise in a relatively 'low risk'
context, this technique is unlikely to be for you.
21
22
Page 23
2. determine the main categories of causes represented by boxes in the Fishbone diagram.
Typically, for a system problem, the categories might be people, equipment,
environment, processes, etc. However, these are chosen to fit the particular context;
3. fill in the possible causes for each major category with branches and sub-branches to
describe the relationship between them;
4. keep asking why? or what caused that? to connect the causes;
5. review all branches to verify consistency and completeness and ensure that the causes
apply to the main effect;
6. identify the most likely causes based on the opinion of the team and available evidence.
The results are displayed as either an Ishikawa diagram or tree diagram.
all potential failure modes of the various parts of a system (a failure mode is what is
observed to fail or to perform incorrectly);
the effects these failures may have on the system;
the mechanisms of failure;
how to avoid the failures, and/or mitigate the effects of the failures on the system.
FMEA/FMECA is a systematic analysis technique that can be used to identify the ways in which
components, systems or processes can fail to fulfil their design intent, highlighting:
FMEA/FMECA also provides qualitative or quantitative information for other types of analysis, such
as fault tree analysis, and is used in quality assurance applications. For example, it can produce a
semi-quantitative measure of criticality known as the risk priority number (RPN) obtained by
multiplying numbers from rating scales (usually between 1 and 10) for (a) consequence of failure, (b)
23
Page 24
likelihood of failure, (c) ability to detect the problem. Note, a failure is given a higher priority if it is
difficult to detect.
24
25
Page 25
2.5.4 HACCP
HACCP a systematic preventive approach to food safety from biological, chemical, and physical
hazards in production processes that can cause the finished product to be unsafe, and designs
measurements to reduce these risks to a safe level.27 HACCP has been recognized internationally as
a logical tool for adapting traditional inspection methods to a modern, science-based, food safety
system.28
HACCP is focused only on the health safety issues of a product ensuring that risks are minimized by
controls throughout the process rather than through inspection of the end product. The seven
HACCP principles are the basis of most food quality and safety assurance systems, and the United
States, HACCP compliance is regulated by 21 CFR part 120 and 123. The HACCP principles are also
included in the international standard ISO 22000 FSMS 2005. This standard is a complete food safety
and quality management system incorporating the elements of prerequisite programmes (GMP &
SSOP), HACCP and the quality management system, which together form an organization's Total
Quality Management system.
Table A.1 Applicability of tools used for risk assessment [see page 22 of ISO 31010], lists the HACCP
technique as "Not Applicable" for analysis of probability or levels of risk.29 However, the principle of
identifying the factors [risks] that can influence product quality, and defining process points where
critical parameters can be monitored and hazards controlled, can be generalized for use other
technical systems.30
26
27
28
Ibid.
29
ISO/IEC 31010:2009, Table A.1 Applicability of tools used for risk assessment, p.22
30
Page 26
A cause-consequence pair is selected, and the layers of protection that prevent the cause
leading to the undesired consequence are identified.
An order of magnitude calculation is then carried out to determine whether the protection is
adequate to reduce risk to a tolerable level.31
LOPA is a less resource-intensive process than a fault tree analysis or a quantitative form of risk
assessment, but is more rigorous than qualitative subjective judgements alone. It focuses efforts on
the most critical layers of protection, identifying operations, systems and processes for which there
are insufficient safeguards and where failure will have serious consequences. However, this
technique looks at one cause-consequence pair and one scenario at a time and, therefore, does not
apply to complex scenarios where there are many cause consequence pairs or where a variety of
consequences affects different stakeholders.
For more information, see:
IEC 61508 (all parts), Functional safety of electrical/electronic/programmable electronic safetyrelated systems
IEC 61511, Functional safety Safety instrumented systems for the process industry sector. [PDF]
Markov analysis
Monte-Carlo analysis
Bayesian analysis
31
32
The Bow Tie Method in 5 Minutes, CGE Risk Management Solutions, YouTube: https://www.youtube.com/watch?v=P7Z6L7fjsi0
Page 27
list of various states that the system, sub-system or component can be in (e.g. fully
operational, partially operation (i.e. a degraded state), failed state, etc);
a clear understanding of the possible transitions that are necessary to be modelled. For
example, failure of a car tyre needs to consider the state of the spare wheel and hence
the frequency of inspection;
rate of change from one state to another, typically represented by either a probability of
change between states for discrete events, or failure rate () and/or repair rate () for
continuous events.35
The output from a Markov analysis is the various probabilities of being in the various states, and
therefore an estimate of the failure probabilities and/or availability, one of the essential
components of a system.
33
34
35
Page 28
2.7.2.1 How does Monte Carlo analysis model the effects of uncertainty?
Systems are sometimes too complex for the effects of uncertainty on them to be modelled using
analytical techniques. However, they can be evaluated by considering the inputs as random variables
Page 29
and running a number N of calculations (so-called simulations) by sampling the input in order to
obtain N possible outcomes of the wanted result.
Monte-Carlo analysis can be developed using spreadsheets, but software tools are readily available
to assist with more complex requirements, many of which are now relatively inexpensive.
Monte-Carlo analysis can be developed using spreadsheets, but software tools are readily available
to assist with more complex requirements, many of which are now relatively inexpensive.
Monte Carlo simulations require you to build a quantitative model of your business activity, plan or
process. This is often done by using Microsoft Excel with a simulation tool plug-in - a relatively
inexpensive set of tools.
To deal with uncertainties using Monte Carlo analysis in your model, you'll replace certain fixed
numbers -- for example in spreadsheet cells -- with functions that draw random samples from
probability distributions. And to analyze the results of a simulation run, you'll use statistics such as
the mean, standard deviation, and percentiles, as well as charts and graphs.36
For risk assessment using the Monte Carlo simulation, triangular distributions or beta distributions
are commonly used.
Note that ISO 31010 Table A.1 Applicability of tools used for risk assessment states this is tool is
strongly applicable for the Evaluation stage of risk assessment but not applicable (NA) for risk
identification or risk analysis.
36
37
ISO/IEC 31010:2009, Table A.1 Applicability of tools used for risk assessment, p.22.
38
Page 30
that explains it best for me comes from the last of these - it is:
"The probability of a hypothesis C given some evidence E equals our initial estimate of the
probability times the probability of the evidence given the hypothesis C divided by the sum
of the probabilities of the data in all possible hypotheses."
Bayesian inference is used in a wide range of fields from medical diagnosis to checking your inbox for
likely spam emails. However, is it any good for risk assessment?
Although it can appear to be objective, this is typically not the case. A Bayesian probability is really a
persons degree of belief in a certain event rather than one based upon physical evidence.
Because the Bayesian analysis approach is based upon the subjective interpretation of probability, it
provides a ready basis for decision thinking and the development of Bayesian nets (or Belief Nets,
belief networks or Bayesian networks).39 The availability of software computing tools and what ISO
31010 terms "intuitive appeal" has led to the widespread adoption of Bayesian nets. However, they
can be valuable wherever there is the requirement for finding out about unknown variables by using
structural relationships and data.
The inputs are similar to the Monte Carlo analysis above; namely:
Bayesian analysis can provide an easily understood model and the data readily modified to consider
correlations and sensitivity of parameters.
This technique could be successfully applied to Quality Management Systems. However, there will
be minimum sample size requirements for control charts that measure non-conformities (errors),
based on the average non-conformity rate in the quality processes being measured.
Lower error rates would therefore require larger sample sizes to make valid inferences because of
the properties of the binomial distribution.
Even so, we would be very interested to hear from Quality Managers who have applied Bayesian
analysis in this way to predict likely error rates in processes!
39
40
Page 31
3.1 Risk based thinking is the new 'preventive actions' for QMS
To briefly recap the position to date:
ISO 9001 Risk-based thinking could (and we am not saying that it should) be demonstrated by one or
more of the risk assessment tools in ISO 31010:2010. However, that still leaves you with the
dilemma of selecting the most appropriate tools to help you to identify, analyse and evaluate risk in
your organizational context and with the resources at your disposal.
In ISO 9001:2015 there is no requirement for risk management. However, organizations can choose
to develop a more extensive risk-based approach, and the Standard refers to ISO 31000, which
provides guidelines that can be appropriate in "certain organizational contexts".
It remains to seen whether assessors for the various Certification Bodies will expect you to produce
documented evidence of risk-based thinking.
How will ISO Assessors attempt to assess RBT in Quality Systems?
The short answer is we do not know at present. However, as we have postulated, there are three
possibilities:
Option 1: They will ignore the risk-based thinking requirements of Clause 6 in the same way that
some claim preventive actions were ignored in the past. The counter to this is that Clause 6 in the
DIS requires "Processes for planning and consideration of risks and opportunities".
Option 2: They will regard the failure to show evidence of risk-based thinking in an organizations
quality processes as a non-conformity (perhaps even a major non-conformity) and will judge the
quality system to be ineffective because it has failed to reduce or eliminate the risks to process
outputs.
Option 3: Auditors will highlight in their report any good practices seen in the application of riskbased thinking to the planning and consideration of quality processes; showing how this has helped
to achieve continual improvement of the system and provide the assurance of conformity to
customer and applicable statutory and regulatory requirements.
You may decide differently, but in our view, Option 3 is more likely in the majority of cases. Ergo, it
cannot hurt your case to show documented evidence of RBT, regardless of whether documented
information is a requirement or not.
However, it will be your assessor that decides this, not us!
Regarding Option 3 above, it is also worth reflecting upon the number uses of the words "continual
improvement" in the clauses of the new Standard.
Page 32
Aside from the definition that appears in Normative References, the term "continual improvement"
is used in Clause 5: Leadership, Clause 6: Planning, Clause 7: Support, Clause 9: Performance
Evaluation, and - unsurprisingly - in Clause 10: Continual Improvement; which states that:
"...the organization shall consider the outputs of analysis and evaluation, and the outputs
from management review, to confirm if there are areas of underperformance or
opportunities that shall be addressed as part of continual improvement."41
There is doubt about which of the three options above best describes the likely future response of
external auditors/assessors, but you can help put your organization in a position where Option 3 is
the more likely outcome, because your quality processes reflect the fact that you have taken
account of the risk and opportunities in your context.
41
42
ISO/IEC 27001 Information Security Management Securing your information assets Product Guide, October 2012 (modified May 2013)
Page 33
integrated management systems as now an organization may apply the same risk assessment
methodology across several disciplines".43
Earlier posts in this series have examined the different risk assessment techniques aligned to ISO
31000 and described fully in ISO 31010:2009.
3.1.2 What actions are required to plan for risks and opportunities?
Clause 6 of ISO 9001:2015 is likely to be explicit about the need for planned actions to address risks
and opportunities in quality systems:
6.1.2 The organization shall plan:
1. actions to address these risks and opportunities;
2. how to:
a. integrate and implement the actions into its quality management system
processes (see 4.4);
b. evaluate the effectiveness of these actions.
Actions taken to address risks and opportunities shall be proportionate to the potential impact on
the conformity of products and services.44
Although not all the processes of the quality management system will represent the same level of
risk in terms of the organizations ability to meet its objectives, - and the consequences of process,
product, service or system nonconformities are not the same for all organizations - there will be risks
that you will need to address through the quality processes.
So how do you go about identifying, considering and planning for risks to quality - and how could risk
analysis help you to achieve your objectives?
The simple answer is that before you can plan processes that address risk, you need to analyze the
relative importance of risks in your system. In a world where risk factors determine the
organization's success or failure, we need a detailed understanding of each of the specific risks
posed to successful outcomes at the various stages of quality processes. With this knowledge, we
can determine appropriate priorities for actions.
This full understanding should result in fewer unpleasant surprises arising and will enable managers
to determine where the greatest effort should be focused in treating identified risks and for quality
assurance purposes.
The alternative to decision-making based on risk analysis is a combination of experience and
intuition. Experience, no matter how extensive, can be out of date and therefore fail to anticipate
4343
Moving from ISO/IEC 27001:2005 to ISO/IEC 27001:2013: The new international standard for information security management
systems, Transition Guide, BSI Group.
44
Page 34
the potential risks in a system. Intuition is the ability to acquire knowledge without inference or the
use of reason and is of questionable value to organizations when planning and considering processes
in order to consistently produce desired outcomes.
By developing a better understanding of risk, risk analysis techniques help organizations facilitate
structured action planning and resource allocation.
The following section of this blog post contains the first part of a Proposal for a formal methodology
for making risk-based decisions when planning and considering quality processes. we have based
some of the ideas on work by Dale F. Cooper et al in the book 'Project management guidelines:
managing risk with ISO 31000 and IEC 62198' (John Wiley and Sons); however, we have simplified
the approach therein as applied to international, large-scale project management. Furthermore, we
have re-engineered these ideas into a method of risk assessment and continual process
improvement for ISO 9001 quality management systems, based on the process improvement model
from ITIL, which itself uses methods from quality management.
The CSI process in ITIL aims to continually improve the effectiveness and efficiency of IT processes
and services, in line with the concept of continual improvement adopted in ISO 20000. It defines the
specific initiatives aimed at improving services and processes, based on the results of service reviews
and process evaluations. The improvement cycle takes into account the business perspective of
service quality, although CSI aims to improve process effectiveness, efficiency and cost effectiveness.
In ITIL 2011, the CSI Register was introduced as a central document or database where all
improvement opportunities and initiatives are recorded. we propose to extend this idea to create a
controlled documented information system (CDIS) for QMS which would contain a Risks and
Opportunities Register (R&O Register), used to record and manage risks to, and improvement
opportunities in, quality management processes throughout their lifecycle.
A key feature of our design for the R&O Register would be outputs from a simple risk assessment
process, following a six-step risk assessment and continual process improvement model, which are:
1.
2.
3.
4.
Page 35
process is repeated after implementation to determine whether the level of risk after risk treatment
has been completed is tolerable; and if this is not the case, then further risk treatment actions are
sought and considered.
6. Monitoring & review
A monitor process is developed for each risk by the risk owners and each relevant control (control
owners). Decisions are made about the time intervals at which the risks and controls will be
reviewed. At the same time, a monitoring process will be put in place for each risk treatment plan
under the direction of the relevant risk owners. Progress will be monitored in respect to the
objectives of the risk treatment plan, and the resulting successes and failures recorded. Periodically,
the team will assess whether new risks are affecting or could affect quality processes and systems as
part of the cycle of continuous quality process improvement (see Figure 1 below).
Page 37
That is to say, context is a term that is used to describe a combination of internal and external
factors and conditions that can have an effect on an organization's (3.01) approach to its products
(3.47), services (3.48), investments, and interested parties (3.02).45
An organization needs to demonstrate its ability to provide products and services that consistently
meet customer and applicable statutory and regulatory requirements and aims to enhance customer
satisfaction.46 Therefore, it is necessary to determine both the external and internal context before
designing and implementing quality processes that take account of the risks and opportunities that
apply in a particular context.
The risk-based approach of ISO 9000:2015 requires the organization to understand its context (see
clause 4.1) and determine the risks and opportunities that need to be addressed (see clause 6.1).
When applying risk-based thinking to the planning and consideration of quality processes, we should
take into account the organization's understanding of the...
external context; which can be facilitated by considering issues arising from legal,
technological, competitive, market, cultural, social, and economic environments, whether
international, national, regional or local.
internal context; which can be facilitated by considering issues related to values, culture
knowledge and performance of the organization.47
The Standard also requires that "...the organization shall maintain documented information to the
extent necessary to support the operation of processes and retain documented information to the
extent necessary to have confidence that the processes are being carried out as planned".48
45
Moving from ISO/IEC 27001:2005 to ISO/IEC 27001:2013: The new international standard for information security management systems,
Transition Guide, BSI Group. 3.24, p.17.
46
47
48
49
Page 38
Establish the external and internal organisational context in which the risk assessment is
taking place (see ISO 9001:2015 Clause 4.1);
Specify the main objectives and outcomes that are uncertain and, therefore, represent a
risk;
Develop criteria against which the consequences and likelihoods of identified risks can be
measured; and
Define the key elements for structuring the risk assessment process.
Process inputs
Key process documents, scope definitions, pre-existing analyses and other relevant documented
information such as organisational policies, processes and structures.
Method
1. Review organisational and process documentation.
2. Review the external and internal contexts.
3. Develop criteria for evaluating consequences and likelihoods.
50
Project management guidelines: managing risk with ISO 31000 and IEC 62198, Dale F Cooper, et al, John Wiley & Sons Inc, March 2014.
51
52
Page 39
organisational objectives;
process objectives;
the internal environment;
the external environment;
the context of the risk management process;
risk criteria.53
However, this list will depend on context and the risks being evaluated.
When defining risk criteria, you should consider:
53
Page 40
For the risk criteria to be adequate to support the decisions made at the risk treatment stage, they
should:
assist in decision-making leading to actions that reduce risk to levels that are as low as
reasonably practicable;
be capable of being communicated, understood and applied within the organization and to
an external organization (ISO 9001:2013, 3.01) where it performs part of an organization's
function (Ibid. 3.25) or process (Ibid. 3.12);
be unambiguous in their formulation;
not evidence any bias towards particular risk treatment options in the way in which risk is
expressed.
Documented information:
Statement of organization context - including its size and complexity, a general outline of the
external and internal risks and opportunities that it needs to address, and how that knowledge is to
be made accessible.
54
historical data;
theoretical analysis;
empirical data and analysis;
informed opinion of the project team and other experts;
Page 41
Method
1. Use one or more of the Look-up and/or Supporting Methods described in ISO/IEC 31010
designed for Risk identification.
These techniques include:
Structured interviews
Brainstorming
Examination of similar quality processes
Delphi technique
SWIFT technique
[See our previous blog post about ISO/IEC 31010 for more information: ISO 31000 Risk management
techniques Attributes of a selection of risk assessment tools ].
Produce a comprehensive list of possible risks to successful outcomes.
Process outputs
See item 2 above.
Steps 3 - 5 will analyse and evaluate these risks and prioritise treatment.
Documented information:
1. Risks and opportunities register (R&O register) - recording identified risks, controls, and
ratings.
2. Risk description worksheet - (for recording risk at process level) listing risk description
process, existing controls, key assumptions, sources of information, document attachments.
55
Adapted from assessing risks to quality from Project management guidelines: managing risk with ISO 31000 and IEC 62198, Dale F
Cooper, et al, John Wiley & Sons Inc, March 2014
Page 42
56
57
Ibid.
Ibid.
Page 43
analysis is the systematic use of available information - including documented information from the
risk identification process in Step 2 - to develop an understanding of the risks to quality objectives.58
This includes:
historical records;
process records; either specific to the kind of process being assessed, or where comparisons
and inferences can be drawn regarding risk scenarios;
industry best practice;
user experience (from quality records and other sources - e.g. customer service records,
social media discussions, consumer satisfaction surveys);
published literature and research reports that contain theory and/or examples relating to
failure modes or equipment reliability;
product brochures and technical manuals;
audit reports.
Process inputs
Information used in qualitative risk analysis and evaluation includes:
58
historical data;
theoretical analysis;
empirical data and analysis;
informed opinion of the project team and other experts;
the concerns of stakeholders.59
59
Adapted from assessing risks to quality from Project management guidelines: managing risk with ISO 31000 and IEC 62198, Dale F
Cooper, et al, John Wiley & Sons Inc, March 2014
Page 44
Note: This simple list is intended to be identical to the list for risk identification in Step 1, although
you can probably add further types of information based on your organization's experience of risks
to outputs.
Method
Steps required for a Qualitative Risk Assessment include:
1. List process controls that are already in place and act to modify each risk and assess their
effectiveness.
2. Determine the kind and level of consequences that characterise each risk.
3. Assess the likelihood of the consequences occurring, given the controls in place.
4. Combine levels of consequences and likelihoods to determine the level of risk.
5. Evaluate the potential exposure for each risk identified to desired quality outcomes.
6. Agree the management priorities for:
risk treatment;
control assurance; and
ensure top management oversight.60
7. In conjunction with Step 5 (Risk Treatment): use risk criteria to determine a) the risk
treatment options available and b) whether any residual risk level in your quality processes
will be tolerable.
Process outputs
A prioritised list of risks that takes account of uncertainty for:
control effectiveness;
consequence;
likelihood;
level of risk; and
potential exposure.
Documented information:
Risks and opportunities register (R&O register) - recording identified risks, controls, and ratings.
60
ISO/DIS 9001:2014, Clause 5.1.1 Leadership and commitment for the quality management system, pp.26-27.
Page 45
Risk description worksheet - (for recording risk at process level) listing risk description process,
existing controls, key assumptions, sources of information, document attachments.
3.5.4 Summary:
In the first three Steps of this risk management process for quality systems, we have addressed
three fundamental requirements of ISO 9001:2015; namely:
1. Understanding the context of the organization, its quality management system and
processes (Clause 4).
2. Processes for planning and consideration of risks and opportunities (Clause 6)
3. Processes for support, including resources, people and information (Clause 7)
As ISO 9001:2015 states, the process for considering and controlling past, existing and additional
knowledge needs to take account of the organization's context, including its size and complexity, the
risks and opportunities it needs to address, and the need for accessibility of knowledge.61 We
propose documented information in the form of (1) Statement of Context, and (2) Risks and
Opportunities Register (R&O register) used to record identified risks, controls, and ratings.
Method
1. Develop an appropriate structure for examining quality system processes;
61
Page 46
2. Use a semi-quantitative risk assessment tool (see example to follow) to assess the
consequences and likelihood of risks arising in each process.
3. Convert the consequence and likelihood of risks arising in each process to an initial priority
level.
4. Determine Risk Factors (RF) for each of the risks analysed - see below:
5. P (Probability) and C (Consequence) values are plotted to show the risk factors affecting
quality processes and their desired outputs.
6. The risk factors, the ranking and the risk profile are then used to decide which of the
identified risks may be deemed acceptable or unacceptable, and to enable resource
priorities to be determined.
Process outputs
A list of risks to outputs prioritised by risk factor; i.e. level of 'riskiness'.
Consequence and likelihood ratings and agreed priorities for each risk.
Risk contour diagrams (see example below) to plot risk factors and iso-contours; i.e., points of equal
RF value, to give an indication of priorities.
62
Project management guidelines: managing risk with ISO 31000 and IEC 62198; Dale F Cooper, et al, John Wiley & Sons Inc, March 2014.
Page 47
=
=
C =
=
RF =
=
3.6.2 What is the value of the Semi-Quantitative approach in Step 4, following the
Qualitative Assessment conducted in Step 3?
To quote Holger Schutz et al, in 'Comparative risk assessments: concepts, problems and
applications.'63 In the qualitative approach to risk assessment, "An event is verbally described in
relation to other events. Absolute reference points and specifications for the bandwidths are lacking
(in which field is the term "high" to be classified?) so that no comparison of various processes /
specifications is possible." In other words, the value of a qualitative risk assessment is limited since
precise data is needed to make more accurate comparisons between the risks being analysed. The
breadth of the classifications needs to be chosen so that "...the findings of imprecise data still lie
63
Comparative risk assessments: concepts, problems and applications; Holger Schutz, Peter M. Wiedemann, Wilfried Hennings, Johannes
Mertens, Martin Clauberg; John Wiley & Sons, July 2006; ISBN: 978-3-527-31667-0.
Page 48
within the bandwidth of the classes". A semi-quantitative classification of the type in the diagram
above can assess the order of magnitude of the importance of individual risk scenarios, either at the
quality process or wider organisational level. Because qualitative terms in this approach have been
given numerical values, the verification of results is made possible by the comprehension of single
steps of awareness,64 enabling high priority risks to be prioritised.
This semi-quantitative approach to assessing risks in a Quality System has the advantage of allowing
comparison of the various risks of non-conformities (minor, major and critical) on one or more risk
attributes by one or more evaluators, resulting in a consensus view of what are the 'real' risks as
measured by risk factors which are plotted on one graph.
In organisational environments where a degree of uncertainty makes it difficult to predict which
risks to quality outcomes are the highest priority, this type of analysis supported by a consensus risk
identification process, provides meaningful outputs to guide planning and consideration of risks.
To read more about the semi-quantitative risk analysis method, see:
Project management guidelines: managing risk with ISO 31000 and IEC 62198; Dale F Cooper, John
Wiley & Sons Inc, March 2014.
Documented information
Assessment sheets, recording likelihood indicators (rated high-low), consequence indictors (rated
high-low); plus the relevant discussions, assumptions and responses; and a risk score for each line
entry made.
Diagrammatic representations of risk - e.g., a risk factor and iso-contour graph used to plot data
from a semi-quantitative risk analysis.
64
65
Page 49
Process inputs
The primary inputs are lists of:
Risks and their priorities from the risk analysis and evaluation step.
Resources, including budget, which can be applied to treating risks.
Method
1.
2.
3.
4.
5.
Process outputs
Risk action plan summaries for each proposed risk treatment action.
66
Project management guidelines: managing risk with ISO 31000 and IEC 62198; Dale F Cooper, et al, John Wiley & Sons Inc, March 2014,
p.363.
67
Ibid., p.363.
68
Page 50
Identify options for addressing the risks. Let's say that the anticipated problem is a backlog in
production indicated by the following RF values:
1. Speed and feed rates too slow: RF = 0.8
2. Machine breakdowns: RF = 0.6
3. High absence and tardiness rate: RF = 0.4
Which risks should take priority? And what options are available to the organization to treat one or
more of the risks using available resources? The highest level risk is number 1. Speed and feed rates
are likely to be too slow at present to meet the delivery schedule of a customer order. The quality
management team working with the operations team has identified, analyzed and evaluated this risk
as having a Risk Factor of 0.8 (on a scale of 0-1). Through brainstorming, they have identified and
analysed a problem with the production operatives' familiarity with new materials. There is a
secondary factor in terms of unfamiliarity with new machines (RF = 0.6). Absence and tardiness are
also potentially an issue as the operatives are reluctant to operate the new machines without proper
training. However, the third anticipated risk: 'high absence and tardiness rate' has been assessed as
a lower risk at RF = 0.4 than the risk factors for risks 1 or 2, so it is decided to prioritise treatment of
1 and 2.
The rationale:
Although lack of familiarity with new materials is thought a higher risk than machine breakdowns or
high absence/tardiness rates, risks 1 & 2 taken together represent an unacceptably high risk within
the context. Machine breakdowns due to poor maintenance by the supplier and/or operator error
are known to have been a problem recently in a competing production facility, and are likely to
reduce output rates at a critical time.
Risk 3 is thought to be a lower priority but merits consideration later on.
So what are the best options?
Strengthening the Operations team with an operator who is familiar with both the new materials
and the machine is one possibility to consider.
Other options are:
Instigating a training programme to familiarise operatives with the new materials and
improve their output performance using the new machines.
Increasing production hours through over-time to compensate for low output until the
operatives are more familiar with the new materials, etc.
Outsourcing the manufacturing of the component made with the new material (either on a
temporary or permanent basis) to avoid the risk.
Obviously, there could be other options available, but let's stick with these for now. The next action
is to determine the potential benefits and costs of the options; and then select the best options to
treat the risks.
The team next look at the possibility of hiring a skilled operative with experience of working with the
materials in question and the machines. Although it is an attractive idea, they cannot be sure that
Page 51
they can hire the right person given the tight timescales they are working to; and although urgent
enquiries could be made through Human Resources with specialist recruitment agencies, the
expectation is that the only two viable options in the short-term are increasing production hours and
outsourcing on a temporary basis. This is because a training programme will take longer to organise
and will require a specialist trainer who has experience of the material and the machines. The trainer
will not be available until over half way through the production of the customer's order. Therefore,
the only options are to increase production hours, accepting a high proportion of scrap that will be
generated while operatives learn to work the material, or outsource to a manufacturer that has
been using the material for two years and has successfully overcome their machine reliability
problems.
The decision is made to avoid the risk by outsourcing in this instance; however, actions to design and
implement a training course are agreed, so that the anticipated production problem will not re-occur
in the future.
This risk treatment plan has removed the risk. It may of course have introduced a new potential risk:
i.e. that the chosen outsourcing company proves to be unreliable and fails to deliver on time, and/or
within budget?
This identified risk will then be duly analysed, evaluated and, if it is thought necessary, treated as
part of a continual review of the risks.
See Step 6: Monitoring and Review below.
Process outputs
Risk action plan summaries for each proposed risk treatment action.
Documented information
1. Risk treatment options worksheet
2. Risk treatment plan summary
It will be necessary to decide how risks and controls will be periodically reviewed, including how
often and when these will take place; who will conduct the reviews, and what is the most
appropriate approach to adopt.
At this final stage, an organization would set up the following processes:
Page 52
69
70
ISO/DIS 9001:2014, 4.4 Quality management system and its processes, p.26.
71
Ibid., p.26.
Page 53
that is consistent with the risk criteria developed as part of establishing the context? [Clause 6.2].
Assuming that you do not have a method in place already: which one should you choose from the
bewildering array?
A risk assessment process needs to include the following task activities:
risk identification;
risk analysis consequence analysis;
risk analysis qualitative, semi-quantitative or quantitative probability estimation;
risk analysis assessing the effectiveness of any existing controls;
risk analysis estimation of the level of risk;
risk evaluation.
The tools necessary to achieve these steps are listed in ISO 31010:2009; especially Table A.1 Tools
used for risk assessment. However, it has to be said that the list is daunting to many quality
professionals who are unfamiliar with risk management processes.
The sheer complexity of some types of risk assessment will render the tool useless in most
organizations employing between 1 and 250 people. However, that does not mean to say that ISO
31010 is not a valuable reference should you ever be required to think about risk in these terms.
We have described a selection of the 31 techniques listed in ISO 31010. We have attempted to link
these tools to QMS processes in a meaningful way; however, our approach is not intended as a
reference, since a great deal will depend on the organization's context and there are a considerable
number of possibilities (potentially many thousands for different types and/or sizes of organization).
There is also no common consensus as yet regarding which ISO 31010 risk assessment techniques
are the most appropriate to apply to ISO 9001:2015 quality processes; although this is certain to be
covered in future books and journal articles on how to comply with the standard.
their interactions; the competence of persons within or working on behalf of the organization; and
its size and organizational structure.
An example statement of context template is available here.
2. Risk identification
This step involves selecting a suitable process for risk identification and for each quality process,
identifying and numbering the risks. This process records the risks in a Risk and Opportunities
Register (R&O Register) that would form an integral part of the Quality Management System.
We offer two supporting templates a Risk Description Brainstorming Sheet and a Risks &
Opportunities Register.
Risk Description Brainstorming Sheet:
5. Risk-treatment
This step considers options for either avoiding or seeking the risk; changing the likelihood; changing
the consequences; sharing the risk; or explicitly accepting the risk without further treatment.
We offer two supporting templates a Risk Treatment Plan Template and a Risk Treatment Options
Worksheet.
Risk Treatment Plan Template:
4.2 Conclusion
The key point in this white paper is that risk-based thinking (RBT) is here to stay in ISO 9001:2015
and other ISO standards.
We believe that it is in your interests to maximise the likelihood of what we term Option 3 that
your ISO auditor will positively note evidence that you have applied RBT.
To do this, you might look to ISO 31000 and its list of risk assessment techniques. This is not as easy
as it sounds.
We have therefore put together a best practices guideline in the form of a proposed six-step
methodology.
Page 57
Company Information
Registered Office:
Cognidox Limited
St Johns Innovation Centre
Cowley Road
Cambridge CB4 0WS
UK
salesinfo@cognidox.com
Telephone
This easy-to-use tool helps break down the barriers to find information, share
solutions and enjoy a faster, more productive development workflow inside
your company. In addition, CogniDox helps you manage and publish
documents and other content to licensed customers. It reduces technical
support load and accelerates your customers' time to market.
https://www.cognidox.com/
Page 58