Journal of Loss Prevention in the Process Industries 28 (2014) 2e9

Contents lists available at SciVerse ScienceDirect

Journal of Loss Prevention in the Process Industries

journal homepage: www.elsevier.com/locate/jlp

Past, present and future of Quantitative Risk Assessment (QRA) and

the incentive it obtained from Land-Use Planning (LUP)
Hans Pasman a, *, Genserik Reniers b, c
a
Mary Kay OConnor Process Safety Center (MKOPSC), Artie McFerrin Department of Chemical Engineering, Texas A&M University, College Station, TX
77843-3122, USA
b
Antwerp Research Group on Safety and Security (ARGoSS), University of Antwerp, Koningstraat 8, 2000 Antwerp, Belgium
c
Centre for Economics and Corporate Sustainability (CEDON), HUB, KULeuven, Stormstraat 2, 1000 Brussels, Belgium

a r t i c l e i n f o a b s t r a c t

Article history: This paper tells the story about the development of Quantitative Risk Assessment, how it was concep-
Received 8 February 2013 tualized in the early 1970s in the nuclear industry, how it was employed within the chemical industry
Received in revised form soon after, and what its status is today. The different purposes of QRA are explained, and we elaborate on
13 March 2013
one of the purposes, that is, Land-Use Planning. The role of Professor Ben Ale, as a process safety pioneer,
Accepted 14 March 2013
is discussed throughout the history. We nally provide some on-going state-of-the-art research and
projects to further improve QRA approaches, and we sketch the future of QRA and its relation with LUP.
Keywords:
2013 Elsevier Ltd. All rights reserved.
Land-Use Planning
Risk policy
Quantitative Risk Assessment

1. Introduction the population called for measures to guarantee safety. This

In Europe, as well as in the rest of the world, process industry
operates to meet with societies daily needs of energy carriers
(fuels) and materials for all kinds of purposes. These activities go
hand in hand with major hazard potential that occasionally results
in major accidents. Obviously, adequate management and control
of such major hazards is essential to guarantee a long-term license
to operate. In Europe, the implementation of a policy for control of
major hazards involving chemical substances is regulated via the
so-called Seveso legislation. In 2012, the European Seveso III
Directive (EU, 2012) was issued. Europe consists of a number of
countries, which are called Member States. The implementation
of the Seveso Directive in national law within the principles and
constraints laid down in the directive, varies from one Member
State to the other, depending on the seriousness of the safety sit-
uation in that Member State, the existing legislative systems, and
the locally available methods and tools that are deemed t to tackle
the problem. For example in the Netherlands, a very large chemical
cluster is situated in the Port of Rotterdam, partly encapsulating
some smaller communities and next to a very densely populated
area with some major cities, such as Rotterdam and The Hague.
When in the 1960s the industry rapidly expanded and scaled up,
* Corresponding author. Tel.: 1 316 30551535.
E-mail address: hjpasman@gmail.com (H. Pasman).
concern was exacerbated by frequent res, explosions, and other
mishaps occurring at the time. This specic situation convinced
authorities in The Netherlands to develop a solution for managing
and controlling major hazards. In the early 1970s, quantitative risk
analysis, developed for nuclear safety, suddenly emerged and was
proposed as the solution to tackle the problem. In a few years time,
quantied risk analysis was introduced and quantied risk criteria
dened and cast into law. Although the application of QRA in the
case of the Netherlands is mainly for Land-Use Planning in the
context of so-called external safety, the method is generally suc-
cessfully applied to analyze and, where necessary, to improve plant
safety and also to perform emergency planning. The approach and
its development during the past decades is expounded in this
article.
The late 1960s and early 1970s was a time of massive and tragic
accidents involving hazardous materials, of which the most well-
known were the Flixborough major accident in the UK in 1974
and the Seveso disaster in Italy in 1976. In the Netherlands, the
disastrous vapour cloud explosion events at Pernis (1968) and Beek
(1975) took place. Ben Ale, at that time working at DSM, a major
Dutch chemical company, developed a clear vision about how to
tackle the problem of safety of population and environment against
severe effects of undesired mishaps within the context of science
and regulations. This article that was written especially for the
Special Issue of JLPPI on Process Safety Pioneers investigates how
Ale had an impact on safety thinking and process industry safety

0950-4230/$ e see front matter 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.jlp.2013.03.004
 H. Pasman, G. Reniers / Journal of Loss Prevention in the Process Industries 28 (2014) 2e9
regulations in the Netherlands and, by extension, in Europe and
even beyond. The paper will therefore not be exhaustive on all pros
and cons of risk assessment for LUP, because the number of risk
assessment related publications makes that prohibitive, but will try
to focus on some crucial points.
2. Research methodology
To investigate the history of Quantitative Risk Assessment, its
contribution to enhancing safety in the process industry, its role in
Land-Use Planning, and the thrust Ben Ale developed as a process
safety pioneer to make it a practical tool, we used a combination of
an in depth literature study and interviews with experts. At the
same time we wanted also to be informed about the current
practices of QRA for LUP within an international context, and to
anticipate further QRA developments. We used the following items
of consideration as an interview guideline:
1. With your experience and knowledge, what is your view about
the application of QRA for the purpose of Land-Use Planning
and in a wider context of safety enhancement in the process
industries?
2. In your viewpoint, what has been the importance of The
Netherlands (and more specically Ben Ale, if possible) in the
state of art of QRA for LUP and why?
3. What do you think about the state of the art of the method of
QRA?
4. What further/future perspectives are to be expected regarding
QRA?
The next section describes the results of our study and presents
at the same time a brief discussion of these ndings.
3. Results and discussion
Although it is impossible to reduce major hazards to zero, or in
other words it is impossible to gain absolute safety, since then no
chemical industrial activity would be possible, authorities and
practitioners have to deal with such hazards in the best possible
and feasible way. As Ale (1991) formulates it himself two central
questions must be answered:
(i) to what extent can a technological hazard be considered
acceptable?
(ii) to what extent should hazards that are considered condi-
tionally acceptable be reduced in the light of societal and
economic factors?
After hazards have been identied and expressed in quantitative
risk, these very important questions are usually dealt within an
assessment by employing the ALARP (As Low As Reasonably Prac-
ticable) or ALARA (As Low As Reasonably Achievable) principles,
the former mainly in the U.K., the latter more in general. In the
Netherlands, the latter principle was mathematically translated by
using a quantitative probabilistic approach. This approach origi-
nated during the time of decision making by the Delta Commission
about the height increase of sea dikes after the ooding disaster in
the early fties, when the probability of coincidence of storm from
a certain direction and a spring tide had to be weighed against the
cost of dike strengthening. When in the 1970s resident and worker
anxieties occurred, the Dutch authorities initiated the Rijnmond
risk assessment study (COVO, 1982) to quantify risks, to formulate
quantied safety goals, and to relate these goals with Land-Use
Planning (LUP) guidelines. Ale, already with a mind-set on a QRA
approach, stepped in during this study, went from industry
3
employment into a governmental one and spent his energy in
further developing QRA for LUP purposes.
3.1. The past of QRA and LUP, and the role of Ben Ale
Since the founding in the Netherlands of a Directorate for the
Environment in the Ministry dealing with Housing and Land Use
Planning (VROM) in 1982, the tasks of the ministries SZW (Ministry
of Social Affairs) and VROM have respectively been focused on safety
of workers (internal safety) and of residents (external safety). This
focus, at that time, entailed a difference in approach: where SZW
tried to foster safety in the process industry by suggesting hazard
identication methods and prevention measures in a deterministic
way, VROM had been examining risks outside the plant premises in a
probabilistic setting. The SZW ofcials emphasized identication of
hazards and measures to prevent, the VROM ofcials focused on the
quantication of failure frequencies and consequences and on
acceptance criteria of risk level tolerability for the public at large.
Ben Ale, from 1980 on no longer working for DSM but working at
VROM, managed to get risk acceptability criteria cast into law and
arranged for funding to develop QRA (Quantitative Risk Assessment)
tools by contracting consultancy agencies such as DNV and TNO.
QRA is in principle a very powerful technique, with ample
application possibilities. The formal beginning of QRA was the
Rasmussen WASH-1400 study published in 1975 for the Nuclear
Regulatory Commission (Rasmussen, 1975). Prof. Norman Ras-
mussen became known in safety engineering communities
worldwide by applying the probabilistic approach to risk assess-
ment to the nuclear industry for the rst time. Under his leadership
the 1975 Reactor Safety Study, also called the Rasmussen Report,
was compiled that studied the public risk of nuclear power plant
accidents and became one of the most inuential elements in nu-
clear safety and licensing internationally. The probabilistic risk
assessment methods (abridged to PRA which is equivalent to QRA)
introduced by that report are still used routinely today in nuclear
power plant safety assessment and in other industries.
In the United Kingdom, the Canvey Island study was carried out
in 1978. This study concerned a proposed extension of chemical
processing on an industrial site in the Thames estuary. In this study,
consideration was given to the acceptability of risk.
In the Netherlands, instigated by the Committee for the Pre-
vention of Disasters, (CPR), and commissioned by the Rijnmond
Council, in 1978 a pilot study was performed into the risks of six
hazardous objects in the Rijnmond area (reported in 1979 by
Cremer and Warner but published three years later as COVO, 1982).
It is interesting to note that a number of weaknesses of the QRA
method of which we are aware today, such as the large uncertainty
margins (at least an order of magnitude) and the problem of taking
account of human error, and other failure causes, have been already
found in this study. Also, a number of knowledge gaps on, e.g.,
consequence calculations were identied. This study concluded,
among other things, that the disaster potential of these objects was
signicant and that it would be advisable to develop a more
automated methodology to perform the many technical and sci-
entic calculations involved in such an analysis.
Since the European Seveso legislation, at that time the Seveso I
Directive (European Commission, 1982), can be fullled with national
legislation, the chemical industry in the Netherlands had few options
and found a modus vivendi with the QRA approach. Ben Ales work at
that time of the early 1980s has thus been to promote QRA as a
methodology and a tool to enhance process safety. Two extensive
early studies with industry participation have been LPG-Integraal and
LNG-aanlanding, which concern, respectively, the safety of produc-
tion, storage, distribution transportation, and consumer use of liq-
ueed petroleum gas (LPG) as a car fuel in comparison to that of
4 H. Pasman, G. Reniers / Journal of Loss Prevention in the Process Industries 28 (2014) 2e9
gasoline, while LNG-aanlanding was about the safety of ship tankers
entering Rotterdam harbour and off-loading to LNG storage tanks
accompanied with re-gasication units to feed the distribution net.
These efforts had a lasting effect and led directly to the start of
developing the series of coloured books1 (the Yellow book on physical
effect calculation, the Red book on statistical methods, the Green book
on damage probits, and the Purple book as a manual on how to
perform QRA), which form the modelling background to present day
QRA. The development of the books stretched over a period of more
than fteen years. An initial version of the Yellow Book drafted by TNO
came out in February 1977. In order to make the use of QRA viable in
policy making and execution on a national scale in the Netherlands,
Ben and his co-workers understood that it was necessary to have
involvement and support of the various groups of stakeholders. In the
Netherlands, the method and data were therefore developed in
committees consisting of representatives and experts from across
government, research, and industry under the auspices of the CPR.
Discussions have been heated at times but nally resulted in publi-
cation of the famous coloured books mentioned, at present well-
known and used throughout the world, but regretfully not updated
anymore.
Following the so-called COVO study by Cremer & Warner in the
early 1980s (COVO, 1982), a project was initiated by the Dutch
government to put these methods into a computerized tool for
general use in a more cost-effective way and in addition improving
accuracy. This way, the studies led to the development of the
necessary supporting software, with a solid and lasting co-
operation between the Dutch authorities and the contractor,
Technica Ltd., acquired by Det Norske Veritas (DNV) in 1990.
Technica provided rst WHAZAN, a spring of ideas for many risk
analysis professionals, and then SAFETI, and later PHAST, which are
today the most widely used tools in risk assessment in the chemical
process industry. The work had taken some 30 man-years of effort,
the nal result being an individual risk plot as well as a societal risk
curve. The reason why Technica Ltd. was chosen for this task is
amongst others the experience and expertise of their London-based
employees in the aftermath of the Flixborough disaster (which took
place in 1974) and the Sizewell B Enquiry (Ale, 2005). As a result,
the approaches and methodologies employed in the UK and in the
Netherlands were very similar.
Because of his responsibility in VROM, the focus of the work
carried out by Ben Ale and his team, has been in off-site risk and
Land-Use Planning. This follows closely the requirements specied
in the Seveso directive. A milestone has been the acceptance by the
Dutch Parliament of the risk acceptance criteria for individual and
group risk proposed by VROM after various orientations. This was
accomplished by acceptance of the Environmental Program of The
Netherlands 1986e1990 (NMP, 1986) with a later Parliamentary
Note elaborating the risk acceptance principle (NMP, 1989).
Following the reasoning developed in the UK (HSE, 1984) the in-
dividual risk was set at 1% of the minimum risk of natural death in
the following year which is 104 at age 10e14, hence a limit value of
106/yr. (until 2004 for existing situations, 105/yr.). Risk is negli-
gible below 108/yr. The Dutch societal or group risk criterion is
rather severe. Initially it was a hard limit determined at a proba-
bility, F of less than 105/yr. at 10 or more immediate fatalities in
the public, N or less than 107/yr. at 100 or more (or according to
the equation FN2 < 103). The negligible level was set at a 100 times
lower frequency value. The steep slope of 2 in a logelog plot is
justied by the disproportionally larger societal disruption of ac-
cidents with higher numbers of fatalities and by taking account of
1
The books are available from the website: http://www.publicatiereeks
gevaarlijkestoffen.nl/.
the growing uncertainty at lower probability values. Later in 1993,
when it was attempted to assess the risks of Amsterdam Schiphol
airport on this basis, the criterion could not be met. In the years
following, the criterion was modied and became instead of a
limiting, an orientation value, while the negligible level was
abandoned. Today, in case of granting a license, the group risk value
serves as a yard stick for a decision maker to judge a potential
disaster situation as controllable or not from the point of view of
emergency response, taking into account self-rescue possibilities.
This is called the authoritys duty of accountability.
Along the way, Ben Ale faced and resolved some difcult issues,
for example how regulation should address the problem of com-
panies in an industrial complex, each with overlapping risk maps,
and the contrast between European regulatory styles with origins
in the Napoleonic code and British style based on precedent. There
is also a contrast between the QRA focused approach, as typied by
the Dutch major hazards regulation with respect to LUP, and the
loss prevention focused approach. One looks at major hazards
problems in the large, with focus on the threats to the communities
situated outside the company fences. The other looks at major
hazards at a detailed level applying HazOp and FMEA tools iden-
tifying with all the minutiae of how equipment failures, errors, and
circumstances come to cause major accidents. Hence, the detailed
identication, so called classical approach, was for the LUP pur-
pose replaced by a top-down one in which all possible scenarios
for a containment are reduced to, e.g., just three: sudden, complete
rupture of vessel or pipe, an emptying of the containment in
10 min, or a continuous leak from a specied hole without regard of
protection measures. The loss prevention approach thus focuses
especially on what one can do to prevent loss of containment oc-
currences and mitigate their effects, while the application of a
concrete, mathematically formulated risk acceptability criterion is
not a requirement.
A separate problem forms the associated containment failure
frequencies. This has been a bone of contention from the start
largely because of a lack of appropriate historical data. First of all, in
the QRA for LUP failure due to human operator error, maintenance
neglect, hence corrosion, but also vibration and thermal cycling are
disregarded, because good management is assumed. Secondly, for a
basic failure rate of, e.g., pressure vessels in the COVO, 1982 study,
use was made of studies and data on steam vessels (e.g., Pasman,
2011). These data still form the basis (Purple Book) despite many
discussions with users and scientists, and despite steel quality, ef-
fect of chemicals, and quite a few other variables of inuence may
be different (Industry keeps its data as far as these are collected and
useful proprietary). The assumptions led to very low failure rate
values, which are challenged by HSE in the UK (HSE, 2012) taking
them an order of magnitude larger. The impression exists that the
low Dutch failure rate values have emerged also under the pressure
of industry, which felt restricted by the severe risk acceptance
criteria.
In the Netherlands, planning authorities are often confronted
with conicting interests when judging the acceptability of devel-
opment proposals near major hazard installations, or proposals to
extend existing plants. Indeed, often only little space is available for
new housing developments, due to industrial parks and population
centres being located together. Such a situation obviously may lead
to important repercussions from new urban developments on the
viability of the local communities. After applying the QRA meth-
odology (which was elaborated during the 1980s) on one particular
case of a proposal/plan for new housing developments in the di-
rection of DSM, a major Dutch chemical site, around 1990, a debate
in the Dutch Parliament took place. The competent authorities
involved came to the following conclusions (Ale, 1991): (i) Some
action will be taken at the plant to improve the safety situation, and
 H. Pasman, G. Reniers / Journal of Loss Prevention in the Process Industries 28 (2014) 2e9
(ii) No new housing developments will be allowed within the 106
contour. The line denoting risks of 108 per year shall be considered
as indicative. This means that when new building activities are
being planned, this safety line must also be taken into account.
Hence, at this point in time the link between QRA and LUP was
set in the Netherlands. Although the uncertainties accompanying
the physical modelling and the consequences calculations on the
one hand, and the probabilistic modelling and the frequencies on
the other hand, were signicant, the results of the QRA modelling
were judged to be close enough to observed effects and frequencies
to be usable in decision making (Ale, 1991).
Initially the regulation applied only to Seveso installations. After
the Enschede reworks explosion in 2000 with 19 fatalities among
residents and 4 emergency responders while 950 injured, the
regulation base was widened to include other hazardous in-
stallations (BEVI, 2004). To boost model robustness, this was fol-
lowed by a standardization of the calculation model to be applied
for a license. Till then, in the free market approach any consultant
could use any model and perform a calculation for a clients safety
report. This led to a large variability in outcomes and also to se-
lective choice of model and data to obtain desired results. There-
fore, one version of the DNV model was declared the ofcial one:
SAFETI-NL, the input data was prescribed in a manual (RIVM,
2013) and the analysts given a training course. This removed
large part of the variability and strengthened the juridical basis,
because in a situation of conicting interests a large variability in
model outcomes does not favour transparent decision making, to
say the least. However, a different question is whether the results
are still realistic and whether the possibility of working to a wishful
result is now excluded.
Another problem with QRA is that the results can be mis-
interpreted, like outside the individual risk zone it is safe and
inside the zone it is unsafe. This is a most common mistake. Apart
from the fact that the calculated risk value is averaged over time
and is determined by the parameters considered while others are
left disregarded, there are many further uncertainties which can
drive the calculated risk value with at least an order of magnitude
up- or downwards such as non-identied risks, model simplica-
tions, and inaccurate or missing data. The standardization effort by
VROM of model and data was of course an important improvement
by reducing variability by analysts but only in a relative sense. The
societal risk criterion triggered much discussion and an improve-
ment program how to apply its results is still running. The public
and even in quite some cases the decision maker does not under-
stand the concept of probability and its uncertainty sufciently to
make a right judgment. In case of a potential large consequence in
combination with a very small probability, perception is focused on
the consequence. Even in the recent case of a carbon dioxide
sequestration pilot planned in a depleted natural gas cavity below
the city of Barendrecht, near Rotterdam, despite a QRA result
allowing a license easily, population objected the plan ercely and
in the end won. Carbon dioxide presents at higher concentrations
in air (10% or more) a hazard and can be lethal. Since carbon dioxide
has no odour and is not visible, fear led to resist the plan. This fear
was fed by media attention to a case of some people falling un-
conscious in the streets of the German city of Mnchen-Gladbach
due to carbon dioxide dispersion after an industrial re extin-
guishing installation was triggered by a re in a low wind inversion
layer condition. Also, it was argued that the price of real estate
property could plunge. Used as a rational argument was the un-
certainty in, e.g., the dispersion model, which was reluctantly
admitted by the experts. But what is the present situation as
regards the use of QRA and LUP, in the Netherlands, and also within
an international context? We discuss this question in the next
section.
5
3.2. The present of QRA and LUP, and the international arena
Quantitative Risk Assessment (QRA), or also sometimes called
Probabilistic Risk Analysis (PRA), is anno 2013 well-known
throughout the world, and used in many organizations and by
many competent authorities. It is always rather easy to explain the
principles to students in process safety; they are catching the
quintessence immediately. The general approach of QRAs today is
actually the same as that of the very early QRAs developed in the
1980s. QRAs have only been more rened, are now more sophisti-
cated and more detailed, are more integrated, and use better and
more accurate models, thanks to more data and better focused in-
formation. The continuous optimization of the software programs
and models (e.g., dispersion calculations, effect simulations, etc.)
goes hand in hand with the advances in computer technology and
the continuous improvements of calculation power with computers.
However, despite all improvements, the limitations are quali-
tatively still the same as already noted in early studies as COVO,
1982. In two bench scale project exercises in 1992 and 2002 the
EU has probed the reproducibility (robustness) by having experi-
enced teams perform an analysis on a given case of an ammonia
release (see for a brief description e.g., Pasman, Jung, Prem, Rogers,
& Yang, 2009). In the rst exercise the scenario was kept the same
for all teams, but despite this simplication, concentration results
differed over two orders of magnitude and risk over four. The 1992
result was reason for the EU to set up a heavy gas dispersion Model
Evaluation Group developing an evaluation protocol which pre-
scribed assessment of the model equations, of validation of results
against experimental data from an authorized database and soft-
ware verication (project SMEDIS, see Duijm, Carissimo, Mercer,
Bartholome, & Giesbrecht, 1997). In the 2002 ASSURANCE exer-
cise the teams had also to identify scenarios. Variability in risk was
still four orders of magnitude in spite of the dispersion models
evaluation. Major sources of variability appeared to be the scenario
descriptions by the analysts, failure rate assessments and choice of
models. In a 2000 HSE report prepared by DNV (HSE, 2000) with
the objective to compare accident experience with QRA method-
ology, the limitations are summarized as follows:
1. Errors, in the formal sense of scientic measurement.
2. Uncertainties in the modelling process and in estimates of
biological effects such as (in human) vulnerability.
3. Uncertainty in whether or not there is indeed an effect to be
incorporated in an estimate.
4. Omissions of possible causes of risk due to:
 Incomplete analysis of the mechanical or engineering sour-
ces of plant failure
 Non-quantication of human error.
 Omission of extreme natural causes.
The omissions include design and construction errors that are not
insignicant and that can appear unexpectedly after many years of
operation. Design errors can be numerous and of much different
types, see e.g., Taylor, 2007. In addition, the mentioned points are
apart from accidents occurring when an installation is in an abnormal
state as start-up, and turn-around, where practice shows that about
40% of all accidents originate. Also, the effectiveness of a plants
emergency response and exposure variability are not factors taken
into account. Hence, the reality value of QRA, whether this is by means
of a top-down scenario denition approach with a standardized
model or the classical one, leaves still much to be desired. The
importance one attaches to this reality value depends on ones view
and the purpose of the analysis. If the results are treated in a relative
sense for the purpose of risk reduction and one just considers fre-
quency values and risk results in orders of magnitude, the problem is
6 H. Pasman, G. Reniers / Journal of Loss Prevention in the Process Industries 28 (2014) 2e9
not dramatic. In fact, the very useful and much applied Layers of
Protection analysis, which is a kind of mini-QRA, is more or less on that
basis. If, on the other hand, the results are used to convince a (usually
non-technical) decision maker and the public at large, by comparison
against a norm, that a plant is safe or rather that the risk level is
tolerable, often a problem will arise. This problem is mainly, because
uncertainties in scenarios and causes of risk boil down to higher
failure rates than assumed, which undermines in the long run the
credibility of the calculated risk value. Uncertainty and lack of accu-
racy were also drivers behind the advice of the meanwhile disbanded
Advisory Council on Hazardous Materials in the Netherlands to
overhaul the QRA approach for LUP with respect to presenting and
using the results perhaps more the way it is adopted in France (see
below). Meanwhile, further model improvements and the availability
of relevant historical failure data shall be sought.
Modern development sees the two previously mentioned ap-
proaches, the QRA focused approach and the loss prevention
focused approach, not just converging, but even becoming ever
more integrated. Sadly, the primary force for movement in this
direction was again a major accident, the Piper Alpha oil platform
disaster in 1988, and the subsequent Cullen enquiry. The heavy
emphasis on safety barriers, safety barrier performance standards,
and schemes of inspection derives to a large extent from this source.
This disaster and following enquiry has forced risk analysts to relate
much more closely to engineering teams, who are the custodians of
the barriers. It has even led to the development of process risk
analysis and safety engineering as an engineering discipline. The
risk analysis component raises what was once known as loss
prevention, into a true, science-based engineering discipline. A
consequence of this has been the advent of risk-based engineering
standards such as IEC 61511, API 752, API 580 and many risk based
engineering guidelines. Following the two EU-projects revealing
the outcome variability mentioned above, the EU sponsored in its
5th Framework Programme the well-structured ARAMIS project
(A Risk Assessment Methodology for Industries in the framework of
the Seveso II directive) with contributions from a number of Eu-
ropean partners. The resulting methodology has been an important
step forward (see the series of articles introduced by Salvi & Debray,
2006). An example of signicant other development is the work of
Cozzanis group in Italy contributing to control of escalation risks
and domino effects (Cozzani, Gubinelli, & Salzano, 2006) and on
natural disaster causing loss of containment events, e.g., earthquake
(Antonioni, Spadoni, & Cozzani, 2007; Campedel, Cozzani, Garcia-
Agreda, & Salzano, 2008) and ooding (Cozzani, Campedel,
Rennia, & Krausmann, 2010). The latter efforts are important to
LUP and culminated in the EU sponsored 7th Framework Pro-
gramme project iNTeg-Risk (Early Recognition, Monitoring and In-
tegrated Management of Emerging, New Technology related Risks;
http://www.integrisk.eu-vri.eu/) which started in December 2008
and runs till May 31, 2013. A contribution performed within this
project by Krausmann, Cozzani, Salzano, & Renni1, 2011 describes in
general the signicance and the way to manage so-called NaTech
risks due to industrial accidents triggered by natural hazards such
as earthquake, lightning and ooding. iNTegRisk has a much wider
scope and aims to identify emerging technology risks such as that of
nanotechnology and to create an integrated scientic and tech-
nology framework for analysis and management of emerging risks.
Individual risk and societal risk acceptability criteria have been
established in various regions around the world. Individual risk for
residential areas is usually acceptable from 106/yr. downwards
(CCPS, 2009). Fig. 1 presents an overview of societal risk criteria.
Although there are countries such as the United Kingdom,
which have for many years followed a risk based approach for LUP
similar to that in the Netherlands, the QRA approach for LUP has not
been adopted everywhere. Even in Europe, important discussions
Fig. 1. Societal risk acceptance criteria in 2013 in various regions of the world: from
top to bottom non-acceptance borderlines of United Kingdom, Flanders (Belgium),
Australia, Hong Kong, Switzerland, the Netherlands, (Meyer & Reniers, 2013). The lines
below which risk is negligible, are parallel to the non-acceptance ones of each country.
(Flanders has no negligible risk line; in the Netherlands the criterion is only a so-called
orientation value: a decision maker can ignore it if he judges it justiable. The Dutch
negligible risk line e the lowest one shown e is from before 2003 and not in use
anymore.)
and debates are on-going. As Christou, Gyenes, and Struckl (2011)
argue, because of historical, cultural, administrative, legislative,
and other reasons, the risk assessment methods applied in EU
Member States to support LUP vary signicantly. The existing
methodologies can be divided into the following categories:
1. Deterministic approaches with implicit judgment of risk
2. Consequence-based approaches
3. Risk-based or probabilistic approaches
4. Semi-quantitative approaches
To illustrate the difference between the different Member
States, some examples of the use of risk assessment approaches for
Land-Use Planning are given for different countries.
In Denmark, the emphasis from the start was different from that
in the Netherlands. The Danish Green book, which provides a
background to the implementation of the Seveso Directive in
Denmark, focuses on hazard identication and HazOp, with QRA
being relegated to a supporting role. The emphasis is on the design
of safe plants, safe operation, and on mitigation. The main tools are
HazOp and quantied safety barrier diagrams. Nevertheless, when
needed, the national guidance on risk acceptance (Qualitative and
Quantitative Criteria for Risk Acceptance) is based for a major part
of its direction on the Dutch model. When Quantitative Risk
Assessment was used for a comparative risk assessment of all the
countrys installations with major hazards potential, it was the
Dutch standardization of consequence modelling that provided a
basis for calculations.
In France, after the introduction in Europe of the Seveso II
Directive in 1996, INERIS had three years to implement the Euro-
pean requested regulations, that is, risk analyses, risk assessments,
modelling, etc. In this regard, LUP was discussed in one of the
Technical Working Groups, and QRA was one of the possible ap-
proaches to perform hazard and risk assessments, and to indicate
how to take into account barriers in risk assessments.
Until the major accident of Toulouse in 2001, there was no real
change in Frances policy regarding risk analysis and it was mainly
deterministic, thus focussing on the consequences and not on the
 H. Pasman, G. Reniers / Journal of Loss Prevention in the Process Industries 28 (2014) 2e9
probabilities of an event. This is the case since the Napoleontic law
of 1810, where companies only receive a permit to operate after
having taken some required LUP measures. After the Toulouse
disaster, risk analyses had to take into account reducing measures
much more extensively. The reasoning is that, if reducing measures
are well considered and implemented, this leads to scenarios that
are better than worst-case scenarios, and they are more effec-
tively handled. Since the Toulouse accident, so-called reference
scenarios, which are the worst-case scenarios and which are
actually the permit scenarios, are used for LUP. Passive safety
equipment can be used that leads to residual risk scenarios for
demonstrating that risks are under control.
In France, since 2001, there has been much discussion about the
data used for QRAs. There is hesitation from companies towards
what they call generic risk assessments, because a lot of generic
data are used, and the outcome of a QRA study largely depends on
the input data used. Therefore, the basis of regulations is still much
more consequence based than probability-driven. To restore the
publics condence in chemical industrial activities after Toulouse,
France uses a so-called buffer zone, which is determined based on
the consequences of accident scenarios. Land is regained and given
back again to the industry: when residential area or land becomes
available around an industrial activity, the industry has the rst
option to buy it. Another feature is the effect of the difference of the
time during which a disaster can develop on the tolerability of the
severityeprobability combination.
Nevertheless, although QRA is still not obliged in France,
employing QRA for LUP helps to inform a decision and is now more
used by companies than before 2001 to gain insight and to un-
derstand the risks at hand.
In Germany, QRA is not used at present. The approach is deter-
ministic, not probabilistic, and only uses appropriate distances.
Although the Seveso directive gave impulse towards a discussion
about deterministic and probabilistic viewpoints, the general ten-
dency within Germany is still deterministic. In general, industrys
initial reluctance to apply a probabilistic approach has diminished
over time. Due to international pressure, that is the international
demand to use probabilistic analyses (for example in Flanders, the
Netherlands, UK, and US), the possibility of probabilistic safety
analysis is, rethought, and reconsidered and the matter is discussed
(e.g., Hauptmanns, 2005). There is however much opposition due to
the claim made by ecologists that using probabilistic analysis would
lower the safety standards. Furthermore, Germany is reluctant to
use QRAs due to not enough data/info being available to obtain
reliable results. Hence, in Germany, as in France, QRA for LUP is
being discussed as possibly being an interesting complementary
exercise for those companies interested, but certainly not a sub-
stitute for the deterministic approach. In Germany, the reason for
reluctance is also administration-related. There has been a signi-
cant reduction in personnel of inspection services in Germany, and
inspection ofcials therefore have (time and competence) dif-
culties with implementing new methods of any kind.
In summary, at present, there is neither a unifying European
policy on the matter of QRA and LUP, nor on the used risk accep-
tance thresholds. Every region and/or country makes its own de-
cision on the use of QRA for gaining insights into individual risks,
societal risks, and on the use of QRA for LUP. On the other hand,
there seem to be no signicant differences noticeable in safety
performance levels of industry working under the umbrella of the
Seveso directive.
3.3. The future of QRA
Despite all the shortcomings of a QRA at present, and despite the
obvious fact that it is not an exact description of reality, it is to date
7
the best available, analytic, predictive tool to assess the risks of
complex processes, storage facilities, and hazardous material
transport systems to contribute to process safety. It is predictive in
the sense that it identies what kinds of accidents can happen and
how probable these accidents are. QRA is analytic in forecasting
potential upsets and not just analytic in hindsight as are accident
investigation methods. It is suited to assess situations where a large
number of possible accident outcome cases are at hand, and at the
same time sophisticated models are required to determine poten-
tial consequences. Life without risk is not possible, and the tool
provides the best method to determine optimum investment in risk
reduction even taking uncertainty into account as e.g., in a Value at
Risk approach (Prem et al., 2010) or to determine the best emer-
gency response given needed capacities. Obviously, even more
renement of the tool is required for achieving its full potential and
for increasing its validity and use throughout the world. Guidance
for presenting and using its results in decision making in view of
the current large uncertainty will be helpful. In LUP application
conveying the right risk level impression to the public requires
further guidelines. The papers of the International Risk Governance
Council (e.g., Renn et al., 2006) are most helpful in this respect.
Some ways for enhancing the technical aspects of QRA are provided
in the next paragraphs.
It will be important in the coming years to build ever more reli-
able QRA software. QRA needs to make much more use of detailed
hazard identication concepts, including those of human reliability,
domino effects, etc. A very promising approach to describe scenario
cause-effect chains effectively is by applying Bayesian networks
(BNs) (for foundations Pearl, 2000; and later editions; for details
Darwiche, 2009; for a more popular description with many types of
application Fenton & Neil, 2013; for use in risk analysis Bedford &
Cooke, 2001;and for application in projects Ale et al., 2009; 2012;
Hanea et al., 2012 and earlier papers). BNs are of the family of acyclic
directed graphs just as Fault trees and Event trees but have a more
extensive causality foundation, are more exible and versatile, e.g.,
taking account of common cause failures, handling probability
density functions and time effects such as warm spares, wear and
repair, etc. BNs are very suitable for diagnosis and have become
practical lately by easily obtainable software, which takes away the
burden of building the joint distributions of the random variable
nodes. It also allows fuzzy relations and is therefore suited for
absorbing expert elicitation results on e.g., human error and man-
agement inuences. As a further step, consequence modelling should
be drastically improved, which will require considerable investment
since tests on a realistic scale are expensive but will be indispensable.
Current consequence models are in general good enough at large
distance (the so-called far eld), and therefore relatively suitable for
land use planning, but in many cases they are too crude to be used for
in-plant engineering purposes. The value of accident mitigation
especially needs to be incorporated into QRA in a way that leads to
adequate engineering applications. Frequency data should also be
largely improved, which so far has proved to be most difcult, since it
requires an abundance of data, expertise, and patience. Predictive
modelling of failure based on rst principles is not possible, even not
on micro-scale level. Random effects always affect failure outcomes.
Failure prediction under uncertainty implies that one needs to
observe relatively large numbers of failures of a certain type or mode
under dened conditions to apply statistics with a sufcient high
condence level. This means that one usually has to go to a rather
detailed component level to collect a sufcient number of samples.
Given sufcient data, cause/consequence analysis and covariate
models would reduce aleatory uncertainty. Until now, only a few
signicant efforts have been made, each within a fairly narrow
domain, and with limited validation of the completeness or accuracy
of the collected data when compared with the true purpose, which is
8 H. Pasman, G. Reniers / Journal of Loss Prevention in the Process Industries 28 (2014) 2e9
the prediction of real accident frequencies. Following the Piper-Alpha
disaster in countries such as Norway and the UK, good work has been
done with respect to offshore installation components resulting in
the OREDA, 2009 database. For use on Land Use Planning cases HSE
(2012) keeps a frequently updated database with carefully selected,
detailed generic component failure data.
As mentioned, data and the right interpretation are crucial.
Because over the last decade companies for predictive life cycle
management of their process equipment assets have turned to risk
based inspection and because of the wide application of IEC 61511
for safety integrity systems from a technical point of view, an
abundance of equipment reliability data should become available.
However, these data shall be mainly proprietary. In the Netherlands
and the UK, over the years many work accidents have been inves-
tigated, not only with hazardous materials but accidents in general.
An accident investigation tool making use of a bow-tie structure,
called Storybuilder, has been developed and applied to 20,000 or
more accident cases which resulted in much information on acci-
dent scenarios and event frequencies (Bellamy et al., 2008). Also,
w60,000 workers risk exposure data have been collected. This in-
formation will greatly help to investigate human error in industrial
accidents. As regards process installations, HSE (UK) and RIVM (NL)
are cooperating to collect improved failure frequencies of installa-
tion components such as piping, pumps, anges, and vessels. This
study is through the analysis of reported plant accidents by means
of the mentioned Storybuilder tool. This tool will identify failure
type, while failure frequency is derived from estimation of the
number of that particular component present in the plant based on
its size.
Because risk is the measure of lack of safety, to monitor safety of
an operation it would be desirable to perform real-time risk analysis,
thereby measuring continually risk level uctuations by tracking
short term effects (e.g., weather, repairs going on, human error, or
large people exposures) or slower, middle, and long term changes
(e.g., delayed inspections, wear, or cultural change). Although there
are thoughts in that direction (Pasman H.J. and Knegtering B., The
Safety Barometer, submitted to JLPPI) it will for the time being, be
utopian. Adding sensors (for detection of vibration, sounds, smells,
unduly energy use, etc.) and making use of the process control sys-
tem (Distributed CS) short-term risk level changes can be monitored.
Slow risk uctuations due to changes in human error levels inu-
enced by management practices and environmental conditions are
difcult to grasp, but in the longer term not impossible.
Human reliability analysis as part of QRA has already quite a
long history, in particular in nuclear safety, resulting in such models
as THERP and ATHENEA. Although Hollnagel in various publications
(e.g., Hollnagel, 2007) doubts whether human behaviour can be
fully decomposed and failure frequencies determined as if it were
equipment components, lately Groth and Mosleh (2012a and
2012b) published results of extracting performance inuencing or
shaping factors from human events databases taking into account
factor interdependencies represented in a BN. Vinnem et al. (2012)
relate historical mishap data via scores with risk inuencing factors
on management performance levels affecting in turn worker per-
formance which shows up as human error level causing risk by
maintenance failure, all modelled in a BN. Ales team has been
working recently with the same goal, namely to account in QRA for
human reliability inuenced by management (Ale et al., 2012;
Hanea et al., 2012), but Ale went for incorporating expert elicitation
and processing results in a BN following the line of thought in an
earlier study on aviation safety (Ale et al., 2009). Another idea
might be to make use of process safety performance indicator
values that are the result of the effectiveness of the safety man-
agement system reecting the culture. Experts could weigh the
relative importance of indicators for the reliability of functioning of
the various components and the installation as a whole directly or it
may be preferable to separate it in a management level inuencing
a human worker level following Vinnem et al. (2012) and relating
the variables in a BN.
QRA might also adapt to system approaches of safety of com-
plex socio-technical systems. Protagonists Leveson (2011) and
Venkatasubramanian (2011) have rightfully emphasized the impor-
tance of a system approach to safety. Classical risk analysis approach
is static. It decomposes a system and focuses on failure events of
components. This approach is not sufcient to explain all what can go
wrong, because it does not grasp the dynamics of unsafe interactions
of well functioning components or faulty risk control actions of hi-
erarchical levels of a system communicating information and feed-
back to each other. Leveson (2011) developed the system approach.
After determining the system boundary, identifying potential losses
and the hazards to the system, the safety constraints must be dened
and the risk controls found. Then, the possible failures in control
actions must be identied. For this purpose, Leveson developed the
System-Theoretic Process Analysis (STPA) tool, which is HazOp like.
Right now it has proven to function very well in accident in-
vestigations, but it obviously has the potential to become predictive
and quantitative given human and automated control system reli-
ability data under a large variety of conditions.
From the point of view of process safety, it may also be helpful to
use the QRA tool together with process simulation to determine how
resilient a process is both in a technical and in an organizational
sense and whether it contains sufcient reserves to absorb unex-
pected disturbances without ending in an unsafe state. For factors
important to resilience, see Dinh, Pasman, Gao, and Mannan (2012).
4. Conclusions
It is a fact that major accidents and disasters are the main drivers
of political and public attention towards safety and risk assessment.
This fact is no different for the elaboration of Quantitative Risk
Assessment and its use for Land-Use Planning in relation with
chemical industrial activities. Disasters such as Flixborough, Pernis,
Beek, Seveso, and Piper Alpha all have played an essential role.
Using QRA for Land-Use Planning has been initiated in the
Netherlands, which is one of the most densely populated, indus-
trialized areas in the world, and has been internationally exported
towards other regions and countries. Because industrial activities
with major hazards are increasingly carried out in ever more
densely populated areas, and land is getting more and more scarce
throughout the world, it is very likely that ever more regions and
countries will use QRA for making Land-Use Planning decisions in
chemical complexes. The importance of QRA goes, however, much
further than just LUP: its main purpose is to strengthen process
safety by risk based safety management and to support the process
industry business by enabling risk management.
It is clear that there is still much to develop and improve in the
eld of process safety and risk analysis, as the continuing sequence
of major hazard accidents illustrates. It is safe to say though that the
number of accidents would have been much larger without the
development of process safety engineering and that fatalities and
injuries amongst workers and public have been greatly reduced
also by the application of quantitative risk techniques over the last
30 years. Ben Ale, presently professor emeritus of the Delft Uni-
versity of Technology, has made a strong and indelible contribution
to this safety success.
Acknowledgements
The authors would like to express their gratitude to the following
experts (in alphabetical order) for their cooperation and inputs to this
 H. Pasman, G. Reniers / Journal of Loss Prevention in the Process Industries 28 (2014) 2e9
study: Ben Ale, Ulrich Hauptmanns, Joy Oh, Olivier Salvi, John Robert
Taylor, and Koos Visser. We also acknowledge the constructive com-
ments and editorial guidance by William Rogers of MKOPSC.
References
Ale, B. J. M. (1991). Risk analysis and risk policy in the Netherlands and the EEC.
Journal of Loss Prevention in the Process Industries, 4, 58e64.
Ale, B. J. M. (2005). Tolerable or acceptable: a comparison of risk regulation in the
United Kingdom and in the Netherlands. Risk Analysis, 25(2), 231e241.
Ale, B. J. M., Bellamy, L. J., Van der Boom, R., Cooper, J., Cooke, R. M., Goossens, L. H. J.,
et al. (2009). Further development of a Causal model for Air Transport Safety
(CATS): building the mathematical heart. Reliability Engineering and System
Safety, 94, 1433e1441.
Ale, B. J. M., Hanea, D., Sillem, S., Lin, P. H., Van Gulijk, C., Hudson, P. (2012).
Modelling risk in high hazard operations: Integrating technical, organisational
and cultural factors. PSAM 11, ESREL 2012, Helsinki, 25e29 June.
Antonioni, G., Spadoni, G., & Cozzani, V. (2007). A methodology for the quantitative
risk assessment of major accidents triggered by seismic events. Journal of
Hazardous Materials, 147, 48e59.
Bedford, T., & Cooke, R. (2001). Probabilistic risk analysis, foundations and methods.
Cambridge: Cambridge University Press, ISBN 0 521 77320 2.
Bellamy, L. J., Ale, B. J. M., Whiston, J. Y., Mud, M. L., Baksteen, H., Hale, A. R., et al.
(2008). The software tool storybuilder and the analysis of horrible stories of
occupational accidents. Safety Science, 46, 186e197.
BEVI. (2004). Besluit van 27 mei 2004, houdende milieukwaliteitseisen voor externe
veiligheid van inrichtingen milieubeheer (Besluit externe veiligheid inrichtingen).
Staatsblad 2004, 250 (translation: External safety installations decree).
Campedel, M., Cozzani, V., Garcia-Agreda, A., & Salzano, E. (2008). Extending the
quantitative assessment of industrial risks to earthquake effects. Risk Analysis,
28(5), 1231e1246.
CCPS. (2009). Guidelines for developing quantitative safety risk criteria 2009. NY
USA: AIChE.
Christou, M., Gyenes, Z., & Struckl, M. (2011). Risk assessment in support to land-use
planning in Europe: towards more consistent decisions? Journal of Loss Pre-
vention in the Process Industries, 24, 219e226.
COVO. (1982). COVO Commission, risk analysis of six potentially hazardous industrial
objects in the Rijnmond area, a pilot study. A Report to the Rijnmond Public
Authoritx, Central Environmental Control Agency, Schiedam, The Netherlands,
1981. Dordrecht, The Netherlands: D. Reidel Publishing Co, ISBN 90 277 1393 6.
Cozzani, V., Campedel, M., Rennia, E., & Krausmann, E. (2010). Industrial accidents
triggered by ood events: analysis of past accidents. Journal of Hazardous
Materials, 175, 501e509.
Cozzani, V., Gubinelli, G., & Salzano, E. (2006). Escalation thresholds in the assessment
of domino accidental events. Journal of Hazardous Materials, A129, 1e21.
Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge:
Cambridge University Press, ISBN 978-0-521-88438-9.
Dinh, L. T. T., Pasman, H. J., Gao, X., & Mannan, M. S. (2012). Resilience engineering of
industrial processes: principles and contributing factors. Journal of Loss Pre-
vention in the Process Industries, 25, 233e241.
Duijm, N. J., Carissimo, B., Mercer, A., Bartholome, C., & Giesbrecht, H. (1997).
Development & test of evaluation protocol for heavy gas dispersion models.
Journal of Hazardous Materials, 56, 273e285.
EU. (2012). Directive 2012/18/EU of the European Parliament and the Council of 4 July 2012
on the control of major-accident hazards involving dangerous substances, amending
and subsequent repealing Council Directive, 96/82/EC, (Seveso Directive III).
European Commission. (1982). Council directive of 24 June 1982 on the major-
accident hazards of certain industrial activities, 82/501/EEC, (Seveso Directive
I). http://www.ess.co.at/HITERM/REGULATIONS/82-501-eec.htm.
Fenton, N., & Neil, M. (2013). Risk assessment and decision analysis with Bayesian networks.
Boca Raton, FL, USA: CRC Press, Taylor & Francis Group, ISBN 978-1-4398-0910-5.
Groth, K. M., & Mosleh, A. (2012a). A data-informed PIF hierarchy for model-based
Human Reliability Analysis. Reliability Engineering and System Safety, 108, 154e174.
9
Groth, K. M., & Mosleh, A. (2012b). Deriving causal Bayesian networks from
human reliability analysis data: a methodology and example model. Pro-
ceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and
Reliability, 0(0), 1e19.
Hanea, D., Hanea, A., Ale, B. J. M., Sillem, S., Lin, P. H., Van Gulijk, C., et al., (2012).
Using dynamic Bayesian networks to implement feedback in a management
risk model for the oil industry. PSAM 11, ESREL 2012, Helsinki, 25e29 June.
Hauptmanns, U. (2005). A risk-based approach to land-use planning. Journal of
Hazardous Materials, A125(2005), 1e9.
Hollnagel, E. (2007). Modelling of failures: From chains to coincidences. http://www.
resist-noe.org/DOC/Budapest/Keynote-Hollnagel.pdf.
HSE. (1984). The control of major hazards, advisory committee on major hazards. Third
Report. London: Health and Safety Executive.
HSE. (2000). A comparison of accident experience with Quantitative Risk Assessment
(QRA) methodology. prepared by Det Norske Veritas Ltd., ISBN 0 7176 1849 8.
Contract Research Report 293/2000, Crown copyright 2000.
HSE. (2012). Failure rate and event data for use within risk assessments, PCAG chp_6K
version 12 e 28/06/12.
Krausmann, E., Cozzani, V., Salzano, E., & Renni1, E. (2011). Industrial accidents
triggered by natural hazards: an emerging risk. Natural Hazards and Earth
System Sciences, 11, 921e929.
Leveson, N. G. (December 2011). Engineering a safer world, systems thinking applied
to safety (pp. 608). The MIT Press. ISBN-10:0-262-01662-1; ISBN-13:978-0-262-
01662-9.
Meyer, Th., & Reniers, G. (2013). Engineering risk management. Publishing House De
Gruyter, ISBN 978-3-11-028516-1. May.
NMP. (1986). Second Chamber of the States-General, Session 1985e86, 19204, no. 1e2,
Indicatief Meerjarenprogramma Milieubeheer 1986e1990. The Hague: VROM.
NMP. (1989). Omgaan met risicos (Dealing with risks), Second Chamber of the States-
General, Session 1988e89, 21137, No. 5. ISSN: 0921-7371: . SDU editor, The
Hague.
OREDA. (2009). Offshore reliability data handbook (5th ed.).. In Topside equipment,
Vol. 1, Norway: prepared by SINTEF and sold by DNV. Vol. 2 e Subsea Equip-
ment. http://www.dnv.com/resources/handbooks/.
Pasman, H. J. (2011). History of Dutch process equipment failure frequencies and the
Purple book. Journal of Loss Prevention in the Process Industries, 24, 208e213.
Pasman, H. J., Jung, S., Prem, K., Rogers, W. J., & Yang, X. (2009). Is risk analysis a
useful tool for improving process safety? Journal of Loss Prevention in the Process
Industries, 22, 769e777.
Pearl, J. (2000). Causality, models, reasoning and Inference (1st ed.). New York, USA:
Cambridge University Press (2nd ed., 2009, ISBN 978-0-521-89560-6).
Prem, K. P., Ng, D., Pasman, H. J., Sawyer, M., Guo, Y., & Mannan, M. S. (2010). Risk
measures constituting a risk metrics which enables improved decision
making: value-at-Risk. Journal of Loss Prevention in the Process Industries, 23,
211e219.
Rasmussen, N. C. (1975). Reactor safety study: An assessment of accident risks in U.S. com-
mercial nuclear power plants. [NUREG-75/014 (WASH-1400)], Rockville, MD, USA.
Renn, O., & Graham, P. (Jan. 2006). White Paper no. 1, risk governance, towards an
integrative approach. Geneva, Switzerland: International Risk Governance
Council. http://www.irgc.org/IMG/pdf/IRGC_WP_No_1_Risk_Governance__
reprinted_version.pdf.
RIVM. (January 2013). Reference manual Bevi risk assessments. Bilthoven, the
Netherlands: National Institute of Public Health and the Environment, RIVM.
http://www.rivm.nl/Onderwerpen/Onderwerpen/S/SAFETI_NL.
Salvi, O., & Debray, B. (2006). A global view on ARAMIS, a risk assessment meth-
odology for industries in the framework of the SEVESO II directive. Journal of
Hazardous Materials, 130, 187e199.
Taylor, J. R. (2007). Understanding and combating design error in process plant
design. Safety Science, 45, 75e105.
Venkatasubramanian, V. (2011). Systemic failures: challenges and opportunities in
risk management in complex systems. AIChE Journal, 57(1), 2e9.
Vinnem, J. E., Bye, R., Gran, B. A., Kongsvik, T., Nyheim, O. M., Okstad, E. H., et al.
(2012). Risk modelling of maintenance work on major process equipment on
offshore petroleum installations. Journal of Loss Prevention in the Process In-
dustries, 25, 274e292.