1

What is INSE 6320?

INSE 6320 -- Week 1

INSE 6320 is an Information and Systems Engineering course

Risk Analysis for Information and Systems Engineering

You will learn how to:

Assess risk for systems engineering using probability theory and statistics
Use the basic tools of risk analysis: fault trees, event trees, simulation models,
and influence diagrams
Model uncertainty and measure risk through various methods

Go over Course Outline

What is Risk?
Introduction to Risk Analysis

Implement quantitative risk analyses, and develop strategies to identify, assess,

monitor and mitigate risk.

Dr. A. Ben Hamza

Concordia University
2

Roadmap of the Course?

Instructor: Dr. A. Ben Hamza

INSE 6320

Office:

EV 7.631

Lectures:

Thursday

5:45 - 8:15 PM
Risk & Uncertainty

Office Hours:

Thursday 1:30 - 3:30 PM or by appointment

E-Mail:

hamza@ciise.concordia.ca

Statistical
Inference

Probability
Distributions
Weibull
Analysis

Fault & Event

Trees

Reliability

Expert Opinion

Decision Theory

Influence
Diagrams

Risk
Management

Risk
Measurement

Midterm Exam

Final Exam

Grading Policy

Administration
Course web page:

Important Dates:

MyConcordia Portal (Moodle)

Its highly advised to check Moodle regularly.

Syllabus, Slides, Assignments, Projects, etc

Go to MyConcordia Portal (Moodle).
Preliminary exam dates and project due date:

Two Assignments

10%

Midterm Exam

30%

Project

15%

Final Exam

45%

February 12, 2015: Assignment #1 due

February 19, 2015: Midterm Exam
April 9, 2015:
Assignment #2 due
April 16, 2015: Project Report due
April ??, 2015: Final Exam

Final Project

Midterm Exam
February 19, 2015 (in class)

Project due
April 16, 2015
Final Exam
April ??, 2015 (TBA)

Final reports due on April 16, 2015

A final project report, completed individually or in pairs, is required.
The term project will have only one component: written report.
For more details: MyConcordia Portal (Moodle)

Recommended Textbook
Probabilistic Risk Analysis: Foundations and Methods
Authors: T. Bedford and R. Cooke
Publisher: Cambridge University Press, 2001
ISBN-13: 978-0521773201

What this course is about?

This course is about Risk Analysis for Information and Systems Engineering.
Engineering systems are almost always designed, constructed, and operated
under unavoidable conditions of risk and uncertainty.

Risk perception of uncertainty in

events that occur and actions taken.
Risks encountered in everyday
decision-making
Multiple ways to consider risks:

Risk as feelings
Risk as analysis
Risk as politics

We primarily evaluate risk intuitively

(as feelings)

It seems every week there is a new

story about some type of security
breach
That new story says that the security
breach costs the organization
thousands or millions of dollars.
Recent Sony security breach cost is
$100 million
The question is: how do they come up
with those numbers?

Systems Engineering

Systems Engineering--One View

Science: Determines what is

Component Engineering: Determines what can be

Systems Engineering: Determines what should be

11

Definition of Systems Engineering

(NASA SE Handbook)
Systems Engineering is a robust approach to
the design, creation, and operation of systems.

Systems Engineering consists of

Identification and quantification of system goals

So what is Systems Engineering?

Many different definitions
All define a process of developing goals and requirements,

Creation of alternative system design concepts

Performance of design trades
Selection and implementation of the best design
(balanced and robust)

designing the system and development, verifying the

requirements are met at each step.

All include successive refinements and iteration on the above

steps.

Verification that the design is actually built and properly integrated in

accordance with specifications

The important thing is that security analysis be integrated into

whatever systems engineering process you use.

Assessment of how well the system meets the goals

What is a System?

There is no standard definition.

Vague definition from a systems engineers perspective:
A system is a combination of interacting components
operating within an external logical and physical
environment.
Each component has attributes that describe what it
does and how it does it.
Components have relationships with other
components which describe how the components
interact to form a system.
A system also interacts with other elements in its
environment
For the Systems Engineer (SE), a system is the part
the SE has some control over; the environment is what
you have to take as is.
A system has relationships with external components
in its environment. These are critical in the SE process

10

12

Information Systems Security (ISS) Engineering

Part of overall systems engineering process

In a simple sense, security is just another source of requirements

Includes architecture, design, development, deployment

System Architecture: where are the security functions performed? Where are new
external interfaces required to support security?
System design includes selection of commercial products: platforms, operating
systems, networks, etc.
Security requirements should be a part of all product selection criteria (not just
selection of security specific components such as firewalls and crypto)
Security design includes designing the management processes and procedures for
individuals that are required to maintain a secure system throughout its life cycle

Risk analysis is a key part of the requirements prioritization--it lets you know what you
might be losing if you relax a security requirement

13

What is Risk?

Risk Applications
Finance
Risk in investments, insurance etc.,
Industrial
Plant failures, accidents, competitive risks
Political
Impact of decisions, probabilities of success etc.
Nuclear
Plant operation, fuel storage, proliferation of fissile material
Aviation
Safety of airplanes, weather conditions, terrorism impact
Medicine
Weighing different treatment options

15

The probability that a particular threat will exploit a particular vulnerability

Risk can be described in terms of probability (the possibility of risk),

consequence (the loss), and time frame

Probability is the likelihood that the consequence will occur

Consequence is the effect of an unsatisfactory outcome

Time Frame refers to when the risk will occur during the product lifecycle, e.g.
long, medium, short, imminent ...

Risks are future events with a probability of occurrence and a potential for loss

Many problems that arise in software development efforts were first known as
risks by someone on the project staff

Caught in time, risks can be avoided, negated or have their impacts reduced

14

Probability

What is Risk?

16

Risk as a science was born in the sixteenth century Renaissance, a time of

discovery
The word risk is derived from the early Italian risicare, which means to dare
Today, risk is defined as the possibility of loss
Loss Unless there is potential for loss, there is no risk
The loss can be either a bad outcome or a lost opportunity
Choice Unless there is a choice, there is no risk management
Definition:

The likelihood that a particular threat using a specific attack, will

exploit a particular vulnerability of a system that results in an
undesirable consequence.
(Definition from National Information Systems Security (INFOSEC) Glossary,
NSTISSI No. 4009, Aug. 1997)

How likely is a future problem to occur?

Often difficult to define precisely
Probability can be defined as a percentage, a phrase or a relative number:

Probability

Uncertainty

Rank

> 80%

Almost certainly, highly likely

61%-80%
41%-60%

Probable, likely, probably, we believe

We doubt, improbable, better than even

4
3

21%-40%

Unlikely, probably not

< 21%

Highly unlikely, chances are slight

17

Risk

The Risk Equation

19

Risk Management Process

What can go wrong

(Initiating Events)?

How Bad
(Consequences)?

How Often
(Likelihood of failure)?

Risk is the
probability that a
specific threat will
successfully
exploit a
vulnerability
causing a loss.

Risk = Probability x Consequence

= Function(Threat, Vulnerability, Consequence)

Aggregate Risk
(Likelihood of consequences calculated for every
possible combination of precipitating events)

Threat : Any person, circumstance or event with the potential to cause

loss or damage.

Vulnerability: Any weakness that can be exploited by an adversary or

through accident.

Consequence: The amount of loss or damage that can be expected from a

successful attack. Also refereed to as impact, loss or cost

Measures to reduce the consequences of risk until they

reach acceptable levels (Benefits > Aggregated Risk)

18

Risk
Example: Driving to Work

How Bad

Cost-Benefit Analysis

Causes

Head on Collision
Side/Rear-end impact
Hit pedestrian
Overturn Car
Carjacking

Fatigue
Poor Judgment
Environmental
Conditions
Failure to see traffic
signals

Death

(Consequences) Insurance Premium

Hike
Vehicle Damage
Injury
Traffic Ticket

Hazard Control
(Reduce likelihood of damage)
License
Proper road & signal construction
Safety Barriers
Police Surveillance & speed control
Obeying traffic rules

Protection & Damage Limitation

(Reduce Consequences)
Having Airbags Installed in Vehicle
Wearing Seatbelts
First Aid & Hospitalization

Total
Risk

What is Risk Analysis?

The process of identifying, assessing, and reducing risks to an
acceptable level
Defines and controls threats and vulnerabilities
Implements risk reduction measures

Employment
Total
Benefit

Risk = Consequence x Likelihood

Potential Accidents

20

An analytic discipline with three parts:

Risk assessment: determine what the risks are
Risk management: evaluating alternatives for mitigating the risk
Risk communication: presenting this material in an
understandable way to decision makers and/or the public
Risk analysis = Risk assessment + Risk management + Risk communication

21

Benefits of Risk Analysis

23

Example Critical Assets

People and skills
Goodwill
Hardware/Software
Data
Documentation
Supplies
Physical plant
Money

Assurance that greatest risks have been identified and addressed

Increased understanding of risks
Mechanism for reaching consensus
Support for needed controls
Means for communicating results

Threats

An expression of intention to inflict evil injury or damage

Attacks against key security services
Confidentiality, integrity, availability

22

Basic Risk Analysis Structure

Evaluate

Value of computing and information assets

Vulnerabilities of the system
Threats from inside and outside
Risk priorities

Risk = Probability x Impact

= Function(Threat,Vulnerability,Impact)

Examine
Availability of security countermeasures
Effectiveness of countermeasures
Costs (installation, operation, etc.) of countermeasures

Implement and Monitor

24

Vulnerabilities

Flaw or weakness in system that can be exploited to violate

system integrity.
Security Procedures
Design
Implementation

Threats trigger vulnerabilities

Accidental
Malicious

25

Controls/Countermeasures

Qualitative Risk Analysis

Mechanisms or procedures for mitigating vulnerabilities

Generally used in Information Security

Hard to make meaningful valuations and meaningful probabilities
Relative ordering is faster and more important

Prevent
Detect
Recover

27

Understand cost and coverage of control

Controls follow vulnerability and threat analysis

Many approaches to performing qualitative risk analysis

Same basic steps as quantitative analysis
Still identifying asserts, threats, vulnerabilities, and controls
Just evaluating importance differently

Example:
The system is weak in this area and we know
that our adversary has the capability and
motivation to get to the data in the system so the
likelihood of this event occurring is high.

26

Types of Risk Analysis: How to Calculate Risk?

28

Qualitative Assessment
Criteria Development

Quantitative
Assigns real numbers to costs of safeguards and
damage
Annual loss exposure (ALE)
Probability of event occurring
Can be unreliable/inaccurate

Qualitative
Judges an organizations risk to threats

Simplified criteria
Severity of Consequence
Low, Medium or High
Probability of Occurrence
Use number between 0 and 1
or L, M or H
Probability of Occurrence

Based on judgment, intuition, and experience

Low or 1

Ranks the seriousness of the threats for the sensitivity

of the asserts

= minimal or unlikely chance of occurrence

Subjective, lacks hard numbers to justify return on

investment

Examples:
One occurrence per 1000 years
One occurrence per 50,000 units produced

29

Qualitative Assessment

31

Qualitative Assessment

Probability of Occurrence
Severity of Consequence

Medium or 2
= medium or somewhat likely chance of occurrence

Medium or 2 = Physical injury or impairment, medium structural

damage (no loss of key components), cost impact of $50,000 to
$500,000, schedule impact of 30 90 days

Examples:
One occurrence per 10 years
One occurrence per 1000 units produced

Severity of Consequence

Probability of Occurrence

High or 3 = Loss of life, major structural damage or

complete destruction, cost impact of >$500,000, schedule
impact of >90 days

High or 3
= maximum or very likely chance of occurrence
Examples:
One occurrence per year
One occurrence per 50 units produced
30

Qualitative Assessment

32

Qualitative Representation of Risk

Consequence of Occurrence
Statement that defines actual impacts of risk occurring

Qualitative risk representations are often used for quick evaluations and screening.

Example:
Project/System = House
Risk = Direct hit from F-4 tornado
Consequence/Impact = People are injured or killed; house is severely
damaged or completed destroyed

Probability
of Occurrence

Consequence of Occurrence
Very Low

Low Moderate High Very High

Very Low
Low

Severity of Consequence
Value that assigns a level of severity to the event
Low or 1 = Minor or no injuries, minimal or no structural
damage, cost impact of <$50,000, schedule impact of <30 days

Moderate
High
Very High
Low Risk

Medium Risk

High Risk

33

Quantitative Risk Analysis

35

Quantitative Risk Analysis

Risk Analysis Definition

Risk analysis involves the identification and assessment

of the levels of risks calculated from the known values of
assets and the levels of threats to, and vulnerabilities of,
those assets.
It involves the interaction of the following elements:

Assets
Vulnerabilities
Threats
Impacts
Likelihoods
Controls

Risk = Risk-impact x Risk-Probability

Loss of car: risk-impact is cost to replace car, e.g. $10,000
Probability of car loss: 0.10
Risk = 10,000 x 0.10 = 1,000

Risk Management is about controlling risk. To control a risk

Reduce the Probability
and/or
Reduce the Impact

Single loss Expectancy (SLE): how much loss for one event?

Risk calculation (per year):

Annual Loss Expectancy (ALE) = SLE x Annual Rate of Occurrence (ARO)

34

Quantitative Risk Analysis

Definitions

Quantitative risk analysis methods are based on statistical

data and compute numerical values of risk. They assign a
dollar value to risk.
By quantifying risk, we can justify the benefits of spending
money to implement controls.
It involves three steps
Estimation of individual risks
Aggregation of risks
Identification of controls to mitigate risk

36

Quantitative Risk Analysis

Risk Analysis Steps

Security risks can be analyzed by the following steps:

1. Identify and determine the value of assets
2. Determine vulnerabilities
3. Estimate likelihood of exploitation

4.

Compute Annual Loss Expectancy

5.
6.

Compute frequency of each attack (with & w/o controls) using statistical
data
Compute exposure of each asset given frequency of attacks

Survey applicable controls and their costs

Perform a cost-benefit analysis
Compare exposure with controls and without
controls to determine the optimum control

37

39

Quantitative Risk Analysis

Quantitative Risk Analysis

Determining Assets & Vulnerabilities

Likelihood of Exploitation

Identification of Assets and Vulnerabilities is the same for

both Qualitative and Quantitative Risk Analysis
The differences in both of these is in terms of valuation:

Difficult to obtain frequency of attacks

using statistical data. Why?
Data is difficult to obtain & often inaccurate

Qualitative Risk Analysis is more subjective and relative

Quantitative Risk Analysis is based on actual numerical costs
and impacts.

If automatic tracking is not feasible,

expert judgment is used to determine
frequency
Approaches
Delphi Approach: Probability in terms of
integers (e.g. 1-10)
Normalized: Probability in between 0 (not
possible) and 1 (certain)

38

40

Quantitative Risk Analysis

Quantitative Risk Analysis

Likelihood of Exploitation

Delphi Approach

Likelihood relates to the stringency of existing

controls
i.e. likelihood that someone or something will evade
controls

Several approaches to computing probability of an

event
classical, frequency and subjective

Probabilities hard to compute using classical

methods
Frequency can be computed by tracking failures that
result in security breaches or create new vulnerabilities
can be identified
e.g. operating systems can track hardware failures, failed
login attempts, changes in the sizes of data files, etc.

Frequency

Ratings

More than once a day

10

Once a day

Once every three days

Once a week

Once in two weeks

Once a month

Once every four months

Once a year

Once every three years

Less than once in three years 1

Subjective probability
technique originally devised to
deal with public policy
decisions
Assumes experts can make
informed decisions
Results from several experts
analyzed
Estimates are revised until
consensus is reached among
experts

41

43

Quantitative Risk Analysis

Quantitative Risk Analysis

Risk Exposure

Summary

Risk is usually measured as $ per annum and is quantified by

risk exposure.

ALE (Annual Loss Expectancy, expressed as: $/year)

If an event is associated with a loss

LOSS = RISK IMPACT ($)

The probability of an occurrence is in the range of:

0 (not possible) and 1 (certain)

Quantifying the effects of a risk by multiplying risk impact by

risk probability yields risk exposure.

Quantitative risk analysis involves statistical data and numerical

values and can be used to justify the benefit of controls.
While asset and vulnerability identification are the same for
qualitative and quantitative methods, qualitative is more
subjective and quantitative is more absolute.
Probabilities can be calculated in multiple ways. This can be
done using calculated values or the Delphi Approach (1-10) and
a Normalized Approach (1,0), which are more subjective.

RISK EXPOSURE = RISK IMPACT x RISK PROBABILITY

= Function(Threat,Vulnerability,Impact)

42

Quantitative Risk Analysis

Intangible Assets

Incorporating intangible assets within Quantitative

Risk Analysis is difficult as it is hard to put a price on
things such as trust, reputation, or human life.
However, it is necessary to put an as accurate a value
as possible when factoring these assets within risk
analysis as they may be even more important than
tangible assets.

44

How to start and quit MATLAB?

PC - a double click on the MATLAB icon on

your desktop
unix system - setup MATLAB (return)
MATLAB
On both system leave a MATLAB session by typing :
>> quit
or by typing
>> exit
at the MATLAB prompt.

45

Getting started with MATLAB

47

Plotting Probability Distributions

>> disttool

46

Statistics with MATLAB

Online help for Statistics Toolbox is available from the Matlab prompt (>> a
double arrow), both generally (listing of all available commands):
>> help stats
[a long list of help topics follows]
and for specific commands:

48

Tips for success

Expect to spend enough time studying the material of the course

Reading: Textbook

Assignment #1

Start every assignment early

Dont fall behind
Ask if you dont know
Do your own work

>> help distool

[a help message on the disttool function follows].
>> help disttool
DISTTOOL Demonstration of many probability distributions.
DISTTOOL creates interactive plots of probability distributions.
This is a demo that displays a plot of the cumulative distribution
function (cdf) or probability distribution function (pdf) of the distributions
in the Statistics Toolbox.

To be posted soon on Moodle