ENGR2140U Problem Solving,

Modeling and Simulation

Lecture-13

So far.

Introduction about the course

Project definition
What is the definition of problem
Real world problems
Effective problem thinkers
Good 6 habits
Tools: Microsoft Project, IDEF0, Flow
Diagram, Formal Languages

So far.

Risk assessment
Paradigm shift
Accident Investigation as problem solving
Exxon Valdez
Common contributors to accidents
Problem solving process, 5 steps
Heuristic Approach
Process Chart

So far.

How to define a problem

Data collection
Triggers
Brainstorming
Fishbone
Solution generation
K-T Analysis, PA, DA,PPA

So far.

Petri Nets
Decision Trees, Binary Decision Trees
FTA
ETA
Solution implementation
Approval
Planning

So far.

Implementation
Planning
MS Project , Critical Path
Approval
Resources
MOC
Onion Model
Follow Up

So far.
Error analysis
Experimental data analysis
Error types

Examine the
need for the
experiment

Define
objectives for
the experiment

Choose
responses to
measure

Identify the
important
variables

Design the
experiment

Basic Building
Blocks for
Experimental
Design
IMPORTANT: Error Analysis

Perform
experiment

Analyze
results

Act upon
results

Report !

Three things I will stress here:

ERROR ANALYSIS
PRESENTATION
OF DATA
REPORT

Experimental Data Analysis

Propagation
of Uncertainty
Bias
Uncertainty

Precision
Uncertainty

Statistical Analysis
Error Distribution

Gaussian distribution
t-distribution
Chi-square distribution

Population
theory

Sample
theory

Some Definitions
Metrology
The science (and art) of measurement

Accuracy
The difference between what you measure and
the true or standard value

Precision
Difference the difference or range in repeated
measurement of the same thing. Also the
number of digits you reasonably report.

Some Definitions
Resolution
Smallest increment of change you can determine;
sometimes called least count.
Resolution is is usually less than or equal to
Precision.

Sensitivity
Change in output per unit change in input.

Range
The interval of the smallest and largest inputs
(measurands) the system can handle without
distortion
Many times range and sensitivity are at odds.

Types of Errors
Bias error (systematic error) are those that occur
the same way each time a measurement is
made. Bias errors do not show a distribution,
therefore no statistical technique may be used.

Calibration errors
Consistent human errors
Defective equipment
Loading errors (effective of sensor on quantity
measured)

Types of Errors (cont)

Precision errors (random error) are different for
each successive measurement but have an
average value of zero. Statistical analysis may
be used to estimate the size of error. Notice
that precision means repeatability.
Human error
Equipment disturbances
Fluctuating experimental conditions

Total error
Bias error

Precision
error
Truth

Measured value

Types of Errors (cont)

Illegitimate errors are those that would not be
expected to exist.
Blunders (outright mistakes)
Computational errors

Error estimates are used

to quantify uncertainty!

Error Analysis
Types of error that should be avoided in
experimental design: Type I and II error
TYPE I: Declare that something has an
effect on the outcome, when really it does
not (False Alarms)
TYPE II: When we fail to discover a real
effect on the outcome (Missed Data)

Schematically
x1 x2 x3 xn

INPUTS

controllable factors

PROCESS to be
Understood

OUTPUT, y

uncontrollable factors
z 1 z2 z3 z m

Schematically
x1 x2 x3 xn

controllable factors

which are important?

INPUTS

PROCESS to be
Understood

OUTPUT, y

uncontrollable factors
z 1 z2 z3 z m

Schematically
x1 x2 x3 xn

controllable factors

which are important?

INPUTS

PROCESS to be
Understood

OUTPUT, y

uncontrollable factors
z1 z2 z3 zm how to minimize these?

Designing an Experiment
A well designed experiment can result in
an experimentally determined model
(Empirical Model)
Good data (and reporting) stands the test
of time.
Galileos drawings of the moons of Jupiter
Mayan observations of the sun

Data Collection
Good data collection is essential
Why it is worth investing in a lab manual,
and recording all observations.

Data Collection
Why it is valuable to include raw data in a
journal report, since it allows others the
opportunity to evaluate the measurements
may come to different conclusions
may notice a problem that you didnt

Data Collection
But before we rush off to collect data we
need to adopt a strategy to follow in order
to collect it
Want a
Strategy of Experimentation

7 Steps for Designing an

Experiment
1.
2.
3.
4.
5.
6.
7.

recognition/statement of problem
choosing response variable
choosing factors, levels, ranges
choose experimental design
do experiment
statistical analysis of data
conclusions/recommendations

Want to Consider this further

x1 x2 x3 xn

INPUTS

controllable factors

PROCESS to be
Understood

OUTPUT, y

uncontrollable factors
z 1 z2 z3 z m

Inputs/Outputs
What are my inputs?
How do we measure the output (y)?
How do we assess improvement in output
(statistics play a big part here)?

Controllable Factors (xis)

Which controllable factors are key?
Which factors influence y most?
Are there combinations of xis that are
desirable?
How do we minimize variability in y?
Do the values of x effect the uncontrollable
factors?
Can this effect be minimized?

Uncontrollable Factors (zis)

How are these minimized or removed from
consideration?

Sample Problem
Goal: Maximize production of C
Reaction:
A+B

catalyst

What is the response being measured?

What factors might influence output?
Uncontrollable factors?

Sample Problem
Factors:
amount of catalyst
T of reaction
stirring/no stirring (degree of agitation)
operator
supplier of A and B

A Method to Solve
try a set of variables measure y
use intuition to change a variable to see
change in measured response

This is the Best-Guess Approach

used a lot!
Question: How does one know if optimum
solution achieved?

Varying One Factor

traditional method;
change one factor at a time
draw conclusions based on measured
response

C (g)

Varying One Factor

Catalyst

Temperature

Agitation

Varying One Factor

Major disadvantage to this approach
NO INTERACTION EFFECTS

Golf Example
Factors identified
type of driver (regular or oversized)
type of ball
walking or riding
drinking water or beer
play AM or PM
play cool or hot
golf shoe spikes (metal or soft)
windy or calm

Interaction Effect

Set of Scores

Determining Effects

Beverage

Three Factor Experiment

Ball
Driver

Improved Approach
Factorial Design

Some Terms

Run
Variable/Factor
Randomization
Replication
Blocking

Run
A measurement from one set of
experimental parameters

Variable/Factor
An experimental value that can be
changed.
Often want a range for the variable:
minimum and maximum temperature
length of experiment (time)
amount of additive
etc.

Randomization
allocation of experimental material and
order of runs is random
ensure that each experimental run is an
independent observation
in other words no carry over effect
between experiments
useful for averaging out effects of
extraneous factors (those zs)

Replication
independent repeat of a run
allows for estimation of experimental error
difference between a repeated
measurement

Blocking
design technique which can improve
precision
used to reduce/eliminate nuisance factors
(something which may influence result but
is not of interest; e.g., two batches of
materials supplied for testing)
Zr/Hf crucibles

Blocking
This is a area requiring more depth

Some Statistical Terms

Mean (central tendency)
Variance (dispersion of a probability
distribution around mean)
Sample Mean
Sample Variance

Sample Mean

Sample Variance

n-1

Regression Model
Linear Model
y = 0 + 1 x +

( is the random error)

Quadratic Model
y = 0 + 1 x + 2 x 2 +

Quartic Model etc.

Want lowest order polynomial
(or the most appropriate)

Factorial Design

Terms
main effects (x1, x2, x3, )
interaction effects (x1x2, x1x3, x2x3, )
coded scale (-1 to +1)

Terms
regression model representation for a twofactor factorial design
y = 0 + 1x1 + 2x2 + 12x1x2 +

response surface

Advantage of Factorial Design

y = 0 + 1x1 + 2x2 + 12x1x2 +

Relative Efficiency

Battery Design Example