Professional Documents
Culture Documents
Basic Concepts
Analysis of Variance (ANOVA)
06/15/15
ANOVA
What is ANOVA?
ANOVA stands for ANalysis Of VAriance
It is a way to decompose a complicated function into a sum of simpler
functions in order to facilitate understanding the complicated function
06/15/15
Basic Concepts
di
f ( x1 , x 2 ,..., x n ) f x
L
f di
x O
P
dx x i
M
x P
M
N Q
L
f di
x O
P
x x id
x x i
d
M
M
Nx x P
Q
i 1
i 1 j 1
Fourier series
Expresses an arbitrary
periodic function as sum of
sines and cosines. System
response to a complicated
input can be found by
adding response to simple
sine and cosine inputs.
f ( x ) a0 a n cos(nx ) bn sin( nx )
06/15/15
1
2
n 1
n 1
Basic Concepts
i 1
f ( x1 , x 2 ,..., x n ) g i ( xi ) hij ( xi , x j )
i 1
i 2 j 1
a constant
functions of one variable
functions of two variables
functions of three variables
(overall mean)
(main effects)
(two-way interactions)
(three-way interactions)
06/15/15
Basic Concepts
A specific example
Suppose we have a simulation with two inputs x1 and x2 that vary over [0,1].
We run the simulation on a 5x5 grid, and obtain these results
X1
1.00
0.00
3.31
4.75
4.31
2.00
06/15/15
Basic Concepts
X1
1.00
0.00
3.31 Average:
4.75 2.38
4.31
2.00
Y - Mean
X1
0.00
0.25
0.50
0.75
1.00
0.00
-2.38
0.44
1.38
0.44
-2.38
0.25
-3.78
-0.84
0.22
-0.59
-3.28
X2
0.50
-2.38
0.69
1.88
1.19
-1.38
0.75
-0.97
2.22
3.53
2.97
0.53
1.00
-2.38
0.94
2.38 Variation
1.94 from
-0.38 Mean
X2
0.50
5.64
0.47
3.52
1.41
1.89
0.75
0.94
4.92
12.47
8.81
0.28
1.00
5.64
0.88
5.64
3.75
0.14
(Y - Mean) ^ 2
X1
0.00
0.25
0.50
0.75
1.00
06/15/15
0.00
5.64
0.19
1.89
0.19
5.64
0.25
14.30
0.71
0.05
0.35
10.77
The constant in
the ANOVA
decomposition is
just the average
of all the values,
here 2.38.
Using only the
constant (mean)
results in a
squared
approximation
error of 96.14
Total
Squared
Deviations
From Mean
96.14
Basic Concepts
0.00
0.25
0.50
0.75
1.00
X1
0.00
-2.38
0.44
1.38
0.44
-2.38
0.25
-3.78
-0.84
0.22
-0.59
-3.28
X2
0.50
-2.38
0.69
1.88
1.19
-1.38
0.75
-0.97
2.22
3.53
2.97
0.53
1.00
-2.38
0.94
2.38
1.94
-0.38
Main Effect of x1
2.5
2
1.5
1
0.5
0
-0.5 0
-1
0.25
0.5
0.75
-1.5
-2
-2.5
-3
06/15/15
x1
Average
(Main
Effect)
-2.38
0.69
1.88
1.19
-1.38
Basic Concepts
Variation explained by x1
X1
X1
X1
0.75
1.41
1.53
1.66
1.78
1.91
1.00
0.00
0.25
0.50
0.75
1.00
{ Y - [ Mean + MainEffect(x1) ] } ^ 2
X2
0.00
0.25
0.50
0.75
0.00
1.98
0.00
1.98
0.06
2.34
0.00
2.34
0.25
2.74
0.00
2.74
0.56
3.17
0.00
3.17
1.00
3.63
0.00
3.63
1.00
0.00
0.06
0.25
0.56
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
1.00
0.00
3.31 Average:
4.75 2.38
4.31
2.00
06/15/15
Variation
from
Mean
Plus
MainEffect(x1)
Total
Squared
Deviations
31.49
Basic Concepts
0.00
0.25
X1
0.50
0.75
1.00
Average = Main Effect
0.00
-2.38
0.44
1.38
0.44
-2.38
-0.50
0.25
-3.78
-0.84
0.22
-0.59
-3.28
-1.66
X2
0.50
-2.38
0.69
1.88
1.19
-1.38
0.00
0.75
-0.97
2.22
3.53
2.97
0.53
1.66
1.00
-2.38
0.94
2.38
1.94
-0.38
0.50
Main Effect of x2
1.5
1
0.5
0
-0.5
0.25
0.5
0.75
-1
-1.5
-2
06/15/15
x2
Basic Concepts
Variation due to x2
X1
X1
X1
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
Y - [ Mean + MainEffect(x2) ]
X2
0.00
0.25
0.50
-1.88
-2.13
-2.38
0.94
0.81
0.69
1.88
1.88
1.88
0.94
1.06
1.19
-1.88
-1.63
-1.38
1.00
0.00
3.31 Average:
4.75 2.38
4.31
2.00
0.75
-2.63
0.56
1.88
1.31
-1.13
1.00
-2.88
0.44
1.88
1.44
-0.88
{ Y - [ Mean + MainEffect(x2) ] } ^ 2
X2
0.00
0.25
0.50
0.75
3.52
4.52
5.64
6.89
0.88
0.66
0.47
0.32
3.52
3.52
3.52
3.52
0.88
1.13
1.41
1.72
3.52
2.64
1.89
1.27
1.00
8.27
0.19
3.52
2.07
0.77
06/15/15
Variation
from
Mean
Plus
MainEffect(x2)
Total
Squared
Deviations
66.21
Basic Concepts
Interaction effects
The function doesnt
have to be completely
explained by the main
effects.
In this case, there is a
residual after the mean
and main effects are
considered. This is due
to effects related to
specific combinations of
(x1,x2), the so-called
interaction effect.
The percent contribution
of the interaction is
1.56 / 96.14 = 2 %
06/15/15
Basic Concepts
If I had showed you this formula, would you have been able
to tell me the relative importance of the variables?
06/15/15
Basic Concepts
06/15/15
Basic Concepts
Number
points
4
9
16
25
36
49
64
81
100
400
900
1,600
2,500
3,600
4,900
6,400
8,100
10,000
Mean
0.50
1.75
2.17
2.38
2.50
2.58
2.64
2.69
2.72
2.87
2.91
2.94
2.95
2.96
2.96
2.97
2.97
2.97
Total Variance
0.75
3.57
3.94
3.85
3.73
3.63
3.55
3.49
3.44
3.22
3.15
3.12
3.10
3.08
3.07
3.06
3.06
3.06
Functional ANOVA
Infinite !
All points !
3.00
3.02
44.20%
54.90%
41
06/15/15
3.01
not reported
42.26%
55.64%
Basic Concepts
Main Effect of x1
0.25
0.5
0.75
Functional ANOVA
Main Effect of x2
x1
2.00000E+00
1.50000E+00
1.00000E+00
5.00000E-01
0.00000E+00
-5.00000E-01
-1.00000E+00
-1.50000E+00
-2.00000E+00
-2.50000E+00
-3.00000E+00
-3.50000E+00
0.00
2.00000E+00
1.5
1.50000E+00
1.00000E+00
0.5
5.00000E-01
0
-0.5 0
0.20
0.40
0.60
0.20
0.40
0.60
0.80
1.00
0.00000E+00
0.25
0.5
0.75
-5.00000E-01
-1
-1.00000E+00
-1.5
-1.50000E+00
-2
x2
-2.00000E+00
0.00
0.80
1.00
06/15/15
Basic Concepts
4
3
2
1
Y - Mean 0
-1
-2
-3
-4
-5
0.25
0.5
0.75
X1
06/15/15
Basic Concepts
3
2
1
Y - Mean 0
-1
-2
-3
-5
-4
0
0.25
0.5
X1
0.75
2
1
Y - Mean 0
-1
-2
-3
06/15/15
0.25
X1
0.50
Basic Concepts
Minitab
Runs in Windows with GUI, available from Knowledge Center
Functional ANOVA
Kriging Wizard
06/15/15
Basic Concepts
06/15/15
Basic Concepts