Advanced Statistics

W3: Multivariate ANOVA

Titis Wijayanto
Industrial Engineering, UGM

Quiz

Find the eigenvalues and eigenvectors of this

matrix:

In this chapter we extend univariate analysis of

variance (ANOVA) to multivariate analysis of variance
(MANOVA), in which we measure more than one
variable on each experimental unit.

Univariate One-Way Analysis of

Variance (ANOVA)

In the balanced one-way ANOVA, we have a random sample of n

observations from each of k normal populations with equal variances, as
in the following layout:

The k samples or the populations from which they arise are sometimes
referred to as groups. The groups may correspond to treatments applied
by the researcher in an experiment.

Univariate One-Way Analysis of

Variance (ANOVA)
We will have such hypothesis like this:

Univariate One-Way Analysis of

Variance (ANOVA)
Example

Univariate One-Way Analysis of

Variance (ANOVA)
Example

In this case, with 2 and 12 degrees of freedom the critical values are 3.89
(p= .05) and 6.93 (p= .01).

Thus what can we say???

Multivariate Analysis of Variance

(MANOVA)
there may be circumstances in which we are interested in several
dependent variables (DV) and in these cases the simple ANOVA
model is inadequate.
Instead, we can use an extension of this technique known as

multivariate analysis of variance(or MANOVA).

ANOVA can be used only in situations in which there is one dependent variable
(or outcome) and so is known as a univariate test (univariate quite obviously
means one variable); MANOVA is designed to look at several dependent

variables (outcomes) simultaneously and so is a multivariate test

(multivariate means many variables).

ANOVA vs MANOVA
Anovas run separately cannot take into account the
pattern of covariation among the dependent measures
It may be possible that multiple Anovas may show no

differences while the Manova brings them out

So, by measuring multiple DVs you increase your chances for

finding a group difference

In this sense, in many cases such a test has more power than the univariate
procedure, but this is not necessarily true as some seem to believe

Also conducting multiple ANOVAs increases the chance for

type 1 error and MANOVA can in some cases help control for
the inflation

ANOVA vs MANOVA

ANOVA vs MANOVA

F table = 3.354

ANOVA vs MANOVA

F table = 3.354

MANOVA
MANOVA uses the same sums of squares as ANOVA.
To understand the relationship between (or among) DVs,
MANOVA will use the cross-products.
Total cross products

Cross product

Model cross products

Residual cross products

MANOVA
Total cross product (CPT)
We have these information

MANOVA
Total cross product (CPT)

()

()

For this case we have

61.47
5.47

5.47
141.47

MANOVA
Model cross product (CPM)
We have these information

MANOVA
Model cross product (CPM)
The model (or hypothesis) sum of squares and cross-product matrix, H, contains the
model sums of squares for each dependent variable and the model cross-product
between the two dependent variables

()

()

For this case we have

10.47
7.53

7.53
19.47

MANOVA
Residual cross product (CPR)

MANOVA
Residual cross product (CPR)
R()

R()

For this case we have

51
13

13
122

MANOVA
So we have 3 matrices
=

61.47
5.47

5.47
141.47

10.47
7.53

7.53
19.47

51
13

13
122

Can you find something

suspicious here???
61.47
5.47
10.47 7.53 51 13
=
+
5.47 141.47 7.53 19.47 13 122

10

MANOVA
In univariate ANOVA we calculate the ratio of the systematic variance
to the unsystematic variance (i.e. we divide SSM by SSR)
The conceptual equivalent would therefore be to divide the matrix H by the matrix E.

???

61.47
5.47 51 13
5.47 141.47 13 122
0.2273 0.082
=
0.1930 0.1794

MANOVA

0.2273 0.082
0.1930 0.1794

Find the eigenvectors and the

eigenvalues

From calculation we get 0.335 and 0.073 for eigenvalues

The eigenvalues are conceptually equivalent to the F-ratio in ANOVA and so
the final step is to assess how large
expect by chance alone.

these values are compared to what we would

11

MANOVA

0.2273 0.082
0.1930 0.1794
0.335
0
0
0.073

Find the eigenvectors and the

eigenvalues
These are the EIGENVALUES

These eigenvalues are conceptually equivalent to the

so the final step is to assess how large
would expect by chance alone.

F-ratio in ANOVA and

these values are compared to what we

MANOVA

The PillaiBartlett trace(also known as Pillais trace) is

given by left equation in which represents the eigenvalues for
each of the discriminant variates and represents the number of
variates.
It is similar to the ratio of SSM/SST, which is known as R2

The HotellingLawley trace (also known as Hotellings

T2) is simply the sum of the eigenvalues for each variate.
This test statistic is the sum of SSM/SSR for each of the variates
and so it compares directly to the F-ratio in ANOVA

12

MANOVA

Wilkss lambda is the product of the unexplained variance on each

of the variates (the symbol is similar to the summation symbol ()
that we have encountered already except that it means multiply
rather than add up).

MANOVA

Roys largest root in a sense it is the same as the HotellingLawley trace

but for the first variate only, that is:

Statistical significance

13

MANOVA

MANOVA

0.2273 0.082
0.1930 0.1794

Find the eigenvectors and the

eigenvalues

From calculation we get this eigenvectors

The eigenvectors show the coefficients

which can be used to the create the discriminant

function

14

MANOVA

Group homework
Create data for a one-way MANOVA with 4 dependent variables and 4 levels
in the way.
Create the data in such a way that there is no difference between the rest
three mean vectors, but there is a difference between the fourth mean vector
and the other three. Prove this with a boxplot, the Wilk's Lambda, Pillai trace,
hoteling T2, and Roys largest root with post-hoc ANOVAs (last part is not
recommended, but just for illustrative purposes).

15

16