Eigenvalues and Eigenvectors

1. Eigenvalues and Eigenvectors

2. Diagonalization
3.Symmetric Matrices and Orthogonal Diagonalization
4.Application of Eigenvalues and Eigenvectors

1. Eigenvalues and Eigenvectors

Eigenvalue problem (one of the most important problems in the

linear algebra):
If A is an nn matrix, do there exist nonzero vectors x in Rn
such that Ax is a scalar multiple of x
(The term eigenvalue is from the German word Eigenwert, meaning
proper value)

Eigenvalue and Eigenvector:

A: an nn matrix
: a scalar (could be zero)
x: a nonzero vector in Rn
Eigenvalue

Ax x
Eigenvector

Geometrical Interpretation

Ex 1: Verifying eigenvalues and eigenvectors

2 0
1
0
A
x1 x 2

1
0
0 1
Eigenvalue

2 0 1 2
1
Ax1
2 2x1

0 1 0 0
0
Eigenvector

In fact, for each eigenvalue, it

has infinitely many eigenvectors.
For = 2, [3 0]T or [5 0]T are
both corresponding eigenvectors.
Moreover, ([3 0] + [5 0])T is still
an eigenvector.

Eigenvalue

2 0 0 0
0
Ax2
1 (1)x2

0 1 1 1
1
Eigenvector

Thm.1: The eigenspace corresponding to of matrix A

If A is an nn matrix with an eigenvalue , then the set of all
eigenvectors of together with the zero vector is a subspace
of Rn. This subspace is called the eigenspace of
Pf:

x1 and x2 are eigenvectors corresponding to

(i.e., Ax1 x1 , Ax2 x2 )
(1) A(cx1 ) c( Ax1 ) c( x1 ) (cx1 )
(i.e., cx1 is also an eigenvector corresponding to )
(2) A(x1 x2 ) Ax1 Ax2 x1 x2 (x1 x2 )
(i.e., x1 x2 is also an eigenvector corresponding to )

Since this space is closed under vector addition and scalar

multiplication, this space is a subspace of Rn

Ex 3: Examples of eigenspaces on the xy-plane

For the matrix A as follows, the corresponding eigenvalues
are 1 = 1 and 2 = 1:
1 0
A

0
1

Sol:

For the eigenvalue 1 = 1, corresponding vectors are any vectors on the x-axis

x 1 0 x x
x Thus, the eigenspace
A
1
corresponding to = 1 is the x

axis, which is a subspace of R2

0 0 1 0 0
0
For the eigenvalue 2 = 1, corresponding vectors are any vectors on the y-axis

0 1 0 0 0 0
A
1

y 0 1 y y y

Thus, the eigenspace

corresponding to = 1 is the yaxis, which is a subspace of R2

If v ( x, y),

1 0 x x
Av

0 1 y y

Geometrically, multiplying a vector (x, y) in R2 by the matrix A corresponds to

a reflection to the y-axis
The above reflection result also can be derived using the eigenvalues and
eigenvectors as follows

x 0
x
x
0
Av A A A A
y
0
y
0 y
x 0 x
1 1
0 y y

Thm.2: Finding eigenvalues and eigenvectors of a matrix AMnn

Let A is an nn matrix.

(1) An eigenvalue of A is a scalar such that det( I A) 0

(2) The eigenvectors of A corresponding to are the nonzero
solutions of ( I A)x 0

Note: following the definition of the eigenvalue problem

Ax x Ax Ix ( I A)x 0 (homogeneous system)
( I A)x 0 has nonzero solutions for x iff det( I A) 0

Characteristic equation of A:
det( I A) 0
Characteristic polynomial of AMnn:

det( I A) ( I A) n cn1 n1

c1 c0

Ex 4: Finding eigenvalues and eigenvectors

2 12
A

Sol: Characteristic equation:

det( I A)

2
1

12
5

2 3 2 ( 1)( 2) 0
1, 2
Eigenvalue: 1 1, 2 2

 3 12 x1 0
(1) 1 1 (1 I A)x

1 4 x2 0
3 12 G.-J. E. 1 4

1
4
0
0

x1 4t 4
t , t 0
x2 t 1
4 12 x1 0

(2) 2 2 (2 I A)x

1 3 x2 0
4 12 G.-J. E. 1 3

1
3
0
0

x1 3s
3
s , s 0
1
x2 s

Ex 5: Finding eigenvalues and eigenvectors

Find the eigenvalues and corresponding eigenvectors for

the matrix A. What is the dimension of the eigenspace of

each eigenvalue?

2 1 0
A 0 2 0

0
0
2

Sol: Characteristic equation:

2 1
0
I A 0
2
0 ( 2)3 0
0
0
2
Eigenvalue: 2

The eigenspace of = 2:

0 1 0 x1 0
( I A)x 0 0 0 x2 0
0 0 0 x3 0
x1 s 1 0
x2 0 s 0 t 0 , s , t 0

x3 t 0 1
1 0

s 0 t 0 s, t R : the eigenspace of A corresponding to 2

0 1

Thus, the dimension of its eigenspace is 2.

Notes:
(1) If an eigenvalue 1 occurs as a multiple root (k times) for
the characteristic polynominal, then 1 has multiplicity k.

(2) The multiplicity of an eigenvalue is greater than or equal

to the dimension of its eigenspace. (In Ex. 5, k is 3 and
the dimension of its eigenspace is 2)

Ex 6Find the eigenvalues of the matrix A and find a basis

for each of the corresponding eigenspaces

1
0
A
1
1

0
1
0
0

0 0
5 10

2 0
0
3

Sol: Characteristic equation:

1 0
0
0
0
1 5
10
I A
1
0
2
0
1
0
0
3
( 1) 2 ( 2)( 3) 0

Eigenvalues: 1 1, 2 2, 3 3

According to the note on

the previous slide, the
dimension of the
eigenspace of 1 = 1 is at
most to be 2
For 2 = 2 and 3 = 3, the
dimensions of their
eigenspaces are at most to
be 1

0
0
(1) 1 1 (1 I A)x
1

0 0 0 x1 0
0 5 10 x2 0

0 1 0 x3 0

0 0 2 x4 0

x1 2t
0 2
x s
1 0
2 s t , s,t 0
x3 2t
0 2


0 1
x4 t

0 2

1 0
,
is a basis for the eigenspace
0 2 corresponding to 1 1
0 1
The dimension of the eigenspace of 1 = 1 is 2

1
0
(2) 2 2 (2 I A)x
1

0 0 0 x1 0
1 5 10 x2 0

0 0 0 x3 0

0 0 1 x4 0

x1 0 0
x 5t 5
2 t , t 0
x3 t 1

x4 0 0

0

5 is a basis for the eigenspace

1 corresponding to 2 2
0
The dimension of the eigenspace of 2 = 2 is 1

2
0
(3) 3 3 (3 I A)x
1

0 0 0 x1 0
2 5 10 x2 0

0 1 0 x3 0

0 0 0 x4 0

x1 0 0
x 5t 5
2 t , t 0
x3 0 0

x4 t 1

0

5 is a basis for the eigenspace

corresponding to 3 3
1
The dimension of the eigenspace of 3 = 3 is 1

Thm.3: Eigenvalues for triangular matrices

If A is an nn triangular matrix, then its eigenvalues are
the entries on its main diagonal
Ex 7: Finding eigenvalues for triangular and diagonal matrices
2 0 0
(a) A 1 1 0
5 3 3

Sol:

1
0

(b) A 0

0
0

0
2
0
0
0

0 0
0 0
0 0
0 4
0 0

0
0
0

0
3

0
0
(a) I A 1
1 0 ( 2)( 1)( 3) 0
5
3 3
1 2, 2 1, 3 3
(b) 1 1, 2 2, 3 0, 4 4, 5 3

2. Diagonalization

Diagonalization problem :
For a square matrix A, does there exist an invertible matrix P
such that P1AP is diagonal?

Diagonalizable matrix :
Definition 1: A square matrix A is called diagonalizable if
there exists an invertible matrix P such that P1AP is a
diagonal matrix (i.e., P diagonalizes A)
Definition 2: A square matrix A is called diagonalizable if A
is similar to a diagonal matrix

Thm. 4: Similar matrices have the same eigenvalues

If A and B are similar nn matrices, then they have the
same eigenvalues
Pf:

A and B are similar B P 1 AP

For any diagonal matrix in the

form of D = I, P1DP = D

Considering the characteristic equation of B:

I B I P 1 AP P 1 IP P 1 AP P 1 ( I A) P
P 1 I A P P 1 P I A P 1P I A
I A
Since A and B have the same characteristic equation,
they are with the same eigenvalues

Ex 1: Eigenvalue problems and diagonalization programs

1 3 0
A 3 1 0

0
0

Sol: Characteristic equation:

1 3
0
I A 3 1
0 ( 4)( 2) 2 0
0
0
2
The eigenvalues : 1 4, 2 2, 3 2

1

(1) 4 the eigenvector p1 1
0

1
0


(2) 2 the eigenvector p 2 1 , p3 0
0
1
1 1 0
4 0 0
P [p1 p 2 p3 ] 1 1 0 , and P 1 AP 0 2 0
0 0 1
0 0 2

Note: If P [p 2

p1 p3 ]

1 1 0
1 1 0
0 0 1

2 0 0
P 1 AP 0 4 0
0 0 2

Thm.5: Condition for diagonalization

An nn matrix A is diagonalizable if and only if it has n
linearly independent eigenvectors

Note that if there are n linearly independent

eigenvectors, it does not imply that there are n distinct
eigenvalues. It is possible to have only one eigenvalue
with multiplicity n, and there are n linearly
independent eigenvectors for this eigenvalue
However, if there are n distinct eigenvalues, then
there are n linearly independent eivenvectors, and thus
A must be diagonalizable

Ex 4: A matrix that is not diagonalizable

Show that the following matrix is not diagonalizable

1 2
A

0
1

Sol: Characteristic equation:

1 2
I A
( 1)2 0
0
1
The eigenvalue 1 1, and then solve (1I A)x 0 for eigenvectors
0 2
1
1I A I A
eigenvector p1

0 0
0

Since A does not have two linearly independent eigenvectors,

A is not diagonalizable.

Steps for diagonalizing an nn square matrix:

Step 1: Find n linearly independent eigenvectors p1 , p2 ,
for A with corresponding eigenvalues 1 , 2 ,

Step 2: Let P [p1 p2

Step 3:

pn ]

1 0
0 2
1
P AP D

0 0

0
0

where Api i pi , i 1, 2,

,n

, n

pn

Ex 5: Diagonalizing a matrix

1 1 1
A 1
3 1

3 1 1
Find a matrix P such that P 1 AP is diagonal.

Sol: Characteristic equation:

1 1
1
I A 1 3 1 ( 2)( 2)( 3) 0
3
1 1

The eigenvalues : 1 2, 2 2, 3 3

1
1 2 1 I A 1
3
x1 t
x 0
2
x3 t

1 1
1 0 1
G.-J. E.
1 1
0 1 0
0 0 0
1 3
1
eigenvector p1 0
1

1 0 14
3 1 1
G.-J. E.
0 1 14
2 2 2 I A 1 5 1
0 0 0
3 1 1
x1 14 t
1
x 1 t eigenvector p 1
2
2 4

x3 t
4

2 1 1
1 0 1
G.-J. E.
0 1 1
3 3 3 I A 1 0 1
3 1 4
0 0 0
x1 t
1
x t eigenvector p 1
3
2

x3 t
1
1 1 1
P [p1 p 2 p3 ] 0 1 1 and it follows that
1 4 1
2 0 0
P 1 AP 0 2 0
0 0 3

Note: a quick way to calculate Ak based on the diagonalization

technique

1 0
0
2
(1) D

0 0

0
1k

0
0
k

n
0

(2) D P 1 AP D k P 1 AP P 1 AP

k
n

2k
0

P 1 AP P 1 Ak P

repeat k times

1k

0
k
k 1
k

A PD P , where D

2k
0

k
n

Thm. 6: Sufficient conditions for

diagonalization
If an nn matrix A has n distinct
eigenvalues, then the corresponding
eigenvectors are linearly independent and
thus A is diagonalizable.

Ex 7: Determining whether a matrix is diagonalizable

1 2 1
A 0 0
1

0 0 3

Sol: Because A is a triangular matrix, its eigenvalues are

1 1, 2 0, 3 3.
According to Thm. 6, because these three values are
distinct, A is diagonalizable.

7.3 Symmetric Matrices and Orthogonal Diagonalization

Symmetric matrix :
A square matrix A is symmetric if it is equal to its transpose:

A AT

Ex 1: Symmetric matrices and nonsymetric matrices

0 1 2
A 1 3 0
(symmetric)

2
0
5

4 3
B
(symmetric)

3
1

3 2 1
C 1 4 0
(nonsymmetric)

1 0 5

Thm.7: Eigenvalues of symmetric matrices

If A is an nn symmetric matrix, then the following properties
are true.

(1) A is diagonalizable (symmetric matrices are guaranteed to

has n linearly independent eigenvectors and thus be
diagonalizable).
(2) All eigenvalues of A are real numbers.
(3) If is an eigenvalue of A with multiplicity k, then has k
linearly independent eigenvectors. That is, the eigenspace
of has dimension k.
The above theorem is called the Real Spectral Theorem, and the set of
eigenvalues of A is called the spectrum of A.

Ex 2:
Prove that a 2 2 symmetric matrix is diagonalizable.

a c
A
c b
Pf: Characteristic equation:
a c
I A
2 (a b) ab c 2 0
c b
As a function in , this quadratic polynomial function has a
nonnegative discriminant as follows

(a b) 2 4(ab c 2 ) a 2 2ab b 2 4ab 4c 2

a 2 2ab b 2 4c 2
(a b) 2 4c 2 0

(1) (a b) 2 4c 2 0

a b, c 0
a c a 0
A

itself is a diagonal matrix.

c b 0 a

(2) (a b) 2 4c 2 0
The characteristic polynomial of A has two distinct real roots,
which implies that A has two distinct real eigenvalues.
According to Thm. 6, A is diagonalizable.

Orthogonal matrix :
A square matrix P is called orthogonal if it is invertible and

P1 PT (or PPT PT P I )

Thm. 8: Properties of orthogonal matrices

An nn matrix P is orthogonal if and only if its column vectors
form an orthonormal set.
Pf: Suppose the column vectors of P form an orthonormal set, i.e.,
P p1 p2

pn , where pi p j 0 for i j and pi pi 1.

p1T p1 p1T p 2
T
T
p
p
p
T
2
1
2 p2

P P

T
T
p n p1 p n p 2

p1T p n p1 p1 p1 p 2

p 2T p1 p 2 p1 p 2 p 2

T
p n p n p n p1 p n p 2

It implies that P1 = PT and thus P is orthogonal.

p1 p n
p 2 p1
In

pn pn

Ex 5: Show that P is an orthogonal matrix.

13
2
P 5
2
3 5

2
3
1
5
4
3 5

0
5
3 5
2
3

Sol: If P is a orthogonal matrix, then P1 PT PPT I

1
3

PPT 25
2
3 5

2
3
1
5
4
3 5

13

0 23
2
5
3 5
3
2
3

2
5
1
5

1 0 0

I
4

0
1
0

3 5

5
0 0 1

3 5
2
3 5

 1
2
2

3
3
3

2
1
Moreover, let p1 5 , p 2 5 , and p3 0 ,
2
4
5
3 5
3 5
3 5
we can produce p1 p 2 p1 p3 p 2 p3 0 and p1 p1
p 2 p 2 p3 p3 1.

So, {p1, p2 , p3} is an orthonormal set.

Thm.9: Properties of symmetric matrices

Let A be an nn symmetric matrix. If 1 and 2 are distinct
eigenvalues of A, then their corresponding eigenvectors x1 and x2
are orthogonal. (Thm. 6 only states that eigenvectors
corresponding to distinct eigenvalues are linearly independent)

Pf:

1 (x1 x2 ) (1x1 ) x2 ( Ax1 ) x2 ( Ax1 )T x2 (x1T AT )x2

because A is symmetric

(x1T A)x2 x1T ( Ax2 ) x1T (2 x2 ) x1 (2 x2 ) 2 (x1 x2 )

The above equation implies (1 2 )(x1 x2 ) 0, and because

1 2 , it follows that x1 x2 0. So, x1 and x2 are orthogonal.

For distinct eigenvalues of a symmetric matrix, their corresponding
eigenvectors are orthogonal and thus linearly independent to each other
Note that there may be multiple x1 and x2 corresponding to 1 and 2

Orthogonal diagonalization :
A matrix A is orthogonally diagonalizable if there exists an
orthogonal matrix P such that P1AP = D is diagonal.
Thm. 10: Fundamental theorem of symmetric matrices
Let A be an nn matrix. Then A is orthogonally diagonalizable
and has real eigenvalues if and only if A is symmetric.

Ex 7: Determining whether a matrix is orthogonally diagonalizable

1 1 1
A1 1 0 1

1
1
1

5 2 1
A2 2 1 8

1
8
0

3 2 0
A3
2 0 1
0 0
A4
0 2

Symmetric
matrix

Orthogonally
diagonalizable

Ex 9: Orthogonal diagonalization

Find an orthogonal matrix P that diagonalizes A.

2 2 2
A 2 1 4
2 4 1

Sol:
(1) I A ( 3) 2 ( 6) 0

1 6, 2 3 (has a multiplicity of 2)
v1
(2) 1 6, v1 (1, 2, 2) u1
( 13 ,
v1
(3) 2 3, v2 (2, 1, 0), v3 (2, 0, 1)

2
3

2
3

Verify Thm. 7.9 that

v1v2 = v1v3 = 0

Linearly independent but not orthogonal

4. Applications of Eigenvalues and Eigenvectors

The rotation for quadratic equation: ax2+bxy+cy2+dx+ey+f = 0

Ex 5: Identify the graphs of the following quadratic equations

(a) 4 x2 9 y 2 36 0

(b) 13x2 10 xy 13 y 2 72 0

Sol:

x2 y 2
(a) In standard form, we can obtain 2 2 1.
3
2
Since there is no xy-term, it is easy
to derive the standard form and it is
apparent that this equation
represents an ellipse.

(b) 13x2 10 xy 13 y 2 72 0
Since there is a xy-term, it is difficult to identify the graph of this equation.
In fact, it is also an ellipse, which is oblique on the xy-plane.
There is a easy way to identify the graph of
quadratic equation. The basic idea is to rotate the
x- and y-axes to x- and y-axes such that there is
no more xy-term in the new quadratic equation.
In the above example, if we rotate the x- and yaxes by 45 degree counterclockwise, the new

( x ')2 ( y ') 2
quadratic equation
2 1 can be
2
3
2

derived, which represents an ellipse apparently.

In Section 4.8, the rotation of conics is achieved by changing basis, but

here the diagonalizating technique based on eigenvalues and
eignvectors is applied to solving the rotation problem

Quadratic form :

ax2 + bxy + cy2

is the quadratic form associated with the quadratic equation
ax2 + bxy + cy2 + dx + ey + f = 0.

Matrix of the quadratic form:

a b / 2
A

b
/
2
c

x
If we define X = y , then XTAX= ax2 + bxy + cy2 . In fact, the

quadratic equation can be expressed in terms of X as follows.

X T AX d

e X f .

Principal Axes Theorem

For a conic whose equation is ax2 + bxy + cy2 + dx + ey + f = 0,
the rotation to eliminate the xy-term is achieved by X = PX,
where P is an orthogonal matrix that diagonalizes A. That is,

1 0
P AP P AP D
,

0 2
where 1 and 2 are eigenvalues of A. The equation for the
rotated conic is given by
1

1 ( x ')2 2 ( y ')2 d e PX f 0.

Pf:
According to Thm. 10, since A is symmetric, we can
conclude that there exists an orthogonal matrix P such that
P1AP = PTAP = D is diagonal.
Replacing X with PX, the quadratic form becomes

X T AX ( PX )T A( PX ) ( X )T PT APX
( X )T DX 1 ( x)2 2 ( y) 2 .
It is obvious that the new quadratic form in terms of X has no xyterm, and the coefficients for (x)2 and (y)2 are the two eigenvalues
of the matrix A.

x
x
x
x
X PX v1 v 2 xv1 yv 2 Since and are
y
y
y
y
the orignal and new coodinates, the roles of v1 and v 2 are like the basis
vectors (or the axis vectors ) in the new coordinate system.

Ex 6: Rotation of a conic
Perform a rotation of axes to eliminate the xy-term in the
following quadratic equation

13x2 10 xy 13 y 2 72 0
Sol:
The matrix of the quadratic form associated with this equation is

13 5
A
.

5 13
The eigenvalues are 1 = 8 and 2 = 18, and the corresponding
eigenvectors are

1
1
x1 and x 2 .
1
1

After normalizing each eigenvector, we can obtain the

orthogonal matrix P as follows.
According to the results in p.

1
2
1
2

1
2 cos 45

1 sin 45
2

sin 45

cos 45

268 in Ch4, X=PX is

equivalent to rotate the xycoordinates by 45 degree to
form the new xy-coordinates,
which is also illustrated in the
figure on Slide 7.62.

Then by replacing X with PX, the equation of the rotated conic

is
8(x) 18( y)2 72 0,

which can be written in the standard form

( x)2 ( y) 2
2 1.
2
3
2
The above equation represents an ellipse on the xy-plane.

In three-dimensional version:
ax2 + by2 + cz2 + dxy + exz + fyz
is the quadratic form associated with the equation of quadric
surface: ax2 + by2 + cz2 + dxy + exz + fyz + gx + hy + iz + j = 0

Matrix of the quadratic form:

d /2
a
A d / 2
b
e / 2 f / 2

e/2
f / 2
c

If we define X = [x y z]T, then XTAX= ax2 + by2 + cz2 + dxy + exz

+ fyz, and the quadratic surface equation can be expressed as

X T AX g h i X j.