Orthogonal Matrices

LINEAR ALGEBRA
W W L CHEN
c _ W W L Chen, 1997, 2008.
This chapter is available free to all individuals, on the understanding that it is not to be used for nancial gain,
and may be downloaded and/or photocopied, with or without permission from the author.
However, this document may not be kept on any information storage and retrieval system without permission
from the author, unless such system is not accessible to any individuals other than its owners.
Chapter 10
ORTHOGONAL MATRICES
10.1. Introduction
Definition. A square matrix A with real entries and satisfying the condition A
1
= A
t
is called an
orthogonal matrix.
Example 10.1.1. Consider the euclidean space R
2
with the euclidean inner product. The vectors
u
1
= (1, 0) and u
2
= (0, 1) form an orthonormal basis B = u
1
, u
2
. Let us now rotate u
1
and u
2
anticlockwise by an angle to obtain v
1
= (cos , sin) and v
2
= (sin, cos ). Then ( = v
1
, v
2
is
also an orthonormal basis.
LINEAR ALGEBRA
W W L CHEN
c W W L Chen, 1997, 2006.
This chapter is available free to all individuals, on the understanding that it is not to be used for nancial gain,
and may be downloaded and/or photocopied, with or without permission from the author.
However, this document may not be kept on any information storage and retrieval system without permission
from the author, unless such system is not accessible to any individuals other than its owners.
Chapter 10
ORTHOGONAL MATRICES
10.1. Introduction
Definition. A square matrix A with real entries and satisfying the condition A
1
= A
t
is called an
orthogonal matrix.
Example 10.1.1. Consider the euclidean space R
2
with the euclidean inner product. The vectors
u
1
= (1, 0) and u
2
= (0, 1) form an orthonormal basis B = {u
1
, u
2
}. Let us now rotate u
1
and u
2
anticlockwise by an angle to obtain v
1
= (cos , sin) and v
2
= (sin, cos ). Then C = {v
1
, v
2
} is
also an orthonormal basis.
u
1
u
2
v
1
v
2
Chapter 10 : Orthogonal Matrices page 1 of 11
Linear Algebra c _ W W L Chen, 1997, 2008
The transition matrix from the basis ( to the basis B is given by
P = ( [v
1
]
B
[v
2
]
B
) =
cos sin
sin cos
.
Clearly
P
1
= P
t
=
cos sin
sin cos
.
In fact, our example is a special case of the following general result.
PROPOSITION 10A. Suppose that B = u
1
, . . . , u
n
and ( = v
1
, . . . , v
n
are two orthonormal
bases of a real inner product space V . Then the transition matrix P from the basis ( to the basis B is
an orthogonal matrix.
Example 10.1.2. The matrix
A =
1/3 2/3 2/3

2/3 1/3 2/3
2/3 2/3 1/3
is orthogonal, since
A
t
A =
1/3 2/3 2/3

2/3 1/3 2/3
2/3 2/3 1/3
1/3 2/3 2/3

2/3 1/3 2/3
2/3 2/3 1/3
1 0 0
0 1 0
0 0 1
.
Note also that the row vectors of A, namely (1/3, 2/3, 2/3), (2/3, 1/3, 2/3) and (2/3, 2/3, 1/3) are
orthonormal. So are the column vectors of A.
In fact, our last observation is not a coincidence.
PROPOSITION 10B. Suppose that A is an n n matrix with real entries. Then
(a) A is orthogonal if and only if the row vectors of A form an orthonormal basis of R
n
under the
euclidean inner product; and
(b) A is orthogonal if and only if the column vectors of A form an orthonormal basis of R
n
under the
euclidean inner product.
Proof. We shall only prove (a), since the proof of (b) is almost identical. Let r
1
, . . . , r
n
denote the row
vectors of A. Then
AA
t
=
r
1
r
1
. . . r
1
r
n
.
.
.
.
.
.
r
n
r
1
. . . r
n
r
n
.
It follows that AA
t
= I if and only if for every i, j = 1, . . . , n, we have
r
i
r
j
=
1 if i = j,
0 if i ,= j,
if and only if r
1
, . . . , r
n
are orthonormal. _
PROPOSITION 10C. Suppose that A is an n n matrix with real entries. Suppose further that the
inner product in R
n
is the euclidean inner product. Then the following are equivalent:
(a) A is orthogonal.
(b) For every x R
n
, we have |Ax| = |x|.
(c) For every u, v R
n
, we have Au Av = u v.
Proof. ((a)(b)) Suppose that A is orthogonal, so that A
t
A = I. It follows that for every x R
n
, we
have
|Ax|
2
= Ax Ax = x
t
A
t
Ax = x
t
Ix = x
t
x = x x = |x|
2
.
((b)(c)) Suppose that |Ax| = |x| for every x R
n
. Then for every u, v R
n
, we have
AuAv =
1
4
|Au+Av|
2
1
4
|AuAv|
2
=
1
4
|A(u+v)|
2
1
4
|A(uv)|
2
=
1
4
|u+v|
2
1
4
|uv|
2
= uv.
((c)(a)) Suppose that Au Av = u v for every u, v R
n
. Then
Iu v = u v = Au Av = v
t
A
t
Au = A
t
Au v,
so that
(A
t
AI)u v = 0.
In particular, this holds when v = (A
t
AI)u, so that
(A
t
AI)u (A
t
AI)u = 0,
whence
(A
t
AI)u = 0, (1)
in view of Proposition 9A(d). But then (1) is a system of n homogeneous linear equations in n unknowns
satised by every u R
n
. Hence the coecient matrix A
t
AI must be the zero matrix, and so A
t
A = I.
_
Proof of Proposition 10A. For every u V , we can write
u =
1
u
1
+ . . . +
n
u
n
=
1
v
1
+ . . . +
n
v
n
, where
1
, . . . ,
n
,
1
, . . . ,
n
R,
and where B = u
1
, . . . , u
n
and ( = v
1
, . . . , v
n
are two orthonormal bases of V . Then
|u|
2
= u, u) =
1
u
1
+ . . . +
n
u
n
,
1
u
1
+ . . . +
n
u
n
) =
n
i=1
n
j=1
j
u
i
, u
j
) =
n
i=1
2
i
= (
1
, . . . ,
n
) (
1
, . . . ,
n
).
Similarly,
|u|
2
= u, u) =
1
v
1
+ . . . +
n
v
n
,
1
v
1
+ . . . +
n
v
n
) =
n
i=1
n
j=1
j
v
i
, v
j
) =
n
i=1
2
i
= (
1
, . . . ,
n
) (
1
, . . . ,
n
).
It follows that in R
n
with the euclidean norm, we have |[u]
B
| = |[u]
C
|, and so |P[u]
C
| = |[u]
C
| for
every u V . Hence |Px| = |x| holds for every x R
n
. It now follows from Proposition 10C that P
is orthogonal. _
10.2. Eigenvalues and Eigenvectors
In this section, we give a brief review on eigenvalues and eigenvectors rst discussed in Chapter 7.
Suppose that
A =
a
11
. . . a
1n
.
.
.
.
.
.
a
n1
. . . a
nn
is an n n matrix with real entries. Suppose further that there exist a number R and a non-zero
vector v R
n
such that Av = v. Then we say that is an eigenvalue of the matrix A, and that v is
an eigenvector corresponding to the eigenvalue . In this case, we have Av = v = Iv, where I is the
n n identity matrix, so that (AI)v = 0. Since v R
n
is non-zero, it follows that we must have
det(AI) = 0. (2)
In other words, we must have
det
a
11
a
12
. . . a
1n
a
21
a
22
a
2n
.
.
.
.
.
.
.
.
.
a
n1
a
n2
. . . a
nn

= 0.
Note that (2) is a polynomial equation. The polynomial det(AI) is called the characteristic polynomial
of the matrix A. Solving this equation (2) gives the eigenvalues of the matrix A.
On the other hand, for any eigenvalue of the matrix A, the set
v R
n
: (AI)v = 0 (3)
is the nullspace of the matrix AI, and forms a subspace of R
n
. This space (3) is called the eigenspace
corresponding to the eigenvalue .
Suppose now that A has eigenvalues
1
, . . . ,
n
R, not necessarily distinct, with corresponding
eigenvectors v
1
, . . . , v
n
R
n
, and that v
1
, . . . , v
n
are linearly independent. Then it can be shown that
P
1
AP = D,
where
P = ( v
1
. . . v
n
) and D =
1
.
.
.
.
In fact, we say that A is diagonalizable if there exists an invertible matrix P with real entries such
that P
1
AP is a diagonal matrix with real entries. It follows that A is diagonalizable if its eigenvectors
form a basis of R
n
. In the opposite direction, one can show that if A is diagonalizable, then it has n
linearly independent eigenvectors in R
n
. It therefore follows that the question of diagonalizing a matrix
A with real entries is reduced to one of linear independence of its eigenvectors.
We now summarize our discussion so far.
DIAGONALIZATION PROCESS. Suppose that A is an n n matrix with real entries.
(1) Determine whether the n roots of the characteristic polynomial det(AI) are real.
(2) If not, then A is not diagonalizable. If so, then nd the eigenvectors corresponding to these eigen-
values. Determine whether we can nd n linearly independent eigenvectors.
(3) If not, then A is not diagonalizable. If so, then write
P = ( v
1
. . . v
n
) and D =
1
.
.
.
,
where
1
, . . . ,
n
R are the eigenvalues of A and where v
1
, . . . , v
n
R
n
are respectively their
corresponding eigenvectors. Then P
1
AP = D.
In particular, it can be shown that if A has distinct eigenvalues
1
, . . . ,
n
R, with corresponding
eigenvectors v
1
, . . . , v
n
R
n
, then v
1
, . . . , v
n
are linearly independent. It follows that all such matrices
A are diagonalizable.
10.3. Orthonormal Diagonalization
We now consider the euclidean space R
n
an as inner product space with the euclidean inner product.
Given any n n matrix A with real entries, we wish to nd out whether there exists an orthonormal
basis of R
n
consisting of eigenvectors of A.
Recall that in the Diagonalization process discussed in the last section, the columns of the matrix P
are eigenvectors of A, and these vectors form a basis of R
n
. It follows from Proposition 10B that this
basis is orthonormal if and only if the matrix P is orthogonal.
Definition. An nn matrix A with real entries is said to be orthogonally diagonalizable if there exists
an orthogonal matrix P with real entries such that P
1
AP = P
t
AP is a diagonal matrix with real
entries.
First of all, we would like to determine which matrices are orthogonally diagonalizable. For those that
are, we then need to discuss how we may nd an orthogonal matrix P to carry out the diagonalization.
To study the rst question, we have the following result which gives a restriction on those matrices
that are orthogonally diagonalizable.
PROPOSITION 10D. Suppose that A is a orthogonally diagonalizable matrix with real entries. Then
A is symmetric.
Proof. Suppose that A is orthogonally diagonalizable. Then there exists an orthogonal matrix P and
a diagonal matrix D, both with real entries and such that P
t
AP = D. Since PP
t
= P
t
P = I and
D
t
= D, we have
A = PDP
t
= PD
t
P
t
,
so that
A
t
= (PD
t
P
t
)
t
= (P
t
)
t
(D
t
)
t
P
t
= PDP
t
= A,
whence A is symmetric. _
Our rst question is in fact answered by the following result which we state without proof.
PROPOSITION 10E. Suppose that A is an n n matrix with real entries. Then it is orthogonally
diagonalizable if and only if it is symmetric.
The remainder of this section is devoted to nding a way to orthogonally diagonalize a symmetric
matrix with real entries. We begin by stating without proof the following result. The proof requires
results from the theory of complex vector spaces.
PROPOSITION 10F. Suppose that A is a symmetric matrix with real entries. Then all the eigenvalues
of A are real.
Our idea here is to follow the Diagonalization process discussed in the last section, knowing that since
A is diagonalizable, we shall nd a basis of R
n
consisting of eigenvectors of A. We may then wish to
orthogonalize this basis by the Gram-Schmidt process. This last step is considerably simplied in view
of the following result.
PROPOSITION 10G. Suppose that u
1
and u
2
are eigenvectors of a symmetric matrix A with real
entries, corresponding to distinct eigenvalues
1
and
2
respectively. Then u
1
u
2
= 0. In other words,
eigenvectors of a symmetric real matrix corresponding to distinct eigenvalues are orthogonal.
Proof. Note that if we write u
1
and u
2
as column matrices, then since A is symmetric, we have
Au
1
u
2
= u
t
2
Au
1
= u
t
2
A
t
u
1
= (Au
2
)
t
u
1
= u
1
Au
2
.
It follows that
1
u
1
u
2
= Au
1
u
2
= u
1
Au
2
= u
1

2
u
2
,
so that (
1

2
)(u
1
u
2
) = 0. Since
1
,=
2
, we must have u
1
u
2
= 0. _
We can now follow the procedure below.
ORTHOGONAL DIAGONALIZATION PROCESS. Suppose that A is a symmetric nn matrix
with real entries.
(1) Determine the n real roots
1
, . . . ,
n
of the characteristic polynomial det(AI), and nd n linearly
independent eigenvectors u
1
, . . . , u
n
of A corresponding to these eigenvalues as in the Diagonaliza-
tion process.
(2) Apply the Gram-Schmidt orthogonalization process to the eigenvectors u
1
, . . . , u
n
to obtain orthogo-
nal eigenvectors v
1
, . . . , v
n
of A, noting that eigenvectors corresponding to distinct eigenvalues are
already orthogonal.
(3) Normalize the orthogonal eigenvectors v
1
, . . . , v
n
to obtain orthonormal eigenvectors w
1
, . . . , w
n
of
A. These form an orthonormal basis of R
n
. Furthermore, write
P = ( w
1
. . . w
n
) and D =
1
.
.
.
,
where
1
, . . . ,
n
R are the eigenvalues of A and where w
1
, . . . , w
n
R
n
are respectively their
orthogonalized and normalized eigenvectors. Then P
t
AP = D.
Remark. Note that if we apply the Gram-Schmidt orthogonalization process to eigenvectors corre-
sponding to the same eigenvalue, then the new vectors that result from this process are also eigenvectors
corresponding to this eigenvalue. Why?
Example 10.3.1. Consider the matrix
A =
2 2 1
2 5 2
1 2 2
.
To nd the eigenvalues of A, we need to nd the roots of
det
2 2 1
2 5 2
1 2 2
= 0;
in other words, (7)(1)
2
= 0. The eigenvalues are therefore
1
= 7 and (double root)
2
=
3
= 1.
An eigenvector corresponding to
1
= 7 is a solution of the system
(A7I)u =
5 2 1
2 2 2
1 2 5
u = 0, with root u
1
=
1
2
1
.
Eigenvectors corresponding to
2
=
3
= 1 are solutions of the system
(AI)u =
1 2 1
2 4 2
1 2 1
u = 0, with roots u
2
=
1
0
1
and u
3
=
2
1
0
which are linearly independent. Next, we apply the Gram-Schmidt orthogonalization process to u
2
and
u
3
, and obtain
v
2
=
1
0
1
and v
3
=
1
1
1
which are now orthogonal to each other. Note that we do not have to do anything to u
1
at this stage,
in view of Proposition 10G. We now conclude that
v
1
=
1
2
1
, v
2
=
1
0
1
, v
3
=
1
1
1
form an orthogonal basis of R

3
. Normalizing each of these, we obtain respectively
w
1
=
1/
6
2/
6
1/
, w
2
=
1/
2
0
1/
, w
3
=
1/
3
1/
3
1/
.
We now take
P = ( w
1
w
2
w
3
) =
1/
6 1/
2 1/
3
2/
6 0 1/
3
1/
6 1/
2 1/
.
Then
P
1
= P
t
=
1/
6 2/
6 1/
6
1/
2 0 1/
2
1/
3 1/
3 1/
and P
t
AP =
7 0 0
0 1 0
0 0 1
.
A =
1 6 12
0 13 30
0 9 20
.
det
1 6 12
0 13 30
0 9 20
= 0;
in other words, ( + 1)( 2)( 5) = 0. The eigenvalues are therefore
1
= 1,
2
= 2 and
3
= 5.
An eigenvector corresponding
1
(A + I)u =
0 6 12
0 12 30
0 9 21
u = 0, with root u
1
=
1
0
0
.
2
(A2I)u =
3 6 12
0 15 30
0 9 18
u = 0, with root u
2
=
0
2
1
.
3
(A5I)u =
6 6 12
0 18 30
0 9 15
u = 0, with root u
3
=
1
5
3
.
Note that while u
1
, u
2
, u
3
correspond to distinct eigenvalues of A, they are not orthogonal. The matrix
A is not symmetric, and so Proposition 10G does not apply in this case.
A =
5 2 0
2 6 2
0 2 7
.
det
5 2 0
2 6 2
0 2 7
= 0;
in other words, ( 3)( 6)( 9) = 0. The eigenvalues are therefore
1
= 3,
2
= 6 and
3
= 9. An
eigenvector corresponding
1
(A3I)u =
2 2 0
2 3 2
0 2 4
u = 0, with root u
1
=
2
2
1
.
2
(A6I)u =
1 2 0
2 0 2
0 2 1
u = 0, with root u
2
=
2
1
2
.
3
(A9I)u =
4 2 0
2 3 2
0 2 2
u = 0, with root u
3
=
1
2
2
.
Note now that the eigenvalues are distinct, so it follows from Proposition 10G that u
1
, u
2
, u
3
are or-
thogonal, so we do not have to apply Step (2) of the Orthogonal diagonalization process. Normalizing
each of these vectors, we obtain respectively
w
1
=
2/3
2/3
1/3
, w
2
=
2/3
1/3
2/3
, w
3
=
1/3
2/3
2/3
.
We now take
P = ( w
1
w
2
w
3
) =
2/3 2/3 1/3

2/3 1/3 2/3
1/3 2/3 2/3
.
Then
P
1
= P
t
=
2/3 2/3 1/3

2/3 1/3 2/3
1/3 2/3 2/3
and P
t
AP =
3 0 0
0 6 0
0 0 9
.
Problems for Chapter 10
1. Prove Proposition 10B(b).
2. Let
A =
a + b b a
a b b + a
,
where a, b R. Determine when A is orthogonal.
3. Suppose that A is an orthogonal matrix with real entries. Prove that
a) A
1
is an orthogonal matrix; and
b) det A = 1.
4. Suppose that A and B are orthogonal matrices with real entries. Prove that AB is orthogonal.
5. Verify that for every a R, the matrix
A =
1
1 + 2a
2
1 2a 2a
2
2a 1 2a
2
2a
2a
2
2a 1
is orthogonal.
6. Suppose that is an eigenvalue of an orthogonal matrix A with real entries. Prove that 1/ is also
an eigenvalue of A.
7. Suppose that
A =
a b
c d
is an orthogonal matrix with real entries. Explain why a

2
+ b
2
= c
2
+ d
2
= 1 and ac + bd = 0, and
quote clearly any result that you use. Deduce that A has one of the two possible forms
A =
cos sin
sin cos
or A =
cos sin
sin cos
,
where [0, 2).
8. Consider the matrix
A =
6

3
6 2

2
3

2 3
.
a) Find the characteristic polynomial of A and show that A has eigenvalues 4 (twice) and 2.
b) Find an eigenvector of A corresponding to the eigenvalue 2.
c) Find two orthogonal eigenvectors of A corresponding to the eigenvalue 4.
d) Find an orthonormal basis of R
3
consisting of eigenvectors of A.
e) Using the orthonormal basis in part (d), nd a matrix P such that P
t
AP is a diagonal matrix.
9. Apply the Orthogonal diagonalization process to each of the following matrices:
a) A =
5 0 6
0 11 6
6 6 2
b) A =
0 2 0
2 0 1
0 1 0
c) A =
1 4 2
4 1 2
2 2 2
d) A =
2 0 36
0 3 0
36 0 23
e) A =
1 1 0 0
1 1 0 0
0 0 0 0
0 0 0 0
f) A =
7 24 0 0
24 7 0 0
0 0 7 24
0 0 24 7
10. Suppose that B is an m n matrix with real entries. Prove that the matrix A = B
t
B has an
orthonormal set of n eigenvectors.

Orthogonal Matrices

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Orthogonal Matrices

Uploaded by

Copyright:

Available Formats

LINEAR ALGEBRA

1/3 2/3 2/3

1/3 2/3 2/3

1/3 2/3 2/3

form an orthogonal basis of R

2/3 2/3 1/3

2/3 2/3 1/3

is an orthogonal matrix with real entries. Explain why a

You might also like