19 Orthogonal projections and orthogonal matrices

19.1 Orthogonal projections

We often want to decompose a given vector, for example, a force, into the sum of two
orthogonal vectors.
Example: Suppose a mass m is at the end of a rigid, massless rod (an ideal pendulum),
and the rod makes an angle with the vertical. The force acting on the pendulum is the
gravitational force mge
2
. Since the pendulum is rigid, the component of the force parallel
mg sin()
The pendulum bob makes an angle
with the vertical. The magnitude of the
force (gravity) acting on the bob is mg.
The component of the force acting in
the direction of motion of the pendulum
has magnitude mg sin().
mg

to the direction of the rod doesnt do anything (i.e., doesnt cause the pendulum to move).
Only the force orthogonal to the rod produces motion.
The magnitude of the force parallel to the pendulum is mg cos ; the orthogonal force has
the magnitude mg sin . If the pendulum has length l, Newtons law (F = ma) reads
ml

= mg sin ,
or

+
g
l
sin = 0.
This is the dierential equation for the motion of the pendulum. For small angles, we have,
approximately, sin , and the equation can be linearized to give

+
2
= 0, where =

g
l
,
which is identical to the equation of the harmonic oscillator.
1
19.2 Algorithm for the decomposition
Given the xed vector w, and another vector v, we want to decompose v as the sum v =
v
||
+v

, where v
||
is parallel to w, and v

is orthogonal to w. See the gure. Suppose is

the angle between w and v. We assume for the moment that 0 /2. Then
is ||v|| cos().
v
w

v
||
If the angle between v and w
||v|| cos()
projection of v onto w
is , then the magnitude of the
||v
||
|| = ||v|| cos = ||v||

w
||v|| ||w||

=
v

w
||w||
,
or
||v
||
|| = v

a unit vector in the direction of w

And v
||
is this number times a unit vector in the direction of w:
v
||
=
v

w
||w||
w
||w||
=

w
w

w.
In other words, if w = (1/||w||)w, then v
||
= (v

w) w. This is worth remembering.

Definition: The vector v
||
= (v

w) w is called the orthogonal projection of v onto w.

The nonzero vector w also determines a 1-dimensional subspace, denoted W, consisting of
all multiples of w, and v
||
is also known as the orthogonal projection of v onto the
subspace W.
Since v = v
||
+v

, once we have v
||
, we can solve for v

algebraically:
v

= v v
||
.
Example: Let
v =

1
1
2

, and w =

1
0
1

.
2
Then ||w|| =

2, so w = (1/

2)w, and
(v

w) w =

3/2
0
3/2

.
Then
v

= v v
||
=

1
1
2

3/2
0
3/2

1/2
1
1/2

.
and you can easily check that v
||

= 0.
Remark: Suppose that, in the above, /2 < , so the angle is not acute. In this case,
cos is negative, and ||v|| cos is not the length of v
||
(since its negative, it cant be a
length). It is interpreted as a signed length, and the correct projection points in the opposite
direction from that of w. In other words, the formula is correct, no matter what the value
of .
Exercises:
1. Find the orthogonal projection of
v =

2
2
0

onto
w =

1
4
2

.
Find the vector v

.
2. When is v

= 0? When is v
||
= 0?
3. This refers to the pendulum gure. Suppose the mass is located at (x, y) R
2
. Find
the unit vector parallel to the direction of the rod, say r, and a unit vector orthogonal
to r, say

, obtained by rotating r counterclockwise through an angle /2. Express
these orthonormal vectors in terms of the angle . And show that F

= mg sin as
claimed above.
4. (For those with some knowledge of dierential equations) Explain (physically) why the
linearized pendulum equation is only valid for small angles. (Hint: if you give a real
pendulum a large initial velocity, what happens? Is this consistent with the behavior
of the harmonic oscillator?)
3
19.3 Orthogonal matrices
Suppose we take an orthonormal (o.n.) basis {e
1
, e
2
, . . . , e
n
} of R
n
and form the nn matrix
E = (e
1
| |e
n
). Then
E
t
E =

e
t
1
e
t
2
.
.
.
e
t
n

(e
1
| |e
n
) = I
n
,
because
(E
t
E)
ij
= e
t
i
e
j
= e
i

e
j
=
ij
,
where
ij
are the components of the identity matrix:

ij
=

1 if i = j
0 if i = j
Since E
t
E = I, this means that E
t
= E
1
.
Definition: A square matrix E such that E
t
= E
1
is called an orthogonal matrix.
Example:
{e
1
, e
2
} =

1/

2
1/

1/

2
1/

is an o.n. basis for R

2
. The corresponding matrix
E = (1/

2)

1 1
1 1

is easily veried to be orthogonal. Of course the identity matrix is also orthogonal. .

Exercises:
If E is orthogonal, then the columns of E form an o.n. basis of R
n
.
If E is orthogonal, so is E
t
, so the rows of E also form an o.n. basis.
(*) If E and F are orthogonal and of the same dimension, then EF is orthogonal.
(*) If E is orthogonal, then det(E) = 1.
Let
{e
1
(), e
2
()} =

cos
sin

sin
cos

.
Let R() = (e
1
()|e
2
()). Show that R()R() = R( + ).
4
If E and F are the two orthogonal matrices corresponding to two o.n. bases, then
F = EP, where P is the change of basis matrix from E to F. Show that P is also
orthogonal.
19.4 Invariance of the dot product under orthogonal transforma-
tions
In the standard basis, the dot product is given by
x

y = x
t
Iy = x
t
y,
since the matrix which represents

is just I. Suppose we have another orthonormal basis,
{e
1
, . . . , e
n
}, and we form the matrix E = (e
1
| |e
n
). Then E is orthogonal, and its the
change of basis matrix taking us from the standard basis to the new one. We have, as usual,
x = Ex
e
, and y = Ey
e
.
So
x

y = x
t
y = (Ex
e
)
t
(Ey
e
) = x
t
e
E
t
Ey
e
= x
t
e
Iy
e
= x
t
e
y
e
.
What does this mean? It means that you compute the dot product in any o.n. basis using
exactly the same formula that you used in the standard basis.
Example: Let
x =

2
3

, and y =

3
1

.
So x

y = x
1
y
1
+ x
2
y
2
= (2)(3) + (3)(1) = 3.
In the o.n. basis
{e
1
, e
2
} =

1
1

,
1

1
1

,
we have
x
e
1
= x

e
1
= 1/

2
x
e
2
= x

e
2
= 5/

2 and
y
e
1
= y

e
1
= 4/

2
y
e
2
= y

e
2
= 2/

2.
And
x
e
1
y
e
1
+ x
e
2
y
e
2
= 4/2 + 10/2 = 3.
This is the same result as we got using the standard basis! This means that, as long as
were operating in an orthonormal basis, we get to use all the same formulas we use in the
standard basis. For instance, the length of x is the square root of the sum of the squares of the
5
components, the cosine of the angle between x and y is computed with the same formula as
in the standard basis, and so on. We can summarize this by saying that Euclidean geometry
is invariant under orthogonal transformations.
Exercise: ** Heres another way to get at the same result. Suppose A is an orthogonal
matrix, and f
A
: R
n
R
n
the corresponding linear transformation. Show that f
A
preserves
the dot product: Ax

Ay = x

y for all vectors x, y. (Hint: use the fact that x

y = x
t
y.)
Since the dot product is preserved, so are lengths (i.e. ||Ax|| = ||x||) and so are angles, since
these are both dened in terms of the dot product.
6