LECTURE V: BILINEAR FORMS AND ORTHOGONALITY

MAT 204 - FALL 2006 PRINCETON UNIVERSITY

ALFONSO SORRENTINO

1. Bilinear forms

Denition.

Let

be a vector space. A bilinear form on

is an application

b : V V R
that satises the two following conditions:

i) ii)

b(c1 u1 + c2 u2 , v ) = c1 b(u1 , v ) + c2 b(u2 , v ), b(u, d1 v1 + d2 v2 ) = d1 b(u, v1 ) + d2 b(u, v2 ),

c1 , c2 R, u1 , u2 , v V ; d1 , d2 R, v1 , v2 , u V .

A bilinear form is said to be symmetric [resp. skew-symmetric] if

b(u, v ) = b(v, u)

[resp. b(u, v ) = b(v, u)] u, v V . b a matrix, b;

it will obviously depend

We would like, as already done for linear transformations, to associate to that encodes all the information we need to know about on our choice of a basis on

V. n and
let

Denition.
matrix

Let

be a vector space of dimension

basis. Given a bilinear form

b : V V R,

we will call matrix of

E = (e1 , . . . , en ) be its b w.r.t. E, the

b(e1 , e1 ) . . . . . . b(e2 , e1 ) . . . . . . A= . . . . . . . . . b(en , e1 ) . . . . . .

Consider two generic vectors

b(e1 , en ) b(e2 , en ) Mn (R). . . . b(en , en ) b

on

Let us see how we can use this matrix to recover the behavior of

V V. x1
. . .

u = Ex

and

v = Ey

in

V.

We want to determine an

expression for

b(u, v ) in terms of x and y ;

we have:

(i.e., with respect to

E).

Set

x=

xn
and

y1
. . .

y=

yn
n n

xi b(ei , v ) = (x1 , . . . , xn )

b(e1 , v )
. . .

b(u, v )

= b
i=1

xi ei , v b(e1 , v )
. . .

=
i=1

b(en , v )

.
1

= xT

b(en , v )

2
For each

ALFONSO SORRENTINO

i = 1, . . . , n, b(ei , v )

we can compute

= b ei ,

b(ei , v ):

yj ej =
j =1 j =1

b(ei , ej )yj = y1
. . .

( i) = A y.

=
Therefore,

(b(ei , e1 ), . . . , b(ei , en ))

yn b(Ex, Ey )
The expression

A(1) y
. . .

A(1)
. . .

T y = x Ay .

= xT

T =x

A(n) y b(Ex, Ey ) = xT Ay bilinear form b w.r.t. E.

A(n)

is called expression of the

Remark.
b
xed

From the expression above, it follows that, once we x a basis

of

V,

is uniquely determined by the matrix

A. b,

Hence, there exists a bijection (w.r.t. a

E)

between the set of bilinear forms on

and

Mn (R).

Let us see what other information about

can be recovered from its matrix.

Proposition 1.
bilinear form on

Let

be a

V.

Fix a basis

n-dimensional E on V and A is

vector space and let

b : V V R a b(Ex, Ey ) = xT Ay . We have that: AT = A ].

b is

symmetric

a symmetric matrix [i.e.,

Proof.

(= ) Since

( =) Observe

b(ei , ej ) = b(ej , ei ), then aij = aji and consequently AT = A. T that, since x Ay is a real number, it coincides with its transpose;

i.e.,

xT Ay = (xT Ay )T = y T AT x = y T Ax , where we used, to get the last equality, that A is symmetric. any u, v V :

Therefore, for

b(u, v ) = b(Ex, Ey ) = xT Ay = y T Ax = b(Ey, Ex) = b(v, u) .

One can verify, in a completely analogous way, that:

Proposition 2.
bilinear form on

Let

be a

V.

Fix a basis

n-dimensional E on V and A is

vector space and let

b : V V R a b(Ex, Ey ) = xT Ay . We have that: AT = A ].

b is
in

skew-symmetric

a skew-symmetric matrix [i.e.,

One natural question that one could ask is: what happens when I change the basis

V? E
and

Let and

be two bases for . We have:

and assume that

E = EC

(with

the formula for the change of basis. Consider two generic vectors

C GLn (R)) is u = Ex = E x

v = Ey = E y
T

x Ay = b(Ex, Ey ) = b(u, v ) = b(E x , E y ) = (x )T A y .

Since

E = EC ,

then

x = Cx

and

y = Cy
T

; therefore:

x Ay = (Cx ) A(Cy ) = ((x ) C )A(Cy ) = (x )T (C T AC )y = (x )T A y .

It follows that

A = C T AC .

LECTURE V: BILINEAR FORMS AND ORTHOGONALITY

Remark.
GLn (R)

Let

such that

alence relation

A, A Mn (R). They are called congruent if there exists C A = C T AC . One can easily verify that congruence is an equivon the set of Mn (R).
Two matrices

In particular (recall what we have proved for similar matrices in Lecture IV):

Proposition 3. Denition.
form on Let

the same bilinear form

A, A Mn (R) are congruent they b : V V R, with respect to dierent bases.

represent

V.

We call rank of

n-dimensional vector space and b : V V R a bilinear b, the rank of its matrix A, with respect to any basis E.

Observe that this denition makes sense (namely, it does not depend on the choice

E), since two congruent matrices have the same rank: rank (A ) = rank (C T AC ) (we have already noticed that multiplication by invertible matrices, does not change the rank). Hence, rank (b) is an invariant of b. A bilinear form is said to be degenerate, if rank (b) < n; otherwise it is said to be
of the basis

non-degenerate.

Proposition 4.
form on
i)

Let

n-dimensional

vector space and

b : V V R

a bilinear

V.

The following conditions are equivalent:

is degenerate;

ii) there exists a non-zero vector iii) there exists a non-zero vector

u0 V , such that b(u0 , v ) = 0 for any v V ; v0 V , such that b(u, v0 ) = 0 for any u V .

Proof.
Let

We show only the equivalence between i) and ii) [the proof of the equivalence We have:

between i) and iii) is pretty similar].

b(Ex, Ey ) = xT Ay . b is
degenerate the rows

rank A < n A(1) , . . . , A(n)

are linearly dependent

there exist

c1 , . . . , cn R (not

all zero), such that

ci A(i) = 0
i=1

there exists

c Mn,1 (R), c = 0,

such that

c A = 0.

Let us verify that:

cT A = 0 cT Ay = 0, y Mn,1 (R).
The implication (= ) is evident. Let us verify (=). In fact:

1 0 = cT A . = cT A(1) . . 0
. . .

0 0 = cT A . = cT A(n) . . . 1

Consequently,

cT A = cT A(1) . . . A(n) = cT A(1) . . . cT A(n) = (0, . . . , 0) = 0 .

ALFONSO SORRENTINO

This allows us to conclude the proof. In fact, using this fact:

b is degenerate rank A < n there exists c Mn,1 (R), c = 0,

there exists there

such that

cT A = 0

c Mn,1 (R), c = 0, such that cT Ay = 0, y Mn,1 (R) exists u0 = Ec V, u0 = 0, such that b(u0 , v ) = 0, v = Ey V .

2. Orthogonality Note: From here on, we will just consider symmetric bilinear forms. In the physical space, it is well known the concept of orhogonal vectors; namely, two vectors are orthogonal if, considered at a same starting point, they form a right angle. We would like to generalize this concept to a general vector space, but it is not evident at all what a natural denition should be. Think for instance at the space of polynomials with real coecients. What might be the meaning of orthogonal polynomials? The main goal of this section is to introduce the concept of orthogonality in a more general framework. Let us rst rephrase the denition of orthogonal physical vectors in way. Dene the application

R2

in a dierent

b : R2 R2 (v1 , v2 )
where

R v1

v2 cos [0, ]
then is the

v1

and

v2

is the euclidean length of the vectors and

smallest angle between these vectors (when we apply them at the same point). One can verify that this is a symmetric bilinear form and that, if

v1 , v2 = 0,

b(v1 , v2 ) = 0
Therefore,

= . 2

v1

and

v2

are orthogonal

b(v1 , v2 ) = 0 . u, v V .
If

This observation justies the following denition.

Denition.
b(u, v ) = 0,
If

Let

b : V V R

be a symmetric bilinear form and

then the two vectors are called

b-orthogonal

(or simply orthogonal) and

u v. S is a non-empty set of V (not necessarily a subspace!), the set S := {u V : b(u, v ) = 0, for all v V } (i.e., the set of vectors that are b-orthogonal to all vectors of S ) is called b-orthogonal subspace of S . It is easy to verify that S is a vector subspace of V [Exercise]. If S = {w }, we will abbreviate w , instead of {w } . Moreover, if W = w1 , . . . , ws ,
it will be denoted by one can easily check that

W = w1 . . . ws . Finally, a vector v V is called b-isotropic (or simply isotropic), if b(v, v ) = 0. The set of all isotropic vectors of V is called b-isotropic cone of V and is denoted by Ib (V ).

Remark.
of

i) In general, the

V.

For example, suppose that

b-isotropic cone Ib (V ) is v1 , v2 Ib (V ) and

not a vector subspace that

b(v1 , v2 ) = 0;

one

has that

v1 + v2 Ib (V ).

In fact:

b(v1 + v2 , v1 + v2 ) = b(v1 , v1 ) + b(v2 , v2 ) + 2b(v1 , v2 ) = 2b(v1 , v2 ) = 0 .

LECTURE V: BILINEAR FORMS AND ORTHOGONALITY

Moreover, observe that if theoretical sense) of

ii)

Ib (V ) = {0}, then it is the union (in the set 1-dimensional vector subspaces of V . In fact, if v Ib (V ) and c R, it is easy to verify that cv Ib (V ). If Ib (V ) = {0}, then b is non degenerate. In fact, using proposition 4, if b(u0 , v ) = 0 for all v V , then in particular b(u0 , u0 ) = 0 and this implies that u0 Ib (V ) = {0}.
The converse is false! Namely, there exist non-degenerate symmetric bilinear forms, whose isotropic cone is not trivial. matrix: For example, consider the

A=
It denes (w.r.t. bilinear form on

0 1 1 1

the canonical basis of

(in fact

R2 ) a non-degenerate symmetric rank (A) = 2), but Ib (R2 ) = {0}; in fact,

be a symmetric bilinear form and

e1 Ib (R2 ).

Proposition 5.
i.e., each vector given by:

Let

b : V V R

uV

non-isotropic vector. Then,

V = u u , v V
can be decomposed uniquely into the sum of a vector parallel (proportional) to

and a vector

v2 b-orthogonal v2 = v

to

u.

v1 These vectors are

v1 =

b(u, v ) u b(u, u)

and

b(u, v ) u. b(u, u)

Proof.

By hypothesis, we know that

b(u, u) = 0. and

We need to verify that

u u = 0
If

u + u = V .

Let us start by proving the rst equality.

w u u , then w = cu (there exists c R) and b(w, u) = 0 . Therefore 0 = b(cu, u) = cb(u, u) and this implies - since b(u, u) = 0 - that c = 0. Hence, w = 0. Let us verify now that u + u = V . In fact, for any v V , if we dene v1 and v2 as above, we have: v1 + v2 = v , v1 u and v2 u . In fact: b(u, v2 ) = b u, v b(u, v ) u b(u, u) = b(u, v ) b(u, u) b(u, v ) = 0. b(u, u)

Denition.

isotropic vector. For any

b : V V R be a symmetric bilinear form and u V a nonb(u,v ) v V , the scalar b (u,u) is called Fourier coecient of v with respect to u and will be denoted by cu (v ). The vector v1 = cu (v )u is called b-orthogonal projection of v in the u direction.
Let Observe that the application

cu : V v
is a linear application [Exercise].

R cu (v )

Proposition 6.

Let

symmetric bilinear form on

n-dimensional V , such that

vector space with basis

E.

Let

be a

b(Ex, Ey ) = xT Ay .
Let

W be a s-dimensional vector subspace of V , with basis {z1 , . . . , zs } and let (z1 . . . zs ) = ED (with D Mn,s (R) and rank (D) = s). Then, the cartesian

6
equations of

ALFONSO SORRENTINO

(i.e., its

b-orthogonal

subspace) with respect to

E,

are given by the

HLS(s, n, R): Moreover, if

is non-degenerate,

DT AX = 0 . then dim W = n dim W . b(z1 , v ) = . . . = b(zs , v ) = 0

T T Ax = . . . = D( D(1) s) Ax = 0

Proof.

Let

v = Ex V . v = Ex W

We have:

DT Ax = 0 . DT AX = 0 . A GLn (R). In this

case,

Hence, If

has cartesian equations given by the HLS(s, n, R)

b is non-degenerate, then rank A = n and therefore rank (DT A) = rank (DT ) = rank (D) = s. Therefore,

dim W = n rank (DT A) = n s = n dim W .

Example.

Let

canonical basis

b be a symmetric bilinear form on R4 dened, E = (e1 e2 e3 e4 ) of R4 , by the matrix 1 0 0 0 0 1 2 1 A= 0 2 0 0 . 0 1 0 1

with respect to the

i) Verify that ii) Set iii) iv)

b is non-degenerate. W = e3 , e1 + e2 . Determine the cartesian Verify that W W = 0. Determine the b-isotropic vectors in W .

equations of

W .

Solution:
i)

is non-degenerate, since

det A = 0. 0 0 with D = 1 0
where

ii) We have:

(e3 e1 + e2 ) = ED,

1 1 . 0 0 D = 0 1 2 1 0 0 2 1
,

W
i.e.,

has cartesian equations

DT AX = 0,

2x2 = 0 x1 + x2 + 2x3 x4 = 0
Therefore, iii) The cartesian equations of

x2 = 0 x1 + 2x3 x4 = 0 .

dim W = 4 2 = 2 and W = (2, 0, 1, 0), (1, 0, 0, 1) . W are obtained directly imposing the condition rank (Dx) = 2: x1 x2 = 0 x4 = 0 .
Hence,

W W has cartesian equations given by: x1 x2 = 0 x1 = 0 x4 = 0 x2 = 0 x = 0 x3 = 0 2 x1 + 2x3 x4 = 0 x4 = 0

LECTURE V: BILINEAR FORMS AND ORTHOGONALITY

this means iv) We have:

W W = 0

2 2 Ex Ib (R4 ) xT Ax = 0 x2 1 + x2 + 4x2 x3 2x2 x4 x4 = 0 .

Ex Ib (R4 ) W if and only if x2 = 0 x2 = 0 x1 + 2x3 x4 = 0 x1 + 2x3 x4 = 0 2 x1 x4 = 0 x1 x2 4 =0

Therefore, In conclusion:

or

x2 = 0 x1 + 2x3 x4 = 0 x1 + x4 = 0 . t R}

Ib (R ) W = {(t, 0, 0, t) R4 ,
4
or equivalently

for all

t R} {(t, 0, t, t) R4 ,

for all

Ib (R4 ) W = (1, 0, 0, 1) (1, 0, 1, 1) .

3. Quadratic forms

Denition.

Let

b : V V R Q:V v

be a symmetric bilinear form. The application

R Q(v ) := b(v, v ) b.

is called quadratic form associated to

Remark.

For any

v1 , v2 V ,

we have:

Q(v1 + v2 )

= b(v1 + v2 , v1 + v2 ) = = b(v1 , v1 ) + b(v2 , v2 ) + 2b(v1 , v2 ) = = Q(v1 ) + Q(v2 ) + 2b(v1 , v2 ) . b

from its quadratic form:

Therefore, we can reconstruct

(b is also called

1 b(v1 , v2 ) = [Q(v1 + v2 ) Q(v1 ) Q(v2 )] 2 polar form of Q) . In particular, it follows that V.

there exists a

11

correspondence between the set of symmetric bilinear forms on quadratic forms on

and the set of

Let

be a

bilinear form on

n-dimensional vector space and E a basis for V . Let b be a symmetric V , with b(Ex, Ey ) = xT Ay . For any vector v = Ex V , we have:
n n

Q(v ) = Q(Ex) = b(Ex, Ex) = xT Ax =

i=1 j =1
this is called expression of the quadratic form

aij xi xj ; E.

with respect to

Remark.

The expression

aij xi xj
i=1 j =1
is a homogeneous polynomial of degree geneous polynomial of degree respect to a xed basis

in

2 in x1 , . . . , xn . Conversely, every homox1 , . . . , xn determines a quadratic form (with bij xi xj

1ij n

E).

In fact, if

P (x1 , . . . , xn ) =

ALFONSO SORRENTINO

is an arbitrary homogeneous polynomial of degree dene a matrix

in

R[x1 , . . . , xn ],

it suces to

A = (aij ) Mn (R), aii = bii aij = aji =

where

bij 2

i = 1, . . . , n for 1 i j n .
for It is easy

Consider now the quadratic form to verify that

Q : V R, such that Q(Ex) = xT Ax. Q(Ex) = P (x1 , . . . , xn ). 11

Hence, there is also a

correspondence between the set of quadratic forms on

and the set of homogeneous polynomials of degree

in

x1 , . . . , xn 2

(and also with

the set of symmetric matrices of order

n). 2x2 4 R[x1 , x2 , x3 , x4 ] .

Example.

Consider the homogeneous polynomial of degree

P (x1 , x2 , x3 , x4 ) =

x2 2

2x1 x3 + x2 x4 + 0 1 0
1 2

It denes the symmetric matrix:

0 0 A= 1 0
The symmetric bilinear form canonical basis) is:

1 0 0 0

0
1 2

M4 (R). 0 2
dened by

b : R4 R4 R

(with respect to the

1 1 b(x, y ) = x2 y2 x1 y3 x3 y1 + x2 y4 + x4 y2 + 2x4 y4 2 2
and the associated quadratic form:

2 Q(x) = x2 2 2x1 x3 + x2 x4 + 2x4 .

4. Inner products

Denition.

Let

be a real vector space and

b : V V R

a bilinear form.

is said to be positive denite, if it satises the following two conditions:

i) ii)
If

b(v, v ) 0 , for all v V b(v, v ) = 0 v = 0 .

satises only

i)

is called positive semidenite.

Analogously, tions:

is said to be negative denite, if it satises the following two condi-

i) ii )
If

b(v, v ) 0 , for all v V b(v, v ) = 0 v = 0 .

satises only

i) b

is called negative semidenite.

A symmetric positive denite bilinear form is called inner product.

Denition.
length) of

Let

be an inner product on

V.

For each

v V,

we call the norm (or

(with respect to

b),

the non-negative real number given by:

v :=
Observe that

b(v, v ) .
Moreover,

is well dened, since

b(v, v ) 0 (b is positive denite). v = 0; for any c R. b)

if

the following properties are obvious:

i) ii)
We will say that a vector non-zero vector

v = 0 cv = |c| v , v
is a versor (w.r.t.

v = 1;

in particular, for any

v V,

the vector

1 v

is a versor.

LECTURE V: BILINEAR FORMS AND ORTHOGONALITY

Moreover one can show that:

Proposition 7.
for any
i) ii)

Let V be a real vector space and b an inner product u, v V : |b(u, v )| u v (Cauchy-Schwartz inequality); | u v | u + v u + v (Triangular inequality).
i) For each

on

V.

Then,

Proof.

tR

we have:

b(u + tv, u + tv ) = u u
2

+ 2b(u, v )t + v 2 t2 0 .

From this inequality, it follows that the polynomial

Q = v 2 t2 +2b(u, v )t + has at most one real root and therefore its discriminant 0; hence: = b(u, v )2 u
2

0. v
.

We can then conclude that

|b(u, v )| =

b(u, v )2 u
2

ii) Let us show the second inequality. Using i), we have:

u+v

= b(u + v, u + v ) = u u
2

+ v 2 + 2b(u, v )
2

+ v2 + u

v ( u + v ) .

Taking the square root on both sides, we obtain the desired inequality. Let us show the rst inequality. We have:

u = u+vv u+v + v = u+v + v v = v+uu u+v + u = u+v + u

and this implies

u v u+v v u u+v

| u v | u+v
where

Denition.
on

A couple

(V, b),

is real vector space and

an inner product

V,

is called euclidean space.

Example. [Canonical inner product on Rn ] Dene

: Rn Rn R (x, y ) = x y := x1 y1 + . . . + xn yn ,
where

x = (x1 , . . . , xn )

and

y = (y1 , . . . , yn ).
T

This is a bilinear form [Exercise] and

its matrix (with respect to the canonical basis of

Rn )

is

In

(the identity):

x y = x In y .
Moreover,

i) ii)
Therefore,

2 x x = x2 1 + . . . + xn 0 ; 2 x x = x2 1 + . . . + xn = 0

x = 0.

is an inner product and is called canonical (or standard) inner product.

From ii) in the denition of positive deniteness, it follows that if

Remark.

is

an inner product, then

Ib (V ) = {0} R
2
) by:

(i.e., no nonzero vector is

b-isotropic).

The

converse is false; for instance, consider the bilinear form on to the canonical basis of

R2

given (with respect

A=
Obviously, that

2 1 1 2

b is not an inner product: b(e1 , e1 ) = 2 < 0; but one can easily show Ib (R2 ) = {0}. In fact, Ib (R2 ) has equation 2x2 + 2xy 2y 2 = 0 and this 2 equation has only one solution in R , given by (0, 0) [to see this, one can denote x 2 t = y and see that the equation 2t + 2t 2 = 0 has not solutions in R].

10
It is also evident, that if rank

ALFONSO SORRENTINO

is an inner product of representing

V,

with

dim V = n, In .

then it has

n,

i.e., it is non-degenerate. In particular, it is always possible to nd a new

basis on is called

V , such that the matrix b-orthonormal basis.

w.r.t. this basis is

Such a basis

Denition.
Let If

Let V be a n-dimensional vector space and E = (e1 . . . en ) a basis. b be an inner product of V , with matrix A (w.r.t. E), i.e., b(Ex, Ey ) = xT Ay . A = In , E is called b-orthonormal basis and one has:

b(Ex, Ey ) = xT In y =
i=1
Hence, with respect to this basis, the canonical inner product on

xi yi

Ex, Ey V . V
as

b Rn .

acts on the coordinates of the vectors of

Remark.

We shall see in the next section, how we can actually compute such a

basis. In the meanwhile, let us just verify the following properties. i) Let

b be an inner product on V . Let us verify that t non-zero vectors u1 , . . . , ut V , pairwise b-orthogonal, are linearly independent. Suppose t that i=1 ci ui = 0; for j = 1, . . . , t:
t t

0 = b(uj , 0) = b uj ,
i=1
and since ii) If

ci ui

=
i=1

ci b(uj , ui ) = cj b(uj , uj )

is a

b(uj , uj ) > 0, it follows that cj = 0 (for all j = 1, . . . , t). b-orthonormal basis of V , then: x1 . b(v, ei ) = xi v = E . . V xn
n n j =1

[in fact, that for

b(v, ei ) = b = j =1 xj ej , ei any vector v V , the following

n

xj b(ej , ei ) = xi ].

It follows

representation holds:

v=
i=1

b(v, ei )v . V , n-dimensional vector C GLn (R). One has:

space. Let

Proposition 8.
E
and

Let

be an inner product on

be two bases with

F = EC ,

where

i) If ii)

E and F are both b-orthonormal, then C On (R) [i.e., it is orthogonal: C T C = In ]. If E is b-orthonormal and C On (R), then F is also b-orthonormal. 11 F.
correspondence between

Therefore, there is a

b-orthonormal

bases and

On (R).

Proof.
of

Let

be the matrix that represents

with respect to

with respect to i) If ii)

We have already observed that

E and B B = C T AC .

the matrix

E and F are both b-orthonormal, then B = A = In and consequently C T In C = In . Therefore C On (R). If E is b-orthonormal and C On (R), then A = In and consequently T T the matrix of b w.r.t. F is B = C In C = C C = In . Hence, F is also b-orthonormal.

LECTURE V: BILINEAR FORMS AND ORTHOGONALITY

5. Gram-Schmidt orthogonalization process Let

11

be a real vector space and x an inner product

b.

What we are going to show

in this section, is that given any (nite or countable) ordered set of vectors of is always possible to vectors, pairwise

V,

it

b-orthonormalize them,

i.e., substitute them with a new set of

b-orthogonal,

that is equivalent (in a certain sense to be specied)

to the original one. The proof of this result will provide a simple algorithm, known as Gram-Schmidt process. Let us start with a denition.

Denition.

Let V be a real vector space with an inner product b and two sets {u1 , . . . , ut , . . .}, {w1 , . . . , wt , . . .} of vectors of V , that are in 1 1 correspondence. We shall say that {w1 , . . . , wt , . . .} is a b-orthogonalization of {u1 , . . . , ut , . . .}, if for any k 1: (ak ) u1 , . . . , uk = w1 , . . . , wk ; (bk ) the vectors w1 , . . . , wk are pairwise b-orthogonal.

Theorem 1 (Gram-Schmidt).
product on

Let

be a real vector space and

an inner

b{w1 , . . . , wt , . . .}. This is essentially unique, in the sense that if {w1 , . . . , wt , . . .} is another b-orthogonalization of {u1 , . . . , ut , . . .}, then wk = wk for any k 1 (i.e., the two vectors are proportional).
Given any set of vectors of there exists a orthogonalization

V.

V {u1 , . . . , ut , . . .},

Proof.
Let

Let us proceed by induction on

t.

If

t = 1,

then it will be sucient to set

w1 = u1

and it is trivial to verify that the properties (a1 ) and (b1 ) of the above

denition hold.

t 2 and suppose that there exist t vectors w1 , . . . , wt that verify the properties 1 k t; we want to construct a vector wt+1 , such that the vectors w1 , . . . , wt+1 verify (at+1 ) and (bt+1 ).
(ak ) and (bk ) for

Let us set:

t
(1) where

wt+1 = ut+1
i=1

ci wi

ci =
[cwi (ut+1 ) is Fourier's coecient

0 if wi = 0 cwi (ut+1 ) if wi = 0 of ut+1 with respect to wi ].

Obviously:

wt+1 w1 , . . . , wt , ut+1
From (at ) it follows that

and ut+1 w1 , . . . , wt , wt+1 .

u1 , . . . , ut+1 = w1 , . . . , wt , wt+1 ;
therefore (at+1 ) holds. Let us verify (bt+1 ). It will be enough to verify that

wt+1

wj , for any j = 1, . . . , t. wj = 0. We have:

If

wj = 0,
t

the claim is trivial. For, let us assume that

b(wt+1 , wj ) = b(ut+1 , wj )
i=1

ci b(wi , wj ) = b(ut+1 , wj ) cj b(wj , wj ) = 0 . t = 1 the claim wk = wk , for

is obvious. any

We want now to discuss the uniqueness matter . If Assume that Since the two

t2

and suppose to have veried that

k t.

We need to show that

wt+1 = wt+1

b-orthogonalizations

satisfy (at+1 ), we have:

w1 , . . . , wt+1 = u1 , . . . , ut+1 = w1 , . . . , wt+1 .

Therefore,

wt+1 = rwt+1 + w,

r R and w w1 , . . . , wt .

12
From (bt+1 ), it follows:

ALFONSO SORRENTINO

wt+1 w1 , . . . , wt wt+1 w1 , . . . , wt
Hence:

= w1 , . . . , wt

and therefore

b(w, wt+1 ) = 0 ;

and therefore

b(w, wt+1 ) = 0 .

b(w, w) = b(wt+1 rwt+1 , w) = b(wt+1 , w) rb(wt+1 , w) = 0 r0 = 0 .

Consequently, since that implies:

is an inner product,

w=0

and it follows that

wt+1 = rwt+1 ,

wt+1 wt+1 .
The other inclusion can be shown similarly. The proof of the above theorem suggests an algorith to nd the of an order set of vectors of

V.

As already done before, we will set

b-orthogonalization c0 (v ) = 0
of

v V. {u1 , . . . , ut , . . .}
for any

It follows from (1) that the Gram-Schmidt is the following:

b-orthogonalization

w1 w2 w3

= u1 ; = u2 cw1 (u2 )w1 ; = u3 cw1 (u3 )w1 cw2 (u3 )w2 ;

. . .

t1

wt

= ut
i=1
. . .

cwi (ut )wi ;

Remark.
a basis of

Let E = (e1 , . . . , en ) be a basis of V . A b-orthogonalization E of E is still V ; in fact, from the above description of the algorithm, it follows easily that E = EC , where C is an upper triangular matrix such that c11 = . . . = cnn = 1; therefore C GLn (R) and in particular E is b-orthogonal. If we divide each vector by its norm, we get an orthonormal basis (i.e., any vector has norm 1 and they are pairwise orthogonal). [See also The Factorization

A = QR,

in

3.4

in the book]

Let us prove this important corollary of Gram-Schmidt's theorem.

Corollary 1.

Let

be a real vector space and

an inner product on it. If

is a

nite dimensional vector subspace, then

V = W W .

Proof.

orthonormal basis of

dim W = t. Using Gram-Schmidt process, one can determine a bW : {w1 , . . . , wt }. Since b is an inner product, then W W = {0}. For, it is sucient to verify that W + W = V . t For any v V , let us consider the vector v = i=1 b(v, wi )wi . Obviously, v W . Since v = v + (v v ), we only need to verify that v v W . In fact, for any j = 1, . . . , t:
Let

b(v v , wj )

= b(v, wj ) b
i=1

b(v, wi )wi , wj

= b(v, wj )
i=1

b(v, wi )b(wi , wj ) =

= b(v, wj ) b(v, wj ) = 0 .

LECTURE V: BILINEAR FORMS AND ORTHOGONALITY

13

Example.
vectors:

In

R3

with the canonical inner product, consider the following ve

u1 = (1, 0, 1), u2 = (1, 1, 0), u3 = (0, 1, 1), u4 = (0, 1, 0), u5 = (1, 2, 0) .

Let us nd a We have:

b-orthogonalization

of these vectors, using Gram-Schmidt process.

w1 w2 w3 w4

= u1 ; = u2 = u3 = u4 u2 w1 1 w1 = u2 u1 = w1 w1 2 1 1 , 1, 2 2
3 2 3 2

; w2 = 0 ; 1 1 1 , , 3 3 3 .

u3 w1 u3 w2 1 w1 w2 = u3 u1 w1 w1 w2 w2 2

u4 w2 0 1 u4 w1 w1 w2 0w3 = u4 w1 3 w2 = w1 w1 w2 w2 2 2

Observe now, that necessarily early independent vectors in

w5

w5 = 0 (otherwise w1 , w2 , w4 , w5 would be R3 ). Indeed, u5 w1 u5 w2 u5 w4 = u5 w1 w2 0w3 w4 = w1 w1 w2 w2 w4 w4 1 = u5 w1 w2 3w4 = 0 . 2

consider the inner product

four lin-

Example.

In

R3

b,

dened - with respect to a basis

E = (e1 , e2 , e3 )

- by the matrix

2 1 1
Using Gram-Schmidt process, nd a

1 1 2 0 . 0 1
basis.

b-orthonormal

[Before solving this exercise, we should ask ourselves a natural question: how can we check that

is an inner product? This question arises the following problem: is

it possible to characterize the inner products (i.e., positive deniteness) in terms of their matrices? We will discuss this problem immediately after this exercise.] Using Gram-Schmidt, we can

b-orthogonalize E:

f1 f2 f3
observing that

= e1 ; 1 b(f1 , e2 ) f1 = e2 e1 ; b(f1 , f1 ) 2 b(f2 , e3 ) b(f1 , e3 ) f1 f2 ; = e3 b(f1 , f1 ) b(f2 , f2 ) = e2

b(f2 , f2 ) =
we obtain:

1 2 1 3 , 1, 0 A 1 = 2 2 0
1 2 3 2

and b(f2 , e3 ) =

0 1 1 , 1, 0 A 0 = , 2 2 1

f3 = e3

1 e1 2

1 e2 e1 2

1 1 1 2 1 = e3 + e1 e2 + e1 = e1 e2 + e3 . 2 3 6 3 3

14
Hence:

ALFONSO SORRENTINO

F = (f1 f2 f3 ) = EC
Since

1 1 2 0 1 with C = 0 0

2 3 1 3

1 1 , 3
1 6 2 6 2 3 3 33

f1 =
a

2,

f2 =

3 2

and

f3 =
1 2

b-orthonormal F = f1 f1

basis is given by:

f2 f2

f3 f3

= EC

with C = 0 0

Let us consider the problem of characterizing the matrices, corresponding to inner products in

(where

is, as usual, a

introduce the following notation: given the determinant of the submatrix (n

= det A)

are called upper-left

n-dimensional vector space). Let us A Mn (R), dene i (for i = 1, . . . , n) A(1, . . . , i|1, . . . , i). These n scalars 1 , . . . , n minors of A (or determinants of the upper-left

submatrices, see
result:

6.2

in the book). We state - without proving it - the following

Theorem 2 (Jabobi-Sylvester).
dimensional vector space

Let

be a symmetric bilinear form on a

n-

b,

with respect to

V. E. Then, b

Let

be a basis for

and

the matrix that denes

is positive denite (i.e., it is an inner product) if and

only if all upper-left minors of

are positive:

1 , . . . , n > 0.

Remark.

Observe that if some of the

i 's

is equal to zero, then the matrix might

not be even positive semidenite. For instance, consider:

A=
in both cases

0 0

0 1

and B = A

0 0 0 1

; B
is negative semi-

1 = 2 = 0,

but while

is positive semidenite,

denite. In order to nd a necessary and sucient condition for positive semideniteness, one cannot consider only the upper-left minors, but needs to consider ALL

principal minors, i.e., all determinants of submatrices of

the same rows and columns:

A(i1 , . . . , ik |i1 , . . . , ik )

for

1 k n.

The criterion in this case becomes (see also

A, obtained by considering 1 i1 < . . . < ik n and 6.2 in the book):

Theorem 3.

Let b be a symmetric bilinear form on a n-dimensional vector space V . Let E be a basis for V and A the matrix that denes b, with respect to E. Then, b is positive semidenite if and only if all its principal minors of A are non-negative (i.e., 0).

Example.

Verify that the symmetric bilinear form b, given in the previous example,

is indeed an inner product:

1 = 2 > 0,

2 1

1 2

=3>0

and 3 =

2 1 1

1 1 2 0 0 1

= 1 > 0.

Remark. [Exercise] Before closing this section, it is interesting to point out that
Jacobi-Sylvester's theorem implies that, performing Gram-Schmidt process, the orthogonal basis

b-

one gets, is such that:

b(f1 , f1 ) = 1 ,
where

b(fi , fi ) =

i , i1 A

i 's,

for all

i = 1, . . . , n

are the upper-left minors of the matrix we called

in the theorem.

LECTURE V: BILINEAR FORMS AND ORTHOGONALITY

15

[See also QR-Factorization, Least squares method and Fourier Analysis, in the book]

Department of Mathematics, Princeton University

E-mail address :

asorrent@math.princeton.edu