Orthonormal Bases for the Four Subspaces using the Singular Value Decomposition

S. K. Hyde November 19, 2007

Contents
1 Introduction 2 Singular Value Decomposition 3 Basis Sets 3.1 Column Space . 3.2 Row Space . . 3.3 Null Space . . . 3.4 Left Null Space 4 Summary 1 1 2 2 2 2 3 3

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

Introduction

The Singular Value Decomposition can be used to get orthonormal bases for each of the four subspaces: the column space C(A), the nullspace N(A), the row space C(A ), and the left nullspace N(tpA).

Singular Value Decomposition

The Singular Value Decomposition of a matrix A is A = U V , where U is a m m orthogonal matrix, is a m n matrix containing the singular values of the matrix A on the main diagonal, and V is a n n orthogonal matrix. The matrix A can also be split into the blocks as follows: A = U V = U1 mr U2 mmr r 0 V1 V2 (1)

r r mr r

r nr mr nr

0 0

r n

nr n

= U1 r
mr mnr

V1 V2

r n

nr n

= U1 r V 1

(2)

mr r r r n

The matrix in (2) is called the Full Rank Singular Value Decomposition. Further, notice that because U1 and V1 contain orthonormal vectors, where r n, it follows that then U1 U1 = Ir and V1 V1 = Ir . 1

Orthonormal Bases for the Four Subspaces using the Singular Value Decomposition, page 2

3
3.1

Basis Sets
Column Space

Theorem 1. Suppose A is any m n matrix, and A = U1 r V1 is the full rank Singular Value Decomposition. It follows that an orthonormal set of basis vectors for C(A), the column space, are the columns of U1 . Proof. Suppose b C(A). It follows that b = Az = U1 r V1 z = U1 z Therefore, b C(U1 ). Since r is invertible and V1 V1 = Ir , then it follows that
1 U1 r V1 = A = U1 r = AV1 = U1 = AV1 r

Suppose b C(U1 ). It follows that

1 b = U1 z = AV1 r z = Az

Therefore, b C(A). It follows that C(A) = C(U1 ). Therefore, the columns of U1 span C(A).

3.2

Row Space

Theorem 2. Suppose A is any m n matrix, and A = U1 r V1 is the full rank Singular Value Decomposition. It follows that an orthonormal set of basis vectors for C(A ), the row space, are the columns of V1 . Proof. Suppose b C(A ). It follows that b = A z = V1 r U1 z = V1 z Therefore, b C(V1 ). Since r is invertible and U1 U1 = Ir , then it follows that
1 V1 r U1 = A = V1 r = A U1 = V1 = A U1 r

Suppose b C(V1 ). It follows that

1 b = V1 z = A U1 r z =A z

Therefore, b C(A ). It follows that C(A ) = C(V1 ). Therefore, the columns of V1 span C(A).

3.3

Null Space

Theorem 3. Suppose A is any m n matrix, and A = U V is the Singular Value Decomposition. Partition U into U1 U2 , where U1 is the rst r columns of U , U2 is the last m r columns of U , and r is the rank of A. Similarly, partition V as V1 V2 . It follows that an orthonormal set of basis vectors for N(A), the null space, are the columns of V2 . Proof. Solve equation (1) for results with =
r r

r 0

r nr mr nr

mr r

r m

U1 U2

A V1
nr

V2 nnr

U1 AV1 U2 AV1

U1 AV2 U2 AV2

(3)

mr m

Orthonormal Bases for the Four Subspaces using the Singular Value Decomposition, page 3

It follows that U1 AV2 U2 AV2 = U1 0 = 0 U2 AV2 = 0 = U AV2 = 0 = AV2 = 0

It follows that V2 N(A). Since the columns of V2 are orthonormal, then they are linearly independent of each other. Since dim N(A) = n r and there are n r columns in V2 , then it follows that the columns of V2 form a basis for N(A).

3.4

Left Null Space

Theorem 4. Suppose A is any m n matrix, and A = U V is the Singular Value Decomposition. Partition U into U1 U2 , where U1 is the rst r columns of U , U2 is the last m r columns of U , and r is the rank of A. Similarly, partition V as V1 V2 . It follows that an orthonormal set of basis vectors for N(A), the null space, are the columns of V2 . Proof. It follows from (3) that U2 AV1 U2 AV2 = 0 0 = U2 A V1 = U2 AV = 0 = U2 A = 0 = A U2 = 0 It follows that U2 N(A ). Since the columns of U2 are orthonormal, then they are linearly independent of each other. Since dim N(A ) = m r and there are m r columns in U2 , then it follows that the columns of U2 form a basis for N(A ). V2 =0

Summary
V1 V2

Dene the Singular Value Decomposition as A = U V It follows that U1 forms a basis for C(A) U2 forms a basis for N(A ) V1 forms a basis for C(A ) V2 forms a basis for N(A) = U1 U2