MAS212 Linear Algebra I Lecture Notes

Francis J. Wright, 2004

4. Dimension, Direct Sum, Coordinates and Ordered Bases

Contents
Dimension Dimensions of Sums of Vector Subspaces Direct Sums of Vector Subspaces Coordinates Ordered Bases A basis completely characterizes a vector space in ways that we now explore. In particular, the size of any basis is one of the determining properties of a vector space and is called its dimension. After formally defining dimension, we investigate the dimension of a sum of two vector subspaces and consider an important special case, the direct sum. Finally, we introduce the notion of coordinates with respect to ordered bases that allow any vector space to be treated like Kn.

Dimension
Definition of Finite-Dimensional A vector space is said to be finite-dimensional if it has a basis consisting of a finite number of vectors. Examples For any finite n, the vector space Kn has a basis consisting of the n standard basis vectors e1, e2, ..., en, so Kn is finite-dimensional. Essentially the same argument shows that, for any finite n, rown( K ) and coln( K ) are finite-dimensional. We now prove two lemmas, the first of which will be used to prove the second, which will then be used to prove a theorem that allows the dimension of a finite-dimensional vector space to be defined. Lemma: Linear Dependence of a Sequence of Vectors The non-zero vectors v1, v2, ..., vn are linearly dependent if and only if one of the vectors vi (1 < i n) is a linear combination of the preceding vectors v1, ..., vi 1. Proof This lemma is really just a convenient restatement of the definition of linear dependence. It involves two statements, "if" and "only if", which we prove separately.

("A if B" means that "B implies A") If vi = k1 v1 + k2 v2 + ... + ki 1 vi 1 then k1 v1 + k2 v2 + ... + ki 1 vi 1 + ( 1 ) vi + 0 vi + 1 + ... + 0 vn = 0, i.e. v1, v2, ..., vn are linearly dependent. ("A only if B" means that "A implies B") If v1, v2, ..., vn are linearly dependent then there exist scalars k1, k2, ..., kn K that are not all zero such that k1 v1 + k2 v2 + ... + kn vn = 0 (*). Let i be the largest value of j for which kj 0. If i = 1 then k1 v1 = 0 with v1 0 so that k1 = 0 , which contradicts the assumption that ki 0. Hence, i > 1 and we may write (*) as k k k1 v + 2 v + ... + i 1 v giving v as a linear combination of v , ..., v . vi = 1 2 i 1 i1 ki i 1 ki ki Lemma: Spanning Sets and Linearly Independent Sets If { v1, v2, ..., vn } is a spanning set and { u1, u2, ..., um } is a linearly independent set in a vector space V then m n. Proof Since { v1, v2, ..., vn } is a spanning set for V we could remove any zero vectors and still have a spanning set containing l vectors with l n. If we can prove that m l then we have proved that m n. Therefore, without loss of generality, let us assume that { v1, v2, ..., vn } is a set of non-zero vectors. The vector space u1, v1, v2, ..., vn must contain the vector space v1, v2, ..., vn = V. Now { u1, v1, v2, ..., vn } is a linearly dependent set since { v1, v2, ..., vn } spans V, i.e. u1 = k1 v1 + k2 v2 + ... + kn vn for some non-zero scalars k1, k2, ..., kn. Then, by the previous lemma, there exists a vector vi that is a linear combination of its predecessors in the sequence of vectors u1, v1, v2, ..., vn. Pick the last vector vi that contributes with a non-zero coefficient ki to the linear combination representing u1, so that conversely vi is a linear combination of u1 and the other vectors vj i. Therefore, { u1, v1, v2, ..., v , ..., vn } spans V, where v means that this vector has been removed,
i i

i.e. we have succeeded in replacing vi by u1 in the spanning set for V. By the same argument, we can now replace vj, j i, by u2 to give a set { u2, u1, v1, ..., v , ..., v , ..., vn }
i j

that spans V. (It cannot be u1 that is replaced because u1 and u2 are linearly independent, so u1 cannot be the last vector that contributes with a non-zero coefficient to the linear combination representing u2.) Continuing this replacement process, after n replacements the set { un, un 1, ..., u1 } must span V. If m > n then un + 1 must be a linear combination of un, un 1, ..., u1. But this contradicts the assumption that { u1, u2, ..., um } is a linearly independent set. Therefore m n. Theorem: All Basis Sets have the Same Size

Let V be a finite-dimensional vector space. If {u1, u2, ..., um} and {v1, v2, ..., vn} are both bases for V then m = n. Proof {v1, v2, ..., vn} is a spanning set (since it is a basis) and {u1, u2, ..., um} is a linearly independent set (since it is a basis). Therefore, by the previous lemma, m n. But this argument applies symmetrically: {u1, u2, ..., um} is also a spanning set and {v1, v2, ..., vn} is also a linearly independent set, therefore n m. Hence, m = n. Definition of Dimension The dimension, dim V, of a finite-dimensional vector space V is the number of elements in a basis. By the previous theorem, this number is independent of the choice of basis since it is the same for all bases, so dimension is well defined. Examples 1. Dim Kn = n since the standard basis {e1, e2, ..., en} is a basis that contains n elements. 2. Consider Mm, n( K ), the vector space of mn matrices over a field K. Let Ei, j Mm, n( K ) denote the matrix with (i, j)-element 1 and all other elements 0. Then the set { Ei, j | 1 i m, 1 j n } is a basis for Mm, n( K ), which contains mn elements. Hence dim Mm, n( K ) = m n. 3. Let U = { in( x, y, z, R3 ) | x + y + z = 0 } R3. We know that dim R3 = 3. We claim that dim U = 2 (because each independent constraint lowers the dimension by 1). To prove this, we need to construct a basis, i.e. we need to find two independent vectors in R3 that satisfy the constraint z = ( x + y ) on U. Two simple choices are u1 = ( 0, 1, 1 ) and u2 = ( 1, 0, 1 ). We must check that these two vectors are linearly independent and span U. Linear independence: a1 u1 + a2 u2 = 0 => ( 0, a1, a1 ) + ( a2, 0, a2 ) = ( a2, a1, a1 a2 ) = ( 0, 0, 0 ) => a1 = a2 = 0. Spanning set: We must be able to express any u U in the form u = a1 u1 + a2 u2, i.e. ( x, y, x y ) = ( a2, a1, a1 a2 ), which is obviously satisfied by choosing a1 = y, a2 = x. Thus, we have constructed a basis for U containing two elements, which proves that dim U = 2. Proposition: Every Linearly Independent Set Extends to a Basis Let V be a finite-dimensional vector space and suppose that { u1, u2, ..., um } is a linearly independent set of vectors in V. Then there exist vectors v1, v2, ..., vr V for some r such that { u1, ..., um, v1, ..., vr } is a basis for V. Proof Let dim V = n and assume m < n (otherwise there is nothing to prove). Every linearly independent

set of vectors in V has at most n elements (by the definition of dimension). Choose a linearly independent set of vectors containing u1, u2, ..., um that is as large as possible, say { u1, ..., um, v1, ..., vr } where r = n m. We have to prove that this is a spanning set. Let v V. Then { u1, ..., um, v1, ..., vr, v } is a linearly dependent set, i.e. there is a linear relation of the form a1 u1 + ... + am um + b1 v1 + ... + br vr + c v = 0 in which not all of the coefficients are zero. The coefficient c cannot be zero, because that would imply a linear relation among the linearly independent set of vectors { u1, ..., um, v1, ..., vr }, so we can write v as the linear combination a1 am b b u + 1 v + ... + r v . v= u1 + ... + 1 m c c c c r The vector v was an arbitrary element of V, so { u1, ..., um, v1, ..., vr } is a spanning set for V. It is also a linearly independent set, therefore it is a basis for V. Example In R3 let u1 = ( 0, 1, 1 ), u2 = ( 1, 0, 1 ). Then {u1, u2} is a set of linearly independent vectors. It may be extended to a basis by including another linearly independent vector such as u3 = ( 0, 0, 1 ), i.e. { u1, u2, u3 } is a basis for R3. [Exercise: Prove this.] Proposition: Dimension of a Finite-Dimensional Vector Subspace If V is a finite-dimensional vector space and U is a vector subspace of V then U is finite-dimensional and dim( U ) dim( V ). If U V then dim( U ) < dim( V ). Proof Let dim V = n and let { v1, v2, ..., vn } be a basis for V. If { u1, u2, ..., um } is any linearly independent set of vectors in U V then m n (by the definition of dimension). Choose m as large as possible; we claim that { u1, u2, ..., um } is a basis set for U. Let u U. Then { u1, u2, ..., um, u } is a linearly dependent set of vectors in U since we chose { u1, u2, ..., um } to be as large as possible. Hence u is a linear combination of u1, u2, ..., um for any u U, so { u1, u2, ..., um } is a spanning set for U. Since { u1, u2, ..., um } was chosen to be a linearly independent set it is also a basis for U, which therefore has dimension m. The observation that m n => dim( U ) dim( V ). If U V we can extend { u1, u2, ..., um } to a basis { u1, ..., um, v1, ..., vr } for V where m + r = n for some r > 0 by the previous proposition, hence m < n, i.e. dim( U ) < dim( V ).

Dimensions of Sums of Vector Subspaces

Given two vector subspaces U, V of a finite-dimensional vector space W, we can construct their sum

space U + V = { u + v | u U, v V }. But what is its dimension? Is it simply dim( U ) + dim( V )? Let's consider a simple geometrical example. Let U and V both be two-dimensional vector subspaces of the real Euclidean space R3, i.e. planes through the origin. Then dim(U) = dim(V) = 2. If the two planes U and V are identical then U + V = U = V and dim( U + V ) = 2:

If the two planes U and V are different then U + V = R3 and so dim( U + V ) = 3:

In this example, dim( U + V ) is either 2 or 3 and is never equal to dim( U ) + dim( V ), which is 4. To understand what is going on we must consider the part of R3 that is common to both U and V, i.e. the vector subspace U V. The dimension of U V contributes to the dimension of both U and V and so is counted twice when we add the dimensions of U and V, so to avoid this we must subtract it from the sum. This suggests that dim( U + V ) = dim( U ) + dim( V ) dim( U V ). If the two planes U and V are identical then U V = U = V so dim( U V ) = 2, giving 2 = 2 + 2 2. If the two planes U and V are different then U V is the line along which they intersect so dim( U + V ) = 1, giving 3 = 2 + 2 1. It only remains to prove this result in general. Theorem: Dimension of a Sum Space If U, V are vector subspaces of a finite-dimensional vector space then dim( U + V ) = dim( U ) + dim( V ) dim( U V ). Proof Let { z1, ..., zr } be a basis for U V. Extend this to a basis { z1, ..., zr, u1, ..., um } for U and a basis

{z1, ..., zr, v1, ..., vn} for V. Thus dim( U V ) = r , dim U = r + m, dim V = r + n. We claim that { z1, ..., zr, u1, ..., um, v1, ..., vn } is a basis for U + V, in which case dim( U + V ) = r + m + n = ( r + m ) + ( r + n ) r = dim( U ) + dim( V ) dim( U V ). Spanning set: If w ( U + V ) then w = u + v for some u U, v V. In terms of the bases for U and V, u = a1 z1 + ... + ar zr + b1 u1 + ... + bm um for some scalars ai, bj, and v = c1 z1 + ... + cr zr + d1 v1 + ... + dn vn for some scalars ci, dj. Hence w = u + v = ( a1 + c1 ) z1 + ... + ( ar + cr ) zr + b1 u1 + ... + bm um + d1 v1 + ... + dn vn is a linear combination of elements of the set { z1, ..., zr, u1, ..., um, v1, ..., vn }, which is therefore a spanning set for U + V. Linear independence: Suppose a1 z1 + ... + ar zr + b1 u1 + ... + bm um + c1 v1 + ... + cn vn = 0 (*) for some scalars ai, bj, ck. Consider a vector of the form w = a1 z1 + ... + ar zr + b1 u1 + ... + bm um, which by (*) can also be expressed as w = c1 v1 ... cn vn. From the bases for U and V it is clear that w U and w V, so w ( U V ). But { z1, ..., zr } is a basis for U V, so w = d1 z1 + ... + dr zr for some scalars di. Combining this and the previous representation for w we have w = d1 z1 + ... + dr zr = c1 v1 ... cn vn, which gives d1 z1 + ... + dr zr + c1 v1 + ... + cn vn = 0. But { z1, ..., zr, v1, ..., vn } is a basis (for V) and therefore a linearly independent set, so d1 = ... = dr = c1 = ... = cn = 0. Setting c1 = ... = cn = 0 in (*) gives a1 z1 + ... + ar zr + b1 u1 + ... + bm um = 0. But { z1, ..., zr, u1, ..., um} is a basis (for U) and therefore a linearly independent set, so a1 = ... = ar = b1 = ... = bm = 0. Hence (*) => a1 = ... = ar = b1 = ... = bm = c1 = ... = cn = 0, therefore { z1, ..., zr, u1, ..., um, v1, ..., vn } is a linearly independent set. Example Let W = R4, u1 = ( 1, 1, 0, 0 ), u2 = ( 3, 7, 2, 1 ), U = u1, u2 , V = { (x1, x2, x3, 0) | xi R }. Find dim U, dim V, dim( U + V ) and dim( U V ). U is spanned by u1, u2 by definition, which are linearly independent because a1 u1 + a2 u2 = 0 => (a1, a1, 0, 0) + (3 a2, 7 a2, 2 a2, 1 a2) = 0 => a2 = 0 => a1 = 0. Therefore { u1, u2 } is a basis for U, so dim( U ) = 2. V has a basis { e1, e2, e3 } consisting of the first three standard basis vectors in R4, namely e1 = ( 1, 0, 0, 0 ), e2 = ( 0, 1, 0, 0 ), e3 = ( 0, 0, 1, 0 ), as is easily proved, so dim( V ) = 3. Now consider U + V and note that (0, 0, 0, 1) = (3, 7, 2, 1) + 3 e1 7 e2 2 e3 is a linear

combination of u2, e1, e2, e3 . Therefore, e4 = ( 0, 0, 0, 1 ) ( U + V ) as are e1, e2, e3, so U + V = R4 and dim( U + V ) = 4. Now we can use the theorem on the dimension of a sum space to find dim( U V ) as dim( U V ) = dim( U ) + dim( V ) dim( U + V ) = 2 + 3 4 = 1. Finally, it is instructive to verify this result directly. A vector in U V is a vector in U that satisfies the constraint on V that the last component is zero, namely a vector of the form a1 u1 + a2 u2 = (a1, a1, 0, 0) + (3 a2, 7 a2, 2 a2, 1 a2) such that a2 = 0, i.e. ( a1, a1, 0, 0 ) = a1 u1. Therefore { u1 } is a basis for U V, so dim( U V ) = 1. Remark: Finding a Basis for a Sum Space If U and V are vector subspaces of a finite-dimensional vector space with bases { u1, ..., um } and { v1, ..., vn }, respectively, then a basis for W = U + V can be constructed as follows. Since by definition w W has the form w = u + v, u U, v V, w can be written as w = a1 u1 + ... + am um + b1 v1 + ... + bn vn for some scalars ai, bj. Therefore { u1, ..., um, v1, ..., vn }, which is the union of the basis sets for U and V, is a spanning set for W. Reducing this spanning set to a maximal linearly independent set gives a basis for W. Aside The formula for the dimension of a sum of two vector subspaces is analogous to the following standard formula (which requires only two-dimensional figures to illustrate) for the cardinality (#) of the union of two sets: #( U V ) = #( U ) + #( V ) #( U V ) Here is an explicit example: U = { a, d, b, c, e }, #( U ) = 5 V = { a, f , g, b }, #( V ) = 4 U V = { a, b }, #( U V ) = 2 #( U ) + #( V ) #( U V ) = 7 U V = { a, f , d, g, b, c , e }, #( U V ) = 7

Direct Sums of Vector Subspaces

An important special case of the sum of two vector subspaces is the sum of two subspaces that have a zero-dimensional intersection. The only vector space that has dimension 0 is the vector space { 0 } and, since every field must contain an additive zero element, the vector space { 0 } is essentially independent of the field over which it is defined. (More precisely, all zero-dimensional vector spaces are trivially isomorphic.) Definition of Direct Sum of Vector Subspaces If U and V are vector subspaces of a vector space such that U V = { 0 } then U + V is called the direct sum of U and V, written U V . Example U = { (x1, x2, 0) | x1, x2 R } and V = { (0, 0, x3) | x3 R } are vector subspaces of R3 such that U V = { 0 } and R 3 = U V . Proposition: Dimension of a Direct Sum Space The dimension of a direct sum is the sum of the dimensions, i.e. dim(U V ) = dim U + dim V . Proof Use the fact that U V = { 0 } => dim( U V ) = 0 in the general formula dim( U + V ) = dim( U ) + dim( V ) dim( U V ). Proposition: Uniqueness of a Direct Sum Space Every vector w W = U V can be written as the unique sum w = u + v, u U, v V. Proof Let w = u + v = u' + v', where u, u' U, v, v' V. Then u u' = v' v . But ( u u' ) U and ( v' v ) V. However, since these two vectors are equal they must be elements of the same vector

space, namely U V = { 0 }. Therefore, u u' = v' v = 0, so u' = u, v' = v and the representation of w is unique.

Aside The above uniqueness property can be used to define a direct sum, in which case U V = { 0 } becomes a consequence of the definition. Proof If w ( W = U + V ) then w = u + v for some u U, v V, and more generally, w = ( u + z ) + ( v z ) for any z such that z U and z V, i.e. z ( U V ). If w = u + v is a unique representation then z must be the zero vector, hence U V = { 0 }. Example
Suppose U, V are vector subspaces of R3 and R3 = U + V = { u + v | u U, v V }. If U is a 2-dimensional vector subspace (a plane through the origin) and V is 1-dimensional vector subspace (a straight line through the origin) not contained in U then R3 is the direct sum of U and V and the representation u + v is unique; essentially the 1-dimensional vector subspace is too small to allow any variation. However, if U and V are distinct 2-dimensional vector subspaces (planes through the origin) then

R3 is the sum (but not the direct sum) of U and V and the representation u + v is not unique; the vector u can be rotated within U and compensated by a rotation of v within V so that u + v does not change, which is equivalent to adding a vector z in the 1-dimensional vector space U V to u and subtracting it from v.

Coordinates
Each vector vi in a basis set {vi | i = 1 .. n} for an n-dimensional vector space V over a field K provides a basis set { vi } for a one-dimensional vector subspace vi , and the intersection of any pair of these one-dimensional vector subspaces is { 0 } since basis vectors are linearly independent. Therefore V can be expressed as the direct sum of n one-dimensional vector subspaces as V = < v1 > < v2 > L < vn > . Hence, any vector v V can be written as the unique sum

v = k1 v1 + k2 v2 + ... + kn vn where k1, k2, ..., kn K are scalars that determine the correct vector ki vi vi for each i = 1 .. n. With respect to a given basis {vi | i = 1 .. n}, the scalars k1, k2, ..., kn K provide a unique representation for the vector v V, which motivates the following definition.

Definition of Coordinates Let v1, ..., vn be a basis for a vector space V over a field K so that any v V can be represented as v = x1 v1 + ... + xn vn. The scalars x1, ..., xn K are called the coordinates of v with respect to the basis v1, ..., vn.

By the uniqueness property of direct sums, the coordinates of a vector with respect to a given basis are unique.

Example
Let V = R3 with (standard) basis vectors e1 = ( 1, 0, 0 ), e2 = ( 0, 1, 0 ), e3 = ( 0, 0, 1 ) and let v = ( x, y, z ). Then v = x e1 + y e2 + z e3, so the components x, y, z of v are also the coordinates of v with respect to the standard basis {e1, e2, e3}. However, x, y, z are not the coordinates of v with respect to any other basis. For example, another basis for V is v1 = ( 0, 1, 1 ), v2 = ( 1, 0, 1 ), x + y + z xy+z x+yz v3 = ( 1, 1, 0 ), with respect to which v = v1 + v2 + v3. 2 2 2

Ordered Bases
It is necessary to ensure that each coordinate of a vector is associated with the correct basis vector, and the standard way to do this is to use an ordered representation for both the coordinates and the basis vectors. When the order of the vectors in a basis matters it is called an ordered basis and the basis is written as a sequence or list; when the order does not matter the basis may be written as a set, which is sometimes done to emphasize that the discussion does not depend on the order. In particular, when coordinates are represented as rows or columns (of field elements) then the representation depends on the order of the basis, so it is essential to use an ordered basis. When linear maps are represented as matrices it is crucial to use ordered bases.

10