Vector spaces

Numbers (Reals)
Real numbers, ℝ, are the set of numbers that we
express in decimal notation, possibly with infinite,
non-repeating, precision.
 Numbers (Reals)
Example: =3.141592653589793238462643383279502884197…

Completeness: If a sequence of real numbers gets

progressively “tighter” then it must converge to a
real number.

Size: The size of a real number aℝ is the square

root of its square norm:
a  a2
 Numbers (Complexes)
Complex numbers, ℂ, are the set of numbers that we
express as a+ib, where a,bℝ and i=  1.

Example: ei=cos+isin
 Numbers (Complexes)
Let p(x)=xn+an-1xn-1+…+a1x1+a0 be a polynomial with
aiℂ.

Algebraic Closure:
p(x) must have a root, x0 in ℂ:
p(x0)=0.
 Numbers (Complexes)
Conjugate: The conjugate of a complex number a+ib
is:
a  ib  a  ib

Size: The size of a real number a+ibℂ is the square

root of its square norm:
a  ib  (a  ib )(a  ib )  a 2  b 2
 Groups
A group G is a set with a composition rule + that
takes two elements of the set and returns another
element, satisfying:
– Asscociativity: (a+b)+c=a+(b+c) for all a,b,cG.
– Identity: There exists an identity element 0G such
that 0+a=a+0=a for all aG.
– Inverse: For every aG there exists an element -aG
such that a+(-a)=0.
If the group satisfies a+b=b+a for all a,bG, then
the group is called commutative.
 Groups
Examples:
– The integers, under addition, are a commutative group.
– The positive real numbers, under multiplication, are a
commutative group.
– The set of complex numbers without 0, under
multiplication, are a commutative group.
– Real/complex invertible matrices, under multiplication
are a non-commutative group.
– The rotation matrices, under multiplication, are a non-
commutative group. (Except in 2D when they are
commutative)
 (Real) Vector Spaces
A real vector space is a set of objects/vectors that can be added together and scaled
by real numbers.
Formally:
A real vector space V is a commutative group with a scaling operator:
(a,v)→av,
aℝ, u,v,wV, such that:
– 1v=v for all vV.
– a(v+w)=av+aw for all aℝ, v,wV.
– (a+b)v=av+bv for all a,bℝ, vV.
– (ab)v=a(bv) for all a,bℝ, vV.
– v+w=w+v for all v,wV.
– u+(v+w)=(u+v)+w for all, u, v,wV.
 (Real) Vector Spaces
Examples:
• The set of n-dimensional arrays with real coefficients is a
vector space.
• The set of mxn matrices with real entries is a vector space.
• The sets of real-valued functions defined in 1D, 2D, 3D,…
are all vector spaces.
• The sets of real-valued functions defined on the circle,
disk, sphere, ball,… are all vector spaces.
• Etc.
 (Complex) Vector Spaces
A complex vector space is a set of objects that can be added
together and scaled by complex numbers.
Formally:
A complex vector space V is a commutative group with a scaling operator:
(a,v)→av,
aℂ, vV, such that:
1. 1v=v for all vV.
2. a(v+w)=av+aw for all aℂ, v,wV.
3. (a+b)v=av+bv for all a,bℂ, vV.
4. (ab)v=a(bv) for all a,bℂ, vV.
 (Complex) Vector Spaces
Examples:
• The set of n-dimensional arrays with complex coefficients
is a vector space.
• The set of mxn matrices with complex entries is a vector
space.
• The sets of complex-valued functions defined in 1D, 2D,
3D,… are all vector spaces.
• The sets of complex-valued functions defined on the
circle, disk, sphere, ball,… are all vector spaces.
• Etc.
 (Real) Inner Product Spaces
A real inner product space is a real vector space V
with a mapping V,V→ℝ that takes a pair of vectors
and returns a real number, satisfying:
 u,v+w= u,v+ u,w for all u,v,wV.
 αu,v=αu,v for all u,vV and all αℝ.
 u,v= v,u for all u,vV.
 v,v0 for all vV, and v,v=0 if and only if v=0.
 (Real) Inner Product Spaces
Examples:
– The space of n-dimensional arrays with real
coefficients is an inner product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
v,w=v1w1+…+vnwn
– If M is a symmetric matrix (M=Mt) whose eigen-
values are all positive, then the space of n-
dimensional arrays with real coefficients is an inner
product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
v,wM=vMwt
 (Real) Inner Product Spaces
Examples:
– The space of mxn matrices with real coefficients is an
inner product space.
 (Real) Inner Product Spaces
Examples:
– The spaces of real-valued functions defined in 1D,
2D, 3D,… are real inner product space.
If f and g are two functions in 1D, then:

f , g   f (x )g (x )dx


– The spaces of real-valued functions defined on the

circle, disk, sphere, ball,… are real inner product
spaces.
If f and g are two functions defined on the circle, then:
2
f , g   f ( )g ( )d
0
 (Complex) Inner Product Spaces
A complex inner product space is a complex vector
space V with a mapping V,V→ℂ that takes a pair of
vectors and returns a complex number, satisfying:
 u,v+w= u,v+ u,w for all u,v,wV.
 αu,v=αu,v for all u,vV and all αℝ.
– u, v  v, u for all u,vV.
 v,v0 for all vV, and v,v=0 if and only if v=0.
 (Complex) Inner Product Spaces
Examples:
– The space of n-dimensional arrays with complex
coefficients is an inner product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
v, w  v1 w1  ...  vn wn
– If M is a conjugate symmetric matrix ( M  M t ) whose
eigen-values are all positive, then the space of n-
dimensional arrays with complex coefficients is an
inner product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
v,wM=vMwt
 (Complex) Inner Product Spaces
Examples:
– The space of mxn matrices with real coefficients is an
inner product space.
 (Complex) Inner Product Spaces
Examples:
– The spaces of complex-valued functions defined in
1D, 2D, 3D,… are real inner product space.
If f and g are two functions in 1D, then:

f , g   f (x )g (x ) dx


– The spaces of real-valued functions defined on the

circle, disk, sphere, ball,… are real inner product
spaces.
If f and g are two functions defined on the circle, then:
2
f , g   f ( )g ( ) d
0
 Inner Product Spaces
If V1,V2V, then V is the direct sum of subspaces V1,
V2, written V=V1V2, if:
– Every vector vV can be written uniquely as:
v  v1  v2
for some vectors v1V1 and v2V2.
 Inner Product Spaces
Example:
If V is the vector space of 4-dimensional arrays, then
V is the direct sum of the vector spaces V1,V2V
where:
– V1=(x1,x2,0,0)
– V2=(0,0,x3,x4)
 Subspace
A subspace S of a vector space V is
nonempty set of vectors in Such that:

1. If a, b S, then a+bS.

2. If aS, c ℝ, a.c S.
A subspace is a subset of a vector space which
is a vector space itself, e.g. the plane z=0 is a
subspace of R3 (It is essentially R2.).
 Linear system & subspaces
 1 0  b1 
  u    • Linear systems define certain subspaces
 2 3     b2  • Ax = b is solvable iff b may be written
 1 3  v   b  as a linear combination of the columns
   3 of A
• The set of possible vectors b forms a
1  0  b1  subspace called the column space of A
     
u  2   v 3    b2 
1  3  b  (1,2,1)
     3
(0,3,3)
 Linear system & subspaces
The set of solutions to Ax = 0 forms a subspace
called the null space of A.

1 0 0
  u   
 2 3     0   Null space: {(0,0)}
 1 3  v   0 
   
 1 0 1  x   0 
    
 2 3 5  y    0   Null space: {(c,c,-c)}
 1 3 4  z   0 
    
 Orthonormal Basis
• Basis: a space is totally defined by a set of
vectors – any point is a linear combination
of the basis
• Ortho-Normal: orthogonal + normal
• Orthogonal: dot product is zero
• Normal: magnitude is one
• Example: X, Y, Z (but don’t have to be!)
6.837 Linear Algebra Review
 Orthonormal Basis
x  1 0 0
T
x y  0
y   0 1 0
T
xz  0
z   0 0 1 yz  0
T

X, Y, Z is an orthonormal basis. We can describe any 3D

point as a linear combination of these vectors.

How do we express any point as a combination of a new basis

U, V, N, given X, Y, Z?

6.837 Linear Algebra Review

 Orthonormal Basis
a 0 0  u1 v1 n1  a  u  b  u  c  u 
 0 b 0 u v2 n2    a  v  b  v  c  v 
  2
 0 0 c  u3 v3 n3  a  n  b  n  c  n 
(not an actual formula – just a way of thinking about it)

To change a point from one coordinate system to

another, compute the dot product of each coordinate row
with each of the basis vectors.

6.837 Linear Algebra Review