Mathematical Methods wk 1: Vectors

John Magorrian, magog@thphys.ox.ac.uk

These are work-in-progress notes for the second-year course on mathematical methods. The most up-to-date
version is available from http://www-thphys.physics.ox.ac.uk/people/JohnMagorrian/mm.

1 Linear vector spaces

A linear vector space (or just vector space for short) consists of
a set V of vectors (the elements of which well usually denote by a, b, ..., ~a, ~b, ... or |ai, |bi, ...);
a set F of scalars (scalars denoted by , , a, b,...),
a rule for adding two vectors to produce another vector,
a rule for multiplying vectors by scalars,
that together satisfy 10 conditions. The four most interesting conditions are the following.
(i) The set V of vectors is closed under addition, i.e.,
|ai+ |bi V

for all |ai, |bi V;

(1.1)

(ii) V is also closed under multiplication by scalars, i.e.,

|ai V

for all |ai V and F.

(1.2)

(iii) V contains a special zero vector, 0 V, for which

|ai+ 0 = |ai for all |ai V;

(1.3)

(iv) Every vector has an additive inverse: for all |ai V there is some |a0 i V for which
|ai+ |a0 i = 0.

(1.4)

The other six conditions are more technical. The addition operator must be commutative and associative:
|ai+ |bi = |bi+ |ai,

(1.5)

(|ai+ |bi) + |ci = |ai+ (|bi+ |ci).

(1.6)

The multiplication-by-scalar operation must be distributive with respect to vector and scalar addition, consistent with the operation of multiplying two scalars and must satisfy the multiplicative identity:
(|ai+ |bi) = |ai+ |bi

(1.7)

( + ) |ai = |ai+ |ai

(1.8)

( |ai) = () |ai

(1.9)

1 |ai = |ai.

(1.10)

For our purposes the set F will usually be either the set R of all real numbers (in which case we have a real
vector space) or the set C of all complex numbers (giving a complex vector space).

Mathematical Methods wk 1: Vectors

1.1 Basic ideas

In a raw vector space there is no notion of the length of a vector or the angle between two vectors.
Nevertheless, there are many important ideas that follow by applying the basic rules (1.1 1.10) above to
linear combinations of vectors, i.e., weighted sums such as
1 |v1 i+ 2 |v2 i+ .

(1.11)

A set of vectors {|v1 i, . . . , |vn i} is said to be linearly independent (abbreviated LI) if the only solution to
the equation
1 |v1 i+ 2 |v2 i+ n |vn i = 0
(1.12)
is if all scalar coefficients i = 0. Otherwise the set is linearly dependent. The dimension of a vector space
is the maximum number of LI vectors in the space.
The span of a list of vectors |v1 i, . . . , |vm iis the set of all possible linear combinations {1 |v1 i+ +m |vm i :
1 , . . . , m F}. A list |e1 i, |e2 i, . . . |en i of vectors forms a basis for the space V if the elements of the list
are LI and span V. Then any |ai V can be expressed as
|ai =

n
X

ai |ei i,

(1.13)

i=1

and the coefficients (a1 , . . . , an ) for which (1.13) holds are known as the components or coordinates of
|ai with respect to the basis vectors |ei i.
Claim: Given a basis |e1 i, . . . , |en i the coordinates ai of |ai are unique.
Proof: suppose that there is another set of coordinates a0i . Then we can express |ai in two ways:
|ai = a1 |e1 i+ a2 |e2 i+ + an |en i
= a01 |e1 i+ a02 |e2 i+ + a0n |en i.

(1.14)

Subtracting,
0 = (a1 a01 ) |e1 i+ (a2 a02 ) |e2 i+ + (an a0n ) |en i.

(1.15)

But the |ei i are LI. Therefore the only way of satisfying the equation is if all ai a0i = 0. So a0i = ai :
the coordinates are unique.
A subset W V is a subspace of V if it satisfies conditions (1.11.4). That is: it must be closed under
addition of vectors and multiplication by scalars; it must contain the zero vector; the additive inverse of each
element must be included. Conditions (1.51.10) are automatically satisfied because they depend only on
the definition of the addition and multiplication operations.

1.2 Examples
Example: Three-dimensional column vectors with real coefficients
The set of column vectors
(x1 , x2 , x3 )T with xi R forms a real vector space under the usual rules of vector addition and multiplication
by scalars. This space is usually known as R3 .
To confirm that this really is a vector space, lets check the conditions (1.11.10). The usual rules of vector
algebra satisfy conditions
For the conditions
(1.11.4)
note that:
(1.51.10).

a1
b1
a1
b1
a1 + b1
(i) For any a2 , b2 R3 , the sum a2 + b2 a2 + b2 R3 .
a3
b3
a3
b3
3 + b
3
a

a1
a1
a1
(ii) Multiplying any vector a2 by a real scalar gives a2 a2 R3 .
a3
a3
a3

I2

I3

Mathematical Methods wk 1: Vectors

(iii) There is a zero
(0, 0, 0)T R3 .
element,

a1
a1
(iv) Each vector a2 has an additive inverse a2 R3 .
a3
a3
So, all conditions (1.11.10) are satisfied. Here are two possible bases for this space:

0
0
1
0,1,0

0
0
1

or

0
0
1
, 1 ,2 .

1
1
6

(1.16)

Each of these basis sets has three LI elements that span R3 . Therefore the dimension of R3 is 3.
Exercise: The set of all 3-dimensional column vectors with real coefficients cannot form a complex
vector space. Why not? (Which of the conditions 1.11.10 is broken?)
Example: Rn and Cn
Similarly, the set of all n-dimensional column vectors with real (complex)
elements forms a real (complex) vector space under the usual rules of vector addition and multiplication by
scalars.
Example: Arrows on a plane
The set of all arrows on a plane with the obvious definitions of addition
of arrows and multiplication by scalars forms a real two-dimensional vector space. There is an natural,
invertible mapping between elements elements x R2 and arrows for which 1 x1 +2 x2 maps to 1 |arrow1 i+
2 |arrow2 iwhenever x1 maps to |arrow1 iand x2 maps to |arrow2 i. An invertible mapping between two vector
spaces that preserves the operations of addition and multiplication is known as a isomorphism.
Example: Perverse arrows in the plane
Let us introduce a new vector space of arrows in the plane
that uses the same definition of vector addition as in the previous example, but in which multiplication by a
complex scalar is defined by increasing the length of the vector by a factor || and rotating it anticlockwise
by an angle arg . Any arrow in the plane can be represented as

|r, i =

r cos
r sin


.

(1.17)

Our new multiplication rule is that



|r, i ||r, + arg .

(1.18)

which is another member of V for all choices of , |r, i. The addition operation is unchanged so this new
definition satisfies the first four conditions (1.11.4). Lets check the more technical conditions:


( |r, i) = ||r, + arg


= ||||r, + arg + arg


= ||||r, + arg + arg


= ||r, + arg

(1.19)

= ( |r, i).
Therefore our new multiplication rule satisfies condition (1.5).
Exercise: Show that all of the conditions (1.51.10) are satisfied. What is the dimension of this space?
Find another vector space to which it is isomorphic.
Example: The set of all m n matrices with complex coefficients
forms a complex vector space
with dimension mn. The most natural basis is

0 1
0 0
0 0
1 0

0 0 ,0 0 ,...,1 0 ,0 1 ,... .
(1.20)
.. ..
.. ..
.. ..
.. ..

. . ...
. . ...
. . ...
. . ...

Mathematical Methods wk 1: Vectors

Example: nth -order polynomials
The set of all nth -order polynomials in a complex variable z forms
an n + 1 dimensional complex vector space. A natural basis is the set of monomials {1, z, z 2 , . . . , z n }.
Example: Trigonometric polynomials
Given n distinct (mod 2) complex constants 1 , . . . , n ,
the set of all linear combinations of ein z forms an n-dimensional complex vector space.
Example: Functions

The set L2 (a, b) of all complex-valued functions

f : [a, b] C

(1.21)

for which the integral

Z

dx|f (x)|2

(1.22)

exists forms a complex vector space under the usual operations of addition of functions and multiplication of
functions by scalars. This space has an infinite number of dimensions. We postpone the issue of identifying
a suitable basis until 4 later.

1.3 Linear maps

A mapping A : V W from one vector space V to another W is a linear map if it satisfies


A |v1 i+ |v2 i = A |v1 i+ A |v2 i,


A |v1 i = A |v1 i,

(1.23)

for all |v1 i, |v2 i V and scalars F. A linear operator is the special case of a linear map from a vector
space V to itself.
Now let n be the dimension of V and m the dimension of W. Choose P
any basis |e1 i, . . . , |en ifor V and another,
n
|e01 i, . . . , |e0m i, for W. Any vector |vi V can be expressed as |vi = i=1 aj |ej i. Using the properties (1.23)
we have that the image of |vi under the linear map A is
A |vi =

n
X

aj A |ej i .

(1.24)

j=1

As this holds for any |vi V, we see that the map A is completely determined by the images A |e1 i, ..., A |en i
of Vs basis vectors. Each of these images A |ei i is a vector that lives in W and so can be expressed in terms
of the basis |e01 i, . . . , |e0m i as
m
X
A |ej i =
Aij |e0i i,
(1.25)
i=1

where Aij is the ith component in the |e01 i, . . . , |e0m i basis of the vector A |ej i. Substituting this into (1.24),

m
n
X
X

A |vi =
Aij aj |e0i i.
i=1

(1.26)

j=1

th
That is, a vector
Pn in V with components a1 , . . . , an maps under A to another vector in W whose i component
is given by j=1 Aij aj . The values of the coefficients Aij depend on the choice of basis for V and W.

I4

I5

Mathematical Methods wk 1: Vectors

1.4 Representation of vectors and linear maps by matrices

Given a basis |e1 i, . . . , |en i for an n-dimensional vector space let us represent

0
1

0
|e2 i =
. ,
..

1
0

0
|e1 i =
. ,
..

...,

0
0

0
|en i =
. .
..
1

(1.27)

Then any vector |vi can be expressed as

a1
a2

|vi = a1 |e1 i+ a2 |e2 i+ + an |en i =

... .

(1.28)

an
Given two vector spaces, V with dimension n and W with dimension m, then any linear map A from V to
W can be represented as the matrix
A11
A21
A=
...

A12
A22
..
.

...
...
..
.

A1n
A2n
,
..
.

Am1

Am2

Amn

with

A11
A21
A |vi =
...

A12
A22
..
.

...
...
..
.

Am1

Am2

A1n
a1
A11 a1 + A12 a2 + + A1n an
A2n a2 A21 a1 + A22 a2 + + A2n an
. =

..
..

. ..
.
Amn
an
Am1 a1 + Am2 a2 + + Amn an

(1.29)

(1.30)

being given by the familiar rules of matrix multiplication.

Linear operators (that is, linear maps of a vector space to itself) are represented by square n n matrices.

Further reading
Linear vector spaces are introduced in RHB 8.1 and linear maps in RHB 8.2. DK II is another good
starting point. Most maths-for-physicists books introduce inner products (see 2 below) at the same time as
vector spaces. Nevertheless, pausing to work out the consequences of the unadorned conditions (1.11.10)
is an supremely useful introduction to mathematical reasoning: many of the statements that we take as
self-evident from our experience in manipulating vectors and matrices are not easy to prove without some
practice. For more on this see, e.g., Linear Algebra by Lang or similar books for mathematicians.

Mathematical Methods wk 1: Vectors

2 Inner-product spaces
The conditions (1.11.10) do not allow us to say whether two vectors are orthogonal, or even what the length
of a vector is. To do these, we need to introduce some additional structure on the space, namely the idea of
an inner product. This is a straightforward generalization of the familiar scalar product. In the following I
use bra-ket notation ha|bi for the inner product of the vectors |ai and |bi.
An inner product is a mapping V V F that takes two vectors and returns a scalar and satisfies
the following conditions for all |ai, |bi, |ci V and F:
hc|di = hc|ai+hc|bi if |di = |ai+ |bi;

(2.1)

hc|di = hc|ai if |di = |ai;

(2.2)

ha|bi = hb|ai ;
ha|ai = 0,
> 0,

(2.3)

only if |ai = 0,
otherwise.

(2.4)

Notice that the inner product is linear in the second argument, but not necessarily in the first.
An inner-product space is simply a vector space V on which an inner product ha|bi has been defined.

Some definitions:
The inner product p
of a vector with itself, ha|ai, is real and non-negative. The length or norm of the
vector |ai is kak ha|ai.
The vectors |ai and |bi are orthogonal if ha|bi = 0.
A set of vectors {|vi i} of V is orthonormal if hvi |vj i = ij .
The condition (2.3) is essential if we want lengths of vectors to be real numbers, but a consequence is that
in general the inner product is not linear in both arguments.
Exercise: Use the properties (2.12.4) above to show that
hd|ci = ? ha|ci+ ? hb|ci

(2.5)

for |di = |ai+ |bi. Some books use the term sesquilinear to describe this property. Under what
conditions is the scalar product linear in both arguments?
Exercise: Show that if ha|vi = 0 for all |vi V then |ai = 0.
Exercise: Show that any n orthonormal vectors in an n-dimensional inner-product space form a basis.
The converse is not true: see 2.4 below.

2.1 Orthonormal bases

Let V be an n-dimensional inner-product space in which the vectors {|ei i} form an orthonormal basis so
that hei |ej i = ij and let
n
n
X
X
|ai =
ai |ei i, and |bi =
bi |ei i
(2.6)
i=1

i=1

be any two vectors in V. Using properties (2.1) and (2.2) of the inner product together with the orthonormality of the basis vectors, we have that the projection of |ai onto the j th basis vector
hej |ai =

n
X
i=1

ai hej |ei i =

n
X
i=1

ai ji = aj

(2.7)

I6

I7

Mathematical Methods wk 1: Vectors

and similarly hej |bi = bj . Therefore the inner product of |ai and |bi is
ha|bi =

n
X

bi ha|ei i =

i=1

n
X

bi hei |ai? =

i=1

n
X

a?i bi .

(2.8)

i=1

This can be written in matrix form as

b1
.
a?n ) .. = a b,
bn

ha|bi = ( a?1

(2.9)

where a is the Hermitian conjugate of the column vector a.

2.2 Duals: bras and kets

There is another way of looking at the inner product that serves as a useful reminder of its sesquilinearity
and helps motivate the unfamiliar ha|bi notation.
Consider the set V ? of all linear maps from ket vectors |vi V to scalars F. Applying any L V ? to an
element |vi = 1 |e1 i+ + n |en i of V, we have, by the linearity of L, that
!
n
n
X
X
i L |ei i.
(2.10)
i |ei i =
L |vi = L
i=1

i=1
?

So, given a basis {|e1 i, . . . , |en i} for V, any L V is completely defined by the n scalar values L |e1 i,. . . ,L |en i.
We can turn V ? into a vector space. Given L1 , L2 V ? , define their sum (L1 + L2 ) as the new mapping
(L1 + L2 ) |vi L1 |vi+ L2 |vi

(2.11)

and the result of multiplying L V ? by a scalar as the mapping (L) defined through
(L) |vi ? L |vi.

(2.12)

It is easy to confirm that the set V with these operations satisfies the conditions (1.11.10).
Inhabitants of the vector space V ? are called bras and are more conventionally written ha|, hb| etc instead
of the L1 , L2 notation used above. V ? has the same dimension as V and is known as the dual space (or
adjoint space) of V. For every ket there is a corresponding dual (or adjoint) bra and vice versa. The
addition and mutliplication rules (2.11) and (2.12) above mean that if kets |ai, |bi have dual bras ha|, hb|
respectively, then
the ket
has dual

|vi = |ai+ |bi

(2.13)

hv| = ? ha| + ? hb| .

Given basis kets |e1 i, . . . , |en i V we may introduce corresponding basis bras he1 | , . . . ,hen | V ? with each
hei | defined through


hei | |ej i = ij .
(2.14)
P
P
Pn
Then given |ai = i=1 ai |ei i and |bi = i=1 bi |ei i, the dual to |ai is ha| = i=1 a?i hei | and we may define

! n
n
X
X
ha|bi (ha|) (|bi) =
a?i hei |
bj |ej i
i=1

n X
n
X
i=1 j=1

j=1

a?i bj hei | |ej i =

n X
n
X
i=1 j=1

a?i bj ij

n
X
i=1

(2.15)
a?i bi ,

Mathematical Methods wk 1: Vectors

in agreement with equation (2.9) from the previous section. It is easy to confirm that this alternative
definition of ha|bi as the result of operating on the ket vector |bi by the bra vector ha| V ? satisfies the
conditions (2.12.4) for an inner product.

2.3 Representation of bras and kets

Here is a brief summary of the results in the last two sections. If we have an orthonormal basis in which we
represent kets by column vectors (1.4),
|vi = 1 |e1 i+ 2 |e2 i+ + n |en i

1
1
0
0
0
1
0 2

= 1
... + 2 ... + + n ... = ... ,
0

(2.16)

then the bra dual to |vi is represented by the Hermitian conjugate of this column vector:
hv| =1? he1 | + 2? he2 | + + n? hen |
=1? ( 1
= ( 1?

0 ...
2?

...

0 ) + 2? ( 0
n?

1 ...

0 ) + + n? ( 0

0 ...

1)

(2.17)

).

The inner productha|biof the vectors |ai = (a1 , . . . , an )T and |bi = (b1 , . . . , bn )T is obtained by premultiplying
|bi by the dual vector to |ai under the usual rules of matrix multiplication:

b1
n
b2 X

=
a?i bi .
(2.18)
ha|bi ha| |bi = ( a?1 a?2 . . . a?n )
.
..
i=1

bn

2.4 GramSchmidt procedure for constructing an orthonormal basis

In an n-dimensional inner-product space any list of n LI vectors |v1 i, . . . |vn i is a basis, but in general this
basis is not orthonormal. There is a simple procedure for constructing an orthonormal basis from the list.
(1) Start with the first vector from the list, |v1 i. The first basis vector |e1 i is defined via
|e01 i = |v1 i
|e1 i = |e01 i/ k|e01 ik .

(2.19)

(2) Take the next vector |v2 i. Subtract any component that is parallel to the previously constructed basis
vector |e1 i. Normalise the result to get |e2 i.
|e02 i = |v2 ihe1 |v2 i|e1 i
|e2 i = |e02 i/ k|e02 ik .

(2.20)

(i) Similarly, work along the remaining |vi i, i = 3, . . . , n, subtracting from each one any component that
is parallel to any of the previously constructed basis vectors |e1 i, . . . , |ei1 i. That is,
|e0i i = |vi i

i1
X

hej |vi i|ej i

j=1

|ei i = |e0i i/ k|e0i ik .

(2.21)

I8

I9

Mathematical Methods wk 1: Vectors

It is easy to see that applying any hek | with k < i to both sides of (2.21) yields hek |ei i = 0: by
construction each new |ei i is orthogonal to all the preceding ones.
The same procedure can be used to construct an orthonormal basis for the space spanned by a list of vectors
|v1 i, . . . , |vm i of any length, including cases where the list is not LI: if |vi i is linearly dependent on the
preceding |v1 i, . . . , |vi1 i then |e0i i = 0 and so that particular |vi i does not produce a new basis vector.
Example: Example
Consider the list |v1 i = (0, i, i, 0)T , |v2 i = (0, 2, 2, 1)T , |v3 i = (1, 1, 1, 1)T and
|v4 i = (2, 1, 1, 0)T . From |v1 i we immediately have that
1
|e1 i = (0, i, i, 0)T .
2

(2.22)

The corresponding basis bra is the row vector

1
he1 | = (0, i, i, 0).
2

(2.23)

The inner product he1 |v2 i = 2 2i, so

|e02 i = |v2 i (2 2i) |e1 i = (0, 0, 0, 1)T = |e2 i.

For |v3 i the necessary inner products are he1 |v3 i = 2i and he2 |v3 i = 1. Then

|e03 i = |v3 i ( 2i) |e1 i |e2 i

(2.24)

(2.25)
= (1, 0, 0, 0)T = |e3 i.

Finally, notice that |e04 i = 0 because |v4 i = 2 |e3 i 2i |e1 i. Therefore the four vectors |v1 i, . . . , |v4 i span a
three-dimensional subspace of the original four-dimensional space. The kets |e1 i, |e2 i and |e3 i constructed
above are one possible orthonormal basis for this subspace.

2.5 Some important relations

2

Recall that kak ha|ai.

Pythagoras

if ha|bi = 0 then
2

k|ai+ |bik = k|aik + k|bik .

(2.26)

Parallelogram law
2

k|ai+ |bik + k|ai |bik = 2 k|aik + k|bik .

(2.27)

k|ai+ |bik k|aik + k|bik .

(2.28)

|ha|bi|2 ha|aihb|bi.

(2.29)

Triange inequality

CauchySchwarz inequality
Proof of (2.29): Let |di = |ai + c |bi, where c is a scalar whose value we choose later. Then hd| =
ha| + c? hb|. By the properties of the inner product,
0 hd|di = ha|ai+ c? hb|ai+ cha|bi+ |c|2 hb|bi.

(2.30)

Now choose c = hb|ai/hb|bi. Then c? = ha|bi/hb|bi and (2.30) becomes

2

0 ha|ai kha|bik /hb|bi,

which on rearrangement gives the required result.

(2.31)

Mathematical Methods wk 1: Vectors

Further reading
Much of the material in this section is covered in 8 of RHB and II of DK. For another introduction to
the concept of dual vectors see 1.3 of Shankars Principles of Quantum Mechanics. (The first chapter of
Shankar gives a succinct summary of the first half of this course.)
Beware in that most books written for mathematicians the inner product ha, bi is defined to be linear in the
first argument.

I10