Matrices
An Introduction
David A. Evans
What is a matrix?
A matrix is a rectangular array of numbers.
All positions in the array contain a number,
there are no gaps. Each individual number is
called an element of the matrix. The dimension
of a matrix is specified by the number of rows
and the number of columns, conventionally, the
number of rows is stated first. Thus the
following matrices are respectively 2×2 and 3×4:
3 0 2 1 
 2.3 3.1   
A=  ;B =  4 2 5 1 
 0. 7 1. 2 
 4 2 1 0
A matrix is a complete entity and is
referred to by a single symbol just as you have
been used to x, y, a, etc. standing for some
number. Printed convention is to use bold
letters to stand for matrices. In handwritten
work the underline is sometimes used thus, A.
We often need to refer to a particular element
(e.g., 2nd row, 3rd column of B) or the general
element (ith row, jth column of A). These have
the symbols b23 and aij, (it is conventional to use
upper-case letters for the matrix and the
corresponding lower-case letter for the
elements). These symbols stand for ordinary
numbers. We can write the equation: b23 = 5.
Note the order of the sub-scripts is row-column.
Addition of matrices
There would be little point in introducing
these entities without defining how they
combine and interact. While it appears trivial,
we must first agree on the meaning of the
statement that two matrices are equal. If A and
B are two matrices, then the statement, A = B
means that the matrices have the same
dimensions and that the corresponding elements
of each are equal. That is, aij = bij, for all possible
values of i and j.
Two matrices can be added to give a third
matrix whose elements are the sums of the
elements in the corresponding row/column
positions of the matrices being added. Thus if
1 0 1   2 1 4
A=  ; B = 0 1 3 
 2 3 1   
Matrices
The matrix A + B is given by
1 + 2 0 + 1 1 + 4 3 1 5 
A +B =  = 
2 + 0 3 + 1 1 + 3   2 4 4 
If we call this matrix C, we can write the
matrix statement or equation
C=A+B
where now the '+' and '=' signs stand for matrix
addition and matrix equality. The subtraction of
two matrices follows in an obvious way from
the definition of addition.
Although any two numbers can always be
added together, the same is not true for
matrices. A moment's thought will show you
that they have to have the same dimensions in
order for addition to work. Thus, if A is a 2×2
matrix and B is 3×2, the symbol A + B is
meaningless. The third row of B does not have
any corresponding elements in A to be added.
Such matrices are not conformable for addition.
Scalar multiple of a matrix
Adding a matrix to itself, A + A, will
produce a matrix whose elements are twice the
corresponding elements in A. It is not
unreasonable to call this matrix 2A. In the same
way, A + A + A will give a matrix with elements
three times those of A and should be written 3A.
By a simple extension of this idea, the symbol kA
stands for a matrix whose elements are each k
times the corresponding ones of A, where k is
any number. We see that this now defines the
multiplication of a matrix by a number. This is
sometimes called scalar multiplication since
ordinary numbers are referred to as scalars to
distinguish them from matrices and other
complicated entities (such as vectors).
Matrix multiplication
The definition of matrix multiplication is
usually considered by students to be the
trickiest and least intuitive of matrix properties.
First of all, the product of two matrices AB is
not, NOT, NOT! given by multiplying the
corresponding elements of A and B!
- 1 of 9 - January, 98
 Let us consider the following simultaneous
equations:
3 x + 2y =6 
 You can't do it because there are 3 elements in
x + 5 y + 2 z = 13 
2x + 2 z = 4 
The solution depends on three items: the array
of coefficients of x and y, the right-hand sides of
the equations, and the symbols x, y that identify
the unknowns. Regroup this information thus:
 3 2 0  x   6 
 1 5 2  y  13 
   =  
 2 0 2  z   4 
This can be expressed in terms of symbols
representing matrices thus
Ax = b
if we agree to interpret the product Ax such that
the element in the first row and first column of
the product is obtained from the elements in the
first row of A and the elements in the first
column of x in the following way: start at the
left of the row and top of the column and take
the product of the elements. Move one element
left in the row and one element down in the
column, multiply these two elements. Continue
moving left in the row and down in the column,
forming products of the elements. Add up all
the products, this gives the value of the element
in the product. The element in the second row,
first column of the product is obtained from the
elements in the second row of A and the first
column of x. As an example the following
product is worked out for you:
the same dimension. Notice also that you can't
always multiply matrices. Write down a couple
of 2×3 matrices and try to find their product.
each row of the first and only 2 elements in each
column of the second. You can't match up the
appropriate elements to find the products
required. In order for the product of two matrices to
exist, the first factor must have the same number of
columns as the second factor has rows. Matrices
which can be multiplied are called conformable
for multiplication. A convenient way of checking
for conformity is to write the dimensions
underneath the matrices, thus:
AB=C
mxn nxp mxp
Note that the inner values must be the same and
that the dimensions of the product are given by
the outer values.
Notice another odd thing, if the two
matrices above are multiplied in the opposite
order, a different matrix results. It doesn't even
have the same dimensions. As a further
example, if A is a 2×3 matrix and B is a 3×4
matrix, you can form the product AB, but in the
order BA, the matrices are not conformable — B
has 4 columns and A has 2 rows.
Thus the order of the factors is crucially
important in matrix multiplication. Unlike the
products of ordinary numbers (scalars), matrix
multiplication does not commute.

1 0 Continued products of more than two

3 − 1 2   
1 0 4  − 1 4 
factors can be done, e.g. ABC, providing the
   2 − 2 numbers of rows and columns match up:
 
 3 × 1 + (− 1 ) × (− 1 ) + 2 × 2 3 × 0 + (− 1 ) × 4 + 2 × (− 2 )
=
 1 × 1 + 0 × (− 1 ) + 4 × 2 1 × 0 + 0 × 1 + 4 × (− 2 ) 
 8 − 8
=
9 − 8
 ABC
mxn nxp pxr
Notice that a 2×3 matrix multiplied by a 3×2
matrix gives a 2×2 matrix. This is an important In this case the final product is an m×r matrix.
thing about matrix multiplication, it can connect
matrices with different dimensions. Addition and
scalar multiplication only work with matrices of

Matrices - 2 of 9 - January, 98
Transpose of a matrix
A matrix can be transposed to create
another matrix. Given a matrix A, its transpose
is written AT (or sometimes A′′ ). The first row of
this matrix contains, in order, the elements in
the first column of A, and likewise for all the
other rows. Thus the number of rows(columns)
in AT is equal to the number of columns(rows)
in A. An example follows:
 1 0 
1 − 2 5  T  
A=  ;A =  − 2 − 1 
 0 − 1 − 3   5 − 3 
Notice that you can always form the products
ATA and AAT for any matrix. The transposing
guarantees that the numbers of rows and
columns match up.
Suppose we have two matrices, A and B
which can be multiplied together to give a
matrix, C = AB. For the sake of example let us
assume that A is 2×3 and B is 3×4, then C is 2×4.
Consider now products of the transposes, AT and
BT. The product ATBT, does not exist since AT
is 3×2 and BT is 4×3. However, the product
BTAT does exist; it is a 4×2 matrix which is in
fact equal to CT. Try this calculation for a
couple of matrices.
We have the general result:
(AB)T = BTAT.
Square matrices
We often encounter matrices with equal
numbers of rows and columns (2×2, 3×3, etc.),
these are quite reasonably called square
matrices. Notice that all square matrices of a
given dimension are conformable for
multiplication, and the products are square
matrices with that same dimension.
It is still true that multiplication does not
commute, AB ≠ BA, in general, even though
both products have the same dimension.
The set of elements on the diagonal of a
square matrix leading from top left to bottom
right is referred to as the leading diagonal.
Matrices
Identity or Unit matrices
Within the realm of square matrices, there
are some special ones which have ones (1) on the
leading diagonal and zeros (0) everywhere else.
The product of one of these matrices with any
conformable matrix, A, (not necessarily square)
gives the same matrix, A. For example,
1 0 0  2 − 3  2 − 3
0 1 0   0
  − 1  =  0 − 1 
0 0 1  − 4 7  − 4 7 
These matrices play a role very similar to unity
(1) in ordinary arithmetic. They are called
identity or unit matrices and are written, In,
where n denotes the dimension, the identity
matrix in the example above is I3. Where there is
little chance of confusion, the subscript is often
omitted. Take the non-identity matrix in the
above example and multiply it by I2, with the
identity matrix as the right-hand or second
factor. You still get the same result. If A is a
square matrix, then IA = AI, i.e. an identity
matrix commutes with any conformable square
matrix.
Matrix inverse
There is no such thing as matrix division.
The symbol A/B cannot be given a meaning.
The reason for this is easily demonstrated. Were
division defined, the matrix equation C = A/B,
would imply that C is a matrix that satisfies the
relation BC = A. This is how division is defined
for ordinary numbers. With matrices there is a
problem. Let
 4 − 7  1 − 2
A=  ;B =  .
− 8 14  − 2 4 
It is easy to verify that
 1 - 2   4 − 3  4 − 7 
- 2 4   0 2  = − 8 14 
    
thus it would appear that
4 − 3 
C= .
0 2 
But it is equally easy to verify that
- 3 of 9 - January, 98
  1 − 2  2 − 1   4 − 7   2 - 4
− 2 4  − 1 3  = − 8 14  - 3 6
     

which would imply instead that does not have an inverse (again, I state this
without proof!). There is no matrix by which
 2 − 1
C= . you can multiply it to give the identity matrix.
− 1 3  Matrices which have no inverse are said to be
Thus a unique value cannot be assigned to the singular.
operation "A/B" for these matrices. In fact, in The inverse of a matrix is analogous to the
this case there are an infinite number of matrices, reciprocal in ordinary numbers. If x is a number,
C that will satisfy the relation BC = A. then its reciprocal x-1 is the number such that
Sometimes, however, C is unique. For the x-1x = 1. One number, 0, has no reciprocal; there
matrices is no number by which you can multiply zero to
obtain 1. Singular matrices are the matrix
 6 − 8 3 − 2 
A=  ;B =  analogues of zero.
 4 − 2 1 0 
I have not indicated how, given the
there is indeed only one matrix: numerical values of the elements of a matrix, to
calculate the elements of the inverse. The
4 − 2
C= discussion of the various techniques for doing
3 1 
this form an important topic in numerical
which satisfies BC = A. methods. The calculations become very complex
for even a 3×3 matrix. One of the powerful
We pose the question: given a square features of using matrix notation is that
matrix A, is there a square matrix, B such that tremendously complicated arithmetic is
the product AB is an identity matrix, i.e. AB = I? encapsulated in the little superscript, -1! For
B must, of course, have the same dimension as example, we have seen that a set of
A. simultaneous equations can be written in matrix
If notation as Ax = b. The equations can be solved
symbolically, thus:
7 5 
A=  Ax = b
4 3 
multiply each side by A − 1 ,
we find that: A − 1 Ax = A − 1b,
 7 5   3 − 5   1 0 but A − 1 A = I, therefore
 4 3  − 4 =
  7  0 1  Ix = A − 1b,
and I state (without proof!) that this is a unique x = A − 1b.
result, like the second example above; no other This shows how the inverse of a matrix is
matrix exists whose product with A is the (2×2) related to the solution of a set of simultaneous
identity matrix. This matrix is called the inverse equations. Note the simplicity of the relation
of A and is written A-1. The result above is expressed in this form even though the
written in matrix notation as AA-1 = I. As an calculation of the inverse and a matrix product
exercise, form the product A-1A, you will find involve many arithmetical operations on the
that this is also the identity matrix. A matrix elements of the matrices.
commutes with its inverse. It is not difficult to
show that the inverse of the product AB, i.e.
Symmetric Matrices
(AB)-1, is equal to B-1A-1.
Consider the matrix
But not every square matrix has an inverse.
For example, the matrix
Matrices - 4 of 9 - January, 98
 − 1 4 0 5 − 7
 4 −3 2 −1 0 
 
 0 2 1 4 − 2
 
 5 −1 4 3 1 
− 7 0 − 2 1 − 5
Notice the symmetrical arrangement of the
numbers. Draw an imaginary line down the
leading diagonal (from upper left to lower
right). This divides the matrix into two
triangles. The numbers in the lower triangle are
the "reflection" in this line of those in the upper
triangle. Another way of putting this is that the
elements in the first (or ith) row are the same
and in the same order as those in the first (or ith)
column. A more succinct way of describing the
arrangement is to note that the transpose of the
matrix is identical to the original matrix. That
is, for the matrix above we can state
AT=A
A matrix with this property is said to be
symmetric. Notice that only square matrices can
be symmetric. Symmetric matrices often appear
in practical applications, for example, in
multivariate statistics the correlation and
covariance matrices are symmetric.
It is not difficult to see that the sum (or
difference) of two symmetric matrices is also
symmetric. The inverse of a symmetric matrix
(if it exists) is symmetric. However, the product
of two symmetric matrices is not necessarily
symmetric. (Can you deduce what property
symmetric matrices A and B must have if their
product AB is also symmetric?)
It was previously stated that for any
matrix, A, we can always make the products
ATA and AAT. Both these products are square
(though not necessarily of the same size) and it
is easy to show that they are symmetric.
A special set of symmetric matrices are
those whose off-diagonal elements are all zero,
e.g.
2 0 0
0 − 1 0
 
0 0 4 
Matrices
These are called diagonal matrices. Note that the
unit matrix is a diagonal matrix in which the
diagonal elements are all unity (1). It is very easy
to find the inverse of a diagonal matrix; it is also
a diagonal matrix whose non-zero elements are
the reciprocals of the corresponding elements.
Thus the inverse of the above matrix is
 12 0 0
 
 0 - 1 0
0 0 1
 4
Check this by multiplying them together.
Another kind of symmetry can occur in
matrices, consider the following matrix,
 0 1 - 3
- 1 0 2

 3 - 2 0
The elements in the lower triangle are the
negative of the corresponding elements in the
upper triangle. The ith column is the negative
of the ith row. The transpose of the matrix gives
the negative of the original matrix, i.e.
AT = –A
Such matrices are called skew-symmetric or
antisymmetric. Notice that the leading diagonal
elements of a skew-symmetric matrix must be
zero.
Determinants
We have seen that a matrix may be
singular, i.e. it has no inverse. Can we detect
this without trying unsuccessfully to invert the
matrix?
A set of linear simultaneous equations
expressed in matrix form as Ax = b, has the
solution expressed as x = A-1b. If A is singular,
A-1 does not exist and so there are no solutions.
Thus the singularity of A is related to the
existence of solutions of a set of simultaneous
equations.
The following equations:
x - 2y = 4

2x - 4 y = 6
- 5 of 9 - January, 98
do not have a solution, i.e. no pair of values of x
and y can satisfy them. The reason is easy to see,
the left hand side of the second equation is just
twice the left hand side of the first. Suppose we
choose any pair of values of x and y that satisfy
the first equation. These values can never
satisfy the second since on substituting the
values of x and y, the left hand side will always
result in the value 8, never 6. The equations are
said to be inconsistent. Conversely, the
equations:
or
ad − bc = 0 .
Here we have a test for the singularity of a 2×2
matrix. The expression ad – bc is calculated, i.e.
(top left)(bottom right) – (top right)(bottom left). If
this is zero the matrix is singular and has no
inverse, otherwise it is non-singular and has an
inverse. The number we have calculated is
called the determinant of the matrix. We use the
symbol

x − 2y = 4 a b
 c d
2 x − 4 y = 8

have an infinite number of solutions. The second to stand for the determinant of the matrix. Note
equation is automatically satisfied for any that the determinant is a number which is a
values of x and y that satisfy the first. We function of the elements of the matrix. We have,
basically have just one equation in two for the example equations above
unknowns. There is no additional information 1 -2
provided by the second equation. The problem, = 1 × (− 4 ) − (− 2 ) × 2 = −4 − (− 4 ) = 0
2 -4
of course, is that the equations are just multiples
of each other. Thus, as promised, the matrix is singular. In the
section dealing with inverse matrices, it was
Since there are no solutions, the matrix of
stated that the matrix
coefficients, A:
 2 − 4
 1 − 2 − 3
2 − 4   6
 
was singular. Checking the determinant we find
is singular. We see that the second row is just
twice the first row. Note a similar dependence 2 −4
= 2 × 6 − (− 4 ) × (− 3 ) = 12 − 12 = 0
between the columns. The second column is the −3 6
negative of twice the first column. It is clear that
showing that the matrix is indeed singular.
the problems will always arise whenever one
row is any multiple of the other. Let us consider This can be extended to square matrices
a general 2×2 matrix with larger dimensions. Any square matrix has
a determinant, a number which is a function of
a b 
 c d the elements. If the determinant is zero, the
  matrix is singular and vice versa. The expression
for the determinant becomes quite complicated
If the matrix is singular, one row is just a
and, at this stage, need not concern us greatly.
multiple of the other:
The determinant of a 3×3 matrix is given by
a = kc;b = kd
a b c
e f d f d e
or d e f =a −b +c
h j g j g h
a b g h j
= =k
c d Each of the 2×2 determinants is then expanded
or using the formula given above. In this way we
can obtain explicitly the value of the
ad = bc

Matrices - 6 of 9 - January, 98
determinant in terms of the elements. This is left
as an exercise. Notice that each of the 2×2
determinants is obtained from the original
matrix by removing the first row and then
removing each column in turn. A similar
relationship can be written to express a 4×4
determinant in terms of four 3×3 determinants,
and this can be continued for the determinants
of larger matrices.
Homogeneous equations
Consider the set of equations:
x + 3 y = 0
 It can be shown that when a determinant is
2 x − y = 0
Both right hand sides are zero; such equations
are called homogeneous. The matrix of
coefficients is nonsingular (check that the
determinant is not zero), the equations have a
unique solution x = 0, y = 0, not a very
interesting state of affairs. In fact, this is called a
trivial solution.
Consider now the following:
x − 2y = 0 

2x − 4y = 0 
where the left hand sides are the same as the
examples given in the last section. There are
now an infinite number of solutions in addition
to the trivial one. Any pair of values where x is
twice y satisfies the set of equations. The set has
the solutions in addition to the trivial one
because the determinant of the matrix of
coefficients is zero. This idea extends to
equations in three, four, etc. unknowns.
We have the result that a set of
homogeneous equations has only the trivial
solution unless the determinant of the matrix of
the coefficients is zero. When the determinant is
zero, there are an infinite number of solutions
where definite relationships exist between the
values of the unknowns.
Consider the following system of
equations:
Matrices
2 − 3 1  x
  
 1 2 − 3  y  = 0
 3 − 1 − 2   z 
The system is homogeneous. It is
straightforward (but labor-intensive!) to check
that:
2 −3 1
1 2 −3 = 0
3 −1 −2
Therefore there are solutions in addition to the
trivial one, x = y = z = 0.
zero, at least one row (or one column) can be
expressed as a linear combination of the
remaining rows (or columns). That is, in the
determinant above, one has that:
row3 = a row1 + b row2
is true for some value of a and b. In our case it is
easy to see that a = 1 and b = 1 so that:
row3 =row1 + row2.
Another way of putting it is that the third
equation is merely obtained by adding the first
two. Therefore it must be satisfied by any
values of x, y, and z that satisfy the first two
equations. It does not bring any new
information to the system; it is redundant.
It is not the third equation that is at fault.
We might equally well write:
row1 =row3 – row2
so that the first row is a linear combination of
the second and the third. The important thing is
that there is only information from two
equations present. We know that two equations
in three unknowns have an infinite number of
solutions.
We solve two of the equations using the
method of transformation to echelon form. We
shall arbitrarily choose the second and third.
- 7 of 9 - January, 98
  1 2 − 3 same. For another set of equations we might
 3 − 1 − 2  , add (-3) × row1 to row2 find that the solutions are all multiples of
 
1 2 − 3 − 1   3   4.7 
→ , divide row2 by - 7      
0 − 7 7   2  or  - 6  or - 9.4  or K
1 2 − 3  1  - 3  - 4.7 
→ 
0 1 − 1 
What is important is the ratios between the
The final matrix corresponds to the equation elements. It is often useful to choose a particular
system: matrix as the basic solution. To do this we must
specify an additional criterion. For example, let
x + 2 y − 3 z = 0
 the first element be “1”. Although this may seem
y − z = 0
reasonable, how would we deal with the
This is re-arranged thus: following solution set

y=z   0 
 − 3  ?
x = −2 y + 3 z   
 2 
Let z = r, where r is an arbitrary number. These
equations then give: No amount of scaling will produce a “1” in the
y=z=r top row. It is preferable to use a criterion which
x = −2 y + 3 z = −2r + 3 r = r does not single out one row over all the others.
For reasons that we shall see later, we define a
So the final solution can be expressed:
“normalized” matrix such that the sum of the
x = r  x  r  1  squares of its elements add to unity. If a matrix

y = r , or  y  = r  = r 1  is denoted by x, then the sum of the squares of
z = r   z  r  1  its elements is given by xT x. For example

− 1  − 1 
Thus all the solutions are multiples of the   T  
column matrix: x =  2  ;x x = [− 1 2 1 ] 2 
 1   1 
1 
1  . = (− 1 ) + 2 + 1 2 = 6
2 2

 
1  Thus if we divide each element of x by 6 we
will get a normalized matrix,
I leave it as an exercise to show that the same
result is obtained with any other choice of two − 1 
 2 6
of the three equations.  6 .
 1 
 6 
Normalization
This works even if some of the elements are
In the above example, we could equally say zero:
that all the solutions are multiples of the matrix,
 0   0 
2  − 74 π − 3  normalizes to - 3 
2  or − 74 or π or K    13 
       2   2 
2  − 74 π  13 

In this particular case it is easy to see that all

that is necessary is that the elements are all the

Matrices - 8 of 9 - January, 98
Rank of a matrix
In the example just given it was shown that
the information content of the matrix really
amounted to two equations rather than three. If
a matrix has a non-zero determinant, none of the
rows or columns can be expressed as a linear
combination of the others. They all bring
independent information. They are called
linearly independent. Such a matrix is said to be
of full rank. In this case the full rank is 3, the
dimension of the square matrix.
If the determinant is zero, the matrix is said
to be not of full rank. The matrix just used
above corresponds to two independent
equations and has a rank of 2.
Consider the following matrix:
− 2 3 5 
 4 − 6 − 10 .
 
− 6 9 15 

The determinant is zero. In this case it is easy to

see that the second and third rows are
respectively, (–2) and 3 times the first row. (Or
equally correctly, the first and third rows are
respectively (–0.5) and (–1.5) times the second
row.) Thus there is only one independent row.
The rank of this matrix is therefore 1.
A non-singular matrix has a non-zero
determinant and has full rank equal to the
number of rows (or columns); its rows (and
columns) are linearly independent. We see that
a singular matrix has a zero determinant and
also has a rank which is less than the dimension
of the matrix. The rank is the number of rows
(or columns) which are linearly independent.
Calculating the rank of a matrix is not a trivial
exercise. We shall see later on that it is closely
related to the eigenvalues of the matrix.

Matrices - 9 of 9 - January, 98