You are on page 1of 41

1.

Systems of Linear Equations and Matrices


1.1 Introduction
One of the most important and frequently occurring problems in science, engineering and the social sciences is the solution of systems of linear equations. It is therefore important that we have available ecient and accurate methods for solving these problems. The simplest and best known procedure for solving systems of linear equations consists of a method known as Gaussian elimination together with back-substitution. We shall describe this procedure with the aid of matrices but rst we look at three types of problems which can be formulated as systems of linear equations. These three problems, which come from quite diverse areas, are presented purely as an illustration of the relevance of systems of linear equations to real life problems. Depending on your background and other coursework your lecturer may refer you to only one or two of these problems. Example 1.1 Flows in Networks Networks consist of branches and nodes and are used as models in diverse elds such as water ow in pipes, trac ow and electrical networks. Consider the diagram below which we will use to represent a simple trac network. The branches represent streets and the nodes represent intersections. The arrows indicate the direction of the trac ow so that the network consists only of one-way streets. We make the assumption that, for each node, the total ow into the node equals the total ow out of the node. The aim is to determine the trac ows in the streets so as to achieve the desired ows into and out of the network as indicated by the numbers in the diagram. This type of analysis can be used by trac engineers in the planning stage to determine whether or not existing streets can cope with the desired trac ow.

200

400

300

300

800

200

200

The rst stage in solving the problem is to assign variables to the unknown trac ows. This is shown in the next diagram. 1

200

400

x1

x2 x5

300

x7
300 D

x3 x4
F 800

x6
E

200

200

From the requirement that the total ow into each node equals the total ow out of the node we obtain the following system of linear equations. 400 = x1 + x7 x1 + x5 = 200 + x2 x2 = x3 + 300 x7 + x6 = 300 + 200 x4 = x5 + x6 800 + x3 = x4 + 200 (Node (Node (Node (Node (Node (Node A) B) C) D) E) F)

(Notice that we have 7 unknowns and only 6 equations.) Example 1.2 Electrical Networks Another problem which reduces to the solution of a system of linear equations is the application of Kirchhos laws to the analysis of an electrical network. These two laws are: 1. Kirchho s Current Law: At any point in the network, the sum of all the inowing currents equals the sum of all the outowing currents. (Compare this with the trac ow problem.) 2. Kirchho s Voltage Law: In any closed loop, the sum of all voltage drops (calculated as V=IR) equals the impressed electromotive force. Consider the electrical network shown in the next diagram. The unknown quantities which we want to calculate are the currents in each segment of the network. If we dene one current for each segment then we have the six currents x1 to x6 as shown in the diagram. The direction for each current can be chosen arbitrarily. If a current comes out to be negative it simply means that we have to reverse the direction of the arrows. 2

40 ohms A

x1
10 ohms

x3

x6

x4
30 ohms 10 ohms 20 ohms 140 volts 20 ohms D

80 volts

x2 x5
250 volts C

We rst apply Kirchhos current law to the four nodes A, B, C and D to obtain the four equations: x2 = x1 + x3 x3 + x6 = x4 x4 = x5 + x6 x1 + x5 = x2 Node Node Node Node A B C D

We now apply Kirchhos voltage law to the left loop, centre loop and right loop to obtain the three equations: 10x1 + 10x2 = 80 10x2 + 40x3 + 20x4 + 20x5 = 250 20x4 + 30x6 = 140 Left Loop Centre Loop Right Loop

This now gives a total of 7 equations in the 6 unknowns x1 , x2 , x3 , x4 , x5 , x6 . Notice there are other closed loops in the network to which we could have applied Kirchhos voltage law (in fact there is a total of six closed loops). These other loops would, however, have just produced redundant equations. For example, when we apply Kirchhos voltage law to the closed loop around the left two-thirds of the network we obtain a single equation which is equivalent to what is obtained by subtracting the equation for the centre loop from the equation for the left loop. Example 1.3 Leontief Input-Output Models The American economist Leontief developed models for analysing the inputs and outputs of an economic system, work for which he later received a Nobel prize. These models, of which we shall consider two main types, are called input-output models. The rst type of model is a closed economic model and is one in which each item produced by one segment of the economy is consumed by other segments of the same economy. Suppose in the model there are n industries which we shall number 1, 2, . . . , n. Over some xed period of time, each industry produces an output of some good or 3

service that is completely utilised in a predetermined manner by the n industries. The input-output matrix is dened to be the n n matrix A, where a11 a12 a1n a21 a22 a2n A= . . . ... . . . . . . an1 an2 ann in which the entry aij (the entry in the ith row and jth column) is the fraction of the total production of industry j required to supply the needs of industry i. Since all of any industrys production must be totally consumed it follows that the sum of the entries in any column of A must be 1. Now denote the value of the total output of the ith industry by xi . We then have the equation xi = ai1 x1 + ai2 x2 + + ain xn since to satisfy the demand for its products the ith industry must produce ai1 x1 units of output for the rst industry, ai2 x2 units of output for the second industry and so on. A similar equation must hold for each value of i, as i goes from 1 to n, and so we obtain the system of linear equations a11 x1 a21 x1 . . . an1 x1 a12 x2 a22 x2 . . . + an2 x2 + + a1n xn a2n xn . . . + + ann xn + + + + x1 x2 . . . = xn = =

which we want to solve for the unknowns x1 , x2 , . . . , xn . In terms of matrices we will see later that this system can be written in the form Ax = x where x1 x2 x = . . . . xn

The other type of model is an open economic model which is similar in structure to the closed model except that we can now have an external demand for each industrys products as well as demands from the other n industries. Again the entry aij of the input-output matrix species the fraction of the total production of industry j required to supply the needs of industry i. The value of the total output of the ith industry is again denoted by xi . If we let di be the external demand for the ith product then we obtain the equation xi = di + ai1 x1 + ai2 x2 + + ain xn . This leads to the system of linear equations d1 d2 . . . dn + + a11 x1 a21 x1 . . . + an1 x1 + + a12 x2 a22 x2 + + a1n xn + + a2n xn . . . . . . + + ann xn 4 = = x1 x2

+ an2 x2

= xn

which we want to solve for the unknowns x1 , x2 , . . . , xn . In terms of matrices this system can be written in the form Ax + d = x where x1 x2 x= . . . xn d1 d2 d = . . . . dn

and

1.2 Gaussian Elimination


Consider the general system of two linear equations in the two unknowns x and y: a1 x + b1 y = d1 a2 x + b2 y = d2 . The graphs of these equations are straight lines - call them 1 and 2 , and the solution of the pair of equations corresponds to points of intersection of 1 and 2 . The diagrams below show the three dierent possibilities that can occur.
y

l1

l2

l1

l 1 and l2

l2

(a) parallel lines no solution

(b) lines intersect one solution

(c) lines coincide innitely many solutions

This illustrates all the possibilities that can occur in solving systems of linear equations. We shall see later that every system of linear equations has either no solution, exactly one solution or innitely many solutions. We say that a system of equations that has no solutions is inconsistent while if there is at least one solution the system is consistent. Now consider a general system of m linear equations in n unknowns which we write in the form a11 x1 + a12 x2 + + a1n xn = b1 a21 x1 + a22 x2 + + a2n xn = b2 . . . . . . . . . . . . am1 x1 + am2 x2 + + amn xn = bm

where the a and b coecients are known constants and the xi are the unknowns. For the constant aij the rst subscript indicates the equation it comes from (the ith equation) and the second subscript indicates the unknown it multiplies (xj ). A convenient 5

notation for representing this system of linear equations can be obtained by using rectangular arrays of numbers called matrices. In the above system the matrix a11 a12 a1n a21 a22 a2n A= . . . ... . . . . . . am1 am2 amn which is made up of the coecients aij and the matrix a11 a21 [A | b] = . . . am1

made up of all the a and b coecients is called the augmented matrix for the system. (The vertical line in the matrix which separates the a coecients from the b coecients is just to emphasise the fact that this is an augmented matrix and represents a system of linear equations.) Example 1.4 The augmented matrix for the system of equations x1 2x1 x1 3x1 is + 4x2 + 3x2 + x2 + x3 x3 3x4 + 4x4 x4 + x4 0 1 . 2 5 = 0 = 1 = 2 = 5

is called the coecient matrix of the system a12 a1n b1 a22 a2n b2 . . . ... . . . . . . am2 amn bm

4x3 1 2 1 3

4 1 3 3 1 4 1 0 1 0 4 1

The method we shall adopt in solving systems of linear equations is to replace the given system by a new system that has the same solutions but which is easier to solve. This is achieved by using the following three types of operations: 1. Interchange two equations. 2. Multiply an equation by a nonzero constant. 3. Add a constant multiple of one equation to another. Note that all the above operations result in a system of equations that has the same solution set as the original system. Since rows of the augmented matrix correspond to equations in the system we perform a corresponding set of operations on rows of the augmented matrix. If we let Ri denote the ith row of the augmented matrix then the three operations are 1. Interchange two rows: Ri Rj . 2. Multiply a row by a nonzero constant: Ri cRi . 3. Add a constant multiple of one row to another Ri Ri + cRj .

We call these operations elementary row operations and again we emphasize that these operations preserve the solution set of the original system of equations. 6

The following example shows the correspondence between operations on the equations and operations on the rows of the corresponding augmented matrix. It also illustrates how to nd elementary row operations so as to eliminate variables from the equations and produce a matrix which has only zeros below the main diagonal. Example 1.5 x + y 2x + 4y 3x + 6y + 2z 3z 5z = 9 = 1 = 0 1 1 2 2 4 3 3 6 5 9 1. 0

add 2 rst equation to the second equation: x + y + 2z = 9 1 1 2 0 2 7 2y 7z = 17 3x + 6y 5z = 0 3 6 5

add 3 rst equation to the third equation: x + y + 2z = 9 1 1 2 0 2 7 2y 7z = 17 3y 11z = 27 0 3 11


3 add 2 second equation to the third equation:

9 17 R2 R2 2R1 0 9 17 27 R3 R3 3R1 9 17 3 3 2 R3 R3 2 R2

x +

y 2y

2z 7z 1 2z

= 9 = 17 3 = 2

1 1 2 0 2 7 0 0 1 2 9 17 3 2

The matrix

in the previous example is an example of a matrix that is in echelon form. An echelon matrix is a matrix with the following properties: 1. All rows consisting entirely of zeros occur at the bottom of the matrix. 2. In any two successive rows, the rst nonzero entry in the lower row occurs to the right of the rst nonzero entry in the higher row. The following are examples of matrices in echelon form. 1 0 0 0 1 0, 0 0 1 2 3 4 0 1 6 0 0 0 0 0 0 5 7 5 0 6 8 , 9 2 0 1 1 0 5 0 0 2 1 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 0 .

1 1 2 0 2 7 0 0 1 2

If a matrix B has been obtained from a matrix A by a sequence of elementary row operations then we say that the matrices A and B are row equivalent and we often 7

write A B to indicate that A and B are row equivalent matrices. From the previous example we see that the matrix (the augmented matrix for the linear system) 1 1 2 9 2 4 3 1 3 6 5 0 is row equivalent to the echelon matrix 1 1 2 0 2 7 1 0 0 2 that is, 9 17 , 3 2

This example leads us to ask the question can every matrix be reduced to an echelon matrix by elementary row operations? The answer to this question is given in the following theorem. Theorem 1.1 Every matrix is row equivalent to an echelon matrix. We shall now describe a systematic procedure for solving systems of linear equations. The procedure consists of two parts. I Systematically reduce the augmented matrix for the system to echelon form using elementary row operations. This is called Gaussian elimination. II Solve the corresponding system of linear equations by a process known as backsubstitution. We will use the following system of linear equations to illustrate the procedure. 2x1 2x1 + 4x2 + 4x2 2x3 10x3 5x3 + 6x4 + 6x4 + 7x5 + 12x5 5x5 12 28 1 = 12 = 28 = 1

1 1 2 9 1 1 2 2 4 3 1 0 2 7 1 3 6 5 0 0 0 2

9 17 . 3 2

The augmented matrix for this system is 0 0 2 0 7 2 4 10 6 12 2 4 5 6 5

The rst stage is to reduce the augmented matrix to echelon form.

Gaussian elimination Step 1. Locate the leftmost column which does not consist entirely of zeros. In our example this is the rst column. Step 2. If the top entry in this column is zero, interchange rows to bring a nonzero entry to the top of the column. In the example we interchange the rst two rows. 2 4 10 6 12 28 R1 R2 0 0 2 0 7 12 2 4 5 6 5 1 8

Step 4. Now ignore the rst row and column and repeat the procedure on the remaining rows. Continue in this way until the entire matrix is in echelon form. 2 4 10 6 12 28 0 0 2 0 7 12 5 1 R 3 R 3 + 2 R2 0 0 0 0 1 2

Step 3. Add appropriate multiples of the rst row to each of the succeeding rows so that the rst nonzero column has a nonzero entry only in the rst row. In the example, we already have a zero in the second row so we add 1 times the rst row to the third row to produce a zero in row three. 2 4 10 6 12 28 0 0 2 0 7 12 0 0 5 0 17 29 R3 R3 R1

The matrix is now in echelon form so the Gaussian elimination part of the procedure is complete. We now complete the solution of the system of linear equations using back-substitution. Back-substitution Step 1. Write down the system of linear equations corresponding to the echelon form of the augmented matrix. For the example we obtain 2x1 + 4x2 10x3 2x3 + 6x4 + 12x5 + 7x5 1 2 x5 = 28 = 12 = 1

Step 2. Solve for the rst variable in each equation in terms of the other variables. x1 x3 x5 = 14 = 6 + = 2 2x2 7 2 x5 + 5x3 3x4 6x5

Step 3. Beginning with the last equation successively substitute each equation into all the above equations. For the example, we rst substitute x5 = 2 into the 2nd equation to get x3 = 1. Then substitute x5 = 2 and x3 = 1 into the rst equation to get x1 = 7 2x2 3x4 . Thus the solution is x1 = 7 2x2 3x4 , x3 = 1, x5 = 2. Notice in the above example there are two independent variables, x2 and x4 , in the solution. These independent variables can take on any values and are usually referred to as parameters. In this case we sometimes say that the solution contains two parameters. Clearly, any system whose solution contains independent variables has innitely many solutions. A system whose solution has no independent variables has exactly one solution, found by the method of back-substitution. We shall see later what happens if the system is inconsistent. 9

Example 1.6 Use Gaussian elimination to show that the augmented matrix for the linear system of equations 2x1 2x1 2x1 3x2 3x2 + 3x2 + x3 + 5x3 4x3 + 3x3 3x4 x4 + 3x4 + 3x4 + 2x5 + x5 + 2x5 9x5 6 2 . 1 0 x4 = 1 3x5 . = 6 = 8 = 3 = 6

Then use back-substitution to obtain the solution x1 = 51 9 3 + x2 x5 , 2 2 8

is row equivalent to the echelon matrix 2 3 1 3 2 0 0 4 2 1 0 0 0 1 3 0 0 0 0 0 7 x3 = x5 , 4

Exercise Set 1.2 1. Write down the augmented matrix for each of the following systems of linear equations. (a) 2x1 + 3x2 x3 = 5 x1 + 4x2 2x3 = 1 x1 + 3x3 = 1 (b) 2x1 + 3x2 = 5 x1 + 7x2 = 1 x2 = 4 augmented ma1 3 1 0 0 3 5 2 1 0

2. Write down a system of linear equations for each of the following trices. 2 3 3 0 4 3 0 5 5 (a) 2 10 4 3 3 (b) 1 1 1 2 0 0 4 1 1 0 0

3. For which values of c does the following system of linear equations have no solutions, exactly one solution, innitely many solutions? 2x y = 3 4x 2y = c 4. What condition on a, b, c and d will guarantee that there will be exactly one solution to the system ax + by = 1 cx + dy = 0? (Try reducing the system to echelon form.) 10

5. Which of the following matrices are in echelon form? 0 1 1 2 3 0 0 0 0 0 A= B = 0 0 0 1 0 0 0 0 0 0 0 2 2 3 7 2 6 0 D= C = 0 0 1 5 2 0 0 0 0 2 0 0 6. Solve the system of equations x1 + 3x2 9x2 + 2x3 7x3 4x3

5 0 0 0 0 1 3 2 0 2 1 5 2 0 0 2 3 0 0 1 5

= 1 = 30 = 12 row equivalent 4 1 3 0 2 1 9 0

7. Suppose the augmented matrix for a system of linear equations is to the given echelon matrix. Solve the system. 2 1 3 2 1 1 2 0 0 2 (a) 0 3 4 15 (b) 0 0 0 0 0 2 6 0 0 0 8. Solve the following systems of linear equations. (a) x1 x2 = 1 3x1 + 2x2 = 1 (b)

3x1 x2 + x3 = 1 x1 + 3x2 x3 = 1 2x1 + 7x2 =0 x1 + 2x2 + 4x3 + 6x4 2x3 + 5x4 2x1 + 4x2 + 7x3 + 4x4 x1 + 2x2 + 3x3 + x4 =6 =5 =5 =2

(c)

x1 + x2 + 2x3 = 8 x1 2x2 + 3x3 = 1 3x1 7x2 + 4x3 = 10

(d)

1.3 Consistent Systems of Linear Equations


We saw earlier that a general system of two linear equations in two unknowns has either no solution, exactly one solution or innitely many solutions and that a system of equations that has no solutions is called inconsistent while if there is at least one solution the system is called consistent. Example 1.7 x1 2x1 5x1 x2 x2 5x2 + 2x2 + x3 2x3 11 + 2x4 + x4 x4 + 3x4 = 1 = 3 = 1 = 1

which

The augmented matrix for the system is 1 1 0 2 2 1 1 1 0 5 2 1 5 2 0 3 1 1 0 2 0 1 1 3 0 5 2 1 0 7 0 7 1 1 0 2 0 1 1 3 0 0 7 14 0 0 7 14 1 1 0 2 0 1 1 3 0 0 7 14 0 0 0 0

the last of which is in echelon form. The equation corresponding to the last row of the echelon form of the augmented matrix is 0x1 + 0x2 + 0x3 + 0x4 = 5 which is clearly impossible and so the system of linear equations is inconsistent and has no solution. In the previous example, the last row of the echelon form of the augmented matrix is of the form [0 0 0 0 | k]

1 1 R2 R2 2R1 1 4 R4 R4 5R1 1 1 6 R3 R3 5R2 11 R4 R4 7R2 1 1 6 5 R4 R4 R3

1 3 1 1

where k is a nonzero real number. It is obvious that if the last nonzero row of the echelon form of the augmented matrix is of the form [0 0 ... 0 | k] where k is a nonzero real number then the system is inconsistent. Otherwise the last nonzero row is of the form [0 ... 0 a ... | k]

where a is the nonzero coecient of the variable xj , so that the equation can then be solved for xj and the system is consistent. Now return to Example 1.6 where there are 4 equations in 5 unknowns. Notice that the echelon form of the augmented matrix has only 3 nonzero rows and that the solution contains 2 independent variables or parameters, x2 and x5 , which can be assigned arbitrary values. The following theorem gives a precise formulation for the number of parameters that arise when solving a consistent system of linear equations a11 x1 a21 x1 . . . am1 x1 + + a12 x2 a22 x2 . . . + am2 x2 + + + + a1n xn a2n xn . . . + + amn xn 12 = = b1 b2 . . . = bm

Theorem 1.2 Suppose that the augmented matrix for a consistent system of linear equations in n unknowns is row equivalent to an echelon matrix with r nonzero rows. Then r n, and the solution of the system contains n r parameters. In Example 1.6 we see that n = 5 and r = 3 so that the solution contains 2 parameters. A consequence of Theorem 1.2 is that if the system consists of m equations in n unknowns and m < n then either the system is inconsistent or the solution contains at least one parameter and hence has an innite number of solutions. Example 1.8 Consider a system of 3 equations in 4 unknowns. What can we deduce about the possible solutions of this system? Either the system is inconsistent (and so has no solutions) or it is consistent. If it is consistent, then by Theorem 1.2, r must equal 1, 2 or 3 and so the solution must contain 4 r, which is 3, 2 or 1, parameters and hence there is an innite number of solutions. Example 1.9 What can we deduce about the solutions of the system x1 2x1 5x1 x2 x2 + 2x2 + 3x3 + x3 + 2x4 + x4 + 3x4 = 0 = 0 = 0

First notice that x1 = x2 = x3 = x4 = 0 is a solution of the system, so the system is consistent. Again by Theorem 1.2, since there are fewer equations than unknowns, the solution must contain at least one parameter and so there is an innite number of solutions. The system in Example 1.9 is a special case of the following system in which the right-hand sides of the equations are all zero. a11 x1 a21 x1 . . . am1 x1 + + a12 x2 a22 x2 . . . + am2 x2 + + + + a1n xn a2n xn . . . + + amn xn = 0 = 0 . . . = 0

This system is called a homogeneous system of linear equations. Every homogeneous system is consistent since x1 = x2 = = xn = 0 is always a solution. This solution is called the trivial solution. If there are other solutions they are called nontrivial solutions. A homogeneous system has either only the trivial solution or has innitely many solutions including the trivial solution. Example 1.10 Solve the system 2x1 3x1 3x1 5x2 5x2 7x2 x3 9x3 3x3 + 4x4 + 11x4 + 7x4 0 0 0 = 0 = 0 = 0

The augmented matrix for the system is 2 5 1 4 3 5 9 11 3 7 3 7 13

by elementary row operations. The corresponding system of equations is 2x1 5x2 x3 + 4x4 = 0 5 x2 15 x3 + 5x4 = 0 2 2 which can be solved to give x1 = 8x3 7x4 , x2 = 3x3 2x4 . The above solution contains 2 parameters and so has an innite number of nontrivial solutions. The trivial solution is obtained when the two parameters are zero. In the example, we started with 3 equations in 4 unknowns and ended with 2 equations in 4 unknowns giving 4 2 = 2 parameters in the solution. We can see that, for homogeneous systems, whenever we start with more unknowns than equations we will end up with at least one parameter in the solution and hence an innite number of solutions. For homogeneous systems, elementary row operations cannot alter the nal column of the augmented matrix so the system of equations corresponding to the row-equivalent echelon matrix must also be homogeneous. Hence, in solving the system by elementary row operations we may ignore the last column of zeros in the augmented matrix and just operate on the coecient matrix. Now let us return and solve the problems posed in Examples 1.1 and 1.2. Example 1.1 A Trac Flow Problem (continued) The augmented matrix for the system of 6 equations in 7 unknowns derived in Example 1.1 is 1 0 0 0 0 0 1 400 1 1 0 0 1 0 0 200 0 1 1 0 0 0 0 300 0 0 0 0 0 1 1 500 0 0 0 1 1 1 0 0 0 0 1 1 0 0 0 600 1 0 0 0 0 0 1 400 0 1 0 0 1 0 1 200 0 0 1 0 1 0 1 100 . 0 0 0 1 1 1 0 0 0 0 0 0 0 1 1 500 0 0 0 0 0 0 0 0

which can be reduced to the echelon matrix 2 5 1 0 5 15 2 2 0 0 0

4 5 0

0 0 0

We now solve the corresponding system of equations to obtain x1 = 400 x7 , x2 = 200 + x5 x7 , x3 = 100 + x5 x7 , x4 = 500 + x5 x7 , x6 = 500 x7 . 14

The solution has two parameters (x5 and x7 ) so there is an innite number of solutions. We can obtain a particular solution by assigning xed values to x5 and x7 . For example, if x5 = 500 and x7 = 350, we obtain the values x1 = 50, x2 = 350, x3 = 50, x4 = 650, x6 = 150.

Example 1.2 An Electrical Network Problem (continued) The augmented matrix for the system of 7 equations in 6 unknowns derived in Example 1.2 is 1 1 1 0 0 0 0 0 0 1 1 0 1 0 0 0 0 1 1 1 0 1 1 0 0 1 0 0 10 10 0 0 0 0 80 0 10 40 20 20 0 250 0 0 0 20 0 30 140 0 1 1 1 0 0 0 0 20 10 0 80 0 0 0 0 0 1 1 0 1 0 . 0 0 0 1 1 1 0 0 0 0 85 20 210 0 0 0 0 0 770 1540 17 17 0 0 0 0 0 0 0 The solution to the corresponding system of equations is x1 = 5, x2 = 3, x3 = 2, x4 = 0, x5 = 2, x6 = 2.

Exercise Set 1.3 1. Solve the following systems of linear equations, if possible. (a) x1 x2 = 0 3x1 + 2x2 = 0 4x1 + x2 = 0 x1 + 3x2 + x3 = 0 3x1 + x2 x3 = 0 (b) x1 + 2x2 + x3 = 1 x2 + 2x3 = 0 x1 + 3x2 + 3x3 = 4 =2 =7 =9 = 4

(c)

(d)

x1 x2 4x3 x4 2x1 + x2 2x3 + x4 x1 + 3x2 x3 + x4 3x1 x2 + 3x3 2x4

(e)

x2 + 2x3 + x4 = 2 3x1 + 3x2 + 3x3 + 5x4 = 4 3x1 + 3x3 2x4 = 3 15

(f)

2x1 x2 + x3 = 0 14x1 + 7x2 + 7x3 = 0 4x1 + x2 2x3 = 0

2. For which value(s) of a does the following system of linear equations have zero, one, innitely many solutions? x1 + x2 + x3 = 4 x3 = 2 (a2 4)x3 = a 2 3. The augmented matrix of a system of linear a 0 b a a 4 0 a 2 equations is 2 4. b

For what values of a and b does the system have (i) a unique solution, (ii) a one-parameter solution, (iii) a two-parameter solution, (iv) no solution?

4. Determine the value or values of k which will make the following systems of linear equations consistent and then solve them. (a) x1 + x2 =1 kx2 + x3 = 1 x1 + x3 = 1 (b) 2x1 x2 + 3x3 x4 x1 x2 + 2x3 x1 + 3x2 x3 + x4 3x1 4x2 + 8x3 2x4 =6 =k =3 =0

5. The augmented matrix of a system of 1 3 0 0 0 0 0 0 x 1 = k1 , x 2 = k2 ,

Student 1 gives the solution of the system as

linear equations, in echelon form, is 2 1 2 4 1 3 . 0 0 0 0 0 0

1 1 x3 = (5 k1 + 3k2 ), x4 = (1 2k1 + 6k2 ) 6 3 where k1 and k2 are parameters which can be assigned arbitrary values. Student 2 gives the solution of the system as x1 = 3c1 6c2 + 5, x2 = c1 , x3 = c2 , x4 = 4c2 3

where c1 and c2 are parameters which can be assigned arbitrary values. Are both answers correct? Support your answer with a brief explanation. 6. Without attempting to solve the systems, determine which of the following homogeneous systems of linear equations have nontrivial solutions. (a) x1 + 3x2 + 5x3 + x4 = 0 4x1 7x2 3x3 x4 = 0 3x1 + 2x2 + 7x3 + 8x4 = 0 16 (b) x1 + 2x2 + 3x3 = 0 x2 + 4x3 = 0 5x3 = 0

(c)

a11 x1 + a12 x2 + a13 x3 = 0 a21 x1 + a22 x2 + a23 x3 = 0

(d)

x1 + x2 = 0 2x1 + 2x2 = 0

7. The following diagram shows a trac network where the numbers show the trac ow in vehicles per hour. Assign variables to the streets and calculate the trac ow.

400

500

400

300

Find the trac ow when 400 vehicles per hour use the street from C to D. What if only 200 vehicles per hour use the street from C to D? 8. The following diagram shows a network of pipes where the numbers represent water ow rates in cubic metres per hour. Write down and solve the six equations which represent the assumption that the total ow into each junction equals the total ow out of each junction and show that the solution is not unique.

400

x1

x2 x5

400

x7
300 D

x3 x4
F 300

x6
E

(i) Find the network ow when x5 = x7 = 100. (ii) Find the network ow when x7 = 300. 9. Find the currents x1 , x2 , x3 in the following electrical network.
3 A

x2
5 5 10 V 2

x3

x1

B 35 V

17

10. Write down a system of linear equations which would enable you to determine the currents x1 , . . . , x6 in the electrical network below. Would you expect there to be a unique solution? Dont attempt to solve this unless you are very persistent as the solution is messy.
20 A B

x3

x5 x1
20

x6
20 10 V C

10 V

x2
D 20

x4

1.4 Matrices and Matrix Operations


In the previous sections matrices were used as a convenient way of representing systems of linear equations. But matrices are of considerable importance in their own right, as we shall see throughout this course. This section introduces some of the fundamental properties of matrices, some of which you are no doubt familiar with. A matrix is just a rectangular array of real numbers. These numbers are called the entries of the matrix. For example, 3 1 3.1 4, [7] , [2 1], 4 2 0 5

are all examples of matrices. The normal convention is to use capital letters to denote matrices and lower case letters to denote their entries. The size of a matrix is dened by specifying the number of rows and columns. For example, the above four matrices have sizes 2 3, 1 2, 3 1 and 1 1, respectively. If A is an m n matrix then the general form of A may be written as a11 a12 a1n a21 a22 a2n A= . . . ... . . . . . . am1 am2 amn

where aij is the entry in the ith row and jth column of A, that is, the (i, j)th entry of A. Matrices with the same number of rows and columns are called square matrices and, as we shall see, these form an important class of matrices. If A is the square n n 18

matrix

then the entries a11 , a22 , . . . , ann form the main diagonal of A. We now dene the operations of addition and multiplication for matrices. To do this we rst need the concept of equality of matrices. Two matrices are said to be equal if they have the same size and the corresponding entries in the two matrices are equal. Example 1 3 A= 5 1.11 2 4, 6 Equality of matrices B= 1 2 3 , 4 5 6 C= 1 2 , 3 4 D= 1 2 , 4 3 E= 1 2 . 3 x

a11 a21 . . . an1

a12 a22 . . . an2

a1n a2n . ... . . ann

Matrices A and B are unequal because they are of dierent sizes. Matrices C and D are of the same size but are unequal because the (2, 1) (and (2, 2)) entries are unequal. Matrices C and E are equal if and only if x = 4. If A and B are any two matrices of the same size then the sum A + B is the matrix obtained by adding together the corresponding entries of A and B. Notice that addition is only dened if the matrices are of the same size. Example 1.12 Addition of matrices 1 2 3 4 A = 3 4, B = 5 6, 5 6 1 0 4 6 A + B = 8 10 4 6

C=

1 2 . 3 4

while A + C and B + C are undened.

If A is any matrix and c is any number then the product cA is the matrix obtained by multiplying each entry of A by c. This operation is often referred to as scalar multiplication. We use A = (1)A to denote the negative of the matrix A. Subtraction of two matrices of the same size is then given by A B = A + (B) = A + (1)B. Example 1.13 Scalar multiplication and subtraction of matrices 3 4 1 2 B = 5 6. A = 3 4, 1 0 5 6 3 6 3 4 0 2 3A = 9 12 , B = 5 6 , 3A B = 4 6 . 15 18 1 0 16 18 19

The third basic operation is matrix multiplication in which two matrices are multiplied together to obtain a matrix product. Although this operation may at rst seem unnatural we will see later that it has many practical applications. If A is an m r matrix and B is an r n matrix, the product AB is the m n matrix whose (i, j)th entry is obtained by multiplying the corresponding entries from row i of A and column j of B together and adding up the resulting products. So if a11 a12 a1r b11 b12 b1n a21 a22 a2r b21 b22 b2n A= . . . and B = . . . ... ... . . . . . . . . . . . . am1 am2 amr br1 br2 brn
r

then the product matrix C = AB is an m n matrix whose (i, j)th entry is given by cij = ai1 b1j + ai2 b2j + + air brj = Example 1.14 Product of matrices 1 2 3 4 A = 2 0 1 1 , 2 5 2 0 7 1 3 0 B= . 4 2 1 1 aik bkj .
k=1

To nd the (1, 1) entry of C we multiply together corresponding entries in the rst row of A and the rst column of B. That is, c11 = (1)(7) + (2)(0) + (3)(4) + (4)(1) = 1. To nd the (1, 2) entry of C we multiply together corresponding entries in the rst row of A and the second column of B. That is, c12 = (1)(1) + (2)(3) + (3)(2) + (4)(1) = 3. To nd the (2, 1) entry of C we multiply together corresponding entries in the second row of A and the rst column of B. That is, c21 = (2)(7) + (0)(0) + (1)(4) + (1)(1) = 17 and so on. The nal result is 1 3 C = AB = 17 5 . 22 21 20

First notice that A is 3 4 and B is 4 2 so the matrix product AB is dened and is a 3 2 matrix. (Also notice that the product BA is not dened.) If we let C = AB, then C will take the form 7 1 1 2 3 4 3 0 = . C = AB = 2 0 1 1 4 2 2 5 2 0 1 1

It is important to understand that for the product of two matrices to be dened, the number of columns of the rst matrix must equal the number of rows of the second matrix. A B A B = mn mr rn Example 1.15 Suppose A is a 4 2 matrix, B a 3 4 matrix and C a 2 3 matrix. Then AC is a 4 3 matrix BA is a 3 2 matrix CB is a 2 4 matrix AB, BC and CA are undened. One practical application of matrix multiplication is in representing a system of linear equations. The system a11 x1 a21 x1 . . . am1 x1 + + a12 x2 a22 x2 . . . + am2 x2 + + + + a1n xn a2n xn . . . + + amn xn = = b1 b2 . . . = bm

is equivalent to the matrix equation a11 x1 + a12 x2 + + a1n xn b1 a21 x1 + a22 x2 + + a2n xn b2 = . . . . . . . . . . . . am1 x1 + am2 x2 + + amn xn bm which can be written as the matrix product Ax = b where b1 x1 a11 a12 a1n b2 x2 a21 a22 a2n A= . . , x = . , b = . . . ... . . . . . . . . . . bm xn am1 am2 amn Notice that x and b are matrices consisting of only one column. Such matrices are often called column vectors and we sometimes use bold lower case letters to denote them instead of upper case letters. Similarly, a 1 n matrix consisting of a single row is often called a row vector. Any matrix A with columns a1 , a2 , . . . , an can be expressed in the form A = [ a1 a2 . . . an ] ,

and this notation is often useful as we shall see later. As a consequence of representing a matrix in this way it follows that, if A is an m n matrix and B is an n r matrix written as B = [ b1 b2 . . . br ] then AB = A [ b1 b2 . . . br ] = [ Ab1 21 Ab2 . . . Abr ] .

Exercise Set 1.4 1. Consider the matrices 3 1 4 2 A = 1 2 , B= 0 2 0 1 1 5 2 5 D = 1 0 1 , E = 1 3 4 2 3 Compute: (a) AB (b) D + E 2. Solve the following equation for the matrix A. 1 0 1 2 A= 0 1 3 4 3. Find conditions on w, x, y and z such that AB = BA for the following matrices. A= w y x z and B = 1 1 1 1

C=

1 4 3 2 1 5

1 3 1 2. 4 1 (d) DE (e) ED.

(c) D 2E

4. Prove that if the products AB and BA are both dened then AB and BA are both square matrices. 5. A square matrix is called a diagonal matrix if all the entries o the main diagonal are zero. Show that the product of diagonal matrices is a diagonal matrix. State a rule for multiplying diagonal matrices. 6. The trace of a square matrix is dened to be the sum of its main diagonal entries. If A and B are square matrices of the same size, prove that trace of A + B equals the trace of A plus the trace of B.

1.5 Properties of Matrix Operations


In the last section we dened the operations of matrix addition, multiplication of a matrix by a number and matrix multiplication. For these operations to be useful we have to determine the basic rules they must obey. Many of these basic rules are similar to the rules of arithmetic for real numbers but there are important exceptions. One important property of real numbers is that the number 0 serves as an additive identity. That is, c + 0 = c for any real number c. We also have an additive identity for matrices called the zero matrix which is any matrix whose entries are all zero. Unlike real numbers where zero is a unique number, there are lots of zero matrices, one for each size of matrix. We will use Omn to denote the m n zero matrix. When the size of the zero matrix is obvious from the context we will usually just denote it by O. 22

Let A, B and C be matrices of the same size. Then we have the following properties of matrix addition. 1. 2. 3. 4. A + B = B + A. (A + B) + C = A + (B + C). A + O = A. A + (A) = O.

These properties can be easily established and as an illustration we prove Property 1. Proof of Property 1: Let A and B be m n matrices with (i, j)th entries aij and bij , respectively. Then (i, j)th entry of A + B = aij + bij = bij + aij = (i, j)th entry of B + A. Hence A + B = B + A. Let A and B be matrices of the same size and c and d any real numbers. Then we have the following properties involving multiplication of a matrix by a number. 1. c(A + B) = cA + cB. 2. (c + d)A = cA + dA. 3. (cd)A = c(dA). These properties are consistent with the corresponding properties for real numbers and can easily be established. Following are some of the rules of matrix multiplication which are consistent with those for real numbers. The sizes of the matrices A, B and C are such that the matrix products are all dened. 1. 2. 3. 4. (AB)C = A(BC). A(B + C) = AB + AC. (A + B)C = AC + BC. c(AB) = (cA)B = A(cB) where c is any real number. Demonstrate that (AB)C = A(BC) where 3 1 2 1 2 2 1 3 A= , B= , C = 2 1 1 . 1 3 1 1 1 4 2 1 AB = 4 3 5 1 2 0 and BC = 20 5 2 . 9 2 2

Example 1.16

Now,

Therefore, (AB)C is the product of a 2 3 and a 3 3 matrix, while A(BC) is the product of a 2 2 and a 2 3 matrix. Forming these products gives (AB)C = 38 9 6 7 1 4 and A(BC) = 38 9 6 . 7 1 4

There are some properties of arithmetic for real numbers which do not carry over to matrices. The rst of these is that matrix multiplication is not commutative. 23

Example 1.17

Noncommutativity of matrix multiplication A= AB = 1 0 , 2 3 1 2 , 11 4 B= 1 2 . 3 0 3 6 . 3 0

BA =

The above example illustrates that, in general, AB = BA. In the example, AB and BA were both of the same size, but it is also possible to have AB and BA of dierent sizes or even to have one product dened and the other not dened. Another important property of matrix multiplication is that it does not have a cancellation law. That is, if C is nonzero and AC = BC we cannot always conclude that A = B. Example 1.18 No cancellation law for matrix multiplication A= AC = 1 3 , 0 1 B= 2 4 , 2 3 C= 1 2 . 1 2

2 4 = BC, 1 2

but A = B.

As a special case of the failure of the cancellation law for matrix multiplication, there are nonzero matrices A and B for which AB = 0. Example 1.19 A= 1 1 , 2 2 but BA = B= 2 1 , 2 1

AB = O,

4 4 . 4 4

An identity matrix behaves like a 1 in multiplication so that, if A is an m n matrix, AIn = A and Im A = A.

A special type of square matrix with 1s on the main diagonal and 0s everywhere else is called an identity matrix. We denote the n n identity matrix by In , or just I when the size is obvious from the context. For example, 1 0 0 1 0 , I3 = 0 1 0 , etc. I1 = [ 1 ] , I2 = 0 1 0 0 1

For repeated multiplication of square matrices we use the same notation as for real numbers, that is, A1 = A, A2 = AA, ,......, Ak = AA A .
k times

24

We can then establish the properties 1. Ap Aq = Ap+q for positive integers p and q. Finally, the transpose of a matrix is formed by interchanging the rows and columns. For example if a11 a12 a1n a21 a22 a2n A= . . . ... . . . . . . am1 am2 amn then the transpose of, denoted by AT , is the n m matrix given by a11 a12 AT = . . . a1n a21 a22 . . . a2n am1 am2 . . ... . . anm 2. (Ap )q = Apq

That is, row 1 of AT is column 1 of A and so on. Example 1.20 Calculate (AB)T , AT B T and B T AT if A= Now AB = 14 10 , 14 9 AT = 1 3 , 3 2 BT = 2 4 , 1 3 1 3 , 3 2 B= 2 1 . 4 3

and so,

(AB)T =

14 14 , 10 9

AT B T =

5 13 , 8 18

B T AT =

14 14 . 10 9

From the previous example it appears that (AB)T = B T AT and not AT B T as we might expect. Here follow some of the properties of the matrix transpose where A and B are matrices of the appropriate size. 1. (AT )T = A. 2. (A + B)T = AT + B T . 3. (AB)T = B T AT . The matrix A in Example 1.20 has the property that AT = A, that is the matrix and its transpose are the same. A matrix with this property is called symmetric and it is clear that a symmetric matrix must be square. 25

Exercise Set 1.5 1. Let A be an m r matrix and B and C be r n matrices. By computing the (i, j) entries, prove the distributive law: A(B + C) = AB + AC. 2. Let A= 3 2 , 1 3 B= 4 0 . 1 5

Verify that (AT )T = A and (AB)T = B T AT . 3. (a) Find 2 2 matrices A and B such that (A + B)2 = A2 + 2AB + B 2 . (b) Show that, if A and B are square matrices such that AB = BA, then (A + B)2 = A2 + 2AB + B 2 . (c) By multiplying out (A + B)(A + B), nd an expansion of (A + B)2 that is valid for all square matrices A and B of the same size. 4. Find, if possible, (a) AB (b) AC given that (c) A2 (d) C 2

(e) BD

(f) CD

(g) ABD,

1 0 1 3 1 3 1 5 4 5 6 1 A= , B= , C= , D = 0 1 1 . 2 6 6 2 0 1 2 1 3 1 0 1 5. Prove by induction that if A = integers n. 6. Let A= 3 4 , 2 3 B= 3 4 , 2 3 C= 1 2 . 5 3 2 1 2n , then An = 0 1 0 1 2n 1 for all positive

Find AB and hence solve the matrix equation XA = C for X. 7. Given is a real number, nd an m m matrix B such that BA = A for all m n matrices A. 8. An n n matrix A is called upper triangular if aij = 0 for all i > j. Prove that the product of two upper triangular matrices is upper triangular. 9. If A is any m n matrix, show that AAT and AT A are both symmetric. 26

1.6 The Inverse of a Matrix


In the real number system, given any nonzero number a, that number b for which ab = ba = 1 is called the (multiplicative) inverse of a, denoted by 1/a or a1 . The same terminology is employed in matrix arithmetic. If A is any square matrix and if there is a matrix B such that AB = BA = I then B is called the inverse of A. Example 1.21 A= AB = Hence, B is the inverse of A. Nonsquare matrices do not have inverses. To see this, note that if A is of size m n and B is of size n m, where m = n, then AB and BA are both dened but are of dierent sizes and so cannot equal each other. Even if A is a square matrix it may still not have an inverse. For example, the matrix A= 1 2 2 4 3 13 , 2 9 B= BA = 9 13 2 3

1 0 , 0 1

1 0 . 0 1

does not have an inverse! If a matrix has an inverse then it is said to be invertible or nonsingular. A matrix that does not have an inverse is called noninvertible or singular. The following theorem tells us that if a matrix does have an inverse, then that inverse is unique. Theorem 1.3 If B and C are both inverses for the matrix A, then B = C. Proof. Since B and C are both inverses of A, we have AB = BA = I hence, it follows that B = BI = B(AC) = (BA)C = IC = C. Because the inverse of an invertible matrix A is unique we call it the inverse of A and denote it by A1 , so that AA1 = A1 A = I. Example 1.22 There is a simple formula that can be derived for the inverse of a 2 2 invertible matrix. Let a b A= . c d Then the following statements can be easily justied. (a) If ad bc = 0 then A is not invertible. 27 and AC = CA = I.

(b) If ad bc = 0 then A is invertible and its inverse is given by A1 = For example, if A= 1 2 , 3 4 then A1 = 1 2 4 2 2 1 = 3 1 . 3 1 2 2 1 d b . c a ad bc

The following theorem lists some of the properties of matrix inverses. Theorem 1.4 Let A and B both be invertible matrices of the same size. Then, 1. A1 is invertible and (A1 )1 = A. 2. AB is invertible and (AB)1 = B 1 A1 . (This says that the product of invertible matrices is invertible and the inverse of the product is the product of the inverses in reverse order.) 3. AT is invertible and (AT )1 = (A1 )T . 4. For any positive integer k, Ak is invertible and (Ak )1 = (A1 )k = A1 A1 A1 .
k times

Proof. We shall prove Property 2 and leave the rest as an exercise for the reader. To prove that B 1 A1 is the inverse of AB we need to show that (AB)(B 1 A1 ) = (B 1 A1 )(AB) = I. This will, at the same time, prove that AB is invertible. Now, (AB)(B 1 A1 ) = A(BB 1 )A1 = AIA1 = AA1 = I. Similarly, (B 1 A1 )(AB) = B 1 (A1 A)B = B 1 IB = B 1 B = I. This proves the result. For invertible matrices we can extend to negative integer powers the same exponential notation we used for positive integer powers of matrices. If A is invertible we dene Ak by Ak = (A1 )k = A1 A1 A1 .
k times

Exercise Set 1.6 1. Assume that A is a square matrix that satises A2 3A + I = 0. Show that A is invertible and that A1 = 3I A. 2. Let A be a square matrix and n a positive integer. (i) Show that if A is invertible then An is invertible. (ii) Show that if An = O then A is not invertible. 28

3. Find the inverse of

cos sin

sin . cos

4. Is the sum of two invertible matrices necessarily invertible? 5. Let A and B be square matrices such that AB = O. Show that if A is invertible then B = O. 6. Prove Part 3 of Theorem 1.4. 7. Prove that if C is invertible and AC = BC then A = B. 8. Let A be a square matrix. (i) By multiplying the expression I + A + A2 + A3 by I A, show that if A4 = O then (I A)1 = I + A + A2 + A3 . (ii) Show that if An+1 = O then (I A)1 = I + A + A2 + + An for any positive integer n. 9. If A is any 2 1 matrix and B is any 1 2 matrix, show that AB cannot have an inverse. 10. Find ve 2 2 matrices A satisfying A2 = I (and discover the surprising fact that there are matrices other than I and I which satisfy A1 = A).

1.7 Calculation of the Matrix Inverse


Before describing a method for calculating the inverse of an invertible matrix we return again to the linear system of equations which can be written in the form Ax = b where A is an mn matrix and x and b are column vectors of size n and m, respectively. First consider the case when b is nonzero. In Section 1.3, we saw that this system is either inconsistent in which case there are no solutions, or consistent in which case there is either a unique solution or innitely many solutions. If A is invertible then m = n and we have the following result. Theorem 1.5 If A is an invertible n n matrix and b is a nonzero column vector of size n, the system of linear equations Ax = b has exactly one solution which is x = A1 b. Proof. To see that A1 b is a solution we substitute it into the equation for x. This gives A(A1 b) = (AA1 )b = Ib = b. This proves that x = A1 b is a solution to the system of linear equations. What remains is to prove that it is the only solution. 29

Suppose that y is another solution, that is, y satises Ay = b. Then multiplying both sides of the equation by A1 we obtain A1 (Ay) = A1 b y = A1 b. Hence A1 b is the only solution. This theorem says that if we know A1 then we can easily obtain the solution of the linear system of equations Ax = b by writing x = A1 b. Now consider the case when b is zero, that is the system of linear equations is homogeneous. In Section 1.3, we saw that this system is always consistent and has either only the trivial solution or has innitely many solutions including the trivial solution. By a similar argument to that used in Theorem 1.5 we can prove the following. Theorem 1.6 If A is an invertible matrix the homogeneous system of linear equations Ax = 0 has only the trivial solution. We now return to the problem of calulating the inverse of an invertible matrix. To illustrate the procedure we will assume that A is a 3 3 invertible matrix. Dene three column vectors 0 0 1 e1 = 0 , e2 = 1 , e3 = 0 1 0 0 Ax1 = e1 , Ax2 = e2 , Ax3 = e3 .

and consider the three systems of linear equations

From Theorem 1.5 these systems of linear equations each have the unique solutions x1 = A1 e1 , x2 = A1 e2 , x3 = A1 e3 . A1 e3 ]

Now form a 3 3 matrix whose columns consist of these three solutions to obtain [ x1 x2 x3 ] = [ A1 e1 A1 e2 = A1 [ e1 e2 e3 ] 1 0 0 = A1 0 1 0 0 0 1 = A1 . This now gives a procedure for calculating the inverse of the matrix A which is to solve the three systems of linear equations Ax1 = e1 , Ax2 = e2 , Ax3 = e3

by using elementary row operations. This procedure is illustrated in the following example. Example 1.23 Let 1 4 2 A = 0 2 1. 3 5 3 30

The augmented matrix for the system Ax = e1 is 1 4 2 1 0 2 1 0 3 5 3 0 which can be reduced to the echelon matrix 1 4 2 0 2 1 1 0 0 2 1 0 3

by elementary row operations. The corresponding system of equations then has the solution 1 x1 = 3 . 6 Similarly we solve the systems Ax = e2 and Ax = e3 to obtain the solutions 0 2 x2 = 3 and x3 = 1 . 2 7 The inverse of A is then given by [ x1 1 2 0 x3 ] = 3 3 1 . 6 7 2

x2

In the example we solved three systems of linear equations to nd the inverse of a 3 3 matrix. The solutions involved reducing the augmented matrix for each of these systems to echelon form during which the matrix A was also reduced to echelon form. We can make the calculations more ecient by reducing A to echelon form only once and this can be achieved by forming the 3 6 matrix [ A | e1 e2 e3 ] = [ A | I ] matrix 4 2 2 1 5 3 1 0 0 0 1 0 0 0 1

where I is the 3 3 identity matrix. The 1 0 [A | I ] = 3

can then be reduced using elementary row operations to the echelon matrix 1 4 2 1 0 0 0 2 1 0 1 0. 1 0 0 2 3 7 1 2 31

Although we could now obtain A1 by successively solving the three corresponding systems of equations by back-substitution as in Example 1.23, it is customary to continue the reduction process until the left-half matrix is transformed to the identity matrix. To illustrate the procedure we will continue with the reduction. 1 0 0 1 0 0 1 0 0 1 0 0 4 2 0 4 1 0 4 1 0 0 1 0 0 0 1 7 2 2 1 2 1 0 0 1 1 0 1 0 R 2 2 R2 2 2 1 6 7 2 R3 2R3 0 13 14 4 R1 R1 2R3 3 1 R2 R2 1 R3 0 3 2 7 2 1 6 0 1 2 0 R1 R1 4R2 0 3 3 1 1 6 7 2 2 1 1 0 3 0 1

The nal matrix is now in the form [ I | B ] where 1 2 0 = 3 3 1 6 7 2

B = A1

as can be seen from Example 1.23. We should now check our calculations by multiplying A by B to conrm that BA = AB = I. The procedure for nding the inverse of an invertible n n matrix A can now be summarised as follows. Step 1. Form the n 2n matrix [ A | I ]. Step 2. Use elementary row operations to transform [ A | I ] to the form [ I | B ]. Then B = A1 . Example 1.24 Find the inverse of 3 4 5 A = 1 1 2 . 2 1 3 32

Hence,

The computations are as 3 1 2 1 3 2 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0

follows: 1 0 0 0 1 0 0 0 1 1 2 0 1 0 R1 R2 4 5 1 0 0 1 3 0 0 1 1 2 0 1 0 7 1 1 3 0 R2 R2 3R1 3 1 0 2 1 R3 R3 2R1 1 2 0 1 0 3 1 1 1 1 7 7 7 0 R2 7 R2 3 1 0 2 1 1 2 0 1 0 1 3 1 1 7 7 0 7 5 0 4 3 7 1 R3 R3 3R2 7 7 0 1 2 0 1 3 0 1 1 1 7 7 7 7 5 7 3 0 1 4 R3 4 R 3 4 4 3 3 7 1 0 2 2 R1 R1 2R3 2 1 1 0 1 4 1 R2 R2 + 1 R 3 4 4 7 5 7 0 1 3 4 4 4 5 7 13 0 0 4 4 R 1 R 1 + R2 4 1 1 1 1 0 4 4 4 5 7 0 1 3 4 4 4 4 5 1 2 1 3 A1 = AA1 5 4


1 4 3 4

7 4 1 4
5 4

13 4 1 4 7 4

Check:

1 0 0 = A1 A = 0 1 0 . 0 0 1

We usually dont know if A is invertible before we try to nd its inverse. If we try to apply the procedure to a matrix that is not invertible we will soon nd that it is impossible to transform [ A | I ] to the form [ I | B ] as a row of zeros will appear on the left-half of the matrix. Example 1.25 Solve the system of equations 3x1 x1 2x1 + 4x2 x2 + x2 + 5x3 + 2x3 + 3x3 33 = 2 = 4 = 1

If we write this system as Ax = b then A is the matrix of the previous example and hence we know A1 . Therefore, 25 5 7 4 4 4 13 2 4 1 1 1 1 4 = 3 . x=A b= 4 4 4 4 3 19 5 7 1 4 4 4 4

Note that if we do not know A1 then it is generally inecient to solve a system of linear equations by rst computing A1 . It is usually much quicker to use Gaussian elimination followed by back-substitution than to calculate a matrix inverse. In Examples 1.23 and 1.24 we checked that the matrix we obtained was in fact the inverse of the original matrix by verifying that it satised AA1 = A1 A = I. The following theorem, which we shall present without proof, tells us that we only need check that either AA1 = I or A1 A = I.

Theorem 1.7 Let A be any square matrix. If there exists a square matrix B which satises either (a) AB = I, or (b) BA = I, then A is invertible and B = A1 . We now take another look at the the Leontief Input-Output models introduced in Example 1.3. Example 1.3 Leontief Input-Output Models - continued First consider the closed economic model in which each item produced by one segment of the economy is consumed by other segments of the same economy. Suppose there are n industries and A is the n n input-output matrix a11 a12 a1n a21 a22 a2n . . . ... . . . . . . an1 an2 ann

in which aij is the fraction of the total production of industry j required to supply the needs of industry i. If xi is the value of the total output of the ith industry then we saw earlier that the unknowns x1 , x2 , . . . , xn satisfy the matrix equation Ax = x. This equation can be written in the form (I A)x = 0 and we know that one possible solution is x = 0 in which nobody produces anything! To obtain other, nonzero solutions, we must have that I A is not invertible (if I A were invertible then the only solution is x = 0). Because of the properties of the matrix A (all its entries are positive and the sum of the entries in each column equals 1), it is possible to prove that I A is not invertible and the system of equations then has an innite number of nontrivial solutions. 34

Now consider the open economic model where we now have an external demand for each industrys products as well as demands from the other n industries. If A is the input-output matrix, xi is the value of the total output of the ith industry and di is the external demand for the ith product then we obtain the system of equations Ax + d = x. This system is equivalent to which will have a unique solution provided I A is invertible. Again, because of the properties of A (all its entries are positive and the sum of the entries in each column is less than or equal to 1 with at least one column sum being strictly less than 1), it is possible to prove that I A is invertible and hence that the system of equations has a unique solution. Example 1.26 A closed economic model Assume the economy has three segments: farming, manufacturing and service. Assume 1 that 3 of the food produced by the farming segment is consumed by the farmers 1 themselves, the manufacturers consume 2 of it and the remaining 1 is consumed by 6 1 the service industry. Also assume that the farmers require 2 of the manufactured goods with the rest split evenly between the other two industries and that all segments have an equal need of services. The input-output matrix is then: f 1 m s
1 2 1 4 1 4 1 3 1 3 1 3

(I A)x = d

where the rst column refers to the consumption of the farming segment, the second to manufacturing and the third to service. Let x represent the dollar values of each segment of the economy which must then satisfy (I A)x = 0. The augmented matrix for this system is 2 2 1 1 3 0 3 2 3 3 1 1 0 0 2 4 3 1 2 0 0 6 1 4 3 5 x1 = x3 , 3 x3 can now be given any (positive) value. Example 1.27 An open economic model A town has three main industries: a coal mining operation, an electrical power plant and a local railway company. To mine $1 of coal, the mining company must purchase 25c of electricity and 25c of rail transport for its shipping needs. To produce $1 of 35
1 2

A =

3 1 2 1 6

f m s

3 8

1 3 7 12 0

The equivalent system of linear equations then has the solution x2 = 14 x3 . 9

0 0. 0

electricity requires 65c of coal, 5c of its own electricity and 5c of rail transportation. To provide for $1 of rail transportation requires 55c of coal and 10c of electricity. In a given week, the mining company receives orders for $50, 000 of coal from outside the town, the power plant receives orders for $25, 000 of electricity from outside the town while the railway company receives no outside orders. How much must each of the three industries produce in order to satisfy their own demand and the outside demand? The solution vector, x satises the equation (I A)x = d where 0 0.65 0.55 A = 0.25 0.05 0.1 , 0.25 0.05 0 50, 000 d = 25, 000 . 0 50, 000 37, 500 . 1, 425, 000

The augmented matrix for this system is 1 0.65 0.55 0.25 0.95 0.1 0.25 0.05 1.0

50, 000 1 0.65 0.55 25, 000 0 0.7875 0.2375 0 0 0 50.3

The equivalent system of linear equations then has the solution x1 = 102, 087, x2 = 56, 163, x3 = 28, 330.

Therefore, the total output of the coal mining operation is $102, 087, the electrical power plant is $56, 163 and the railway company is $28, 330. Example 1.28 Cryptography Messages can be encoded and decoded using matrix multiplication. The following procedure is one way of accomplishing this. We will illustrate the procedure by applying it to the message I HAVE THE GOLD. 1. Assign the numbers 1 to 26 to each letter of the alphabet with 0 assigned to a blank space. 2. Partition the message into groups (row vectors) of size n. For example if n = 3 our message becomes: [ 9 0 8 ] [ 1 22 5 ] [ 0 20 8 ] [ 5 0 7 ] [ 15 12 4 ] I H A V E T H E G O L D 3. To encode the message choose an invertible n n matrix A and multiply each row vector (on the right) by A. For our message we will use the matrix of Example 1.23: 1 4 2 A = 0 2 1 3 5 3 36

so that [9 0

[ 1 22

[ 0 20

[5 0

[ 15 12 The encoded message is then

1 0 8] 3 1 5]0 3 1 8]0 3 1 7]0 3 1 0 4] 3

4 2 2 1 = [ 33 5 3 4 2 2 1 = [ 16 5 3 4 2 2 1 = [ 24 5 3 4 2 2 1 = [ 26 5 3 4 2 2 1 = [ 27 5 3

76 42 ]

73 39 ]

80 44 ]

55 31 ]

104 54 ]

[ 33 76 42 ] [ 16 73 39 ] [ 24 80 44 ] [ 26 55 31 ] [ 27 104 54 ] . 4. Once the message is sent, the receiver can decode it by multiplying each encoded row vector (on the right) by A1 . Referring again to Example 1.23, 1 2 0 A1 = 3 3 1 6 7 2 so the decoded row vectors are [ 33 76 1 42 ] 3 6 1 39 ] 3 6 1 44 ] 3 6 1 3 31 ] 6 2 0 3 1 = [ 9 7 2 2 0 3 1 = [ 1 7 2 2 0 3 1 = [ 0 7 2 2 0 3 1 = [ 5 7 2

0 8]

[ 16 73

22 5 ]

[ 24 80

20 8 ]

[ 26 55

0 7]

from which the original message is recovered. 37

1 2 0 [ 27 104 54 ] 3 3 1 = [ 15 12 4 ] 6 7 2

An important feature of this method of cryptography is that several occurrences of the same letter in the plain text become dierent in the encryption, for example E = 5 is encoded as 39 and 26. Thus this method resists attack by analysis of the frequency of variuous letters. Exercise Set 1.7 1. Determine whether the following matrices are invertible, exists. 1 0 3 5 2 2 1 , 2 (a) (b) 1 5 7 1 2. Find the inverse 1 1 (a) 1 3 2 (d) 5 2 of the following matrices. 3 3 1 2 1 3 , 4 3 (b) 3 4 2 4 4 2 7 1 3 3 2 1 , (e) 7 3 9 2 3 2 3 3 3 3, 3 1 2 3 4 and nd the inverse if it 6 5 1 4. 4 3

1 3 5 5

2 1 1 (c) 1 3 2 , 1 2 1 1 4 . 5 8

3. Find a matrix C such that AC = B where 1 0 2 2 1 3 B = 1 1 0. A = 1 1 2 , 0 0 1 3 2 4 4. If the matrix 1 1 b b2 is invertible, what can be said about b?

5. If a closed economic system has an input-output matrix 0.2 0.1 0.3 A = 0.4 0.8 0.3 , 0.4 0.1 0.4

nd nonnegative vectors giving the values of each segment of the economy.

6. An industrial system has two industries with the following inputs. (a) To produce $1s worth of output, Industry A requires 0.40c of its own product and 30c of Industry Bs product. (b) To produce $1s worth of output, Industry B requires 0.20c of its own product and 30c of Industry As product. Find the input-output matrix for the system and determine how much each industry must produce if Industry A has an external demand of $50, 000 and Industry B has an external demand of $30, 000. 38

7. A civil engineer (CE), a mechanical engineer (ME) and an electrical engineer (EE) each have a consulting rm. As their consulting is of a multidisciplinary nature they must use each others services. For every $1s consulting CE does, she buys 10c of MEs services and 30c of EEs services. For every $1s consulting ME does, he buys 20c of CEs services and 40c of EEs services. For every $1s consulting EE does, she buys 30c of CEs services and 40c of MEs services. In a given week, CE receives outside consulting orders of $500, ME receives outside consulting orders of $700 and EE receives consulting orders of $600. What is the value of the consulting each engineer does in that week? 8. Let A= 1 2 . 3 5

(a) Use A to encode the message PLEASE SEND MONEY. (b) Find A1 and use it to decode the cryptogram 11 21 64 112 25 50 29 53 23 46 40 75 55 92

1.8 Gaussian Elimination and Partial Pivoting


Consider the system of n linear equations in n unknowns: Ax = b where A is an n n invertible matrix. We know by Theorem 1.5 that this system has a unique solution. From a practical viewpoint we are interested in solving this system of equations as eciently and accurately as possible. One of the most ecient means of solving linear systems is to use Gaussian elimination with back-substitution. As mentioned previously, it is not necessary or advisable to rst compute A1 and then calculate the solution as x = A1 b as this takes nearly 3 times the number of operations (additions and multiplications) as Gaussian elimination with back-substitution. In practical problems, we may be faced with solving systems of equations involving 100s or even 1, 000s of variables. Obviously these problems cannot be solved by hand, so the procedure for solving linear systems, that is Gaussian elimination with backsubstitution, needs to be coded as a computer program. This can be easily achieved and all the standard numerical software packages will contain such a program. However, when one is using computers to solve problems, and in particular problems involving systems of linear equations, the question of numerical accuracy becomes important. Computers are limited in the number of digits they can use to represent a number. 2 For example, a computer which can store 8 decimal digits might represent 3 as either 0.66666667 or 0.66666666. In either case an error is introduced called a roundo error. Roundo errors can occur whenever numbers are stored or arithmetic operations are performed on calculator or computer. Recall that, in Gaussian elimination, if the top entry in the leftmost nonzero column is zero then we interchange rows so as to bring a nonzero entry to the top of the column. For example, the augmented matrix 1 1 1 4 1 1 1 4 1 1 2 3 0 0 1 1 R2 R2 R1 . 2 4 4 5 0 2 2 3 R3 R3 2R1 39

The entries in bold are called pivots and it is clear that we can deal with zero pivots simply by interchanging rows. However, intuition should tell us that it may be necessary to perform row interchanges not only when a pivot is exactly zero, but also when it is nearly zero. Consider the following example. Example 1.29 Consider the system 108 x + y x + y = 1 = 2

We now have to interchange rows 2 the second column. This gives 1 1 0 2 0 0

and 3 because there is a zero entry at the top of 1 2 1 4 3 R2 R3 1

whose solution, accurate to 7 digits, is x = y = 1. We will rst solve the system by Gaussian elimination rounding o all computations to 7 signicant digits. The augmented matrix for the system is 108 1 1 1 1 108 2 0 1 108 1 108 R2 R2 108 R1 .

(Notice that the calculation to obtain the (2, 2) entry is 1 108 = 99, 999, 999. If we now round this to 7 digits we obtain 100, 000, 000 = 108 .) The solution by back-substitution is x = 0, y = 1 which has considerable error! Now consider what happens when we interchange rows 1 and 2 at the rst step. This gives 108 1 1 1 1 1 2 108 1 1 2 1 1 1 0 1 2 1 R2 R2 108 R1 .

(This time the calculation to obtain the (2, 2) entry is 1 108 = 0.99999999 = 1 after rounding to 7 digits.) The solution by back-substitution is now x = 1, y = 1. This example shows that a small pivot can cause large errors in the solution. The eect of the small pivot is to magnify any roundo errors. To reduce the eect of roundo error we use a process called partial pivoting in which the rows are interchanged at each stage so that the pivot is chosen as the entry whose absolute value is the largest in the unreduced part of the column. All respectable numerical software packages automatically include partial pivoting when solving systems of linear equations. Example 1.30 Use partial pivoting and Gaussian elimination to solve the system x1 4x1 2x1 + x2 + 16x2 + 4x2 + x3 + 64x3 + 8x3 = 2 = 100 = 6

The augmented matrix for the system is 1 1 1 4 16 64 2 4 8 40

2 100 . 6

and then, proceeding with 4 0 0

Partial pivoting requires that the entry a21 = 4 rst be brought to the top of the rst column: 4 16 64 100 R1 R2 1 1 1 2 2 4 8 6 the elimination we obtain 16 64 100 3 15 27 R2 R2 1 R1 . 4 4 24 44 R3 R3 1 R1 2

This matrix is now in echelon form and back-substitution gives the solution x1 = 3, Exercise Set 1.8 1. Solve the following system using partial pivoting. 2x1 + 3x2 + x3 = 12 4x1 8x2 + 3x3 = 28 x1 + x2 + x3 = 1 x2 = 1, x3 = 2.

We now examine the second column and see that the largest entry in magnitude in the unreduced part of the column is the entry a32 = 4. Hence we interchange rows 2 and 3 and then proceed with the elimination: 100 4 16 64 100 4 16 64 0 4 24 44 R2 R3 0 4 24 44 . 3 6 R3 R3 4 R2 0 0 3 0 3 15 27

2. Solve the following systems: (i) without partial pivoting and (ii) with partial pivoting. Perform all computations to four signicant digits. (a) 0.00001x + y = 2 x+y =3 (b) 0.00002x y = 1 4x + 3y = 5

3. The following system of linear equations has the exact solution x = 1, y = 1. x + 1.04y = 2.04 6x + 6.2y = 12.2 (i) Use Gaussian elimination without partial pivoting to solve the system, rounding all intermediate calculations to three signicant digits. (ii) Use Gaussian elimination with partial pivoting to solve the system, again rounding all intermediate calculations to three signicant digits. In both cases compare your solutions with the exact solution. 41

You might also like