Lecture Notes #1: Review of Matrix Algebra

ECON 222 Spring, 2007

This note describes how vectors and matrices are used as a means of storing information. It also describes dierent types of operations that can be performed on vectors and matrices. Evatually, we will use matrices and vectors to represent a variety of econometric estimators.

1
1.1

Vectors
Denition

A number can be used to represent a point on a line. A pair of numbers can be used to represent a point in two dimensional space. Three numbers can be used to represent a point in 3-space. Similarly, we can consider 4-space, 5-space or n-space (though it becomes more dicult to visualize these points) x = (x1 , x2 , x3 , x4 , ..., xn ) For example, x1 , x2 , x3 , x4 could represent earnings in years 2000, 2001, 2003 and 2004. (In physics, the 4th dimension might be time). Denote n-space Rn . x Rn . EXAMPLES: Suppose a=(1,2) and b=(-3,5) represent two vectors. Then a+b=(-2,7). Note also that a+b=b+a Suppose x = (2, 1, 5) and c = 7 (a constant). Then c x = (14, 7, 35)

1.2

Some Properties of Vectors:

Suppose A and B are vectors and c, c1 ,c2 are some constants. (i) c(A + B) = cA + cB (ii) (c1 + c2 )A = c1 A + c2 B (iii) c1 c2 A = c2 c1 B 1

Suppose A = (a1 , a2 ) and B = (b1 , b2 ). Dene the dot product (or scalar product) A B to be a1 b1 + a2 b2 EXAMPLE: A = (1, 3, 2) and B = (1, 4, 3) A B = 1 + 12 + 6 = 17 Properties of scalar products (i) A B = B A (ii) A (B + C) = A B + A C (iii) If x is a number, then (xA) B = x(A B) A (x B) = x(A B) (iv) If A = 0 is the zero vector, then A A = 0, otherwise A A > 0 The norm of a vector is dened as ||A|| = AA

Matrices
2x + y + z 5x y + 7z = 1 = 0

Recall linear equations

where the goal is to solve for x,y and z. We will now consider how to write these equations in terms of vectors and matrices. Let n and m be two integers 1

2.1

Denition

A matrix is an array of numbers with n columns and m rows a11 ... a1n . . .. . . . . . am1 ... amn EXAMPLE is a 2 3 matrix. Each row of the matrix is a vector. The rows are (1,1,-2) and (-1,4,-5). Also, each column of the matrix is a vector. The column vectors are (1,-1), (1,4) and (2,-5). We call the ij cell of a matrix, aij , the ij element or ij component. Matrices are used in economics as a convenient way of representing (and storing) data. For example, 1..m might refer to m individuals and 1..n to n characteristics of these individuals (such as education, years of labor market experience, place of residence, age, race) Suppose we are given the following data on 4 individuals on their years of education, their years of labor market experience, race and gender: 12 20 white male 16 5 Afr. Amer. female 14 10 white male 12 15 Hispanic female A matrix representation of the data is 12 20 1 16 5 2 14 10 1 12 15 3 0 1 0 1 1 1 2 1 4 5

where numbers are used to denote the categorical variables. If m = n then we say that the matrix is a square matrix. There are some special matrices that have been given special names. The zero matrix is a matrix containing all zeros 3

The identity matrix is a matrix that 0s everywhere else 1 0 0

0 . . . 0

... .. . ...

has 1s along the diagonal elements and 0 0 1 0 0 1

0 . . . 0

Let A = (aij ) be an m n matrix. The n m matrix B = (bji ) such that bji = aij is called the transpose of A, also denoted A0 . A matrix that equals its transpose (A = A0 ), is called symmetric. All symmetric matrices must be square. EXAMPLE 2 1 0 A = 1 3 5 2 1 A0 = 1 3 0 5

2.2

Matrix multiplication

Let A = (aij ) be an m n matrix, i = 1..m and j = 1..n. Let B = (bjk ) be a n s matrix, j = 1..n and k = 1..s a11 ... a1n . . .. . A = . . . . am1 ... amn b11 ... b1s . . .. . B = . . . . bn1 ... bns We dene the product AB to be the m s matrix whose ik coordinate is
n X j=1

aij bjk = ai1 b1k + ai2 b2k + ... + ain bnk

In other words, if A1 , ..., Am are the row vectors of the matrix A and B 1 , ..., B S are the column vectors, then the ik-coordinate of the product AB equals Ai B k . A1 B 1 ... A1 B s . . .. . . AB = . . . Am B 1 ... Am B s EXAMPLE #1 AB = EXAMPLE #2 C BC = 2 1 5 1 3 2 3 4 1 2 = 15 15 4 12 2 1

1 3 1 1 3 4 1 5 1 3 = 1 2 = 3 5 1 1 2 1 1 5

2.2.1

Properties of Multiplication

(a) A(B + C) = AB + AC (Distributive Law) (b) (AB)C = A(BC) (Associative Law) (c) (AB)0 = B0A0 NOTE: Commutative Law does not in general hold (AB 6= BA) 2.2.2 Inverse of a Matrix

An inverse for A is a matrix B such that AB = BA = I Note that A must be a square matrix (n n). / Lemma 1 If an inverse exists, there is only one. Proof: Suppose both B and C are inverses. Then AB AC = BA = I = CA = I 5

and BAC = (BA)C = IC and BAC = B(AC) = BI = B so B = C. Since there is only one inverse, denote the inverse of A by A1 . AA1 = I A1 A = I Remark: The transpose of the inverse is the inverse of the transpose ((A )1 = 1 0 (A ) ) Finding the inverse of a matrix can be tedious (especially for large matrices), but computers can do the work for you. EXAMPLE: Solving for the inverse of a matrix 2 3 x11 x12 1 0 = 0 5 0 1 x21 x22 Need to solve the following: 2x11 + 3x21 2x12 + 3x22 0x11 + 5x21 0x12 + 5x22 Get A
1
0

= = = =

1 0 0 1

1/2 3/10 0 1/5

In R, you can nd the inverse of a matrix x using the command solve: y < solve(x)

Representing a system of linear equations

Matrices provide a convenient way of writing linear equations. Suppose we have the set of equations 3x 2y + 3z x + 7y 4z 6 = 1 = 5

where we have two equations and 3 unknowns. We can write this set of equations in matrix notation as x 3 2 3 1 y = 1 7 4 5 z Let A= 3 2 3 1 7 4

and let x = (x, y, z)0 and b = (1, 5)0 . Then we can write the set of equations as A = b. x

Markov Matrices

Suppose there are three cities: L.A., Chicago and Philadelphia. Each year, each city loses some of its population to one of the other cities. That is, 1/4 of the population of LA goes to Phil, 1/7th goes to Chicago. 1/5th of the population of Chicago goes to LA and 1/3 to Philadelphia. 1/6th of the population of Philadelphia goes to LA and 1/8th to Chicago. The total fraction leaving LA each year is 1/4+1/7 = 11/28. Let xn , yn , zn be the population in the 3 cities in year n. Then the population in LA in year n + 1 will be 17 1 1 xn + yn + zn 28 5 6 We similarly obtain equations for Chicago, Philadelphia xn+1 = yn+1 zn+1 or
17 28 1 7 1 4 1 5 7 15 1 3

= =
1 6 1 8 17 24

which we can write as

1 7 1 yn + yn + zn 7 15 8 1 1 17 xn + yn + zn 4 3 24 xn xn+1 yn = yn+1 zn zn+1

An = xn+1 . x If we want to know what population is left after two years xn+2 = An+1 x 2 = A xn

The population that is left after w years is given by Aw xn . 7

References
[1] Serge Lang An Introductory Course in Linear Algebra