MA 106 Linear Algebra: Lecture 1

Krishna Kaipa

January 2, 2012

Krishna Kaipa

MA 106 Linear Algebra: Lecture 1

Course Plan

Week 1: Matrix Mechanics. Solving Linear equations Weeks 2, 3, 4: Linear Algebra: What is the Physics/Geometry behind all those Matrix manipulations? Vector spaces and Linear Transformations Week 5: Determinants: What is the geometry behind that algebraic gadget known as determinant? Week 6: Dot Product generalized, Orthogonality Week 7: Eigenvalues, Eigenvectors and Diagonalization of Symmetric Matrices Applications will be interspersed.

Krishna Kaipa

MA 106 Linear Algebra: Lecture 1

Policy for MA 106

As per the Senate rule, the attendance in the rst week is Compulsory; failing to which you will be dis-enrolled from the course. Starting second week, there will be weekly 10 minute quizzes of 10 marks at the beginning of each tutorial. It will have 2 questions; one from tutorial sheet and the second from the material covered in the class during previous week. Mid semester exam will be of 50 marks. The textbook Introduction to linear algebra (2nd edition) by Serge Lang will be followed for the course. Indian edition of this book is available.

3 4

Krishna Kaipa

MA 106 Linear Algebra: Lecture 1

Matrix Terminology
For integers m, n > 0, an m n matrix A is an array of numbers (mostly real, sometimes complex) a11 a12 . . . a1n a21 a22 . . . a2n A= . . . . . . . . ... . am1 am2 . . . amn We write A = (aij ), where the row index i runs from 1 to m, and, the column index j runs from 1 to n. aij itself is called the ij-th entry of A. Sometimes we will use the notation A:,j for the j-th column of A, and Ai,: for the i-th row.

Krishna Kaipa

MA 106 Linear Algebra: Lecture 1

Some special Matrices and Operations

A row vector (x1 , , xn ) is just a 1 n matrix x1 . column vector . is just a n 1 matrix. . xn A square matrix (of order n) is just a n n matrix The zero matrix O (of any size m n) is simply a matrix will all its entries being zero. The sum A + B is dened if A, B have the same size m n. The ij-th entry of A + B is simply the sum of the ij-th entries of A and B. Obviously A + 0 = A for all matrices A Scalar multiple cA for a scalar c and a matrix A, is simply the matrix whose ij-th entry is c times the ij-th entry of A. If c = 1 the matrix obtained is denoted A. Obviously A + (A) = 0
Krishna Kaipa MA 106 Linear Algebra: Lecture 1

The Transpose Operation

The Transpose of an m n matrix A is the matrix whose ij-th entry is the ji-th entry of A. The transpose is denoted as At , and has size n m. Example Let A = 1 3 1 2 2 0 . . Then At = 2 3 0 1 2 1

Clearly the rows/columns of A become the columns/rows of At . Clearly taking the transpose twice does nothing to a matrix . A symmetric matrix is a matrix which equals its transpose. (It has to be a square matrix) Using the transpose notation, a column vector of size n 1 can be written as (x1 , , xn )t
Krishna Kaipa MA 106 Linear Algebra: Lecture 1

Multiplication of Matrices
Recall that the dot product of two column vectors (of length n), X = (x1 , , xn )t and Y = (y1 , , yn )t is the scalar
n

X Y =
i=1

Xi Yi

Given a matrix A of size m n, and a matrix B of size n p we dene the product AB to be the m p matrix whose ij-th entry is the dot product of the i-th column of At with the j-th col. of B. We see that the denition is not symmetric in A and B, so that BA need not equal AB even if they have the same size. Let us put down the formula for the ij-th entry of AB
n

(AB)ij =
k=1

aik bkj .

Krishna Kaipa

MA 106 Linear Algebra: Lecture 1

Some properties of matrix multiplication

1

Writing B = [B1 , B2 , , Bp ] where bi are the coulumns of the n p matrix B. We claim: AB = [AB1 , AB2 , , ABp ] To see this, observe that the ij-the entry of l.h.s is by defn., the dot product of row i of A with Bj . But the same dot product is also the i-th entry of ABj .

We claim that distributive law holds for matrix products: A(B + C ) = AB + AC To prove this we just have to show that the j-th entries of l.h.s and r.h.s agree. Let u be the (transpose of) the i-th row of A, let v , w be the j-th columns of B, C resp. Then, we must show u (v + w ) = u v + u w , and this we borrow from the distributive law for dot products.
Krishna Kaipa MA 106 Linear Algebra: Lecture 1

Getting familiar with matrix multiplication

If X , Y are column vectors of length n, then X Y = X t Y where l.h.s is the dot product, and r.h.s a matrix product Let ei denote the column vector (of length n) having a 1 for the i-th component and zeros elsewhere. We will frequently refer to {e1 , e2 , , en } as the standard basic vectors of Rn Let A be an m n matrix. How is Aej more familiar Answer: Aej = A:,j = j-th column of A. Work it out! Similarly, if {f1 , , fm } are the standard basic vectors of Rm , then fi t A = Ai,: = i-th row of A. Let x = (x1 , , xn )t and y = (y1 , , ym )t be column vectors. Prove (using x = n xi ei and distributive law) i=1 Ax = x1 A:,1 + x2 A:,2 + + xn A:,n

y t A = y1 A1,: + y2 A2,: + + ym Am,:

Krishna Kaipa MA 106 Linear Algebra: Lecture 1

Examples
Let A = AB = 2 1 1 0 3 1 1 6 0 2 and B = 2 1 1 2. Then 2 0 1 1

2 11 2 1 . (and BA cannot be dened) 8 3 2 5 Consider a system of two linear equations in three unknowns 2x + y z 3y + z
2 0 1 3

= 1 = 5
1 1 1 5

we can rewrite this as a vector equation: x+ y+ z=

which we can cast as a matrix equation A x = b where A matrix is as in example above, the vector x = (x, y , z)t and b = (1, 5)t . It is clear that m linear equations in n unknowns can be cast as A x = b where A is m n and x and b are column vectors of length n and m, resp.
Krishna Kaipa MA 106 Linear Algebra: Lecture 1

Example of Rotation matrices

For each 0 < 2, let R() = cos sin . sin cos

Question Take a vector v = (x, y )t R2 . How does v R() v transform v ? [Also answer: what is the product R() R() ?] Answer It rotates v by radians anti-clockwise. One way to see this, is to use complex notation. (If you dont want complex notation, simply use polar coordinates) Set z = x + y , then the vector x cos sin R() =x +y y sin cos which in complex notation is xe + y e = ze Now, well do something weird. We will let the rotation angle become imaginary, set = where = 1 and is real. We make sense out of this as follows. Fact: The power series e z = 1 + z + z 2 /2! + z 3 /3! + converges for all complex numbers. For a real number , Eulers formula is:
Krishna Kaipa MA 106 Linear Algebra: Lecture 1

Rotation by imaginary angle

cos = (e + e )/2 sin = (e e )/(2) Now, we simply remove the restriction that is real, in order to dene cos and sin for all complex numbers. In particular cos = (e + e )/2 =: cosh sin = (e e )/(2) =: sinh
sinh Let = tanh = cosh . Now, recall/check that cosh2 sinh2 = 1. Since cosh > 0, we get can express cosh, sinh in terms of tanh as: cosh = 1/ 1 2 and sinh = / 1 2 . Using this we get:

x y

= x cosh y sinh = (x y )/ = x sinh + y cosh = (x + y )/

Krishna Kaipa

1 2 1 2

MA 106 Linear Algebra: Lecture 1

Rotation by imaginary angle continued

Now, assume we have just one spatial dimension namely x. Suppose, (x, t) are the coordinates of an event in an inertial frame of reference, and there is another frame of ref. (denoted primed frame) moving w.r.t the former frame with uniform speed v . Then the coordinates (x , t ) of an event in the primed frame are related to (x, t) by a Lorentz transformation. If we set y = ct and = v /c the above equations are just the Lorentz transformation. x t = (x vt)/ 1 (v /c)2

= (vx/c 2 + t)/ 1 (v /c)2

Krishna Kaipa

MA 106 Linear Algebra: Lecture 1

Markov matrices and an Introduction to Googles Pagerank Algorithm

A probability vector is a vector (x1 , , xn )t with xi 0 and x1 + + xn = 1. A Markov matrix is an n n matrix all whose columns are probability vectors. Exercise Prove that x Ax sends the set of probability vectors to itself (Hint: A vector x with xi 0 is a probability vector i its dot product with (1, 1, , 1)t is 1.) A Markov Chain consists of iterating an initial probability vector v0 to obtain a sequence vn = An v0 . They nd wide application in Physics, Chemistry, Computer Science, Information sciences, Operations Research... Fact about Markov matrices with positive entries: There is a unique probability vector v such that Av = v . Moreover, for any initial probability vector v0 , limk Ak v0 is v (For n = 2, we will prove this fact in Tutorial Sheet 1. For general n, one has to use a theorem from Linear Algebra called Perron-Frobenius theorem.)
Krishna Kaipa MA 106 Linear Algebra: Lecture 1

Basic idea of Googles Pagerank Algorithm:

Robot web crawlers from Google index and catalogue webpages. When you query something on Google, it selects the relevant webpages, and displays them according to their ranking. The ranking is computed by the Pagerank Algorithm created by Google founders Sergey Brin and Larry Page, during their Ph.D. at C.S. dept. of Stanford Univ. Let n 3 109 be the number of webpages on WWW. The Google matrix G will be an n n Markov matrix. Let Out(j) = {i | page j links to page i} Let us introduce RS (for Random Surfer), who is an innitely dedicated and compulsive surfer. When on page j, RS clicks on any one of the outlinks of j with equal likelihood. More realistically RS does that 85% of the time, the remaining 15% of time, RS is bored with the outlinks of j and picks a webpage randomly. If Out(j) is empty, RS picks a webpage randomly. Let gij = Pr(j i), the probability that RS visits page i immediately after page j.
Krishna Kaipa MA 106 Linear Algebra: Lecture 1

continued
Problem: Determine gij and prove G is Markov (with all gij > 0). Answer: If Out(j) = , then gij = 1/n. If Out(j) = , and i Out(j), then gij = 0.85/#Out(j) + 0.15/n If Out(j) = , and if i Out(j) then gij = 0.15/n. / Check that each column sum is 1. Question What is the probability that RS lands at page i after 2 clicks from page j. Generalize for 2 replaced with m Answer: ij-th entry of G 2 , G m . To see this, we observe that the desired probability. is the sum over all pages k,of the probability of going from j k times, the probability of going from k i. In 2 other words n gik gkj (which is Gij ) k=1 The j-th column of G m is just G m ej . As m , the fact on previous frame implies that all columns of G will converge to a xed v .It follows that the probability of reaching page i after m clicks is virtually (v )i for all suciently large m (independent of the starting page).
Krishna Kaipa MA 106 Linear Algebra: Lecture 1

continued
The pagerank of page i is dened to be (v )i Interpretation: Pick m very large such that the columns of G m are virtually equal to v . Let RS start surng. Consider the n pages RS visits between the m-th and m + n-th clicks, the fraction of visits to page i will tend to v (i) as n (intuitively believable, but also provable by using a theorem in probability theory called the law of large numbers). It is reasonable to say that page i has higher rank if this fraction is large. Alternatively: If we set a large number of random surfers (with identical strategy) to task, then the fraction of these surfers who will be at page i after m clicks (for all m suciently large) is virtually (v )i . (intuitively believable by the relative frequency interpretation of probability, and provable by law of large numbers). The larger this fraction, the more important we consider page i to be. Hence using this as page ranking is reasonable.
Krishna Kaipa MA 106 Linear Algebra: Lecture 1