Coding The Matrix

Coding the Matrix!
Divya Padmanabhan
Intelligent Systems Lab,
Dept. of CSA,
Indian Institute of Science,
Bangalore
June 28, 2013
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
How does the most popular search engine work?
Google it!
Search more than 8 billion pages ..
Results in less than a second..
How is it possible???
What is Google doing in that 1 second?
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
How does Facebook manage information about you?
Friend network matrix

U Un
1
U1
U2
..
.
Un
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
How does Flipkart recommend products for you?
How does Flipkart know what you require?

How does Flipkart keep track of so many users and find their
interests?
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Some BASICS!!!
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Preliminaries
Vectors
eg. How well can you sing? paint? play? ..

Professional singer : 1
. Amateur : 0.1
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Preliminaries
Vectors
eg. How well can you sing? paint? play? ..

Professional singer : 1
. Amateur : 0.1
q
Norm |~x | = x12 + x22 + + xn2
Dot product ~x .~y = |~x ||~y | cos = x1 y1 + x2 y2 + + xn yn

Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Orthogonality and Projections

Figure: Orthogonal vectors ~x T ~y = ||x||||y || cos = 0
~x
~y
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Orthogonality and Projections

Figure: Orthogonal vectors ~x T ~y = ||x||||y || cos = 0
~x
~y
Figure: Projection of ~x on ~y =
~
x .~
y
||~
y ||
~x
Z
O
Divya Padmanabhan
~y
~x.~y
||y||
Coding the Matrix!
June 28, 2013
Linear Combination and Independence
Linear Combination
c1~x + c2~y , c1 R, c2 R
Linear Independence of vectors ~x and ~y

c1~x + c2~y = 0 c1 = 0, c2 = 0
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Linear Combination
c1~x + c2~y , c1 R, c2 R

c1~x + c2~y = 0 c1 = 0, c2 = 0
Cant express ~x in terms of ~y .
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Linear Combination
c1~x + c2~y , c1 R, c2 R

c1~x + c2~y = 0 c1 = 0, c2 = 0
Cant express ~x in terms of ~y .

1
2
,
, c1 = 0, c2 = 0
Eg.
0
3
Linearly
dependent :
1
2
,
, c1 = 2, c2 = 1
2
4
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Matrix Multiplication
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Matrix Multiplication
z }| { z}|{
z }| {
2 3 1
8
=
4 5 2
14

2
3
8
+2
=
1
4
5
14
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
Row and column vectors
2 1 0
A=
3 1 4
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
10

2 1 0
A=
3 1 4

2
3
Row vectors = r 1 = 1 , r 2 = 1
0
4
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
10

2 1 0
A=
3 1 4

2
3
Row vectors = r 1 = 1 , r 2 = 1
0
4

2
1
0
Column vectors = c1 =
, c2 =
, c3 =
3
1
4
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
10
Fundamental spaces
Row space and column space of A

Row Space: Linear combination of r 1, r 2.
Column space : Linear combination of c1, c2, c3
Null space: Vectors orthogonal to row space of A.
Row Space (A) = Col space (AT )
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
11
Representatives of a space
Basis and dimensions of a space
Basis : Smallest no. of linearly independent vectors whose linear
combination gives the entire space.

2
1
0
= 2
+1
1
0
1
Dimension : No. of vectors in the basis.
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
12
Redundancy elimination
Rank and nullity of matrix A

Rank(A)= no. of linearly independent column vectors of A.
Nullity(A) = dimension of nullspace of A.
Rank measures redundancy in the matrix.
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
13


1 0 4 28 15
0 1 1 2 6
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
13


1 0 4 28 15
0 1 1 2 6
Rank-Nullity Theorem: Rank(A) + Nullity(A) = n
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
13
Eigen values and eigen vectors

Publi
Speaking
0.8
1
Resear h
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
14

Publi
Speaking
A22
0.8
1
Resear h
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
14

Publi
Publi
Speaking
Speaking
Ax = x
A22
0.8
1.6
0.8
Divya Padmanabhan
2
Resear h
Resear h
Coding the Matrix!
June 28, 2013
14

Publi
Publi
Speaking
Speaking
Ax = x
A22
0.8
1.6
0.8
2
Resear h
Resear h
How do you determine the x and given a matrix A?

Ax = x, x 6= ~0 ( or x T AT = x T , x 6= ~0 )
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
14

Publi
Publi
Speaking
Speaking
Ax = x
A22
0.8
1.6
0.8
2
Resear h
Resear h

Ax = x, x 6= ~0 ( or x T AT = x T , x 6= ~0 )
(A I )x = 0
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
14

Publi
Publi
Speaking
Speaking
Ax = x
A22
0.8
1.6
0.8
2
Resear h
Resear h

Ax = x, x 6= ~0 ( or x T AT = x T , x 6= ~0 )
(A I )x = 0
det(A I ) = 0
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
14

Publi
Publi
Speaking
Speaking
Ax = x
A22
0.8
1.6
0.8
2
Resear h
Resear h

Ax = x, x 6= ~0 ( or x T AT = x T , x 6= ~0 )
(A I )x = 0
det(A I ) = 0
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
14
Now for the FUN stuff!!!
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
15
A Typical Search Activity : Random surfer
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
16
A Typical Search Activity : Random surfer
Jump
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
16
Importance of pages
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
17
Importance of pages
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
17
Page Rank Algorithm
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
18
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Divya Padmanabhan
Coding the Matrix!
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0
June 28, 2013
18
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0
Anything particular about H?
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
18
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

H : row stochastic
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
18
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

H : row stochastic
Vt (A) : Probability of being
t
at page A at time

Vt : row vector. eg. 0.1 0.3
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
18
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

H : row stochastic
t
at page A at time

Vt (A) = Vt1 (A)HAA + Vt1 (B)HBA + . . .
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
18
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

H : row stochastic
t
at page A at time

Vt (A) = Vt1 (A)HAA + Vt1 (B)HBA + . . .
Vt = Vt1 H
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
18
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0
H : row stochastic, guaranteed to have eigen value 1, under some

conditions.

1/4
1/4
V0 =
1/4,
1/4
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
19
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

conditions.

1/4
9/24
1/4
5/24
V0 =
1/4, V1 = V0 H = 5/24 ,
1/4
5/24
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
19
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

conditions.

1/4
9/24
15/48
1/4
5/24
11/48
V0 =
1/4, V1 = V0 H = 5/24 , V2 = V1 H = 11/48
1/4
5/24
11/48
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
19
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

conditions.
..
.
Vn = Vn1 H
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
20
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

conditions.
..
.
Vn = Vn1 H
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
20
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

conditions.
..
.
Vn = Vn1 H
..
.
vH = 1v
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
20
Page Rank Algorithm

Transition
A B
A 0 13
1
B
2 0
C1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0

conditions.
..
.
Vn = Vn1 H
..
.
vH = 1v
v is the eigen vector of H corresponding to eigen value 1 !!!
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
20
Page Rank Algorithm - conditions on H

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Divya Padmanabhan
Coding the Matrix!
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0
June 28, 2013
21

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0
Row stochastic
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
21

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0
Row stochastic
Irreducible :Possible to reach a page from any other page.
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
21

Transition
A B
A 0 13
1
B
2 0
C 1 0
D 0 12
Matrix H
C D
1
1
3
0
0
1
2
3
1
2
0
0
Row stochastic
Irreducible :Possible to reach a page from any other page.
Aperiodic : Not periodic
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
21
Page Rank Algorithm - Dangling nodes

Transition
A B
A 0 13
1
B
21 0
C2 0
D 0 0
Divya Padmanabhan
Coding the Matrix!
Matrix H
C D
1
1
3
0
0
0
3
1
2
1
2
June 28, 2013
22
Page Rank Algorithm - Dangling nodes

Transition
A B
A 0 13
1
B
21 0
C2 0
D 0 0
Matrix H
C D
1
1
3
0
0
0
3
1
2
1
2
S =H +Y
A
C
D
Divya Padmanabhan
1
3
1
3
1
2
1
2
1
4
1
4
1
4
1
3
1
2
1
2
1
4
Coding the Matrix!
June 28, 2013
22
Page Rank Algorithm - Entering a new destination
A
B
S=
C
D
A
0
1
2
1
2
1
4
1
3
1
3
0
0
0
0
1
4
1
4
1
3
1
2
1
2
1
4
1/n
B:
1/n
1/n
1/n
nn
Google matrix!
Surfer follows the link structure 60 % of time
Otherwise to a random page!
G = S + (1 )B
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
23
Page Rank Algorithm
Search Engines
Get pages containing query words.
Use page rank to determine the order of pages
to be displayed.
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
24
Page Rank Algorithm
Search Engines
Get pages containing query words.
Use page rank to determine the order of pages
to be displayed.
Ford
0.1
0.7
Maruti
Volkswagen 0.05
Hyundai
0.15
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
24
Recommender Systems
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
25
Recommender Systems
Matrix Factorization
Original matrix Xdn (d users, n movies)
= Mdk Hkn
X X
Find M, H.
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
26
References
Gilbert Strang, Linear Algebra and its Applications, Fourth Edition.
Langville and Meyer,Googles PageRank and Beyond.
Gene H. Golub, Charles F. Van Loan, Matrix Computations.
Howard Anton, Chris Rorres, Elementary Linear Algebra.
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
27
THANK YOU!
Divya Padmanabhan
Coding the Matrix!
June 28, 2013
28

Coding The Matrix

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Coding The Matrix

Uploaded by

Copyright:

Available Formats

Coding the Matrix!

June 28, 2013