Lecture 12: Chain Matrix Multiplication

CLRS Section 15.2 Outline of this Lecture

Recalling matrix multiplication.

The chain matrix multiplication problem.

A dynamic programming algorithm for chain matrix multiplication.

 B  @ @ @ A9    

 4"

 3 3

) ) 10)  & $8

 & 56  " $2

 4"  4"

 & 56  " $2

    

Example: The following is a

!  " 47

) ) 10) ) ) 10)

 &  & ('5!  &  " ('$#!

 "  & %5!     "  " %$#!

Matrix: An dimensional array

which has

. .

Recalling Matrix Multiplication

rows and

. .

matrix

columns.

. .

matrix:

. .

is a two-

2
3 S B " Q R" & @ @ @ 9 I D C I H G D G F I G PF I F H  & E"   3 Q I D D F !

D B  @ 9 I F "

D B I 3 I G " @ 9 H 

D H F " 

  


"

 


"

  

      

   

The product matrix is a

Recalling Matrix Multiplication

of a matrix matrix given by

and a

for

and

then

Example: If

3
S B " Q H @ A9 Q R" F Q R" & Q R"  Q C H C

Remarks on Matrix Multiplication

00) ) )

If

Quite possible that

Multiplication is recursively dened by



Matrix multiplication is associative , e.g.,



so parenthenization does not change result.

) 0) )

is dened,

may not be dened.

Direct Matrix multiplication

Complexity of Direct Matrix multiplication:

Note that has entries and each entry takes time to compute so the total procedure takes time.

for

and

  

      

  

Given a matrix way of multiplying

   

and a matrix , the direct is to compute each

"

 


"

Given a matrix , a matrix and a matrix , then can be computed in two ways and : The number of multiplications needed are:


A big difference! Implication: The multiplication sequence (parenthesization) is important!!

"

When

and

, then

Direct Matrix multiplication of

&

The Chain Matrix Multiplication Problem Given dimensions corresponding to matrix sequence , , , where has dimension , determine the multiplication sequence that minimizes the number of scalar multiplications in computing . That is, determine how to parenthisize the multiplications.

Question: Any better approach?

Exhaustive search:

. Yes DP

S S 01S

 

  

S S 01S

10) ) )

 

Developing a Dynamic Programming Algorithm Step 1: Determine the structure of an optimal solution (in this case, a parenthesization). Decompose the problem into subproblems: For each pair , determine the multiplication sequence for that minimizes the number of multiplications.

Original Problem: determine sequence of multiplication for .

Clearly,

is a

matrix.

10) ) )

 2

"

Developing a Dynamic Programming Algorithm Step 1: Determine the structure of an optimal solution (in this case, a parenthesization). High-Level Parenthesization for For any optimal multiplication sequence, at the last step you are multiplying two matrices and for some . That is,

S

Example
S

Here

01) ) )

01) ) )

Developing a Dynamic Programming Algorithm Step 1 Continued: Thus the problem of determining the optimal sequence of multiplications is broken down into 2 questions:

How do we decide where to split the chain (what is )? (Search all possible values of )
 

How do we parenthesize the subchains and ? (Problem has optimal substructure property that and must be optimal so we can apply the same procedure recursively)

10

Developing a Dynamic Programming Algorithm Step 1 Continued: Optimal Substructure Property: If nal optimal soinvolves splitting into and lution of at nal step then parenthesization of and in nal optimal solution must also be optimal for the subproblems standing alone: If parenthisization of was not optimal we could replace it by a better parenthesization and get a cheaper nal solution, leading to a contradiction. Similarly, if parenthisization of was not optimal we could replace it by a better parenthesization and get a cheaper nal solution, also leading to a contradiction.

11

Developing a Dynamic Programming Algorithm Step 2: Recursively dene the value of an optimal solution. As with the 0-1 knapsack problem, we will store the solutions to the subproblems in an array. For , let denote the minimum number of multiplications needed to compute . The optimum cost can be described by the following recursive denition.

   4

12

"

Developing a Dynamic Programming Algorithm Step 2: Recursively dene the value of an optimal solution.
) 0
13

for some , where the sequences of multiplications for and also are optimal. Hence
S

Proof: Any optimal sequence of multiplication for is equivalent to some choice of splitting

' (

# 

% $ &

  (" 

 

"  !   

 

   

Developing a Dynamic Programming Algorithm Step 2 Continued: We know that, for some

  (" 

We dont know what is, though But, there are only possible values of so we can check them all and nd the one which returns a smallest cost.

3 

14

) 0

Therefore

' (


#  # % $ & #

 

!     

 

   

 &  (  F  A   IB'&$"##!HG1%$1%$%  & (  ( &  (  A     8$E'&%DCBB'&$"##!1%$@1%9%  & (  ( & (  ( &  (      87%6'&542$32'&3210)'&%$##"!

Step 3: Compute the value of an optimal solution in a bottom-up fashion.

we must have already evaluated and For both cases, the corresponding length of the . Hence, the algorithm matrix-chain are both less than should ll the table in increasing order of the length of the matrixchain.

'  #

 #

% $ & #

   !  

Our Table: . only dened for

to calculate

. . .

The important point is that when we use the equation

That is, we calculate in the order

Developing a Dynamic Programming Algorithm

15

 & 8%  &  ( & 8$5PB%

Dynamic Programming Design Warning!! When designing a dynamic programming algorithm there are two parts: 1. Finding an appropriate optimal substructure property and corresponding recurrence relation on table items. Example:

16
 4  

2. Filling in the table properly. This requires nding an ordering of the table elements so that when a table item is calculated using the recurrence relation, all the table values needed by the recurrence relation have already been calculated.


In our example this means that by the time is calculated all of the values and were already calculated.


   ("

 

 

  

       ("

Example for the Bottom-Up Computation Example: Given a chain of four matrices and , with , , , . Find . S0: Initialization

, and

A1 p0 p1

A2 p2

A3 p3

A4 p4

0 7

m[i,j] 2 3 4 i

17

&

 " (2

S Q & "

 & $'

& 6 

 &  (8$"

  " (2
 &  " ('$#

Stp 1: Computing

Example Continued

 "  " 4%(2

 &  " ('$#

120

p0

A1

p1

A2

p2

By denition

A3

p3

m[i,j]

A4

p4

18

S C F

 $ 

 ( $" 

 

& 6

 ( 5  &

Stp 2: Computing

Example Continued

 & $'

& 6

120

48

p0

A1

p1

A2

p2

By denition

A3

p3

m[i,j]

A4

p4

19

 ( 5  &

S C

C 

 $"

C  

 
C   5

Stp3: Computing

Example Continued

 $ 

120

48

By denition

84

p0

A1

p1

A2

p2

A3

p3

m[i,j]

A4

p4

20

C 

 5

 &  " $8$2  4" $2  "

 $ 56  &

$5 

 ( $" 

  " (2
 ( $#  "

Stp4: Computing

Example Continued

S F F

88

120

48

By denition

84

p0 A1

p1 A2

p2 6 A3 0

p3

m[i,j]

A4

p4 7

21
 ( $#  "

 

C C 

C  5

 $ 56  &  &  & $856 & 6 C   & 5

  C  

 $"

 

Stp5: Computing

Example Continued

S C

Q R"

88

By denition

104

120

48

84

p0 A1

p1 A2

p2 6 A3 0

p3

m[i,j]

A4

p4 7

22
C   & 5

 C  

C C C 

C  5  & 5

 ( $#  "  &  " ('$#  "  " %$#

 $"

  " (2
C 

 " $#

St6: Computing

Example Continued

158

S D F

"

) 

88

By denition

104

120

48

84

We are done!

p0

A1

p1

A2

p2

A3

p3

m[i,j]

A4

p4

23

C   " $#

Step 4: Construct an optimal solution from computed information extract the actual sequence.

Idea: Maintain an array , where denotes for the optimal splitting in computing . The array can be used recursively to recover the multiplication sequence.

Do this recursively until the multiplication sequence is determined.

     '  % % !% " !    #      $' % % " % % !

      ' %  %  !%  % ! '  % % %      '  % % ! ' % %

! !

 

How to Recover the Multiplication Sequence?

S S

S S

" 

" 2

 4  

S S

S S

" 7

" #

. . .

Developing a Dynamic Programming Algorithm

. . .

24

Developing a Dynamic Programming Algorithm Step 4: Construct an optimal solution from computed information extract the actual sequence. Example of Finding the Multiplication Sequence: Consider . Assume that the array has been computed. The multiplication sequence is recovered as follows.
 I S S " 7 I S S " 2

Hence the nal multiplication sequence is

S

   

 

"

 ( $#  "

I $#  "

I 

25

The Dynamic Programming Algorithm

for (

to

Complexity: The loops are nested three deep.

26

 

Hence the time complexity is

. Space complexity

'

'

Each loop index takes on

values. .

return

and ; (Optimum in

if (

) ; ;

for (

to

) ;


)

Matrix-Chain( ) for ( to ) for ( to )

The actual multiplication code uses the value to determine how to split the current sequence. Assume that the matrices are stored in an array of matrices , and that is global to this recursive procedure. The procedure returns a matrix.

Mult(

if (

) ; , where ;


  ("

and

else return

To compute

, call Mult(

).
27

 " $7

is now return ; multiply matrices

10) ) )

is now

is

Constructing an Optimal Solution: Compute

 4  

   

   

   

00) ) )

  

   

10) ) )

 

   

S S

" 2

' '

! ' ' ! ' ! ! ! ' ' ' ! '  ! ! ' % ! ! ' ' ! ' ! ! ' ' ' ! '  ! ! ' % ! ! ' ' ! ' ! ! ' '  ! ' % ! ! !' % ! ! ' ! ' '  ! ' % !

'

. Assume Consider the example earlier, where that the array has been computed. The multiplication sequence is recovered as follows.

Constructing an Optimal Solution: Compute

Example of Constructing an Optimal Solution: Compute .

Hence the product is computed as follows


" #

'

! ! ! ! !

S S

S S

" %

Mult Mult Mult Mult Mult

28
S

 

' ' '  '

" "