CS 306 DAA, IIITDM Jabalpur

11/7/2012

Dynamic Programming - II
Atul Gupta

DP: Finding Sub-Problems

Finding the right sub-problems takes creativity and experimentation Standard choices

CS 306 DAA, IIITDM Jabalpur

11/7/2012

DP: Finding Sub-Problems

Standard Choice #1
The input is x1, x2 , . . . , xn and a sub-problem is x1 , x2, . . . , xi

The number of sub-problems are linear Ex: LIS

DP: Finding Sub-Problems

Standard Choice #2
The input is x1, . . . , xn , and y1, . . . , ym. A subproblem is x1, . . . , xi and y1, . . . , yj

The number of sub-problems are (m*n) Ex: Edit Distance

CS 306 DAA, IIITDM Jabalpur

11/7/2012

DP: Finding Sub-Problems

Standard Choice #3
The input is x1, . . . , xn and a sub-problem is xi, xi+1, . . . , xj

The number of sub-problems are (n2)

DP: Finding Sub-Problems

Standard Choice #4
The input is a rooted tree. A sub-problem is a rooted sub-tree

The number of sub-problems are (?)

CS 306 DAA, IIITDM Jabalpur

11/7/2012

Knapsack Problem
Also referred as 0/1 Knapsack There are n items to pick from, of weight w1, . . . , wn and dollar value v1, . . . , vn. Find the optimal solution to fill the knapsack of size W, such that
max subjected to xi = 1 or 0 =< W

Knapsack Problem
There are two versions of this problem
Unlimited quantities of each item Fix quantities

CS 306 DAA, IIITDM Jabalpur

11/7/2012

Knapsack with Repetitions

What are the sub-problems?
Option 1: look at smaller knapsack capacities w W, or Option 2: look at fewer items (for instance, items 1, 2, . . . , j, for j n)

Consider the first option

K (w) = maximum value achievable with a knapsack of capacity w. How to express K(w) into smaller problems?

Knapsack with Repetitions

Lets assume that the optimal solution includes an item i, then
K(w) = , for some i, i.e.

CS 306 DAA, IIITDM Jabalpur

11/7/2012

Knapsack with Repetitions

The Algorithm

The Complexity of the Algorithm

O(n . W) Polynomial? NO !

Matrix Chain Multiplication

CS 306 DAA, IIITDM Jabalpur

11/7/2012

Matrix Chain Multiplication

Matrix Chain Multiplication

How many such trees are possible?

All full binary trees with n leaves !!!

CS 306 DAA, IIITDM Jabalpur

11/7/2012

Matrix Chain Multiplication

A DP solution
For the full binary tree to be optimal, its sub-trees must also be optimal

What are sub-problems?

The internal nodes They are the products of the form

MCM: A DP Solution
The DP relation The Algorithm

CS 306 DAA, IIITDM Jabalpur

11/7/2012

Shortest Paths
All pair shortest path(APSP)
Dijkestras algorithm can be used for all vertices Time complexity of Dijketras algorithm is O(|V| |E|) (Worst case is O(n3) considering a complete graph of n nodes) Therefore, the time complexity of APSP will be O((|V|2 |E|) (Worst case is O(n4)) Can we do it better?

APSP: Yes, We Can

A DP Solution - The Idea:
The shortest path between (u, v) may go through some intermediate vertices, possibly none So, we start with zero intermediate nodes, and gradually allow to include other nodes in a sequence

CS 306 DAA, IIITDM Jabalpur

11/7/2012

APSP: Yes, We Can

A DP Solution
Sub-problems: Let D(i, j, k) represent shortest distance between I and j using (1..k) number of nodes For, the recursive computation, we can think of a lower order node k can be included in the path or not iff

APSP The DP Solution

The Algorithm
for i = 1 to n: for j = 1 to n: dist(i, j, 0) = for all (i, j) E : dist(i, j, 0) = w (i, j) for k = 1 to n: for i = 1 to n: for j = 1 to n: dist(i, j, k) = min dist(i, k, k 1) + dist(k, j, k 1), dist(i, j, k 1)

The Time Complexity?

O(|V|3)

10

CS 306 DAA, IIITDM Jabalpur

11/7/2012

Travelling Salesperson (TSP)

Another famous hard problem (NP-C) Nave complexity is O(Factorial (n-1)) A DP solution (still exponential) is follows

TSP
A DP Solution
Sub-Problems? let C (S, j) be the length of the shortest path visiting each node in S exactly once, starting at 1 and ending at j For |S| > 1, we define

11

CS 306 DAA, IIITDM Jabalpur

11/7/2012

Travelling Salesperson A DP Solution

The Algorithm

Time Complexity?

Summary: Dynamic Programming

A technique to optimize the Backtrack search Two important considerations
What are the sub-problems How the solution of a larger sub-problem is expressed in terms of solutions of the lower-order sub-problems The Recursive Relation

The arrangements of all sub-problems can be thought of various nodes of a DAG, and solution lies of computing some properties of this DAG

12

CS 306 DAA, IIITDM Jabalpur

11/7/2012

References
Chapter 6, Algorithms by S. Dasgupta, C.H. Papadimitriou, and U.V. Vazirani

13