Professional Documents
Culture Documents
2
1 2
W = 20
9 10
3 4
3 4
1
0-1 Knapsack problem: Brute-force
0-1 Knapsack problem approach
Problem is to find
Since there are n items, there are 2n possible
max bi subject to wi W combinations of items.
iT iT We go through all combinations and find the one with the
most total value and with total weight less or equal to W
Running time will be O(2n)
5 6
5 6
Defining a Subproblem
0-1 Knapsack problem: Brute-force approach
If items are labeled 1..n, then a subproblem would be to
If items are labeled 1..n, then a subproblem
find an optimal solution for Sk = {items labeled 1, 2, .. k}
would be to find an optimal solution for
This is a valid subproblem definition.
Sk = {items labeled 1, 2, .. k}
Can we describe the final solution (Sn ) in terms of
subproblems (Sk)?
7 8
7 8
2
Defining a Subproblem
w1 =2 w2 =4 w3 =5 w4 =3
Weight Benefit Defining a Subproblem (continued)
b1 =3 b2 =5 b3 =8 b4 =4 Item w bi
# The solution for S4 is not part of the solution for S5
Max weight: W = 20 1 i2 3
For S4:
S4
Definition of a subproblem is flawedneed another
Total weight: 14; 2 3 4
total benefit: 20
S5
one.
3 4 5
Let’s add another parameter: w, which will represent the
4 5 8
exact weight for each subset of items
w1 =2 w2 =4 w3 =5 w4 =9 5 9 10 The subproblem then will be to compute B[k,w]
b1 =3 b2 =5 b3 =8 b4 =10
Solution for S4 is
For S5:
Total weight: 20 not part of the
total benefit: 26
solution for S5
9 10
9 10
3
Example Example (2)
i\W 0 1 2 3 4 5
Let’s run our algorithm on the 0 0 0 0 0 0 0
following data: 1
2
3
n = 4 (# of elements) 4
W = 5 (max weight)
Elements (weight, benefit): for w = 0 to W
V[0,w] = 0
(2,3), (3,4), (4,5), (5,6)
13 14
Items:
1: (2,3)
Example (3) Example (4) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 0 0 0 0 0 0 0 bi=3
1 0 1 0 0
wi=2
2 0 2 0
w=1
3 0 3 0
w-wi =-1
4 0 4 0
if wi <= w // item i can be part of the solution
for i = 1 to n if bi + V[i-1,w-wi] > V[i-1,w]
V[i,0] = 0 V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
15 16
4
Items: Items:
1: (2,3) 1: (2,3)
Example (5) 2: (3,4) Example (6) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=1 i\W 0 1 2 3 4 5 i=1
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 bi=3 0 0 0 0 0 0 0 bi=3
1 0 0 3 1 0 0 3 3
wi=2 wi=2
2 0 2 0
w=2 w=3
3 0 3 0
w-wi =0 w-wi =1
4 0 4 0
if wi <= w // item i can be part of the solution if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w] if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi] V[i,w] = bi + V[i-1,w- wi]
else else
V[i,w] = V[i-1,w] V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w else V[i,w] = V[i-1,w] // wi > w
17 18
Items: Items:
1: (2,3) 1: (2,3)
Example (7) 2: (3,4) Example (8) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=1 i\W 0 1 2 3 4 5 i=1
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 bi=3 0 0 0 0 0 0 0 bi=3
1 0 0 3 3 3 1 0 0 3 3 3 3
wi=2 wi=2
2 0 2 0
w=4 w=5
3 0 3 0
w-wi =2 w-wi =3
4 0 4 0
if wi <= w // item i can be part of the solution if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w] if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi] V[i,w] = bi + V[i-1,w- wi]
else else
V[i,w] = V[i-1,w] V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w else V[i,w] = V[i-1,w] // wi > w
19 20
5
Items: Items:
1: (2,3) 1: (2,3)
Example (9) 2: (3,4) Example (10) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=2 i\W 0 1 2 3 4 5 i=2
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 bi=4 0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3 1 0 0 3 3 3 3
wi=3 wi=3
2 0 0 2 0 0 3
w=1 w=2
3 0 3 0
w-wi =-2 w-wi =-1
4 0 4 0
if wi <= w // item i can be part of the solution if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w] if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi] V[i,w] = bi + V[i-1,w- wi]
else else
V[i,w] = V[i-1,w] V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w else V[i,w] = V[i-1,w] // wi > w
21 22
Items: Items:
1: (2,3) 1: (2,3)
Example (11) 2: (3,4) Example (12) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=2 i\W 0 1 2 3 4 5 i=2
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 bi=4 0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3 1 0 0 3 3 3 3
wi=3 wi=3
2 0 0 3 4 2 0 0 3 4 4
w=3 w=4
3 0 3 0
w-wi =0 w-wi =1
4 0 4 0
if wi <= w // item i can be part of the solution if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w] if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi] V[i,w] = bi + V[i-1,w- wi]
else else
V[i,w] = V[i-1,w] V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w else V[i,w] = V[i-1,w] // wi > w
23 24
6
Items: Items:
1: (2,3) 1: (2,3)
Example (13) 2: (3,4) Example (14) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=2 i\W 0 1 2 3 4 5 i=3
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 bi=4 0 0 0 0 0 0 0 bi=5
1 0 0 3 3 3 3 1 0 0 3 3 3 3
wi=3 wi=4
2 0 0 3 4 4 7 2 0 0 3 4 4 7
w=5 w= 1..3
3 0 3 0 0 3 4
w-wi =2
4 0 4 0
if wi <= w // item i can be part of the solution if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w] if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi] V[i,w] = bi + V[i-1,w- wi]
else else
V[i,w] = V[i-1,w] V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w else V[i,w] = V[i-1,w] // wi > w
25 26
Items: Items:
1: (2,3) 1: (2,3)
Example (15) 2: (3,4) Example (16) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=3 i\W 0 1 2 3 4 5 i=3
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 bi=5 0 0 0 0 0 0 0 bi=5
1 0 0 3 3 3 3 1 0 0 3 3 3 3
wi=4 wi=4
2 0 0 3 4 4 7 2 0 0 3 4 4 7
w= 4 w= 5
3 0 0 3 4 5 3 0 0 3 4 5 7
w- wi=0 w- wi=1
4 0 4 0
if wi <= w // item i can be part of the solution if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w] if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi] V[i,w] = bi + V[i-1,w- wi]
else else
V[i,w] = V[i-1,w] V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w else V[i,w] = V[i-1,w] // wi > w
27 28
7
Items: Items:
1: (2,3) 1: (2,3)
Example (17) 2: (3,4) Example (18) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=4 i\W 0 1 2 3 4 5 i=4
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 bi=6 0 0 0 0 0 0 0 bi=6
1 0 0 3 3 3 3 1 0 0 3 3 3 3
wi=5 wi=5
2 0 0 3 4 4 7 2 0 0 3 4 4 7
w= 1..4 w= 5
3 0 0 3 4 5 7 3 0 0 3 4 5 7
w- wi=0
4 0 0 3 4 5 4 0 0 3 4 5 7
if wi <= w // item i can be part of the solution if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w] if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi] V[i,w] = bi + V[i-1,w- wi]
else else
V[i,w] = V[i-1,w] V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w else V[i,w] = V[i-1,w] // wi > w
29 30
8
How to find actual Knapsack
Items
All of the information we need is in the table.
V[n,W] is the maximal value of items that can be
placed in the Knapsack.
Let i=n and k=W
if V[i,k] V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 // Assume the ith item is not in the knapsack
// Could it be in the optimally packed knapsack?
33 34
Items: Items:
1: (2,3) 1: (2,3)
Finding the Items 2: (3,4) Finding the Items (2) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=4 i\W 0 1 2 3 4 5 i=4
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 k= 5 0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6 1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5 2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7 3 0 0 3 4 5 7 V[i,k] = 7
V[i1,k] =7 V[i1,k] =7
4 0 0 3 4 5 7 4 0 0 3 4 5 7
i=n, k=W i=n, k=W
while i,k > 0 while i,k > 0
if V[i,k] V[i1,k] then if V[i,k] V[i1,k] then
mark the ith item as in the knapsack mark the ith item as in the knapsack
i = i1, k = k-wi i = i1, k = k-wi
else else
i = i1 35
i = i1 36
9
Items: Items:
1: (2,3) 1: (2,3)
Finding the Items (3) 2: (3,4) Finding the Items (4) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=3 i\W 0 1 2 3 4 5 i=2
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 k= 5 0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=5 1 0 0 3 3 3 3 bi=4
2 0 0 3 4 4 7 wi=4 2 0 0 3 4 4 7 wi=3
3 0 0 3 4 5 7 V[i,k] = 7 3 0 0 3 4 5 7 V[i,k] = 7
V[i1,k] =7 V[i1,k] =3
4 0 0 3 4 5 7 4 0 0 3 4 5 7
k wi=2
i=n, k=W i=n, k=W
while i,k > 0 while i,k > 0
if V[i,k] V[i1,k] then if V[i,k] V[i1,k] then
mark the ith item as in the knapsack mark the ith item as in the knapsack
i = i1, k = k-wi i = i1, k = k-wi
else else
i = i1 37
i = i1 38
Items: Items:
1: (2,3) 1: (2,3)
Finding the Items (5) 2: (3,4) Finding the Items (6) 2: (3,4)
3: (4,5) 3: (4,5)
i\W 0 1 2 3 4 5 i=1 i\W 0 1 2 3 4 5 i=0
4: (5,6) 4: (5,6)
0 0 0 0 0 0 0 k= 2 0 0 0 0 0 0 0 k= 0
1 0 0 3 3 3 3 bi=3 1 0 0 3 3 3 3
2 0 0 3 4 4 7 wi=2 2 0 0 3 4 4 7
3 0 0 3 4 5 7 V[i,k] = 3 3 0 0 3 4 5 7 The optimal
V[i1,k] =0 knapsack
4 0 0 3 4 5 7 4 0 0 3 4 5 7
k wi=0 should contain
i=n, k=W i=n, k=W {1, 2}
while i,k > 0 while i,k > 0
if V[i,k] V[i1,k] then if V[i,k] V[i1,k] then
mark the ith item as in the knapsack mark the nth item as in the knapsack
i = i1, k = k-wi i = i1, k = k-wi
else else
i = i1 39
i = i1 40
10
Items:
1: (2,3)
Finding the Items (7) 2: (3,4) Memorization (Memory Function Method)
3: (4,5)
i\W Goal:
0 1 2 3 4 5 4: (5,6)
0 0 0 0 0 0 0 » Solve only subproblems that are necessary and solve it only once
Memorization is another way to deal with overlapping subproblems
1 0 0 3 3 3 3
in dynamic programming
2 0 0 3 4 4 7
With memorization, we implement the algorithm recursively:
3 0 0 3 4 5 7 The optimal
» If we encounter a new subproblem, we compute and store the solution.
knapsack
4 0 0 3 4 5 7 » If we encounter a subproblem we have seen, we look up the answer
should contain
i=n, k=W {1, 2} Most useful when the algorithm is easiest to implement recursively
while i,k > 0 » Especially if we do not need solutions to all subproblems.
if V[i,k] V[i1,k] then
mark the nth
item as in the knapsack
i = i1, k = k-wi
else
i = i1 41 42
43 44
11