You are on page 1of 49

0/1 knapsack problem

The subset-sum problem


knapsack problem
0/1 knapsack problem
Given a instance I of items {1,2,,n}, item i has weight
w
i
and value v
i
. the knapsack can hold items of total
weight up to W.
Knapsack problem is to choose the subset of items with
maximum value of total weight W.
0/1 knapsack problem:
Assume that all weights are integers.
Items cannot be taken partially.
Greedy strategy:
1. picking the next items with maximum
weight.
2. picking the next items with maximum
value.
3. picking the next items with maximum
value/weight ratio
The optimal solution cannot be guaranteed
by above strategies.
Dynamic Programming based on
maximum value
1. List the items in any order.
2. We consider subproblems of the
following form: V[i, w]: maximum
value of items in the set {1,2,,i} that
fit in a knapsack of weight w.
3.We want to compute V[n,W].
The recursive formula for V[i, w].

s +
=
otherwise w i V
w w v w w i V w i V
w i V
i i
] , 1 [
} ] , 1 [ ], , 1 [ max{
] , [

s
=
otherwise
w w v
w V
0
] , 1 [
1 1
DPWKnapsack (I ,W)
1 f or w 0 to w
1
- 1 do V[1, w] 0
2 for w w
1
to W do V[1, w] v
1

3 for i 2 to n do
4 for w 0 to W do
5 if w
i
> w then
6 V[i,w] V[i-1, w]
7 else
8 V[i,w] max{V[i-1, w], V[i-1,w-w
i
]+v
i
}
Analysis
The algorithms runs in time O(nW) if I has n items.
It produces optimal solution.
The algorithm is not strictly polynomial.
Its running time depends on magnitudes of item
weights and profits.
A poly algorithms time should depend only on
the number of items, not their magnitudes.
DPWKnapsack (I ,W) is called pseudo-polynomial.
Dynamic programming based on
minimum weight
1. List the items in any order.
2. We consider subproblems of the following form: W[i,
v]: minimum weight of items in the set {1,2,,i} that fit
in a knapsack of value v.
3. We want to compute W[n,V], where
.
1

=
=
n
i
i
v V
The recursive formula for W[i, v]:





At the end, the maximum value with weight less than or
equal to W can be found as
Maximum value = max{v|W[n,v]W}.

s +
=
otherwise v i W
v v w v v i W v i W
v i W
i i i
] , 1 [
} ] , 1 [ ], , 1 [ min{
] , [

=
=
otherwise
v v w
v W
1 1
] , 1 [
DPVKnapsack (I ,V)
1 for v 0 to V do W[1, v]
2 W[1, v
1
] w
1

3 for i 2 to n do
4 for v 0 to V do
5 if v
i
> v then
6 W[i,v] W[i-1, v]
7 else
8 W[i,v] min{W[i-1, v], W[i-1,v-v
i
]+w
i
}
Analysis
The algorithms runs in time O(nV) if I has n
items
It produces optimal solution.
The algorithm is not strictly polynomial.
A poly algorithms time should depend only
on the number of items, not their
magnitudes.
Poly-time Approximation algorithm
Sort the items in descending order of value/weight.
Assume v
1
/w
1
v
2
/w
2
v
n
/w
n
Scan the items in this order, taking each item if
there is still room in the knapsack.
Quit when we cant take an item.
Let A
G
(I) denote the value produced by this greedy
algorithm, OPT(I)

denote the optimal value.
Approximation Ratio
Unfortunately, this approximation ratio is unbounded.
Consider two items{1,2}: (1, 1.01), and (W, W).
The greedy will take only item 1, while the
optimal will take the second item.
The ratio is W/1.01, which can be made
arbitrarily large.
Modified Greedy strategy
Main idea:
Scan the items in sorted order, taking each item if there
is still room in the knapsack.
When the next item doesnt fit, output the current
knapsack solution A
G
(I) .
Let V
max
= max{v
1
,, v
n
}.
If A
G
(I) > V
max
then A(I) = A
G
(I)
else A(I) = V
max



O(nlogn)
2-approximation algorithm
Theorem: MGreedy3(I) is 2-approximation algorithm.
Proof. First consider a relaxed version of knapsack, in
which items can be taken fractionally, if needed. Let A(I)
denote the value produced by this fractional algorithm.
Then, OPT(I)A(I) . Because the fractional algorithm
can never do worse than the optimal of 0/1 knapsack.
Fractional optimal solution will take items in sorted
order; only the last item may be fractional.
If j was the first item in the list that
MGreedy3(I) couldnt accept, then
A(I) A
G
(I) + v
j
, thus,
OPT(I)

A(I)
A
G
(I) + v
j
A
G
(I) + V
max
2A(I)
MGreedy3(I) never returns a solution worse
than half of the optimal.
PTAS
A smarter approach to the knapsack problem involves
brute-forcing part of the solution and then using the
greedy algorithm to finish up the rest. In particular,
consider all O(kn
k
) possible subsets of objects that have
up to k objects, where k is some fixed constant and
belongs to [1,n]. Then for each subset, use the greedy
algorithm to fill up the rest of the knapsack in O(n) time.
Pick the most profitable subset S. The total running time
of this algorithm is thus O(kn
k+1
).
PTASKnapsack(I, k)
1 sort the items in descending order of v
i
/w
i
2 for each feasible subset of at most k items do
3 fill the subset in the knapsack
4 fill the rest items in the knapsack by the sorted list
one by one, while there is enough available capacity
5 pick the most profitable subset S
6 return S
Theorem Let A(I) denote the value achieved by
PTASKnapsack(I, k) and OPT(I) be the value achieved
by the optimal set S*, then
.
1
1
) (
) ( OPT
k I A
I
+ s
Proof. If the optimal set S* has size less than or equal to
k, then this algorithm returns the optimal solution because
S* will be considered in the brute-force step. Otherwise,
let H = {a
1
, . . . , a
k
} be the set of k most profitable items
in S*. Since all subsets of size k items are considered in
the brute-force step, H must be one of them. For this
subset, filling in the rest of the knapsack using the greedy
algorithm will yield a value that satisfies the desired
bound.
Let S
1
= S*- H = {a
k+1
, . . . , a
|S*|
}, the remaining
items in S* in decreasing order of ratio of value
density. Let m be the index of the first item in S
1

which is not picked by the greedy algorithm step after
picking H in the brute-force step. The reason the item
a
m
is not picked must be because its size is larger than
the remaining empty space W
r
. This also means that
the greedy algorithm step has only picked items that
have value density of at least v(a
m
)/w(a
m
) because it
picks items in decreasing order of value density. At
this point, the knapsack contains H, a
k+1
, . . . , a
m1
,
and some items not in S*.
Let G be the items packed by the greedy algorithm step.
As mentioned before, all items in set G have value
density of at least v(a
m
)/w(a
m
). The items in G\ S*, or
the items in G that are not in the optimal set S*, have
total weight


Since all these items have value density of at least
v(a
m
)/w(a
m
), we can bound the value of A
G
(I):

=
+ = A
1
1
)) ( (
m
i
i r
a w W W
.
) (
) (
) ( ) (
1
1

+ =
A + >
m
k i
m
m
i G
a w
a v
a v I A
We can write the total value of the items in S* as:
) ( ) ( ) (
) (
) (
) ( ) (
) (
) (
)) ( ( )
) (
) (
) ( ( ) (
) ( ) ( ) ( ) ( OPT
1
1
1
1
| * |
1
m G
m
m
r G
m
i
m
m
i
m
m
G
m
k i
S
m i
i i
k
i
i
a v I A H V
a w
a v
W I A H V
a w
a v
a w W
a w
a v
I A H V
a v a v a v I
+ + s + + =
+ A + s
+ + =

+ = = =
The best found subset S returned after both the brute-force
and the greedy algorithm steps will have at least as much
value as that of H and G combined since we know that the
union of H and G must have been one of the considered
subsets. Namely, A(I) V(H)+A
G
(I).
Since OPT(I) V(H)+ A
G
(I)+v(a
m
), and then OPT(I)
A(I) +v(a
m
). Because the k items in H all have value at
least as large as that of a
m
, we have that
OPT(I)(k+1)v(a
m
), so
(k+1) OPT(I) (k+1) A(I) +(k+1)v(a
m
)
(k+1) A(I) + OPT(I),
and this gives the approximation ratio.
To obtain the polynomial time approximation scheme or
PTAS, we have a 1+-approximation where = 1/k . The
resulting running time is O((1/)nn
1/
), so the
approximation scheme is polynomial in n but not 1/.
FPTAS
From the pseudo-polynomial time algorithm, we see that
if the profits of the objects were all small numbers, i.e.
polynomially bounded in n, then we would have a
regular polynomial time algorithm. We will use this to
obtain an FPTAS for the knapsack problem.
Algorithm that depend on the values in a pseudo-
polynomial way can often be used to design polynomal-
time approximation schemes.
Key idea:
We will scale and round the values of the items so that
they are polynomially bounded. ThenRunning time
O(nV
max
) will be polynomial in input size.
More precisely, for each item i, let its rounded value
be for each item i we have


Error from rounding will be small
. K
K
v
v
i
i
(

= '
.
i i i
v v K v s
'
s
Let

The knapsack problem with and the scaled problem
with have the same set of optimum solutions, the
optimum value differs exactly by a factor of K, and the
scaled values are integral.
Given an instance I of 0/1 Knapsack problem and >0,
let , FPTAS is as follows:


.
(

=
'
= ' '
K
v
K
v
v
i i
i
i
v
'
i
v
' '
} { max
1
max i
n i
v V
'
=
'
s s
Theorem The algorithm FPTASKnapsack (I , ) runs in
polynomial time for any fixed .
Proof. The running time complexity of this algorithm is
O(n ), .


So O(n )=O(n
3

-1
), the theorem is proved.
V
' '
c
2
max 1
1
n
K
V n
K
v
v V
n
i
i
n
i
i
s
'
s
'
=
' '
=
' '

=
=
V
' '
Theorem let S* as the optimal set for the original
problem instance I. Let S be the optimal set under the
new problem instance I, then ,where

Proof. For every item i :

so each item T


Namely,

. ' 1
K
v
v
K
v
i
i
i
s s
. ' | |
K
v
v T
K
v
T i
i
T i
i
T i
i

e
e
e
s s
. ' | |

e e e
s s
T i
i
T i
i
T i
i
v v k T K v
c + s1
) (
) ( OPT
I A
I
. ) ( OPT , ) (
*

e ' ' e
= =
S i
i
S i
i
v I v I A
for S,


Due I and I have the same set of optimum solutions
namely S= S


namely,
so

Due to

. *
* *
S K v v K v K v
S i
i
S i
i
S i
i
S i
i
>
'
>
'
>

e e ' e ' e
Kn I
S K I I A
>
>
) ( OPT
* ) ( OPT ) (
Kn I A I s ) ( ) ( OPT
) ( ) (
) ( ) ( OPT
I A
Kn
I A
I A I
s

) (
max
I A v v V
S i
i
S i
i
= s
'
s
'

' e ' e
There are other variations as well, notably the
multiple knapsack problem, in which you have
more than one knapsack to fill.
c =
'
s s

max
) ( ) (
) ( ) ( OPT
V
Kn
I A
Kn
I A
I A I
so
so it is FPTAS.
The subset-sum problem
An instance of the subset-sum problem is a pair (S, t),
where S is a set {x
1
, x
2
, ..., x
n
} of positive integers and t
is a positive integer.
This decision problem asks whether there exists a subset
of S that adds up exactly to the target value t.
The optimization variant asks for a subset of S whose
sum is as large as possible, but not larger than t
An exponential-time exact algorithm
ExactSubsetSum(S, t)
1 n |S|
2 L
0
0
3 for i 1 to n do
4 L
i
MergeLists(L
i-1
, L
i-1
+ x
i
)
5 remove from L
i
every element that is
greater than t
6 return the largest element in L
n
To see how ExactSubsetSum(S, t) works, let P
i

denote the set of all values that can be obtained by
selecting a (possibly empty) subset of {x
1
, x
2
, ..., x
i
}
and summing its members.
For example, if S = {1, 4, 5}, then
P
1
= {0, 1} ,
P
2
= {0, 1, 4, 5} ,
P
3
= {0, 1, 4, 5, 6, 9, 10}
) (
1 1 i i i i
x P P P + =


A fully polynomial-time approximation
scheme
We can derive a fully polynomial-time
approximation scheme for the subset-sum
problem by "trimming" each list L
i
after it is
created.


To trim a list L by means to remove as
many elements from L as possible, in such
a way that if L' is the result of trimming L,
then for every element y that was removed
from L, there is an element z still in L' that
approximates y, that is,


y z
y
s s
+o 1
example, if = 0.1 and L = <10,
11, 12, 15, 20, 21, 22, 23, 24, 29>
then we can trim L to obtain
L' = <10, 12, 15, 20, 23, 29>

Trim(L, )
1 m |L|
2 L' y
1
3 last y
1
4 for i 2 to m do
5 if y
i
> last (1 +) then
6 append y
i
onto the end of L'
7 last y
i
8 return L'
ApproxSubsetSum(S, t )
1 n |S|
2 L
0
0
3 for i 1 to n do
4 L
i
MergeLists(L
i-1
, L
i-1
+ x
i
)
5 L
i
Trim(L
i
, /(2n))
6 remove from L
i
every element that is greater than t
7 let A(I) be the largest value in L
n
8 return A(I)
S =<104, 102, 201, 101> with t = 308 and = 0.40. The
trimming parameter is /8 = 0.05.
line 2: L
0
= <0>,
line 4: L
1
= <0, 104>,
line 5: L
1
= <0, 104>,
line 6: L
1
= <0, 104>,
line 4: L
2
= <0, 102, 104, 206>,
line 5: L
2
= <0, 102, 206>,
line 6: L
2
= <0, 102, 206>,
line 4: L
3
= <0, 102, 201, 206, 303, 407>,
line 5: L
3
=< 0, 102, 201, 303, 407>,
line 6: L
3
=< 0, 102, 201, 303>,
line 4: L
4
= <0, 101, 102, 201, 203, 302, 303, 404>,
line 5: L
4
= <0, 101, 201, 302, 404>,
line 6: L
4
= <0, 101, 201, 302>.
Theorem ApproxSubsetSum(S, t )
is a fully polynomial-time approximation scheme for the
subset sum problem.
Proof. The operations of trimming L
i
in line 5 and
removing from L
i
every element that is greater than t
maintain the property that every element of L
i
is also a
member of P
i
. Therefore, the value A(I) returned in line
8 is indeed the sum of some subset of S. Let OPT(I)P
n

denote an optimal solution to the subset-sum problem.
Then, from line 6, we know that A(I) OPT(I) By the
definition of PTAS, we need to show that OPT(I) /A(I)
1+. We must also show that the running time of this
algorithm is polynomial in both 1/ and the size of the
input.
y z
n
y
i
s s
+ ) 2 1 ( c
The above equation must hold for OPT(I) P
n
, and
therefore there is a z L
n
such that
) OPT(
) 2 1 (
) OPT(
I z
n
I
i
s s
+c
and thus


Since there is a z L
n
fulfilling the above
inequality, the inequality must hold for A(I) , which
is the largest value in L
n
; that is,


It remains to show that OPT(I)/A(I) 1+ . We do
so by showing that (1 + /2n)
n
1 + .
45
i
n
z
I
) 2 1 (
) OPT(
c + s
i
n
I A
I
) 2 1 (
) (
) ( OPT
c + s
c
c c
c
c
+ s
+ + s
s +
1
)
2
(
2
1
) 2 1 (
2
2
e n
i
So
. 1 ) 2 1 (
) (
) ( OPT
c c + s + s
i
n
I A
I
To show that ApproxSubsetSum is a fully polynomial-time
approximation scheme, we derive a bound on the length of
L
i
. After trimming, successive elements z and z' of L
i
must
have the relationship z' /z > 1+ /2n. That is, they must
differ by a factor of at least 1+ /2n.
Each list, therefore, contains the value 0, possibly the
value 1, and up to log
1+ /2n
t additional values. The
number of elements in each list L
i
is at most
2
ln 4

2
ln ) 2 1 ( 2

2
) 2 1 ln(
ln
2 log
2 1
+ s
+
+
s
+
+
= +
+
c
c
c
c
c
t n
t n n
n
t
t
n
Homework
Exercises: Page 123
2,4
Experiments:
1. Implement PTASKnapsack and
FPTASKnapsack algorithms
2. Implement SubsetSum algorithm
Problem
Maximum profit two-dimensional knapsack packing
problem
Given a set of n rectangles j = 1, . . . , n, each having a
width w
j
, height h
j
and profit p
j
and a rectangular plate
having width W and height H. The maximum profit
two-dimensional knapsack packing (2DKP) problem
assigns a subset of the rectangles onto the plate such
that the associated profit sum is maximized, please
design an approximation algorithm or meta heuristic
algorithm for it.(Scores 10)

You might also like