A

Summations

When an algorithm contains an iterative control construct such as a while or for

loop, we can express its running time as the sum of the times spent on each execution of the body of the loop. For example, we found in Section 2.2 that the j th
iteration of insertion sort took time proportional to j in the worst case. By adding
up the time spent on each iteration, we obtained the summation (or series)
n
X
j :
j D2

When we evaluated this summation, we attained a bound of .n2 / on the worstcase running time of the algorithm. This example illustrates why you should know
how to manipulate and bound summations.
Section A.1 lists several basic formulas involving summations. Section A.2 offers useful techniques for bounding summations. We present the formulas in Section A.1 without proof, though proofs for some of them appear in Section A.2 to
illustrate the methods of that section. You can nd most of the other proofs in any
calculus text.

A.1 Summation formulas and properties

Given a sequence a1 ; a2 ; : : : ; an of numbers, where n is a nonnegative integer, we
can write the nite sum a1 C a2 C C an as
n
X
ak :
kD1

If n D 0, the value of the summation is dened to be 0. The value of a nite series

is always well dened, and we can add its terms in any order.
Given an innite sequence a1 ; a2 ; : : : of numbers, we can write the innite sum
as
a1 C a2 C

1146

Appendix A
1
X

Summations

ak ;

kD1

which we interpret to mean

lim

n
X

n!1

ak :

kD1

If the limit does not exist, the series diverges; otherwise, it converges. The terms
of a convergent series cannot always be added in any order. We can, P
however,
1
rearrange the terms P
of an absolutely convergent series, that is, a series kD1 ak
1
for which the series kD1 jak j also converges.
Linearity
For any real number c and any nite sequences a1 ; a2 ; : : : ; an and b1 ; b2 ; : : : ; bn ,
n
X
kD1

.cak C bk / D c

n
X
kD1

ak C

n
X

bk :

kD1

The linearity property also applies to innite convergent series.

We can exploit the linearity property to manipulate summations incorporating
asymptotic notation. For example,
!
n
n
X
X
.f .k// D
f .k/ :
kD1

kD1

In this equation, the -notation on the left-hand side applies to the variable k, but
on the right-hand side, it applies to n. We can also apply such manipulations to
innite convergent series.
Arithmetic series
The summation
n
X
k D1C2C
kD1

Cn;

is an arithmetic series and has the value

n
X
1
n.n C 1/
k D
2
kD1

D .n2 / :

(A.1)
(A.2)

A.1 Summation formulas and properties

1147

Sums of squares and cubes

We have the following summations of squares and cubes:
n
X
n.n C 1/.2n C 1/
;
k2 D
6

(A.3)

kD0

n
X
kD0

k3 D

n2 .n C 1/2
:
4

(A.4)

Geometric series
For real x 1, the summation
n
X
xk D 1 C x C x2 C C xn
kD0

is a geometric or exponential series and has the value

n
X
x nC1 1
:
xk D
x 1

(A.5)

kD0

When the summation is innite and jxj < 1, we have the innite decreasing geometric series
1
X
1
:
(A.6)
xk D
1 x
kD0

Harmonic series
For positive integers n, the nth harmonic number is
1
1
1 1
Hn D 1 C C C C C
2 3
4
n
n
X
1
D
k
kD1

D ln n C O.1/ :

(We shall prove a related bound in Section A.2.)

(A.7)

Integrating and differentiating series

By integrating or differentiating the formulas above, additional formulas arise. For
example, by differentiating both sides of the innite geometric series (A.6) and
multiplying by x, we get

1148

Appendix A
1
X
kD0

Summations

kx k D

x
.1

(A.8)

x/2

for jxj < 1.

Telescoping series
For any sequence a0 ; a1 ; : : : ; an ,
n
X
.ak ak 1 / D an a0 ;

(A.9)

kD1

since each of the terms a1 ; a2 ; : : : ; an 1 is added in exactly once and subtracted out
exactly once. We say that the sum telescopes. Similarly,
n 1
X

.ak

kD0

akC1 / D a0

an :

As an example of a telescoping sum, consider the series

n 1
X
kD1

1
:
k.k C 1/

Since we can rewrite each term as

1
1
1
D
;
k.k C 1/
k kC1
we get

n 1
n 1
X
X
1
1
1
D
k.k C 1/
k kC1
kD1

kD1

D 1

1
:
n

Products
We can write the nite product a1 a2
n
Y
ak :
kD1

an as

If n D 0, the value of the product is dened to be 1. We can convert a formula with

a product to a formula with a summation by using the identity
!
n
n
Y
X
ak D
lg ak :
lg
kD1

kD1

A.2 Bounding summations

1149

Exercises
A.1-1
Pn
Find a simple formula for kD1 .2k
A.1-2 ? P
Show that nkD1 1=.2k
series.

1/.

p
1/ D ln. n/ C O.1/ by manipulating the harmonic

A.1-3
P1
Show that kD0 k 2 x k D x.1 C x/=.1

A.1-4 ? P
1
Show that kD0 .k

x/3 for 0 < jxj < 1.

1/=2k D 0.

A.1-5 ?
P1
Evaluate the sum kD1 .2k C 1/x 2k .

A.1-6
Pn
Pn
Prove that kD1 O.fk .i // D O
kD1 fk .i / by using the linearity property of
summations.
A.1-7
Qn
Evaluate the product kD1 2 4k .
A.1-8 ?
Qn
Evaluate the product kD2 .1

1=k 2 /.

A.2 Bounding summations

We have many techniques at our disposal for bounding the summations that describe the running times of algorithms. Here are some of the most frequently used
methods.
Mathematical induction
The most basic way to evaluate a series is to use
Pn mathematical induction. As an
example, let us prove that the arithmetic series kD1 k evaluates to 12 n.n C 1/. We
can easily verify this assertion for n D 1. We make the inductive assumption that

1150

Appendix A

Summations

it holds for n, and we prove that it holds for n C 1. We have

nC1
X
kD1

k D

n
X
kD1

k C .n C 1/

1
n.n C 1/ C .n C 1/
2
1
.n C 1/.n C 2/ :
D
2
You dont always need to guess the exact value of a summation in order to use
mathematical induction. Instead, you can use induction to proveP
a bound on a summation. As an example, let us provePthat the geometric series nkD0 3k is O.3n /.
n
More specically, let us prove that kD0 3k c3n for some constant c. For the
P0
initial condition n D 0, we have kD0 3k D 1 c 1 as long as c 1. Assuming
that the bound holds for n, let us prove that it holds for n C 1. We have
D

nC1
X
kD0

3k D

n
X
kD0

3k C 3nC1

c3n C 3nC1
(by the inductive hypothesis)

1
1
C
c3nC1
D
3
c
c3nC1

Pn
as long as .1=3 C 1=c/ 1 or, equivalently, c 3=2. Thus, kD0 3k D O.3n /,
as we wished to show.
We have to be careful when we use asymptotic notation
Pn to prove bounds by induction. Consider the following fallacious proof that kD1 k D O.n/. Certainly,
P1
kD1 k D O.1/. Assuming that the bound holds for n, we now prove it for n C 1:
nC1
X
kD1

k D

n
X
kD1

k C .n C 1/

D O.n/ C .n C 1/
D O.n C 1/ :

wrong!!

The bug in the argument is that the constant hidden by the big-oh grows with n
and thus is not constant. We have not shown that the same constant works for all n.
Bounding the terms
We can sometimes obtain a good upper bound on a series by bounding each term
of the series, and it often sufces to use the largest term to bound the others. For

A.2 Bounding summations

1151

example, a quick upper bound on the arithmetic series (A.1) is

n
X

kD1

n
X

kD1
2

D n :

In general, for a series

n
X
kD1

Pn
kD1

ak , if we let amax D max1kn ak , then

ak n amax :

The technique of bounding each term in a series by the largest term is a weak
methodPwhen the series can in fact be bounded by a geometric series. Given the
n
0, where 0 < r < 1 is a
series kD0 ak , suppose that akC1 =ak r for all k
constant. We can bound the sum by an innite decreasing geometric series, since
ak a0 r k , and thus
n
X

ak

kD0

1
X

a0 r k

kD0

D a0
D a0

1
X

rk

kD0

1
1

P1
k
We can apply this method to bound the summation
kD1 .k=3 /. In order to
P1
kC1
start the summation at k D 0, we rewrite it as kD0 ..k C 1/=3 /. The rst
term (a0 ) is 1=3, and the ratio (r) of consecutive terms is
.k C 2/=3kC2
.k C 1/=3kC1

for all k
1
X
kD1

k
3k

1 kC2
3 kC1
2
3

0. Thus, we have
D

1
X
kC1
kD0

3kC1

1
1
3 1 2=3
D 1:

1152

Appendix A

Summations

A common bug in applying this method is to show that the ratio of consecutive terms is less than 1 and then to assume that the summation is bounded by a
geometric series. An example is the innite harmonic series, which diverges since
1
X
1
k
kD1

n
X
1
n!1
k

lim

kD1

D lim .lg n/
n!1
D 1:

The ratio of the .k C1/st and kth terms in this series is k=.k C1/ < 1, but the series
is not bounded by a decreasing geometric series. To bound a series by a geometric
series, we must show that there is an r < 1, which is a constant, such that the ratio
of all pairs of consecutive terms never exceeds r. In the harmonic series, no such r
exists because the ratio becomes arbitrarily close to 1.
Splitting summations
One way to obtain bounds on a difcult summation is to express the series as the
sum of two or more series by partitioning the range of the index and then to bound
each of the resulting series.
Pn For example, suppose we try to nd a lower bound
on the arithmetic series kD1 k, which we have already seen has an upper bound
of n2 . We might attempt to bound each term in the summation by the smallest term,
but since that term is 1, we get a lower bound of n for the summationfar off from
our upper bound of n2 .
We can obtain a better lower bound by rst splitting the summation. Assume for
convenience that n is even. We have
n
X
kD1

k D

n=2
X
kD1
n=2
X
kD1

kC
0C

D .n=2/2
D
.n2 / ;

n
X

kDn=2C1
n
X

.n=2/

kDn=2C1

Pn
which is an asymptotically tight bound, since kD1 k D O.n2 /.
For a summation arising from the analysis of an algorithm, we can often split
the summation and ignore a constant number of the P
initial terms. Generally, this
n
technique applies when each term ak in a summation kD0 ak is independent of n.

A.2 Bounding summations

1153

Then for any constant k0 > 0, we can write

n
X
kD0

k0 1

ak D

X
kD0

ak C

D .1/ C

n
X

ak

kDk0

n
X

ak ;

kDk0

since the initial terms of the summation are all constant andPthere are a constant
n
number of them. We can then use other methods to bound kDk0 ak . This technique applies to innite summations as well. For example, to nd an asymptotic
upper bound on
1
X
k2
kD0

2k

we observe that the ratio of consecutive terms is

.k C 1/2 =2kC1
k 2 =2k

if k

.k C 1/2
2k 2
8
9

3. Thus, the summation can be split into

1
X
k2
kD0

2k

2
X
k2

1
X
k2

2k
2
1
X
9X 8 k
k2

C
2k
8 kD0 9
kD0
kD0

2k

kD3

D O.1/ ;

since the rst summation has a constant number of terms and the second summation
is a decreasing geometric series.
The technique of splitting summations can help us determine asymptotic bounds
in much more difcult situations. For example, we can obtain a bound of O.lg n/
on the harmonic series (A.7):
n
X
1
:
Hn D
k
kD1

We do so by splitting the range 1 to n into blg nc C 1 pieces and upper-bounding

the contribution of each piece by 1. For i D 0; 1; : : : ; blg nc, the ith piece consists

1154

Appendix A

Summations

of the terms starting at 1=2i and going up to but not including 1=2i C1 . The last
piece might contain terms not in the original harmonic series, and thus we have
n
X
1
k
kD1

blg nc 2i 1
XX
i D0 j D0
blg nc 2i 1
XX
i D0 j D0
blg nc

2i

1
Cj

1
2i

i D0

lg n C 1 :

(A.10)

Approximation by integrals
Pn
When a summation has the form kDm f .k/, where f .k/ is a monotonically increasing function, we can approximate it by integrals:
Z nC1
Z n
n
X
f .x/ dx
f .k/
f .x/ dx :
(A.11)
m 1

kDm

Figure A.1 justies this approximation. The summation is represented as the area
of the rectangles in the gure, and the integral is the shaded region under the curve.
When f .k/ is a monotonically decreasing function, we can use a similar method
to provide the bounds
Z n
Z nC1
n
X
f .x/ dx
f .k/
f .x/ dx :
(A.12)
m

kDm

m 1

The integral approximation (A.12) gives a tight estimate for the nth harmonic
number. For a lower bound, we obtain
Z nC1
n
X
1
dx
kD1

D ln.n C 1/ :

For the upper bound, we derive the inequality

Z n
n
X
1
dx

k
x
1
kD2

D ln n ;

(A.13)

A.2 Bounding summations

1155

f (x)

n2

f (n)

m+2

f (n1)

m+1

f (n2)

f (m+2)

f (m+1)

f (m)
m 1

n1

n+1

(a)

f (x)

n2

n1

f (n)

f (n1)

m+2

f (n2)

m+1

f (m+2)

f (m+1)

f (m)

m 1

n+1

(b)
P
Figure A.1 Approximation of nkDm f .k/ by integrals. The area of each rectangle is shown
within the rectangle, and the total rectangle area represents the value of the summation. The inis represented
Rtegral
Pnby the shaded area under the curve. By comparing areas in (a), we get
n
f
.x/
dx

f .k/, and then by shifting the rectangles one unit to the right, we get
m 1
R nC1kDm
P
n
f
.k/

f
.x/
dx in (b).
kDm
m

1156

Appendix A

Summations

which yields the bound

n
X
1
ln n C 1 :
k

(A.14)

kD1

Exercises
A.2-1
Pn
Show that kD1 1=k 2 is bounded above by a constant.
A.2-2
Find an asymptotic upper bound on the summation
blg nc

n=2k :

kD0

A.2-3
Show that the nth harmonic number is

.lg n/ by splitting the summation.

A.2-4
Pn
Approximate kD1 k 3 with an integral.
A.2-5
Pn
Why didnt we use the integral approximation (A.12) directly on kD1 1=k to
obtain an upper bound on the nth harmonic number?

Problems
A-1 Bounding summations
Give asymptotically tight bounds on the following summations. Assume that r
and s 0 are constants.
a.

n
X

kr .

kD1

b.

n
X
kD1

lgs k.

Notes for Appendix A

c.

n
X

1157

k r lgs k.

kD1

Appendix notes
Knuth [209] provides an excellent reference for the material presented here. You
can nd basic properties of series in any good calculus book, such as Apostol [18]
or Thomas et al. [334].