1

CMPS 102
Introduction to Analysis of Algorithms
Fall 2003

Asymptotic Growth of Functions

We introduce several types of asymptotic notation which are used to compare the performance and
efficiency of algorithms. As well see, the asymptotic run time of an algorithm gives a simple, and
machine independent, characterization of its complexity.

Definition Let ) (n g be a function. The set )) ( ( n g O is defined as

} ) ( ) ( 0 : , 0 , 0 | ) ( { )) ( (
0 0
n cg n f n n n c n f n g O > > = .

In other words, )) ( ( ) ( n g O n f if and only if there exist positive constants c, and
0
n , such that for all
0
n n , the inequality ) ( ) ( 0 n cg n f is satisfied. We say that ) (n f is Big O of ) (n g , or that ) (n g
is an asymptotic upper bound for ) (n f .

We often abuse notation slightly by writing )) ( ( ) ( n g O n f = to mean )) ( ( ) ( n g O n f . Actually
)) ( ( ) ( n g O n f is also an abuse of notation. We should really write ) (g O f since what we have
defined is a set of functions, not a set of numbers. The notational convention )) ( ( n g O is useful since
it allows us to refer to the set ) (
3
n O say, without having to introduce a function symbol for the
polynomial
3
n . Observe that if )) ( ( ) ( n g O n f = then ) (n f is asymptotically non-negative, i.e. ) (n f is
non-negative for all sufficiently large n, and likewise for ) (n g . We make the blanket assumption from
now on that all functions under discussion are asymptotically non-negative.

In practice we will be concerned with integer valued functions of a (positive) integer n (
+ +
Z Z : g ).
However, in what follows, it is useful to consider n to be a continuous real variable taking positive
values and g to be real valued function (
+ +
R R : g ).

Geometrically )) ( ( ) ( n g O n f = says:

) (n cg


) (n f







0
n

2
Example ) 300 10 ( 100 40
2
+ + = + n n O n . Observe that 300 10 100 40 0
2
+ + + n n n for all 20 n ,
as can be easily verified. Thus we may take 20
0
= n and 1 = c in the definition.
300 10
2
+ + n n



300 100 40 + n




100


10 20


Note that in this example, any value of
0
n greater than 20 will also work, and likewise any value of c
greater than 1 works. In general if there exist positive constants
0
n and c such that ) ( ) ( 0 n cg n f
for all
0
n n , then infinitely many such constants also exist. In order to prove that )) ( ( ) ( n g O n f = it
is not necessary to find the smallest possible
0
n and c making the ) ( ) ( 0 n cg n f true. It is only
necessary to show that at least one pair of such constants exist.

Generalizing the last example, we will show that ) (
2
e dn cn O b an + + = + for any constants a-e, and in
fact )) ( ( ) ( n q O n p = whenever ) (n p and ) (n q are polynomials with ) deg( ) deg( q p .

Definition Let ) (n g be a function and define the set )) ( ( n g to be

} ) ( ) ( 0 : , 0 , 0 | ) ( { )) ( (
0 0
n f n cg n n n c n f n g > > = .

We say ) (n f is big Omega of ) (n g , and that ) (n g is an asymptotic lower bound for ) (n f . As before
we write )) ( ( ) ( n g n f = to mean )) ( ( ) ( n g n f . The geometric interpretation is:

) (n f


) (n cg







0
n

3

Lemma )) ( ( ) ( n g O n f = if and only if )) ( ( ) ( n f n g = .
Proof: If )) ( ( ) ( n g O n f = then there exist positive numbers
1
c ,
1
n such that ) ( ) ( 0
1
n g c n f for all
1
n n . Let
1 2
/ 1 c c = and
1 2
n n = . Then ) ( ) ( 0
2
n g n f c for all
2
n n , proving )) ( ( ) ( n f n g = .
The converse is similar and we leave it to the reader. ///

Definition Let ) (n g be a function and define the set )) ( ( )) ( ( )) ( ( n g n g O n g = . Equivalently

} ) ( ) ( ) ( 0 : , 0 , 0 , 0 | ) ( { )) ( (
2 1 0 0 2 1
n g c n f n g c n n n c c n f n g > > > = .

We write )) ( ( ) ( n g n f = and say the ) (n g is an asymptotically tight bound for ) (n f , or that ) (n f
and ) (n g are asymptotically equivalent. We interpret this geometrically as:

) (
2
n g c

) (n f

) (
1
n g c






0
n

Exercise Prove that if c is a positive constant, then )) ( ( ) ( n f n cf = .

Exercise Prove that )) ( ( ) ( n g n f = if and only if )) ( ( ) ( n f n g = .

Example Prove that ( ) n n = +10 .
Proof: According to the definition, we must find positive numbers
0 2 1
, , n c c , such that the inequality
n c n n c
2 1
10 0 + holds for all
0
n n . Pick 1
1
= c , 2
2
= c , and 10
0
= n . Then if
0
n n we
have:
0 10 and n 10
n ) 1 1 ( 10 and n ) 1 2 ( 10
n c ) 1 ( 10
2
1
and n c ) 1 ( 10
2
2

10
2
1
+ n n c and n c n
2
2
10 + ,
n c n n c
2
2
2
1
10 + ,
n c n n c
2 1
10 + ,

as required. ///

4
The reader may find our choice of values for the constants
0 2 1
, , n c c in this example somewhat
mysterious. Adequate values for these constants can usually be obtained by working backwards
algebraically from the inequality to be proved. Notice that in this example there are many valid
choices. For instance one checks easily that 2 / 1
1
= c , 2 / 3
2
= c , and 20
0
= n work equally well.

Exercise Let a, b be real numbers with 0 > b . Prove directly from the definition (as above) that
) ( ) (
b b
n a n = + . (In what follows we learn a much easier way to prove this.)

Lemma If ) ( ) ( n h n f for all sufficiently large n, and if )) ( ( ) ( n g O n h = , then )) ( ( ) ( n g O n f = .
Proof: The above hypotheses say that there exist positive numbers c and
1
n such that ) ( ) ( n cg n h
for all
1
n n . Also there exists
2
n such that ) ( ) ( 0 n h n f for all
2
n n . (Recall ) (n f is assumed
to be asymptotically non-negative.) Define ) , max(
2 1 0
n n n = , so that if
0
n n we have both
1
n n and
2
n n . Thus
0
n n implies ) ( ) ( 0 n cg n f , and therefore )) ( ( ) ( n g O n f = . ///

Exercise Prove that if ) ( ) ( ) (
2 1
n h n f n h for all sufficiently large n, where )) ( ( ) (
1
n g n h = and
)) ( ( ) (
2
n g O n h = , then )) ( ( ) ( n g n f = .

Example Let 1 k be a fixed integer. Prove that ) (
1
1
+
=
=

k
n
i
k
n i .
Proof: Observe that ) (
1 1
1 1
+ +
= =
= = =

k k k
n
i
k
n
i
k
n O n n n n i , and
( (
(
) ( ) 2 / 1 ( ) 2 / )( 2 / ( ) 2 / ( 2 / ) 2 / (
1 1 1
2 / 2 / 1
+ + +
= = =
= =

k k k k k
n
n i
k
n
n i
k
n
i
k
n n n n n n n i i .
By the result of the preceding exercise, we conclude ) (
1
1
+
=
=

k
n
i
k
n i . ///

When asymptotic notation appears in a formula such as ) ( ) 2 / ( 2 ) ( n n T n T + = we interpret ) (n to
stand for some anonymous function in the class ) (n . For example ) ( 3 1 2 4 3
2 3 2 3
n n n n n + = + + .
Here ) (
2
n stands for 1 2 4
2
+ n n , which belongs to the class ) (
2
n .

The expression

=

n
i
i
1
) ( can be puzzling. On the surface it stands for ) ( ) 3 ( ) 2 ( ) 1 ( n + + + + ,
which is meaningless since ) constant ( consists of all functions which are bounded above by some
constant. We interpret ) (i in this expression to stand for a single function ) (i f in the class ) (i ,
evaluated at n i , , 3 , 2 , 1 = .

Exercise Prove that ) ( ) (
2
1
n i
n
i
=

=
. The left hand side stands for a single function ) (i f summed
for n i , , 3 , 2 , 1 = . By the previous exercise it is sufficient to show that ) ( ) ( ) (
2
1
1
n h i f n h
n
i

=
for
all sufficiently large n, where ) ( ) (
2
1
n n h = and ) ( ) (
2
2
n O n h = .

5
Definition } ) ( ) ( 0 : , 0 , 0 | ) ( { )) ( (
0 0
n cg n f n n n c n f n g o < > > = . We say that ) (n g is a
strict Asymptotic upper bound for ) (n f and write )) ( ( ) ( n g o n f = as before.

Lemma )) ( ( ) ( n g o n f = if and only if 0
) (
) (
lim =

n g
n f
n
.
Proof: Observe that )) ( ( ) ( n g o n f = if and only if c
n g
n f
n n n c < > >
) (
) (
0 : , 0 , 0
0 0
, which is the
very definition of the limit statement 0
) (
) (
lim =

n g
n f
n
. ///

Example ) ( ) lg( n o n = since 0
) lg(
lim =

n
n
n
. (Apply lHopitals rule.)

Example ) (
n k
b o n = for any 0 > k and 1 > b since 0 lim =

n
k
n
b
n
. (Apply lHopitals rule
(
k times.) In
other words, any exponential grows strictly faster than any polynomial.

By comparing definitions of )) ( ( n g o and )) ( ( n g O one sees immediately that )) ( ( )) ( ( n g O n g o .
Also no function can belong to both )) ( ( n g o and )) ( ( n g , as is easily verified (exercise). Thus
= )) ( ( )) ( ( n g n g o , and therefore )) ( ( )) ( ( )) ( ( n g n g O n g o .

Definition } ) ( ) ( 0 : , 0 , 0 | ) ( { )) ( (
0 0
n f n cg n n n c n f n g < > > = . Here we say that ) (n g is
a strict asymptotic lower bound for ) (n f and write )) ( ( ) ( n g n f = .

Exercise Prove that )) ( ( ) ( n g n f = if and only if =

) (
) (
lim
n g
n f
n
.

Exercise Prove = )) ( ( )) ( ( n g O n g , whence )) ( ( )) ( ( )) ( ( n g n g n g .

The following picture emerges:



)) ( ( n g O )) ( ( n g



)) ( ( n g o )) ( ( n g )) ( ( n g






6
Lemma If L
n g
n f
n
=

) (
) (
lim , where < L 0 , then )) ( ( ) ( n g O n f = .
Proof: The definition of the above limit is < > > L
n g
n f
n n n
) (
) (
: , 0 , 0
0 0
. Thus if we let
1 = , there exists a positive number
0
n such that for all
0
n n :
1
) (
) (
< L
n g
n f

1
) (
) (
1 < < L
n g
n f

1
) (
) (
+ < L
n g
n f

) ( ) 1 ( ) ( n g L n f + < .

Now take 1 + = L c in the definition of O , so that )) ( ( ) ( n g O n f = as claimed. ///

Lemma If L
n g
n f
n
=

) (
) (
lim , where < L 0 , then )) ( ( ) ( n g n f = .
Proof: The limit statement implies L
n f
n g
n
=

) (
) (
lim , where L L / 1 = and hence < L 0 . By the
previous lemma )) ( ( ) ( n f O n g = , and therefore )) ( ( ) ( n g n f = . ///

Exercise Prove that if L
n g
n f
n
=

) (
) (
lim , where < < L 0 , then )) ( ( ) ( n g n f = .

Although )) ( ( n g o , )) ( ( n g , and a certain subset of )) ( ( n g are characterized by limits, the full sets
)) ( ( n g O , )) ( ( n g , and )) ( ( n g have no such characterization as the following examples show.

Example A Let n n g = ) ( and n n n f + = )) sin( 1 ( ) ( .
) ( 2 n g


) (n f






Clearly )) ( ( ) ( n g O n f = , but ) sin( 1
) (
) (
n
n g
n f
+ = , whose limit does not exist. This example shows that
the containment )) ( ( )) ( ( )) ( ( n g n g O n g o is in general strict since )) ( ( ) ( n g n f (exercise).
Therefore )) ( ( ) ( n g n f , so that )) ( ( )) ( ( ) ( n g n g O n f . But )) ( ( ) ( n g o n f since the limit does
not exist.
7

Example B Let n n g = ) ( and n n n f + = )) sin( 2 ( ) ( .
) ( 3 n g


) (n f


) (n g




Since n n n n 3 )) sin( 2 ( + for all 0 n , we have )) ( ( ) ( n g n f = , but ) sin( 2
) (
) (
n
n g
n f
+ = whose
limit does not exist.

Exercise Find functions ) (n f and ) (n g such that )) ( ( )) ( ( ) ( n g n g n f , but
) (
) (
lim
n g
n f
n
does not
exist (even in the sense of being infinite), so that )) ( ( ) ( n g n f .


The preceding limit theorems and counter-examples can be summarized in the following diagram.
Here L denotes the limit
) (
) (
lim
n g
n f
L
n
= , if it exists.


)) ( ( n g O )) ( ( n g
)) ( ( n g

)) ( ( n g o )) ( ( n g
0 = L < < L 0 = L



Ex A Ex B



In spite of the above counter-examples, the preceding limit theorems are a very useful tool for
establishing asymptotic comparisons between functions. For instance recall the earlier exercise to
show ) ( ) (
b b
n a n = + for real numbers a, and b with 0 > b . The result follows immediately from
1 1 1 lim
) (
lim = =
|
.
|

\
|
+ =
+

b
b
n
b
b
n
n
a
n
a n
,
since < <1 0 .
8
Exercise Use limits to prove the following:
a. ) ( ) lg(
2
n o n n = (here ) lg(n denotes the base 2 logarithm of n.)
b. ) ( 2
10 5
n n
n
= .
c. If ) (n P is a polynomial of degree 0 k , then ) ( ) (
k
n n P = .
d. )) ( ( )) ( ( ) ( n f n f o n f = + . (One can always disregard lower order terms)
e. ) ( ) (log

n o n
k
= for any 0 > k and 0 > . (Polynomials grow faster than logs.)
f. ) (
n
b o n =

for any 0 > and 1 > b . (Exponentials grow faster than polynomials.)

There is an analogy between the asymptotic comparison of functions ) (n f and ) (n g , and the
comparison of real numbers x and y.

)) ( ( ) ( n g O n f = ~ y x
)) ( ( ) ( n g n f = ~ y x =
)) ( ( ) ( n g n f = ~ y x
)) ( ( ) ( n g o n f = ~ y x <
)) ( ( ) ( n g n f = ~ y x >

Note however that this analogy is not exact since there exist pairs of functions which are not
comparable, while any two real numbers are comparable. (See problem 3-2c, p.58.)