Growth of functions

Function Growth
The running time of an algorithm as input size
approaches infinity is called the asymptotic running
time
We study different notations for asymptotic efficiency.
In particular, we study tight bounds, upper bounds and
lower bounds.

Growth of Functions 2

The sets and their use big Oh

Big oh - asymptotic upper bound on the
growth of an algorithm
When do we use Big Oh?
1. Theory of NP-completeness
2. To provide information on the maximum
number of operations that an algorithm
performs

Insertion sort is O(n2) in the worst case

This means that in the worst case it performs at

most cn2 operations

Insertion sort is also O(n6) in the worst caseGrowth of Functions

since it also performs at most dn6 operations

The sets and their use Omega

Omega - asymptotic lower bound on the
growth of an algorithm or a problem*
When do we use Omega?
1. To provide information on the minimum
number of operations that an algorithm
performs
Insertion sort is (n) in the best case
This means that in the best case its instruction count
is at least cn,

It is (n2) in the worst case

This means that in the worst case its instruction
count is at least cn2

Growth of Functions 4

The sets and their use Omega

cont.
2. To provide information on a class of algorithms that
solve a problem
Sort algorithms based on comparison of keys are (nlgn) in
the worst case
This means that all sort algorithms based only on comparison of keys
have to do at least cnlgn operations

Any algorithm based only on comparison of keys to find the

maximum of n elements is (n) in every case
This means that all algorithms based only on comparison of keys to find
maximum have to do at least cn operations

Growth of Functions 5

The sets and their use - Theta

Theta - asymptotic tight bound on the growth rate of an
algorithm
Insertion sort is (n2) in the worst and average
cases
The means that in the worst case and average
cases insertion sort performs cn2 operations
Binary search is (lg n) in the worst and average
cases
The means that in the worst case and average
cases binary search performs clgn operations
Note: We want to classify an algorithm using Theta.
In Data Structures used Oh
Little oh - used to denote an upper bound that is not
asymptotically tight. n is in o(n3). n is not in o(n)
Growth of Functions 6

The functions
Let f(n) and g(n) be asymptotically nonnegative
functions whose domains are the set of natural numbers
N={0,1,2,}.
A function g(n) is asymptotically nonnegative, if g(n)0
for all nn0 where n0N

Growth of Functions 7

Asymptotic Upper Bound: big

Graph shows that
for all n N,
f(n) c*g(n)

c g (n)
f (n)

Why only for n N ?

What is the purpose of multiplying by c > 0?
N

f (n) = O ( g ( n ))
Growth of Functions 8

Asymptotic Upper Bound:

Definition:
Let f (n) and g(n) be asymptotically non-negative
functions. We say
f (n) is in O ( g ( n )) if there is a real positive
constant c and a positive Integer N such that for
every n N
0 f (n) c g (n ).
Or using more mathematical notation
O ( g (n) ) =

{ f (n )| there exist positive constant c and

a positive integer N such that
0 f( n) c g (n ) for all n N }
Growth of Functions 9

O(n )

n2 + 10 n

Why?
take c = 2
N = 10

2n2 > n2 + 10 n
for all n>=10
1400
1200
1000
800
600
400
200
0

n2 + 10n
2 n2

10

20

30

Growth of Functions 10

Does 5n+2 O(n)?

Proof: From the definition of Big Oh, there must exist
c>0 and integer N>0 such that 0 5n+2cn for all
nN.
Dividing both sides of the inequality by n>0 we get:
0 5+2/nc.
2/n 2, 2/n>0 becomes smaller when n increases
There are many choices here for c and N.
If we choose N=1 then c 5+2/1= 7.
If we choose c=6, then 0 5+2/n6. So N 2.
In either case (we only need one!) we have a c>o and
N>0 such that 0 5n+2cn for all n N. So the
definition is satisfied and 5n+2 O(n)
Growth of Functions 11

Does n2 O(n)? No.

We will prove by contradiction that the definition cannot

be
satisfied.
Assume that n2 O(n).
From the definition of Big Oh, there must exist c>0 and
integer N>0 such that 0 n2cn for all nN.
Dividing the inequality by n>0 we get 0 n c for all nN.
n c cannot be true for any n >max{c,N }, contradicting
our assumption
So there is no constant c>0 such that nc is satisfied for
all nN, and n2 O(n)

Growth of Functions 12

( g (n) ) =
{ f (n )| there exist positive constant c
and positive integer N such that
0 f( n) c g (n ) for all n N }
1,000,000 n2 O(n2) why/why not?
(n - 1)n / 2 O(n2) why /why not?
n / 2 O(n2) why /why not?
lg (n2) O( lg n ) why /why not?
n2 O(n) why /why not?
Growth of Functions 13

Asymptotic Lower Bound, Omega:

f (n)

c * g (n)

f(n) = ( g ( n ))
Growth of Functions 14

Asymptotic Lower Bound:

Definition:
Let f (n) and g(n) be asymptotically non-negative
functions. We say
f (n) is ( g ( n )) if there is a positive real constant c
and a positive integer N such that for every n N
0 c * g (n ) f ( n).
Or using more mathematical notation
( g ( n )) =

{ f (n) | there exist positive constant c and

a positive integer N such that
0 c * g (n ) f ( n) for all n N }

Growth of Functions 15

Is 5n-20 (n)?
Proof: From the definition of Omega, there must exist c>0 and
integer N>0 such that 0 cn 5n-20 for all nN
Dividing the inequality by n>0 we get: 0 c 5-20/n for all
nN.
20/n 20, and 20/n becomes smaller as n grows.
There are many choices here for c and N.
Since c > 0, 5 20/n >0 and N >4
For example, if we choose c=4, then 5 20/n 4 and N
20
In this case we have a c>o and N>0 such that 0 cn 5n-20
Growth of Functions
for all n N. So the definition is satisfied and 5n-20 (n)

16

( g ( n )) =
{ f (n) | there exist positive constant c and
a positive integer N such that
0 c * g (n ) f ( n) for all n N }
1,000,000 n2 (n2) why /why not?
(n - 1)n / 2(n2) why /why not?
n / 2(n2) why /why not?
lg (n2) ( lg n ) why /why not?
n2 (n) why /why not?
Growth of Functions 17

Asymptotic Tight Bound:

d g (n)

f (n)

c g (n)

f (n) = ( g ( n ))

N
Growth of Functions 18

Asymptotic Bound Theta:

Definition:
Let f (n) and g(n) be asymptotically non-negative
functions. We say f (n) is ( g ( n )) if there are
positive constants c, d and a positive integer N
such that for every n N
0 c g (n ) f ( n) d g ( n ).
Or using more mathematical notation
( g ( n )) =
{ f (n) | there exist positive constants c, d and
a positive integer N such that
0 c g (n ) f ( n) d g ( n ). for all n

N }
Growth of Functions 19

More on
We will use this definition:
(g (n)) = O( g (n) ) ( g (n) )

Growth of Functions 20

1 2
Does n 3n (n 2 ) ?
2
We show:

1.

2.

1 2
2
n 3n O (n )
2
1 2
2
n 3n (n )
2
Growth of Functions 21

1 2
Does n 3n O(n 2 ) ?
2
From the definition there must exist c 0,
and N 0 such that
1
0 n 2 3n cn 2 for all n N .
2
Dividing the inequality by n 2 0 we get :
1 3
0 c for all n N .
2 n
Since 3 / n 0 for finite n, c 1/2.
Choose c 1 / 4.
So

1 3 1
, and N 12
2 n 4
Growth of Functions 22

1 2
Does n 3n (n 2 ) ?
2
There must exist c 0 and N 0 such that
1
0 cn 2 n 2 3n for all n N
2
Dividing by n 2 0 we get
1 3
.
2 n
1 3
Since c 0, 0 and N 6.
2 N
Since 3 /n 0 for finite n, c 1 / 2. Choose c 1/4.
1 1 3
for all n 12.
4 2 n
So c 1/ 4 and N 12.
0c

Growth of Functions 23

More
1,000,000 n2 (n2) why /why not?

(n - 1)n / 2(n2) why /why not?

n / 2(n2) why /why not?
lg (n2) ( lg n ) why /why not?
n2 (n) why /why not?
Growth of Functions 24

Qualified Notation
Omega Notation
best case - describes a lower bound for all input (it
can't get any better than this). Example: the array is
already correctly sorted.
worst case - describes a lower bound for worst case
input, possibly greater than best case. Example: the
array is sorted in reverse order.
just - same as best case

Qualified Notation
O Big-O Notation
best case O - describes an upper bound for best case
input, possibly lower than worst case. Example: the
array is already correctly sorted.
worst case O - describes an upper bound for all input (it
can't get any worse than this). Example: the array is
sorted in reverse order.
just O - same as worst case O

Qualified Notation
Theta Notation
best case - not used
worst case - describes asymptotic bounds for worst
case input
just - same as worst case

Comparing functions

Similarly for
Reflexivity
Symmetry
Transpose symmetry