Algorithms

Ch 4: Recurrences

Ming-Te Chi

Ch4 Recurrences

Recurrences

When an algorithm contains a recursive

call to itself, its running time can often
described by a recurrence.
A recurrence is an equation or
inequality that describes a function in
terms of its value of smaller inputs.

Ch4 Recurrences

Recurrencesexamples

Summation
n

n 1

i 1

i 1

i n i

Factorial

n ! n * ( n 1)!
Ch4 Recurrences

Recurrencesexamples

Summation

S ( n ) n S (n 1)

Factorial

n ! n * ( n 1)!
Ch4 Recurrences

Termination conditions
(boundary conditions)

Summation

S ( n ) n S ( n 1)
S (1) 1

Factorial

n ! n * ( n 1)!
1! 1
Ch4 Recurrences

RecurrencesCS

oriented

examples

Fibonacci number

F ( n ) F (n 1) F ( n 2)

Merge sort

n
T ( n ) 2T ( ) ( n )
2
Ch4 Recurrences

Influence of the boundary

conditions

Fibonacci number

F ( n ) F (n 1) F ( n 2)

Merge sort

n
T ( n ) 2T ( ) ( n )
2
Ch4 Recurrences

Recurrences
T ( n ) aT ( n / b ) f ( n )

Substitution method
Recursion-tree method
Master method

Ch4 Recurrences

Technicalities

Neglect certain technical details when

stating and solving recurrences.
A good example of a detail that is
often glossed over is the assumption
of integer arguments to functions.
Boundary conditions is ignored.
Omit floors, ceilings.

Ch4 Recurrences

The maximum-subarray
problem

Ch4 Recurrences

10

Ch4 Recurrences

11

Ch4 Recurrences

12

Matrix multiplication

Ch4 Recurrences

13

A simple divide-and-conquer
algorithm

Ch4 Recurrences

14

Ch4 Recurrences

15

Strassens method

Ch4 Recurrences

16

Ch4 Recurrences

17

The substitution method:

Mathematical induction

The substitution method for

solving recurrence has two steps:
1. Guess the form of the solution.
2. Use mathematical induction to find
the constants and show that the
solution works.

Powerful but can be applied only in

cases when it is easy to guess the
form of solution.
Ch4 Recurrences

18

Can be used to establish either

upper bound or lower bound on a
recurrence.

Ch4 Recurrences

19

General principle of
the Mathematical Induction

True for the trivial case,

problem size =1.
If it is true for the case of
problem at step k,
we can show that it is true for the case
of
problem at step k+1.

Ch4 Recurrences

20

Example
Determine the upper bound on the
recurrence
T ( n ) 2T ( n / 2 ) n

T (1 ) 1

(We may omit the initial condition later.)

Guess
T (n) O(n log n)
Prove

T (n ) cn log n

for some c > 0.

Ch4 Recurrences

21

Inductive hypothesis: assume this bound

holds for

T ( n / 2 ) c n / 2 log n / 2

The recurrence implies

n
T ( n ) 2T ( ) n
2
n
n
n
2[c log(
)] n cn log n

2
2
2
cn log n cn log 2 n cn log n (if c 1.)
Ch4 Recurrences

22

Initial condition

T (1) 1
T (1) cn log1 0 ( )

However

T (2) 4
cn log 2 (if c 4)
Ch4 Recurrences

23

Making a good guess

T ( n ) 2T ( n / 2 17 ) n

We guessT ( n ) O( n log n )
Making guess provides loose upper
bound and lower bound. Then
improve the gap.
Ch4 Recurrences

24

Ch4 Recurrences

25

Ch4 Recurrences

26

Subtleties
T ( n ) T ( n / 2 ) T ( n / 2 ) 1

Guess T ( n ) O( n )
AssumeT ( n ) cn
T ( n ) c n / 2 c n / 2 1 cn 1 cn

T ( n ) cn b
However, assume
T ( n ) ( c n / 2 b ) ( c n / 2 b ) 1
cn 2 b 1 cn b ( Choose b 1 )
Ch4 Recurrences

27

Avoiding pitfalls
T ( n ) 2T ( n / 2 ) n

T (1 ) 1
AssumeT ( n ) O( n )

Hence T ( n ) cn

T (n) 2(c n / 2 ) n cn n O(n)

(Since c is a constant)

(WRONG!) You cannot find such a

c.
Ch4 Recurrences

28

Changing variables

Ch4 Recurrences

29

4.4 the Recursion-tree

method
T (n) 3T ( n / 4 ) (n 2 )

Ch4 Recurrences

30

Ch4 Recurrences

31

n
Subproblem size for a node at depth i 4i

Total level of tree log 4 n 1

i
3
Number of nodes at depth i

n 2
c
(
)
Cost of each node at depth i
i
4
n
3
Total cost at depth i 3i c( i ) 2 ( )i cn 2
4
16

Last level, depthlog 4 n

, has3log4 n n log4 3

Ch4 Recurrences

nodes
32

log 4 n

Prove of 3
log 4 n

log 4 3

log 4 3

(log 4 n )(log 4 3)
(log 4 3)(log 4 n )
log 4 n

We conclude,

log 4 3

log 4 n

Ch4 Recurrences

log 4 3
33

The cost of the entire tree

2

3 2 3
3
2
T (n) cn cn
cn ...

16
16
16
2

log 4 n 1

i 0

3
log
2
cn

16

log 4 n 1

cn 2 (n log 3 )
4

(3 / 16)log n 1 2

cn (nlog 3 ).
(3 / 16) 1
4

Ch4 Recurrences

34

T ( n)

log 4 n 1

i 0
i

3
log
2
cn

16

3
log 3
2
cn n
i 0 16
1

cn 2 nlog
1 (3 / 16)
16 2
cn (nlog 3 )
13

O(n2 )
Ch4 Recurrences

35

Substitution method
We want to Show that T(n) dn2 for some
constant d > 0.
Using the same constant c > 0 as before, we
have
T ( n)
3T ( n / 4 ) cn 2

3d n / 4

cn 2

3d (n / 4) 2 cn 2
3 2

dn cn 2
16

dn 2 ,

Where the last step holds as long as d (16/13)c.

Ch4 Recurrences

36

T (n) T (n / 3) T (2n / 3) cn

Ch4 Recurrences

37

2 i
( i) n
Subproblem size for a node at depth
3

Total level of tree

log 3/ 2 n 1

Ch4 Recurrences

38

substitution method
T (n) T (n / 3) T (2n / 3) cn

d (n / 3) lg(n / 3) d (2n / 3) lg(2n / 3) cn

(d (n / 3) lg n d (n / 3) lg 3) (d (2n / 3) lg n d (2n / 3) lg(3 / 2)) cn

dn lg n d ((n / 3) lg 3 (2n / 3) lg(3 / 2)) cn

dn lg n d ((n / 3) lg 3 (2n / 3) lg 3 (2n / 3) lg 2 cn

dn lg n dn(lg 3 2 / 3) cn

dn lg n,

As long as d c/(lg3 (2/3)).

Ch4 Recurrences

39

4.5 Master method

Master Theorem

Ch4 Recurrences

40

Remarks 1:

In the first case, f(n) must be polynomailly

smaller than n logb a

That is, f(n) must be asymptotically

logb a
smaller
than
by a factor
n
n
of
Similarly, in the third case, f(n) must be
logb a
polynomailly larger than
n

Also, the conditiona f ( n / b) c f ( n )

must be hold.
Ch4 Recurrences

41

Remarks 2:

The three cases in the master theorem do

not cover all the possibilities.
There is a gap between cases 1 and 2
logb a than
when f(n) is smaller
but not
n
polynomailly smaller.
Similarly, there is a gap between cases 2
ba
and 3 when f(n) is larger
but
n logthan
not polynomailly larger (or the additional
condition does not hold.)
Ch4 Recurrences

42

Example 1:
T ( n ) 9T ( n / 3 ) n
a 9, b 3, f ( n ) n
n log3 9 n2 ,

f ( n ) O( n log3 9 1 )

Case 1 T ( n ) ( n2 )

Ch4 Recurrences

43

Example 2:
T( n ) T( 2n / 3 ) 1
a 1, b 3 / 2 , f ( n ) 1
n log3 / 2 1 n0 1 f ( n ),
Case

2 T ( n ) (log n )

Ch4 Recurrences

44

Example 3:
T (n) 3T (n / 4) n log n
a 3, b 4, f (n) n log n
nlog4 3 n 0.793 , f (n) (n log4 3 )
Case 3
Check
af(n/b)=3(

n
n
3n
) log ( )
log n=cf(n)
4
4
4

3
for c= , and sufficiently large n
4
T ( n ) ( n log n )
Ch4 Recurrences

45

Example 4:

Ch4 Recurrences

46

Example 5:

Ch4 Recurrences

47

Remarks 3:

The master theory does not apply to

the recurrence
T (n) 2T (n / 2) n lg n,
even though it has the proper form: a
log a
= 2, b=2, f(n)= n lgn,
n and n.
It might seem that case 3 should
apply, since f(n)= n lgn is
asymptotically
nlog a n. larger than
But it is not polynomially larger.
b

Ch4 Recurrences

48

Remarks 3 (continue)

However, the master method

augments the conditions

Case 2 can be applied.

Ch4 Recurrences

49

Example 6:

Ch4 Recurrences

50

Example 7:

Ch4 Recurrences

51

Ch4 Recurrences

52

Ch4 Recurrences

53