Fundamental Algorithms

CSCI-GA.1170-001/Summer 2016
Solution to Homework 1
Problem 1. (2 points) Give an algorithm to compute the index of the maximum element in
an array. Prove correctness of your algorithm using a loop invariant (refer to pages 19-20 in
section 2.1 of CLRS for an example), state the running time.
Solution:
Algorithm pseudocode:

def find_max (a ):
if len (a) == 0:
return -1
max = 0
for i in range (1 , len (a )):
if a[i] > a[ max ]:
max = i
return max
We prove the algorithm correct using the following loop invariant:
At the start of each iteration of the for loop on lines 5-7, a[max] is the largest
element in subarray a[0..i-1].
Initialization: Before the first loop iteration, max = 0, i = 1 and therefore a[max] is
correctly the largest (and only) element in a[0..0]. Note that the array is guaranteed
to have at least one element by the guard condition on lines 2-3.
Maintenance: Assuming that a[max] is correctly the largest element in a[0..i-1] at the
start of iteration for i = k, let us show that this property holds at the start of iteration
for i = k+1. There are two possible executions of the iteration for i = k:
1. a[k] > a[max]. Given that a[max] is the largest element in a[0..k-1], a[k] is the
new largest element in a[0..k]. After line 7 updates max and i is incremented to
k+1 in line 5, a[max] is correctly the largest element in a[0..k] and the invariant
holds.
2. a[k] <= a[max]. a[k] is no greater that the largest element in a[0..k-1] and
thus cannot be the largest element in a[0..k]. max is not updated, so after i is
incremented to k+1 in line 5, a[max] is still correctly the largest element in a[0..k]
and the invariant holds.
Termination: By maintenance, at the beginning of iteration for i = len(a), a[max] is the
largest element in subarray a[0..len(a)-1]. i = len(a) causes the loop to terminate
without executing the body, max is not changed, and the invariant holds.

1
2
3
4
5
6
7
8

Observing that a[0..len(a)-1] is the entire array and a[max] is thus the largest element
in the entire array, we conclude that the algorithm is correct.
Passing in an empty array will execute a constant number of operations on lines 2-3, so
the algorithm runs in (1) time in best case.
In all other cases, a constant number of operations will be executed on lines 4 and 8,
and at most linear in len(a) number of operations on lines 5-7. Note that while line 7 is
executed conditionally, lines 5-6 are always executed a linear in len(a) number of times.
Therefore, we can provide a tight bound of (len(a)) for both average and worst cases.
Problem 2. (2 points) The Lucas numbers are defined as follows:

if n = 0
2
Ln = 1
if n = 1

L n1 + L n2 if n > 1
Prove by induction the closed-form expression for the n-th Lucas number:
L n = n + (1 )n
Where is the golden ratio:
p
1+ 5
=
2
Solution:
The inductive step for L n+1 will have to rely on L n and L n1 , so we need to show that the
statement holds for two base cases:
for n = 0, L0 = 0 + (1 )0 = 2,
for n = 1, L1 = 1 + (1 )1 = 1.
Similarly, the inductive hypothesis needs to include two cases. We assume that for some n and
n + 1:
L n = n + (1 )n ,
L n+1 = n+1 + (1 )n+1 .
For the inductive step let us show that:
L n+2 = n+2 + (1 )n+2 .
By definition of L n and the inductive hypothesis:
L n+2 = L n+1 + L n
= n+1 + (1 )n+1 + n + (1 )n .
2

(IS.1)

Observing that:
2 = + 1,
k 2 = k ( + 1),
k+2 = k+1 + k .
We can complete the inductive step by rewriting (IS.1) as:
L n+2 = n+1 + n + (1 )n+1 + (1 )n
= n+2 + (1 )n+2 .
With both the base case and the inductive step in place, we conclude that the statement holds
for all natural numbers n. 
Problem 3. (3 points) State formally, then prove or disprove the following conjectures:
(a) Being in is an equivalence relation.
(b) Maximum of two functions is in of their sum.
(c) Sum of two functions is in of their maximum.
Solution:
(a) To show that binary relation "is in of" is an equivalence relation we need to show that it
is reflexive, symmetric, and transitive.
Reflexivity: f (n) = ( f (n)). Observe that:
0 c1 f (n) f (n) c2 f (n)
for c1 = c2 = 1 and any n0 > 0.
Symmetry: f (n) = (g(n)) if and only if g(n) = ( f (n)). Let us show that:
If f (n) = (g(n)) then g(n) = ( f (n)).
By definition of :
0 c10 g(n) f (n) c20 g(n)
for all n

n00

and some positive

(S.1)

c10 , c20 , n00 .

Dividing (S.1) by c10 gives:

0 g(n)

c20
1
f
(n)

g(n).
c10
c10

(S.2)

1
f (n) g(n).
c20

(S.3)

Dividing (S.1) by c20 gives:

0

c10
c20

g(n)

Combining (S.2) and (S.3) gives:

0

1
1
f (n) g(n) 0 f (n).
0
c2
c1

Or, in canonical form:

0 c1 f (n) g(n) c2 f (n)
1
1
for all n n0 and c1 = 0 , c2 = 0 , n0 = n00 .
c2
c1
The proof for converse is analogous.
Transitivity: If f (n) = (g(n)) and g(n) = (h(n)) then f (n) = (h(n)). By definition
of :
0 c10 g(n) f (n) c20 g(n)
for all n

n00

and some positive

0 c100 h(n) g(n) c200 h(n)

for all n

n000

and some positive

(T.1)

c10 , c20 , n00 .

(T.2)

c100 , c200 , n000 .

Multiplying (T.2) by c20 gives:

0 c20 c100 h(n) c20 g(n) c20 c200 h(n).
Combining the above with (T.1) gives:
f (n) c20 g(n) c20 c200 h(n),
f (n) c20 c200 h(n).

(T.3)

Multiplying (T.2) by c10 gives:

0 c10 c100 h(n) c10 g(n) c10 c200 h(n).
Combining the above with (T.1) gives:
c10 c100 h(n) c10 g(n) f (n),
c10 c100 h(n) f (n).
Combining (T.3) and (T.4) gives:
0 c10 c100 h(n) f (n) c20 c200 h(n).
Or, in canonical form:
0 c1 h(n) f (n) c2 h(n)
for all n n0 and c1 = c10 c100 , c2 = c20 c200 , n0 = max(n00 , n000 ).
4

(T.4)

(b) Let us show that max( f (n), g(n)) = ( f (n) + g(n)):

Observe that for non-negative f (n) and g(n):
0 f (n) + g(n) 2 max( f (n), g(n)),
0 max( f (n), g(n)) f (n) + g(n).
Assuming f (n) 0 for all n n00 and g(n) 0 for all n n000 , we can write:
0 c1 ( f (n) + g(n)) max( f (n), g(n)) c2 ( f (n) + g(n))
1
for c1 = , c2 = 1, and all n max(n00 , n000 ).
2
(c) The symmetry of relation "is in of" established in (a):
max( f (n), g(n)) = ( f (n) + g(n))
if and only if
f (n) + g(n) = (max( f (n), g(n))).
Taken with (b) implies:
f (n) + g(n) = (max( f (n), g(n))).
Problem 4. (4 points) Rank the following function forms by order of growth: constant, exponential, linear, linearithmic, logarithmic, polynomial. Formulate your answer as five comparisons of the form:
Informal statement about relation between orders of growth of two function forms.
Formal statement in terms of limits or asymptotic notation (feel free to switch between
the two if it makes your proofs easier).
Proof of the formal statement.
Solution: We rank the function forms using -notation and the equivalent limit definition:
f (n) = (g(n)) if and only if lim

f (n)
= .
g(n)

LR

(= indicates application of LHopitals rule.)

Logarithmic functions grow asymptotically faster than constant functions.
log n = (c) for all c > 0:
log n
= .
n
c
lim

 Linear functions grow asymptotically faster than logarithmic functions.

n = (log n):
n LR
1
= lim 1 = lim n = .
n
n log n
n
n
lim

Linearithmic functions grow asymptotically faster than linear functions.

n log n = (n):
n log n
= lim log n = .
n
n
n
lim

Polynomial functions grow asymptotically faster than linearithmic functions.

nd = (n log n) for all d > 1:
nd
nd1 LR
(d 1)nd2
= (d 1) lim nd1 = .
lim
= lim
= lim
1
n
n n log n
n log n
n
n
Exponential functions grow asymptotically faster than polynomial functions.
a n = (nd ) for all a > 1 and d > 0, k = dde:
k
a n LR ln a
a n LR ln2 a
a n LR
LR ln a
=
lim
=
lim
=

=
lim a n = .
n nk
k n nk1
k(k 1) n nk2
k! n

lim