Divide and Conquer Algorithms

- Example
- Binary Search
- Merge Sort
- Quick Sort

1
 Divide and Conquer

Divide the problem into a number of subproblems

There must be base case (to stop recursion).

Conquer (solve) each subproblem recursively

Combine (merge) solutions to subproblems into a solution to the

original problem

2
 Divide-and-Conquer

Most common usage.

Break up problem of size n into two equal parts of size n.
Solve two parts recursively.
Combine two solutions into overall solution in linear time.

Consequence.
Brute force: n2.
Divide-and-conquer: n log n.

3
 Closest Pair of Points

Closest pair. Given n points in the plane, find a pair

with smallest Euclidean distance between them
(closest pair).

Fundamental geometric primitive.

Graphics, computer vision, geographic information systems,
molecular modeling, air traffic control.
Special case of nearest neighbor, Euclidean MST, Voronoi.
Brute force. Check all pairs of points p and q with
(n2) comparisons.
1-D version. O(n log n) easy if points are on a line.

Assumption. No two points have same x

coordinate.

4
 Closest Pair of Points: First Attempt

Divide. Sub-divide region into 4 quadrants.

5
 Closest Pair of Points: First Attempt

Divide. Sub-divide region into 4 quadrants.

Obstacle. Impossible to ensure n/4 points in each piece.

6
 Closest Pair of Points

Algorithm.
Divide: draw vertical line L so that roughly n points on each
side.

7
 Closest-Pair Problem:
Divide and Conquer

Brute force approach requires comparing every point with

every other point
Given n points, we must perform 1 + 2 + 3 + + n-2 + n-1
comparisons.
n 1
(n 1) n
k
k 1 2
Brute force O(n2)
The Divide and Conquer algorithm yields O(n log n)
Reminder: if n = 1,000,000 then
n2 = 1,000,000,000,000 whereas
n log n = 20,000,000
 Closest-Pair Algorithm

Given: A set of points in 2-D

Closest-Pair Algorithm

Step 1: Sort the points in one D

Closest-Pair Algorithm
Lets sort based on the X-axis
O(n log n) using quicksort or mergesort

4
9

6 13
2
14
11
3 7 10
5
1 8
12
Closest-Pair Algorithm
Step 2: Split the points, i.e.,
Draw a line at the mid-point between 7 and 8

4
9

6 13
2
14
11
3 7 10
5
1 8
12

Sub-Problem 1 Sub-Problem 2
Closest-Pair Algorithm
Advantage: Normally, wed have to compare
each of the 14 points with every other point.
(n-1)n/2 = 13*14/2 = 91 comparisons

4
9

6 13
2
14
11
3 7 10
5
1 8
12

Sub-Problem 1 Sub-Problem 2
Closest-Pair Algorithm
Advantage: Now, we have two sub-problems of
half the size. Thus, we have to do 6*7/2
comparisons twice, which is 42 comparisons
solution d = min(d1, d2)

4
9

d1 6 13
2 d2
14
11
3 7 10
5
1 8
12

Sub-Problem 1 Sub-Problem 2
Closest-Pair Algorithm
Advantage: With just one split we cut the
number of comparisons in half. Obviously, we
gain an even greater advantage if we split the
sub-problems.
d = min(d1, d2)
4
9

d1 6 13
2 d2
14
11
3 7 10
5
1 8
12

Sub-Problem 1 Sub-Problem 2
Closest-Pair Algorithm
Problem: However, what if the closest two
points are each from different sub-problems?

4
9

d1 6 13
2 d2
14
11
3 7 10
5
1 8
12

Sub-Problem 1 Sub-Problem 2
Closest-Pair Algorithm
Here is an example where we have to compare
points from sub-problem 1 to the points in sub-
problem 2.

d1 6 13
2 d2
9 14
7 11
3 10
5 8
1
12

Sub-Problem 1 Sub-Problem 2
Closest-Pair Algorithm
However, we only have to compare points
inside the following strip.

d = min(d1, d2)
4
9

d1 d d 13
2 6
d2
14
11
3 7 10
5
1 8
12

Sub-Problem 1 Sub-Problem 2
Closest-Pair Algorithm
Step 3: But, we can continue the advantage by
splitting the sub-problems.

4
9

6 13
2
14
11
3 7 10
5
1 8
12
Closest-Pair Algorithm
Step 3: In fact we can continue to split until
each sub-problem is trivial, i.e., takes one
comparison.

4
9

6 13
2
14
11
3 7 10
5
1 8
12
Closest-Pair Algorithm
Finally: The solution to each sub-problem is
combined until the final solution is obtained

4
9

6 13
2
14
11
3 7 10
5
1 8
12
Closest-Pair Algorithm
Finally: On the last step the strip will likely
be very small. Thus, combining the two largest
sub-problems wont require much work.

4
9

6 13
2
14
11
3 7 10
5
1 8
12
Closest-Pair Algorithm
In this example, it takes 22 comparisons to find the
closets-pair.
The brute force algorithm would have taken 91
comparisons.
But, the real advantage occurs when there are millions of
points.
4
9

6 13
2
14
11
3 7 10
5
1 8
12
 Closest Pair Divide & Conquer
dimension d = 2
Partition two dimensional set S into subsets S1 and S2 by a
vertical line l at the median x coordinate of S.
Solve the problem recursively on S1 and S2.
Let {p1, p2} be the closest pair in S1 and {q1, q2} in S2.
Let 1 = distance(p1,p2) and 2 = distance(q1,q2)
Let = min(1, 2)

l
S1 S2

p1

1
p2
q1

q2 2
 Closest Pair Divide & Conquer
dimension d = 2
In order to merge we have to
determine if exists a pair of points {p, P1 l P2
q} where p S1, q S2 and
distance(p, q) < .
If so, p and q must both be within of
l. p1
Let P1 and P2 be vertical regions of q1
1
the plane of width on either side of l. p2 S2
If {p, q} exists, p must be within P1 S1
2
and q within P2.
However, every point in S1 and S2 may q2
be a candidate, as long as each is
within of l, which implies: O(n/2)
O(n/2) = O(n2)

Can we do better ?
 Closest Pair Divide & Conquer
dimension d = 2
For a point p in P1, which portion of P1 l P2
P2 should be checked?

We only need to check the points
that are within of p.
Thus we can limit the portion of P2.
The points to consider for a point
p must lie within 2 rectangle R. S1 S2
p
At most, how many points are
there in rectangle R?
 Closest Pair Divide & Conquer
dimension d = 2

How many points are there in P1 l P2

rectangle R?
Since no two points can be

closer than , there can only

be at most 6 points
Therefore, 6 O(n/2)O(n)
S2
Thus, the time complexity is
S1
p
O(n log n)
R

How do we know which 6 points to check?

 Closest Pair Divide & Conquer
dimension d = 2

How do we know which 6 points to check?

Project p and all the points of S2 within P2 onto l.
Only the points within of p in the y projection
need to be considered (max of 6 points).
After sorting the points on y coordinate we can
find the points by scanning the sorted lists. Points
are sorted by y coordinates.
To prevent resorting in O(n log n) in each merge,
two previously sorted lists are merged in O(n).
Time Complexity: O(n log n)
 Counting Inversions

Music site tries to match your song preferences with others.

You rank n songs.
Music site consults database to find people with similar tastes.

Similarity metric: number of inversions between two rankings.

My rank: 1, 2, , n.
Your rank: a1, a2, , an.
Songs i and j inverted if i <= j, but ai > aj.

Songs

A B C D E
Inversions
Me 1 2 3 4 5
3-2, 4-2
You 1 3 4 2 5

Brute force: check all (n2) pairs i and j.

29
 Counting Inversions: Divide-and-Conquer

Divide-and-conquer.

1 5 4 8 10 2 6 9 12 11 3 7

30
 Counting Inversions: Divide-and-Conquer

Divide-and-conquer.
Divide: separate list into two pieces.

Divide: O(1).
1 5 4 8 10 2 6 9 12 11 3 7

1 5 4 8 10 2 6 9 12 11 3 7

31
 Counting Inversions: Divide-and-Conquer

Divide-and-conquer.
Divide: separate list into two pieces.
Conquer: recursively count inversions in each half.

Divide: O(1).
1 5 4 8 10 2 6 9 12 11 3 7

1 5 4 8 10 2 6 9 12 11 3 7 Conquer: 2T(n / 2)

5 blue-blue inversions 8 green-green inversions

5-4, 5-2, 4-2, 8-2, 10-2 6-3, 9-3, 9-7, 12-3, 12-7, 12-11, 11-3, 11-7

32
 Counting Inversions: Divide-and-Conquer

Divide-and-conquer.
Divide: separate list into two pieces.
Conquer: recursively count inversions in each half.
Combine: count inversions where ai and aj are in different halves,
and return sum of three quantities.

Divide: O(1).
1 5 4 8 10 2 6 9 12 11 3 7

1 5 4 8 10 2 6 9 12 11 3 7 Conquer: 2T(n / 2)

5 blue-blue inversions 8 green-green inversions

9 blue-green inversions
Combine: ???
5-3, 4-3, 8-6, 8-3, 8-7, 10-6, 10-9, 10-3, 10-7

Total = 5 + 8 + 9 = 22.

33
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.

18 7 14 10 3 19 17 25 16 2 23 11

How many blue-green inversion : 13

34
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 6 j = 6

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

auxiliary array

current Inversions:
Total: 0

35
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 6 j = 5

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 auxiliary array

current Inversions: 6
Total: 6

36
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 6 j = 5

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 auxiliary array

current Inversions: 6
Total: 6

37
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 5 j = 5

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 auxiliary array

current Inversions: 6
Total: 6

38
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 5 j = 5

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 auxiliary array

current Inversions: 6
Total: 6

39
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 4 j = 5

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 auxiliary array

current Inversions: 6
Total: 6

40
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 4 j = 5

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 auxiliary array

current Inversions: 6
Total: 6

41
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 3 j = 5

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 auxiliary array

current Inversions: 6
Total: 6

42
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 3 j = 5

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 auxiliary array

current Inversions: 6
Total: 6

43
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 3 j = 4

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 auxiliary array

Inversions: 6 + 3
Total: 9

44
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 3 j = 4

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 auxiliary array

Inversions: 6 + 3
Total: 9

45
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 2 j = 4

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 auxiliary array

Inversions: 6 + 3
Total: 9

46
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 2 j = 4

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 auxiliary array

Inversions: 6 + 3
Total: 9

47
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 2 j = 3

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 auxiliary array

Inversions: 6 + 3 + 2
Total: 11

48
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 2 j = 3

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 auxiliary array

Inversions: 6 + 3 + 2
Total: 11

49
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 2 j = 2

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 auxiliary array

Inversions: 6 + 3 + 2 + 2
Total: 13

50
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 2 j = 2

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 auxiliary array

Inversions: 6 + 3 + 2 + 2
Total: 13

51
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 1 j = 2

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 18 auxiliary array

Inversions: 6 + 3 + 2 + 2
Total: 13

52
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
i = 1 j = 2

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 18 auxiliary array

Inversions: 6 + 3 + 2 + 2
Total: 13

53
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
first half i = 0 j = 2
exhausted

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 18 19 auxiliary array

Inversions: 6 + 3 + 2 + 2
Total: 13

54
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
first half i = 0 j = 2
exhausted

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 18 19 auxiliary array

Inversions: 6 + 3 + 2 + 2
Total: 13

55
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
first half i = 0 j = 2
exhausted

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 18 19 23 auxiliary array

Inversions: 6 + 3 + 2 + 2 + 0
Total: 13

56
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
first half i = 0 j = 1
exhausted

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 18 19 23 auxiliary array

Inversions: 6 + 3 + 2 + 2 + 0
Total: 13

57
 Merge and Count

Merge and count step.

Given two sorted halves, count number of
inversions where ai and aj are in different
halves.
Combine two sorted halves into sorted
whole.
smallest smallest
first half i = 0 second half j = 0
exhausted exhausted

3 7 10 14 18 19 2 11 16 17 23 25 N/2 = 6

2 3 7 10 11 14 16 17 18 19 23 25 auxiliary array

Inversions: 6 + 3 + 2 + 2 + 0 + 0
Total: 13

58