Professional Documents
Culture Documents
Merge Sort
2
Divide-and-Conquer
• Divide-and conquer is a general • Merge-sort is a sorting
algorithm design paradigm: algorithm based on the
– Divide: divide the input data S divide-and-conquer paradigm
in two disjoint subsets S1 and • Like heap-sort
S2
– It uses a comparator
– Recur: solve the subproblems
– It has O(n log n) running
associated with S1 and S2
time
– Conquer: combine the
• Unlike heap-sort
solutions for S1 and S2 into a
solution for S – It does not use an
auxiliary priority queue
• The base case for the recursion are
subproblems of size 0 or 1 – It accesses data in a
sequential manner
(suitable to sort data on a
disk)
3
Merge-Sort
• Merge-sort on an input Algorithm mergeSort(S, C)
sequence S with n Input sequence S with n
elements, comparator C
elements consists of
Output sequence S sorted
three steps: according to C
– Divide: partition S into if S.size() > 1
two sequences S1 and S2 (S1, S2) partition(S, n/2)
of about n/2 elements mergeSort(S1, C)
each mergeSort(S2, C)
– Recur: recursively sort S1 S merge(S1, S2)
and S2
– Conquer: merge S1 and S2
into a unique sorted
sequence
4
Merge-Sort
• MergeSort is a recursive sorting procedure that uses at most
O(n lg(n)) comparisons.
• To sort an array of n elements, we perform the following
steps in sequence:
• If n < 2 then the array is already sorted.
• Otherwise, n > 1, and we perform the following three steps
in sequence:
1. Sort the left half of the the array using MergeSort.
2. Sort the right half of the the array using MergeSort.
3. Merge the sorted left and right halves.
5
Merge-Sort
• The performance of any divide-and-conquer
algorithm will be good if the resultant subproblems
are as evenly sized as possible.
6
Performance of Merge Sort
• The performance of any divide-and-conquer algorithm will be
good if the resultant sub-problems are as evenly sized as
possible.
• Let T(N) denote the worst-case running time of mergesort to
sort N numbers.
• Assume that N is a power of 2.
• Divide step: O(1) time
• Conquer step: 2 T(N/2) time
• Combine step: O(N) time
• Recurrence equation: T(1) = 1 ; if N < 2
T(N) = 2T(N/2) + N ; if N 2
7
Merge sort
1. Algorithm MergeSort(low, high)
2. / / a[low : high] is a global array to be sorted.
3. / / Small(P) is true if there is only one element
4. / / to sort. In this case the list is already sorted.
5. {
6. if (low < high) then / / If there are more than one element
7. {
8. / / Divide P into subproblems.
9. / / Find where to split the set.
10. mid:= (low + high)/2;
11. / / Solve the subproblems.
12. MergeSort(low, mid);
13. MergeSort(mid + 1, high);
14. / / Combine the solutions.
15. Merge(low, mid, high);
16. }
17. }
8
Algorithm Merge(low, mid, high) : 1/2
9
Algorithm Merge(low, mid, high): 2/2
Quick Sort 10
Procedure Merge
Merge(A, p, q, r)
1 n1 q – p + 1
2 n2 r – q Input: Array containing sorted
3 for i 1 to n1 subarrays A[p..q] and A[q+1..r].
4 do L[i] A[p + i – 1]
5 for j 1 to n2
Output: Merged sorted subarray
6 do R[j] A[q + j] in A[p..r].
7 L[n1+1]
8 R[n2+1]
9 i1
10 j1
11 for k p to r Sentinels, to avoid having to
12 do if L[i] R[j]
13 then A[k] L[i]
check if either subarray is
14 ii+1 fully copied at each step.
15 else A[k] R[j]
16 jj+1
11
Merge – Example
A … 61 86 26
8 32
9 1
26 9
32 42 43 …
k k k k k k k k k
L 6 8 26 32
R 1 9 42 43
i i i i i j j j j j
12
Correctness of Merge
Merge(A, p, q, r)
1 n1 q – p + 1
Loop Invariant for the for loop
2 n2 r – q At the start of each iteration of the
3 for i 1 to n1 for loop:
4 do L[i] A[p + i – 1] Subarray A[p..k – 1]
5 for j 1 to n2 contains the k – p smallest elements
6 do R[j] A[q + j] of L and R in sorted order.
7 L[n1+1]
8 R[n2+1]
L[i] and R[j] are the smallest elements of
9 i1 L and R that have not been copied back into
10 j 1 A.
11 for k p to r
12 do if L[i] R[j]
13 then A[k] L[i] Initialization:
14 ii+1 Before the first iteration:
15 else A[k] R[j]
16 jj+1 •A[p..k – 1] is empty.
•i = j = 1.
•L[1] and R[1] are the smallest
elements of L and R not copied to A.
13
Correctness of Merge
Merge(A, p, q, r) Maintenance:
1 n1 q – p + 1
2 n2 r – q Case 1: L[i] R[j]
3 for i 1 to n1 •By LI, A contains p – k smallest elements
4 do L[i] A[p + i – 1] of L and R in sorted order.
5 for j 1 to n2 •By LI, L[i] and R[j] are the smallest
6 do R[j] A[q + j] elements of L and R not yet copied into A.
7 L[n1+1] •Line 13 results in A containing p – k + 1
8 R[n2+1]
9 i1 smallest elements (again in sorted order).
10 j 1 Incrementing i and k reestablishes the LI
11 for k p to r for the next iteration.
12 do if L[i] R[j] Similarly for L[i] > R[j].
13 then A[k] L[i]
14 ii+1 Termination:
15 else A[k] R[j] •On termination, k = r + 1.
16 jj+1 •By LI, A contains r – p + 1 smallest
elements of L and R in sorted order.
•L and R together contain r – p + 3 elements.
All but the two sentinels have been copied
back into A.
14
Working of Merg algorithm
• Input: Array A and indices p, q, r such that
p≤q<r
– Subarrays A[p . . q] and A[q + 1 . . r] are sorted
• Output: One single sorted subarray A[p . . r]
p q r
1 2 3 4 5 6 7 8
2 4 5 7 1 2 3 6
15
Merging
• Idea for merging: p q r
1 2 3 4 5 6 7 8
A1 A[p, q]
A[p, r]
A2 A[q+1, r] 16
Example: MERGE(A, 9, 12, 16)
p q r
17
Example: MERGE(A, 9, 12, 16)
18
Example (cont.)
19
Example (cont.)
20
Example (cont.)
Done!
21
Merge - Pseudocode
Algorithm: MERGE(A, p, q, r)
1. Compute n1 and n2
2. Copy the first n1 elements into
L[1 . . n1 + 1] and the next n2 elements into R[1 . . n2 +
1]
3. L[n1 + 1] ← ; R[n2 + 1] ← p q r
4. i ← 1; j ← 1 1 2 3 4 5 6 7 8
2 4 5 7 1 2 3 6
5. for k ← p to r
6. do if L[ i ] ≤ R[ j ] n1 n2
7. then A[k] ← L[ i ] p q
8. i ←i + 1
L 2 4 5 7
9. else A[k] ← R[ j ]
j←j+1
q+1 r
10.
R 1 2 3 6
22
Merging Two Sorted Sequences
• The conquer step of merge-sort consists of
merging two sorted sequences A and B into a
sorted sequence S containing the union of the
elements of A and B
• Merging two sorted sequences, each with n/2
elements and implemented by means of a doubly
linked list, takes O(n) time
23
Recurrence Equation Analysis
• The conquer step of merge-sort consists of merging two
sorted sequences, each with n/2 elements and
implemented by means of a doubly linked list, takes at
most bn steps, for some constant b.
• Likewise, the basis case (n < 2) will take at b most steps.
• Therefore, if we let T(n) denote the running time of
merge-sort:
b if n 2
T (n )
2T (n / 2) bn if n 2
• We can therefore analyze the running time of merge-
sort by finding a closed form solution to the above
equation.
– That is, a solution that has T(n) only on the left-hand side.
24
Iterative Substitution
• In the iterative substitution, or “plug-and-chug,” technique, we
iteratively apply the recurrence equation to itself and see if we
can find a pattern:
T ( n ) 2T ( n / 2) bn
2( 2T ( n / 2 2 )) b( n / 2)) bn
2 2 T ( n / 2 2 ) 2bn
2 3 T ( n / 2 3 ) 3bn
2 4 T ( n / 2 4 ) 4bn
...
2i T ( n / 2i ) ibn
• Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n.
• So,
T (n) bn bn log n
• Thus, T(n) is O(n log n).
25
WORKING OF MERGE SORT
26
Example – n Power of 2
1 2 3 4 5 6 7 8
Divide 5 2 4 7 1 3 2 6 q=4
1 2 3 4 5 6 7 8
5 2 4 7 1 3 2 6
1 2 3 4 5 6 7 8
5 2 4 7 1 3 2 6
1 2 3 4 5 6 7 8
5 2 4 7 1 3 2 6
27
Example – n Power of 2
1 2 3 4 5 6 7 8
Conquer 1 2 2 3 4 5 6 7
and
Merge 1 2 3 4 5 6 7 8
2 4 5 7 1 2 3 6
1 2 3 4 5 6 7 8
2 5 4 7 1 3 2 6
1 2 3 4 5 6 7 8
5 2 4 7 1 3 2 6
28
Example – n Not a Power of 2
1 2 3 4 5 6 7 8 9 10 11
4 7 2 6 1 4 7 3 5 2 6 q=6
Divide
1 2 3 4 5 6 7 8 9 10 11
q=3 4 7 2 6 1 4 7 3 5 2 6 q=9
1 2 3 4 5 6 7 8 9 10 11
4 7 2 6 1 4 7 3 5 2 6
1 2 3 4 5 6 7 8 9 10 11
4 7 2 6 1 4 7 3 5 2 6
1 2 4 5 7 8
4 7 6 1 7 3
29
Example – n Not a Power of 2
1 2 3 4 5 6 7 8 9 10 11
Conquer 1 2 2 3 4 4 5 6 6 7 7
and
1 2 3 4 5 6 7 8 9 10 11
Merge 1 2 4 4 6 7 2 3 5 6 7
1 2 3 4 5 6 7 8 9 10 11
2 4 7 1 4 6 3 5 7 2 6
1 2 3 4 5 6 7 8 9 10 11
4 7 2 1 6 4 3 7 5 2 6
1 2 4 5 7 8
4 7 6 1 7 3
30
Complexity of MergeSort
Pass Number of Merge list # of comps /
Number merges length moves per
merge
1 2k-1 or n/2 1 or n/2k 21
2 2k-2 or n/4 2 or n/2k-1 22
3 2k-3 or n/8 4 or n/2k-2 23
. . . .
. . . .
. . . .
k = log n
k–1 21 or n/2k-1 2k-2 or n/4 2k-1
k 20 or n/2k 2k-1 or n/2 2k
31
Complexity of MergeSort
32
Variants of merge sort
33
Insertion Sort
• Insertion Sort Insertion sort inserts each item into its proper
place in the final list.
• It consumes one input at a time and growing a sorted list.
• It is highly efficient on small data and is very simple and easy
to implement.
• The worst case complexity of insertion sort is O(n²). When
data is already sorted it is the best case of insertion sort and it
takes O(n) time
34
Insertion Sort
35
Time complexity of Insertion Sort
• The complexity of the algorithm is result of the statements
within the domain of the while loop.
• This loop can executed minimum zero times to a maximum of
j times. The J varies from 2 to n, hence worst case time
complexity of the insertion sort can be computed as :
𝑛 𝑛+1
𝑗 = − 1 = (𝑛2 )
2
2𝑗𝑛
36
Revised version of merge sort with Insertion
sort with elements are stored in linked list
37
Modified Merge Sort: MergSort1
38
Merging link list of sorted elements
39
Merge sort on linked list
100
108
ADDR DATA LINK
100 11 108 ADDR DATA LINK
108 3 116 100 11 156
116 39 124 108 3 164
124 50 132 116 39 140
132 21 140 124 50 NULL
140 42 148 132 21 172
148 7 156 140 42 124
156 18 164 148 7 100
164 6 172 156 18 132
172 36 NULL 164 6 148
172 36 116
40
92
92
ADDR DATA LINK
92 100 ADDR DATA LINK
100 11 108 92 108
108 3 116 100 11 156
116 39 124 108 3 164
124 50 132 116 39 140
132 21 140 124 50 NULL
140 42 148 132 21 172
148 7 156 140 42 124
156 18 164 148 7 100
164 6 172 156 18 132
172 36 NULL 164 6 148
172 36 116
41
Sorting Challenge 1
42
Sorting Files with Huge Records and Small Keys
• Selection sort?
43
Sorting Challenge 2
Problem: Sort a huge randomly-ordered file of small records
Application: Process transaction record for a phone company
44
Sorting Huge, Randomly - Ordered Files
• Selection sort ?
– NO, always takes quadratic time
• Bubble sort?
– NO, quadratic time for randomly-ordered keys
• Insertion sort?
– NO, quadratic time for randomly-ordered keys
• Mergesort?
– YES, it is designed for this problem
45
Sorting Challenge 3
Problem: sort a file that is already almost in order
Applications:
– Re-sort a huge database after a few changes
– Doublecheck that someone else sorted a file
Which sorting method to use?
A. Mergesort, guaranteed to run in time NlgN
B. Selection sort
C. Bubble sort
D. A custom algorithm for almost in-order files
E. Insertion sort
46
Sorting Files That are Almost in Order
• Selection sort?
– NO, always takes quadratic time
• Bubble sort?
– NO, bad for some definitions of “almost in order”
– Ex: B C D E F G H I J K L M N O P Q R S T U V W X Y Z A
• Insertion sort?
– YES, takes linear time for most definitions of “almost in
order”
• Mergesort or custom method?
– Probably not: insertion sort simpler and faster
47
Exercises
Chapter 3 : Divide & Conqure
49
Suggested Exercises
1. Is merge sort a stable sorting algorithm?
2. How would you implement merge sort without using
recursion?
3. Determine the running time of merge sort for (i) sorted
input, (ii) reverse-ordered input, and (iii) random input.
4. The worst case running time of mergesort is O(n logn). What
is its best-case time? Can we say that the time for merge
sort is ( n log n)?
5. Consider the following variation on MERGESORT for large
values of n. Instead of recursing until n is sufficiently small
recur at most a constant r times, and then use insertion sort
to solve the 2r resulting sub-problems. What is the
(asymptotic) running time of this variation as a function of n?
50
Suggested Exercises
6. Can we make a general statement that "if a problem can be
split using D&C strategy in almost equal portion at each
stage, then it is a good candidate for recursive
implementation, but if it cannot be easily divided into equal
portions, than it is better to be implemented iteratively“?
Explain with the help of an example.
52
53
Thanks for Your Attention!
54