Lecture 2

Sorting Algorithms
 Sorting Algorithms

Input:
A sequence of n numbers <a1, a2, .., an>

Output:
A permutation (reordering) <a1, a2, ..,
an> of the input sequence such that a1
a2 ..an

The numbers we wish to sort are known

keys.
 Sorting Classes
Sorting Algorithms are usually divided
into 2 classes: internal sorting
algorithms and external sorting
algorithms.
Internal sorting algorithms assume

that data is stored in an array in main

memory
External sorting algorithms

assume data is stored on disk or

some other device that is best
accessed sequentially
 Properties
In-place
The algorithm uses no additional array
storage and hence (other than perhaps the
systems recursion stack) it is possible to
sort very large lists without the need to
allocate additional working storage.
Stable
The algorithm is stable if two elements
that are equal remain in the same relative
position after sorting is completed.
This is of interest, since in some sorting
applications, you sort first on one key and
then on another. It is nice to know that
two items that are equal on the second
key, remain sorted in the first key
Comparison-based Sorting
Algorithms
Determine the sorted order of
an input array by comparing
elements
Insertion Sort
Heapsort

Mergesort

Quicksort
 Insertion Sort

Inserts each item into its proper place

in the final sorted list.
Efficient algorithm for sorting a small
number of elements
Stable and in-place sorting algorithm
Easiest and simplest implementation:
2 list structures
source list
list into which sorted items are inserted.

In-place approach: moving the current

item past the already sorted items and
repeatedly swapping it with the
preceding item until it is in-place
 Insertion Sort Algorithm
For j 2 to length [A] do
key A[j]
/* insert A[j] into the sorted sequence A[1..j-1] */

ij1
while i > 0 and A[i] > key do
A[i+1] A[i]
ii1
A[i+1] key

*Consider A = <5, 2, 4, 6, 1, 3> & A=<31, 41,59, 26, 41, 58>.

 Analysis - Insertion Sort
Time taken by the algorithm is
dependent on the input and the number
of comparisons
Best Case:
Sorted List
n-1 Comparisons
Worst Case:
Inverse sorted list
Consider there are n-1 insert operations.
The j-th insert operation inserts a new
element into a sorted array of j elements
and hence takes at most j comparisons.
Thus the total number of comparisons is
1 + 2 + 3 +... + (n-1) = n*(n-1)/2 = O(n2)
Useful Links
http://www.cs.brockport.edu/cs/java/apps/sor
ters/insertsort.html
http://linux.wku.edu/~lamonml/algor/sort/ins
ertion.html
http://www.epaperpress.com/sortsearch/
http://www.animal.ahrgr.de/en/Animation33
.html
http://www.animal.ahrgr.de/en/Animation15
9.html
http://www.animal.ahrgr.de/en/Animation16
0.html
http://www.csse.monash.edu.au/~lloyd/tilde
AlgDS/Sort/Insertion/
 HeapSort
Based on a data structure called Heap, which
is an efficient implementation of a priority
queue data structure.
Heap data structure
An array object that can be viewed as a nearly
complete binary tree
Each node of the tree corresponds to an element of
the array that stores the value in the node.
The tree is completely filled on all levels except
possibly the lowest, which is filled from the left up to
a point.
Height is O(lg n) since it is based on a complete
binary tree
In-place sorting algorithm
Array is used to represent a heap
Length [A] = number of elements in the array
Heap-size[A] = number of elements in the heap
stored within array A.
Kinds of Binary Heaps:
max-heap
for every node i other than the root node,
H[PARENT(i)] H[i].
The value of a node is at most the value of its
parent.
Largest element is stored at the root
Subtree rooted at a node contains values no
larger than that contained at the node itself.
min-heap
For every nodeHi other than the root node,
H[PARENT(i)] H[i]
The smallest element is at the root
The root of the tree is at A[i]
Index of Parent (i)
return i/2
Index of Left (i)
return 2i
Index of Right (i)
return 2i + 1
A nearly complete
binary tree; max-heap

16
2 3

14 10
4 6
5 7

8 7 9 3
8 9 10

2 4 1
 buildHeap extractMax

Heap
BUILD-MAX-HEAP(H) produces a max-heap from an
unordered input array; runs in linear time

MAX-HEAPIFY(H, i) maintains max-heap property;

O(lg n) or based on the height O(h)

HEAPSORT sorts an array in place but not stable

O(n lg n)

MAX-HEAP-INSERT, MAX-EXTRACT-MAX, HEAP-

INCREASE KEY AND HEAP MAXIMUM allows
heap structures to be used as a priority queue;
O(lg n)
 Max-Heapify (A, i) O(lg n)
L Left (i)
R Right (i)
If L <= heapsize [A] and A[L] > A[i] then
largest L
Else
largest i
If R <= heapsize [A] and A[R] > A[largest] then
largest R
If largest <> i then
exchange A[i] and A[largest]
Max-heapify (A, largest)

Build-Max Heap(A) O(n)

Heap-size [A] length [A]
For I length [A]/2 downto 1
do Max-Heapify (A, I)
 HeapSort (A) O(n lg n)

Build-Max-Heap (A)
For K length [A] downto 2 do
exchange A[1] and A[K]
heapsize[A] heapsize [A] 1
Max-Heapify (A, 1)
 Heaps
H = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7}

H 1
4
2 3
1 3
4 5 6 7
2 16 9 10
8 9 10
14 8 7

H 1
4
2 3
MAX-HEAPIFY(H, 5) 1 3
4 5 6 7
2 16 9 10
8 9 10
14 8 7
 H 1
4
2 3
1 3
4 5 6 7
2 16 9 10
8 9 10
14 8 7

MAX-HEAPIFY(H, 4) H 1
4
2 3
1 3
4 5 6 7
2 16 9 10
8 9 10
14 8 7
 H 1
4
2 3
1 3
4 5 6 7
2 16 9 10
8 9 10
14 8 7

MAX-HEAPIFY(H, 3) H 1
4
2 3
1 3
4 5 6 7
14 16 9 10
8 9 10
2 8 7
 H 1
4
2 3
1 3
4 5 6 7
14 16 9 10
8 9 10
2 8 7

H 1
MAX-HEAPIFY(H, 2)
4
2 3
1 10
4 5 6 7
14 16 9 3
8 9 10
2 8 7
 H 1
4
2 3
1 10
4 5 6 7
14 16 9 3
8 9 10
2 8 7

MAX-HEAPIFY(H, 1) H 1
4
2 3
16 10
4 5 6 7
14 7 9 3
8 9 10
2 8 1
 H 1
4
2 3
16 10
4 5 6 7
14 7 9 3
8 9 10
2 8 1

H 1
FINAL TREE 16
2 3
14 10
4 5 6 7
8 7 9 3
8 9 10
2 4 1
Exercises:
1. Illustrate the operation of BUILD-MAX-
HEAP on array
H = {5, 3, 17, 10, 84, 19, 6, 22, 9}
2. Illustrate the operation of BUILD-MAX-
HEAP on array
H = {27, 17, 3, 16, 13, 10, 1, 5, 7, 12, 4,
8, 9, 0}
3. Illustrate the operation of Heapsort on
the array H = {5, 13, 2, 25, 7, 17, 20, 8,
4}
 Priority Queues (Max)
A priority Queue is a data structure for
maintaining a set S elements, each with an
associated value called Key.
Operations:
(a) Max-Heap-Insert (S, x) inserts the
element x into the set S.
(b) Heap-Maximum (S) returns the element
of S with the largest key
(c) Heap-Extract-Max (S) removes and
returns the element of S with the largest
key
(d) Heap-Increase-Key(S, x, k) increases the
value of element xs key to the new value
k, which is assumed to be at least as large
as xs current key value.
Heap-Extract-Max (A) O(lg n)

if (HEAP-SIZE[A] < 1) then

printf(Heap Underflow!\n);
else {
max = A[1];
A[1] = A[HEAP-SIZE[A]]
HEAP-SIZE [A]= HEAP-SIZE[A] 1;
MAX-HEAPIFY(A, 1);
return max;
}
 HEAP-EXTRACT-MAX(PQ)

PQ 1
16
2 3
14 10
4 5 6 7
8 7 9 3
8 9 10
2 4 1

PQ 1
1
Extracting PQ[1], MAX-HEAPIFY(PQ, 1) 2 3
14 10
4 5 6 7
8 7 9 3
8 9
2 4
max = 16
 PQ 1
1
2 3 max = 16
14 10
4 5 6 7
8 7 9 3
8 9
2 4

PQ 1
MAX-HEAPIFY(PQ, 2)
14
2 3
1 10
4 5 6 7
8 7 9 3
8 9
2 4
max = 16
 PQ 1
14 max = 16
2 3
1 10
4 5 6 7
8 7 9 3
8 9
2 4

PQ 1
MAX-HEAPIFY(PQ, 4) 14
2 3
8 10
4 5 6 7
1 7 9 3
8 9
2 4

max = 16
 PQ 1
14
2 3
max = 16
8 10
4 5 6 7
1 7 9 3
8 9
2 4

PQ 1
14
MAX-HEAPIFY(PQ, 9) 2 3
8 10
4 5 6 7
4 7 9 3
8 9
2 1

max = 16
 PQ 1
14
2 3
8 10
4 5 6 7
4 7 9 3
8 9
2 1

max = 16

The resulting heap. HEAP-EXTRACT-MAX(PQ) will return 16.

Exercise:
Illustrate the operation of HEAP-
EXTRACT-MAX on the heap H =
{15, 13, 9, 5, 12, 8, 7, 4, 0, 6, 2, 1}.
 Heap-Maximum (A)
Return A[1]

Heap-Increase-Key (A, i, key) O(lg n)

If key < A[i] then error: new key is smaller than current key
A[i] key
While i > 1 and A[Parent(i)] < A[i] do
exchange A[i] and A[Parent(i)]
i Parent (i)
Or simply stated as
Change value of <node> to <value>

Reposition <node>

While Parent (<node>) != root_node and

(value of Parent (<node>) < <node>)

Swap contents of Parent (<node>) and
<node>
<node> = Parent (<node>)
Max-Heap-Insert (A, key) O(lg n)
Heap-size[A] heap-size [A] + 1
A[heap-size[A]] -
Heap-Increase-Key (A, heap-size[A], key)

Or simply stated as
Add a new node to the next available
position of the heap
Assign an arbitrary value to it
Heap-Increase-Key (<heap>,
<new_node>, <key>)
Max-Heap-Insert (PQ, x) inserts an element x
into the set PQ
Ex. Insert(PQ, 15)
PQ 1
16
2 3
14 10
4 5 6 7
8 7 9 13
8
2

PQ
16
2 3
14 10
4 5 6 7
8 7 9 3
8 9
2 15
 PQ 1
16
2 3
14 10
4 5 6 7
15 7 9 3
8 9
2 8

PQ 1
16
2 3
15 10
4 5 6 7
14 7 9 3
8 9
2 8
 PQ 1

16
2 3

15 10
4 5 6 7

14 7 9 3
8 9

2 8
Exercise:
Illustrate the operation of MAX-HEAP-
INSERT(H, 10) on the heap H = {15, 13,
9, 5, 12, 8, 7, 4, 0, 6, 2, 1}.

Note:
Max-heaps are used in Heapsort.
Heapsort is an efficient sorting algorithm
with a worst case time complexity of O(n
log n) as compared to bubble sort, balloon
sort, insertion sort, and selection sorts
O(n2).
 Max-Heap-Delete (A, key)

Swap the <key> with the content of

last element of the heap
Take note of the index of <key>

Delete the last element of the heap

Max-Heapify (A, index of <key>)

Useful Links

http://linux.wku.edu/~lamonml/al
gor/sort/heap.html
http://www.mkssoftware.com/do
cs/man3/heapsort.3.asp
http://www.csse.monash.edu.au/
~lloyd/tildeAlgDS/Sort/Heap/
http://www.cs.northwestern.edu/
academics/courses/311/notes/1
8.pdf