Professional Documents
Culture Documents
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Sorting Algorithms
Dr. Iftikhar Ahmad
ia@giki.edu.pk
Faculty of Computer Science and Engineering, GIKI, Topi
Sorting Algorithms
1 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Acknowledgement & Questions
Acknowledgement
Questions
Dr. Iftikhar Ahmad
Sorting Algorithms
2 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Acknowledgement & Questions
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Introduction
Theoretical Results
Can we do better?
Introduction
Sorting :: Definition
Sorting is the process of arranging the data in some particular
(ascending/descending) order.
Assumption
For the remainder of this lecture, we assume that we want to order
the list in ascending order.
Given
A sequence of n numbers < a1 , a2 , . . . , an >
Output
A re-ordering < a10 , a20 , . . . , an0 > of the input sequence such
that a10 a20 an0
Dr. Iftikhar Ahmad
Sorting Algorithms
3 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Introduction
Theoretical Results
Can we do better?
Definition
A comparison based sorting algorithm takes as input an array
[a1 , a2 , . . . , an ] of n items, and can only gain information about the
items by comparing pairs of them. Each comparison ( is ai > aj )
returns YES or NO and counts a 1 time-step. The algorithm may
also for free reorder items based on the results of comparisons
made. In the end, the algorithm must output a permutation of the
input in which all items are in sorted order.
Sorting Algorithms
4 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Introduction
Theoretical Results
Can we do better?
Sorting Algorithms
5 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Introduction
Theoretical Results
Can we do better?
Theorem
Any deterministic-based sorting algorithm must perform (n log n)
comparisons to sort n elements in the worst case. Specifically, for
any deterministic comparison-based sorting algorithm A, n 2
there exists an input I of size n such that A makes at least
(n log n) comparisons to sort I .
Sorting Algorithms
6 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Introduction
Theoretical Results
Can we do better?
Can we do better?
NO
Lets relax our assumption
Assume that all keys are in some specific range , say l and u
Can we do better than n log n?
YES
Sorting Algorithms
7 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Acknowledgement & Questions
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
Sorting Algorithms
8 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
Index
Data
1
3
2
8
3
1
4
3
5
9
6
4
7
5
8
3
9
9
10
7
11
6
12
2
Exampled
Sort the array.
Sorting Algorithms
9 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
Index
Data
1
3
2
8
3
1
4
3
5
9
6
4
7
5
8
3
9
9
10
7
11
6
9
2
10
0
12
2
Step 1
Find frequency of each item
Index
Frequency
1
1
2
1
3
3
4
1
5
1
6
1
7
1
8
1
Sorting Algorithms
10 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
3
5
9
6
4
7
5
8
3
9
9
4
1
5
1
6
1
7
1
8
1
4
6
5
7
6
8
7
9
8
10
10
7
11
6
9
2
10
0
12
2
Step 2
Re-use for Distribution
Index
Frequency
1
1
2
2
3
5
Sorting Algorithms
9
12
10
12
11 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
Index
Frequency
2
8
1
1
3
1
2
2
4
3
3
5
5
9
4
6
6
4
7
5
5
7
8
3
9
9
6
8
7
9
8
10
10
7
9
12
11
6
12
2
10
12
Next
Find an appropriate location for 2
Index
Temp
2
2
10
Sorting Algorithms
11
12
12 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
5
5
9
4
6
6
4
5
7
7
5
6
8
8
3
9
9
7
9
8
10
8
6
10
7
9
12
11
6
12
2
10
12
Next
Find an appropriate location for 6
Index
Temp
2
2
10
Sorting Algorithms
11
12
13 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
5
5
9
4
6
6
4
5
7
7
5
6
7
8
3
9
9
7
9
8
10
8
6
9
7
10
7
9
12
11
6
12
2
10
12
Next
Find an appropriate location for 7
Index
Temp
2
2
10
Sorting Algorithms
11
12
14 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
5
5
9
4
6
6
4
5
7
7
5
6
7
8
3
9
9
7
8
8
10
8
6
9
7
10
7
9
12
11
6
12
2
10
12
Next
Find an appropriate location for 9
Index
Temp
2
2
10
Sorting Algorithms
11
12
9
15 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
5
5
9
4
6
6
4
7
5
5
7
8
3
9
9
6
7
7
8
8
10
8
6
9
7
10
7
9
11
11
6
12
2
10
12
Next
Find an appropriate location for 3
Index
Temp
2
2
5
3
10
Sorting Algorithms
11
12
9
16 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
4
5
9
4
6
6
4
7
5
5
7
8
3
9
9
6
7
7
8
8
10
7
5
8
6
9
7
10
7
9
11
11
6
12
2
10
12
Next
Find an appropriate location for 5
Index
Temp
2
2
5
3
10
Sorting Algorithms
11
12
9
17 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
4
5
9
4
6
6
4
7
5
5
6
8
3
9
9
6
7
7
8
8
10
7
5
8
6
9
7
10
7
9
11
11
6
12
2
10
12
Next
Find an appropriate location for 4
Index
Temp
2
2
5
3
6
4
10
Sorting Algorithms
11
12
9
18 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
1
3
2
8
Index
Frequency
1
1
3
1
2
1
4
3
3
4
5
9
4
5
6
4
5
6
7
5
6
7
8
3
9
9
7
8
8
10
8
6
9
7
10
7
9
11
11
6
12
2
10
12
Next
Find an appropriate location for 11
Index
Temp
2
2
5
3
6
4
7
5
10
11
9
12
9
and so on ...
Dr. Iftikhar Ahmad
Sorting Algorithms
19 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
Algorithm
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
Sorting Algorithms
20 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
Analysis
Sorting Algorithms
21 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Algorithm
Analysis
Definition
A sorting algorithm is stable if it always leaves elements with equal
keys in their original order.
Examples
Counting Sort, Merge Sort, Insertion Sort
Quick Sort is NOT stable.
Sorting Algorithms
22 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Acknowledgement & Questions
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Analysis
Radix Sort
Basic Idea
Sort the keys digit by digit, starting with the least significant digit.
Sorting Algorithms
23 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Analysis
Sorting Algorithms
24 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Analysis
Sorting Algorithms
25 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Example
Analysis
Analysis
Worst-Case Complexity
O(nk)
n is the number of elements
k is the (average) length of elements
Question
Is RadixSort stable?
Sorting Algorithms
26 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Acknowledgement & Questions
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Algorithm
Example
Conquer:
Sort the two sub-arrays recursively
Combine:
The array data[left, right] is in sorted form.
Dr. Iftikhar Ahmad
Sorting Algorithms
27 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Algorithm
Example
}
swap ( data [ i +1] , data [ right ]) ;
r e t u r n ( i +1) ;
}
Time Complexity
Worst Case : O(N 2 )
Ave Case : O(N log N)
Sorting Algorithms
28 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Basic Idea
Algorithm
Example
Example Run
Given:
An unsorted list (array) with the following data;
28713564
Sorting Algorithms
29 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Acknowledgement & Questions
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Sorting Type
Selection Sort
Bubble
Quick Sort (Worst Case)
Quick Sort (Ave Case)
Counting Sort
Radix Sort
Time Complexity
O(n2 )
O(n2 )
O(n2 )
O(n log n)
O(n + m)
O(nk)
Sorting Algorithms
30 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Acknowledgement & Questions
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Acknowledgement
Questions
Acknowledgement
Source Acknowledgement
The material (including figures and algorithms) presented in this
lecture is adopted from the following books;
Cormen, Thomas, Charles Leiserson, Ronald Rivest, and
Clifford Stein. Introduction to Algorithms.
Leviten, Anany. Introduction to the Design and Analysis of
Algorithms.
www.inf.ed.ac.uk/teaching/courses/ads/Lects/lecture8.pdf
Radix Sort Example is from Wikipedia
Sorting Algorithms
31 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Acknowledgement
Questions
Questions
Questions?
Sorting Algorithms
32 / 32