Sorting Algorithms

Notation:
O(x) = Worst Case Running Time
Ω(x) = Best Case Running Time
Θ(x)= Best and Worst case are the same.

Sorting Implementation Comments Type Stable? Asymptotic

Algorithm Summary Complexities
On each pass the Good for nearly Insertion Yes
current item is sorted lists, very
inserted into the bad for out of order
sorted section of the lists, due to the
list. It starts with the shuffling.
last position of the
sorted list, and moves
backwards until it
finds the proper place
of the current item.
That item is then
inserted into that
place, and all items
after that are shuffled
to the left to
accommodate it. It is
Best Case: O(n).
Straight for this reason, that if
Insertion the list is already
Worst Case: O(n2)
sorted, then the
sorting would be O(n)
because every
element is already in
its sorted position. If
however the list is
sorted in reverse, it
would take O(n2)
time as it would be
searching through the
entire sorted section
of the list each time it
does an insertion, and
shuffling all other
elements down the
list..
 This is an extension This is better than Insertion Yes
of the Straight the Strait Insertion
Insertion as above, if the comparisons
however instead of are costly. This is
doing a linear search because even
each time for the though, it always
correct position, it has to do log n
does a binary search, comparisons, it
which is O(log n) would generally
instead of O(n). The work out to be less
only problem is that than a linear
Binary it always has to do a search. Best Case: O(n log n).
Insertion binary search even if
Sort the item is in its Worst Case: O(n2)
current position. This
brings the cost of the
best cast up to
O(n log n). Due to the
possibility of having
to shuffle all other
elements down the
list on each pass, the
worst case running
time remains at
O(n2).
On each pass of the In general this is Exchange Yes.
data, adjacent better than
elements are Insertion Sort I NOTE: Preiss uses a
compared, and believe, because it bad algorithm, and
switched if they are has a good change claims that best and
out of order. eg. of being sorted in worst case is O(n2).
e1 with e2, then e2with much less than
e3 and so on. This O(n2) time, unless We however using a
means that on each you are a blind little bit of insight,
pass, the largest Preiss follower. can see that the
Bubble
element that is left following is correct of
Sort
unsorted, has been a better bubble sort
"bubbled" to its Algorithm (which
rightful place at the does Peake agree
end of the array. with?)
However, due to the
fact that all adjacent Best Case: O(n).
out of order pairs are
swapped, the Worst Case: O(n2)
algorithm could be
finished sooner.
 Preiss claims that it
will always take
O(n2) time because it
keeps sorting even if
it is in order, as we
can see, the algorithm
doesn't recognise
that. Now someone
with a bit more
knowledge than
Preiss will obviously
see, that you can end
the algorithm in the
case when no swaps
were made, thereby
making the best case
O(n) (when it is
already sorted) and
worst case still at
O(n2).
I strongly recommend A complicated but Exchange No
looking at effective sorting
the diagram for this algorithm.
one. Thecode is also
useful and provided
below (included is
Best Case: O(n log n).
theselectPivot method
even though that
Worst Case: O(n2)
probably won't help
you understanding
Refer to page 506 for
anyway).
more information
The quick sort
about these values.
operates along these
Quicksort Note: Preiss on page
lines: Firstly a pivot
524 says that the
is selected, and
worst case is
removed from the list
O(n log n)
(hidden at the end).
contradicting page
Then the elements are
506, but I believe that
partitioned into 2
it is O(n2), as per page
sections. One which
506.
is less than the pivot,
and one that is
greater. This
partitioning is
achieved by
exchanging values.
 Then the pivot is
restored in the
middle, and those 2
sections are
recursively quick
sorted.

This one, although A very simple Selection No

not very efficient is algorithm, to code,
very simply. and a very simple
Basically, it one to explain, but Unlike the Bubble
2
does n linear passes a little slow. sort this one is
on the list, and on truly Θ(n2), where
each pass, it selects Note that you can best case and worst
Straight
the largest value, and do this using the case are the same,
Selection
swaps it with the last smallest value, and because even if the
Sorting.
unsorted element. swapping it with list is sorted, the same
This means that it the first unsorted number of selections
isn't stable, because element. must still be
for example a 3 could performed.
be swapped with a 5
that is to the left of a
different 3.
This uses a similar This utilises, just Selection No
idea to the Straight about the only
Selection Sorting, good use of heaps,
except, instead of that is finding the
using a linear search maximum element,
Best Case: O(n log n).
for the maximum, a in a max heap (or
heap is constructed, the minimum of a
Worst Case:
and the maximum can min heap). Is in
O(n log n).
easily be removed every way as good
Ok, now I know that
(and the heap as the straight
looks tempting, but
Heap Sort reformed) in selection sort, but
for a much more
log n time. This faster.
programmer friendly
means that you will
solution, look at
do npasses, each time
Merge sort instead,
doing a log n remove
for a better O(n log n)
maximum, meaning
sort .
that the algorithm
will always run
in Θ(n log n) time, as
it makes no
difference the
 original order of the
list.

It is fairly simple to Now isn't this Merge Yes

take 2 sorted lists, much easier to
and combine the into understand that
another sorted list, Heap sort, its really
simply by going quite intuitive. This
through, comparing one is best explain
the heads of each list, with the aid of
removing the smallest thediagram, and if
to join the new sorted you haven't
list. As you may already, you should
guess, this is an O(n) look at it.
operation. With 2
way sorting, we apply
this method to an
single unsorted list.
In brief, the algorithm
recursively splits up
the array until it is
2 Way fragmented into pairs
Best and Worst
Merge of two single element
Case: Θ(n log n)
Sort arrays. Each of those
single elements is
then merged with its
pairs, and then those
pairs are merged with
their pairs and so on,
until the entire list is
united in sorted order.
Noting that if there is
every an odd number,
an extra operation is
added, where it is
added to one of the
pairs, so that that
particular pair will
have 1 more element
than most of the
others, and won't
have any effect on the
 actual sorting.
Bucket sort initially This sufferers a Distribution No
creates a "counts" limitation that
array whose size is Radix doesn't, in
the size of the range that if the possible
of all possible values range of your
for the data we are numbers is very
sorting, eg. all of the high, you would
values could be need too many
between 1 and 100, "buckets" and it
therefore the array would be
would have 100 impractical. The
Best and Worst
elements. 2 passes are other limitation
case:Θ(m + n) where
then done on the list. that Radix doesn't
m is the number of
The first tallies up the have, that this one
possible values.
occurrences of each does is that
Obviously this is O(n)
of number into the stability is not
for most values of m,
"counts" array. That maintained. It does
so long as misn't too
is for each index of however
Bucket large.
the array, the data outperform radix
Sort
that it contains sort if the possible
The reason that these
signifies the number range is very small.
distribution sorts
of times that number
break
occurred in list. The
the O(n log n) barrier
second and final pass
is because no
goes though the
comparisons are
counts array,
performed!
regenerating the list
in sorted form. So if
there were 3 instance
of 1, 0 of 2, and 1 of
3, the sorted list
would be recreated to
1,1,1,3.
This diagram will
most likely remove
all shadows of doubt
in your minds.
 This is an extremely This is the god of Distribution Yes
spiffy implementation sorting algorithms.
of the bucket sort It will search the
algorithm. This time, largest list, with the
several bucket like biggest numbers,
sorts are performed and has a is
(one for each digit), guaranteed O(n)
but instead of having time complexity.
a counts array And it ain't very
representing the complex to
range of all possible understand or
values for the data, it implement.
represents all of the My
possible values for recommendations
each individual digit, are to use this one
which in decimal wherever possible.
numbering is only 10.
Firstly a bucked sort
is performed, using
only the least
significant digit to
sort it by, then
Best and Worst
Radix another is done using
Case: Θ(n)
Sort the next least
Bloody awesome!
significant digit, until
the end, when you
have done the number
of bucket sorts equal
to the maximum
number of digits of
your biggest number.
Because with the
bucket sort, there are
only 10 buckets (the
counts array is of size
10), this will always
be an O(n) sorting
algorithm! See below
for a Radix Example.
On each of the
adapted bucket sorts
it does, the count
array stores the
numbers of each
digit. Then the offsets
are created using the
 counts, and then the
sorted array
regenerated using the
offsets and the
original data.

Radix Sort Example:

First Pass:
Data: 67 50 70 25 93 47 21

Buckets on first pass (least significant digit):

index 0 1 2 3 4 5 6 7 8 9
count 2 1 0 1 0 1 0 2 0 0
offset 0 2 3 3 4 4 5 5 7 7

Data after first pass

50 70 21 93 25 67 47

That data is created by doing a single pass on the unsorted data, using the offsets to
work out at where each item belongs.
For example, it looks at the first one 67, then at the offsets for the digit 7, and inserts it
into the 5th position. The offset at 7 is then incremented, so that the next value
encountered which has a least significant digit of 7 is placed into the 6th position.
Continuing the example, the number 50 would then be looked at, and inserted into the
0th position, its offset incremented, so that the next value which is 70 would be
inserted into the 1st position, and so on until then end of the list.
As you can see, this data is sorted by its least significant digit.

Second Pass:
Data after first pass
50 70 21 93 25 67 47

Buckets on first pass (most significant digit):

index 0 1 2 3 4 5 6 7 8 9
count 0 0 2 0 1 1 1 1 0 1
offset 0 2 3 3 4 4 5 5 7 7

Data after second pass (sorted)

21 25 47 50 67 70 93

Look at this diagram for another example, noting that the "offsets" array is
unnecessary
Images:
Straight Insertion - Figure 15.2
Bubble Sorting - Figure 15.3
Quick Sorting - Figure 15.4
Program 15.7 AbstractQuickSorter code
Program 15.9 MedianOfThreeQuickSorter class selectPivot method
Straight Selection Sorting - Figure 15.5
Building a heap - Figure 15.7
Heap Sorting - Figure 15.8
Two-way merge sorting - Figure 15.10

Bucket Sorting - Figure 15.12

Radix Sorting - Figure 15.13