SortingI

(Insertionsort,Shellsort,Heapsort,MergeSort,Quicksort),
Quickselect,
Worstandaveragecaseanalysis

Sortingalgorithms
Comparisonbasedsorting
STLversions:

sort(v.begin(),v.end());
sort(v.begin(),v.begin()+(v.end()v.begin())/2);
sort(v.begin(),v.end(),greater<int>());
Also:stable_sort()
()

InsertionSort
Passp=1throughN1:

itemsinpositions0throughpareinsortedorder.

InsertionSort

InsertionSort(STLimplementation)

Insertion Sort (STL implementation)

InsertionSort(STLimplementation)

InsertionSortAnalysis
Worstcase:(N2)
Bestcase(presortedinput):O(N)

Lower bound for simple sorting

Lowerboundforsimplesorting
algorithms
Invertion inanarrayisorderedpair(i,j):
i<jand
a[i]>a[j]
[]
[j]
Examplearray:34,8,64,51,32,21
9inversions:(34,8),(34,32),(34,21),
(64,51),(64,32),(64,21),
(51,32),(51,21),
(32,21)

Theseareexactlytheswapsperformedbyinsertionsort!
=>Swappingtwoadjacentitemsthatareoutofplacereducesoneinversion
=>Sortedarrayneedsnoinversions

Lower bound for simple sorting

Lowerboundforsimplesorting
algorithms
Averagenumberofinversionsinpermutation

ofNintegers(assumenoduplicates)provides
Averagerunningtimeforinsertionsortandfor
A

i ti f i
ti t df
allalgorithmsthatarebasedonswapping
adjacentelements!

Lower bound for simple sorting

Lowerboundforsimplesorting
algorithms
Theorem:Averagenumberofinversionsinanarray

ofNdistinctelementsisN(N1)/4
Proof:
ConsiderlistLofNintegersanditsreverseLr.
g
Totalnumberofinversionsforbothlistsis?
Averagenumberofinversionsis?
g
Therefore: Anyalgorithmthatsortsbyexchanging
adjacentelementsis(N2)onaverage.

Shellsort (byDonaldShell)
Improvesaveragetimebyexchangingnonadjacentelements
Diminishingincrementsort
Usesincrementsequenceh1,h2,,htwithh1=1
Allsequenceswork,butsomeperformbetter!
Idea:
d

Foreveryhk (fromhtdowntoh1)
[ ll l
[allelementsspacedh

dhk apartaresorted]==

d]
[a[i]a[i+hk]foreveryi ]==
[ f
[performinsertionsortinh
i
ti ti hk subarrays]
b
]

Shellsort
Arrayindex:01234567891011 12

Atincrementhk:
foreverypositioni=hk,h
hk+1,h
+1 hk+2,,N1
+2 N1
finditsrightspotamongi,ihk,i2hk,i3hk ,

N h h h
Notethatath1wehaveregularinsertionsort
l i
i

5sort:5subarrays
5 so t 5 suba ays
0123456789101112

81941196123517952858417515
8 6 8 8
813541
354181
941775
177594
119515
111595
9628
2896
1258
1258

35171128124175159658819495

3sort:3subarrays
3 so t 3 suba ays
0123456789101112

35171128124175159658819495
3528755895
2835587595
17121581
12151781
11419694
11419496
6

28121135154158179475819695

1sort:1subarray
so t
suba ay
0123456789101112

28121135154158179475819695
11121517283541587581949596
Laststep:insertionsort
subarraysareconceptual(i.e.noneedforextrastorage)
Implementationisextremelysimple
Bestalgorithmforsmall(upto10000elements)collections

Shellsort
Originalincrementsequence:

ht=floor(N/2),hk=floor(hk+1/2)
Notagoodselection

Shellsort

WorstCaseAnalysis
WorstcaserunningtimeusingShellsincrementsis(N2)
Proof:

A)Provideanexampleinputthattheboundexists.
ConsiderNispowerof2
PuttheN/2largestnumbersatevenpositions
N/2smallestnumbersatoddpositions
B)Theoreticalupperbound:atpasshk:hkinsertionsorts=>
)
O(hk (N/hk)2)=O(N2 /hk)perpass
T li O(S (N2/hk)forallk)==O(N
TotalisO(Sum(N
)f llk)
O(N2)

Worstcaseanalysis
Positions12345678910111213141516
P iti
6 8 6

Inputthatprovidesworstbehavior

Betterincrements
1,3,7,,2k1(Hibbard)
Consecutiveincrementshavenocommonfactors
Theorem: Worstcaserunningtimeis(N3/2)
Proof: Inbook,buthard.
Improvementoveroriginalinsertionsort!
Provingaveragecaseisstillopen.

http://www.cs.princeton.edu/~rs/shell/animate.html

From,AlgorithmsinC++,RobertSedgewick

Heapsort

Usesbinaryheapimplementedasarray
O(NlogN)timebound[worstandaverage]
I
Inplacealgorithm
l l ith
Simple:
A)BuildbinaryheapfromNelement:O(N)
B)PerformdeleteMax()fromNarray
//
//
fromN1arrayy

//
from2array
<=2NlogN

Afterbuildheap

Afterbuildheap

AfterfirstdeleteMax()

Heapsort example
From,AlgorithmsinC++,RobertSedgewick

From,AlgorithmsinC++,
RobertSedgewick

Mergesort
O(NlogN)worstcaserunningtime
Nearoptimalcomparisons
Recursive
Key:O(N)formergingtwosortedlists
Bad:needforextraO(N)space

Merge
Merge
two
sorted
lilists

From,AlgorithmsinC++,
From
AlgorithmsinC++
RobertSedgewick

AnalysisofMergesort

T(N)
T(N):runningtimeofmergesortforarrayofsizeN
i ti f
tf
f i N

Analysisofmergesort

AnalysisofMergesort

Quicksort
FastestforC++
O(NlogN)average,O(N2)worst
TosortarrayS:

quicksort(S)
If|S|=0,or1thenreturn
Pick anyelementvinS.Thisisthepivot.
PartitionS{v}intotwodisjointgroups:
P titi S { }i t t di j i t

S1={xinS{v}|xv}andS2={xinS{v}|xv}
Returnquicksort(S1),v,quicksort(S2)
Returnquicksort(S1) v quicksort(S2)

Pickingthepivot

8,1,4,9,6,3,5,2,7,0

Takecareofmedian

Whilei <j
movei totherightskippingoverelementssmallerthanpivot
movejtotheleftskippingoverelementslargerthanpivot
j
pp g
g
p
Ifi<jswap a[i]witha[j]

Whilei <j
movei totherightskipping
overelementssmallerthan
pivot
movej
j totheleftskipping
h l f ki i
overelementslargerthanpivot
Ifi<jswap a[i]witha[j]

Finally,swapa[i]withpivot

Importantdetail
HowtohandleitemsthatareEQUALtopivot?
Shouldi stopatelementthat=thepivot?
Shouldjstopatelementthat=thepivot?
816963527|6

44444444|4

Importantdetail
Ifbothstop:manyswaps,buti,j willcrossinmiddle
Good:evenpartitioning
Ifnonestop:
Preventthemfromcrossingboundaries
i willtouchtheend,andjwilltouchthebeginning
illt
hth d dj illt
hth b i i
Pivotwillswapa[i]
Bad:unevenpartitioning
Thisdetailnotassillyasitseems.

Smallarrays
ForN<=20,wecanuseinsertionsort=>reduce

numberofrecursivecalls.

AnalysisofQuicksort
RunningtimeforarrayofNelements:

T(N)=T(i)+T(Ni1)+cN,i=|S1|
T(0)=T(1)=1

WorstCase
T(N)=T(0)+T(N1)+cN =>

T(N)=T(N1)+cN,N>1

T(N1)=T(N2)+c(N1)

T(2)=T(1)+c(2)
==========================
T(N)=T(1)+c(N+(N1)++2)
=>T(N)=O(N2)

BestCase
T(N)=T(N/2)+T(N/2)+cN=>

T(N)=2T(N/2)+cN
sameasmergesort
T(N)=O(NlogN)

AverageCase
T(N)=T(i)+T(Ni1)+cN,i=sizeofS1,N>=2
T(0)=T(1)=1

Howdowedefineaveragecase?
SizesofS1isequallylikely,i.e.
S1canbeofsize0,1,2,,orN1withequalprob 1/N:
Prob{|S1|=k}=1/Nforeveryk=o,,N1

AverageCase
T(N)=T(i)+T(Ni1)+cN,i=sizeofS1,N>=2
T(0)=T(1)=1

Howdowedefineaveragecase?
ItfollowsthatsizesofS2 areequallylikelyalso,i.e.
S2canbeofsize0,1,2,,N1withequalprob 1/N:
Prob{|S2|=k}=1/Nforeveryk=o,,N1

AverageCase
So,average valueofT(i)andofT(Ni1)is

<S1><S2>
(1/N)[T(0)+T(1)+T(N1)]=(1/N)j=0N1 T(j)

AverageCase
Recurrencerelationbecomes:

T(N)=(2/N)[j=0N1 T(j)]+cN,N>=2
or
NT(N)=2[j=0N1 T(j)]+cN2
ofcoursethisistrueforN1aswell:
N T(j)]+c(N1)
(N )T(N ) [j=0N2
(N1)T(N1)=2[
T(j)] (N )2

AverageCase
Bysubtraction:

NT(N) (N1)T(N1)=2[j=0N1 T(j)] 2[j=0N2 T(j)]+

cN2 c(N1)
( )2
=2T(N1)+2cN c=>(cnotsignificant)
NT(N)=(N+1)T(N1)+2cN=>
T(N)=((N+1)/N)T(N1)+2c=>
( )
((
) ) ( )
T(N)/(N+1)=T(N1)/N+2c/(N+1)

Averagecase
T(N)/(N+1)=T(N1)/N+2c/(N+1)[N]

Telescope:
T(N1)/N=T(N2)/(N1)+2c/N
[N1]
T(N2)/(N1)=T(N3)/(N2)+2c/(N1)[N2]
.
T(2)/3=T(1)/2+2c/3[2]
+all
T(N)/(N+1)=T(1)/2+2c/(N+1)+2c/N++2c/3

Averagecase
T(N)/(N+1)=T(1)/2+2c/(N+1)+2c/N++2c/3

Compute:2c/(N+1)+2c/3=2c*(1/3+1/4++1/(N+1))
1/3+1/4++1/(N+1)=loge(N+1)+3/2,
~0.577(Eulersconstant)
Finally!!T(N)/(N+1)=O(logN)or
( ) (
) (
)

T(N)=O(NlogN)

LinearExpectedTimeforSelection
Selectionproblem
Usingheapfindkth largest(orsmallest)in

O(N kl N) i O(Nl N)f di

O(N+klogN)time=>O(NlogN)formedian

LinearExpectedTimeforSelection
quickselect(S,k)
If|S|=1,thenk=1andreturnelementofS
PickapivotvinS
PartitionS{v}intoS1andS2aswasdoneinquicksort
Ifk<=|S1|,returnquickselect(S1,k)
Elseifk=|S1|+1,returnpivot
Else,returnquickselect(S2,k|S1|1)

Quickselect analysis
Worstcase?
Averagecase?