DATA STRUCTURES

ALGORITHM AND ANALYSIS

Rajdeep Chatterjee
Assistant Professor
School of Computer Engineering
KIIT University
OVERVIEW
 Algorithm
 Analysis of Algorithm
 Space Complexity
 Time Complexity
 Step Counts
 Asymptotic Notations
 Big-oh Notations
 Rate of Growth
 Types of Time Complexity
 Best Case Complexity
 Worst Case Complexity
 Average Case Complexity
ALGORITHM
 Algorithm is a finite set of well defined
computational instructions written in a proper
sequence in order to solve a problem.

 Criteria-
 Input
 Output
 Definiteness
 Finiteness
 Effectiveness
EXAMPLES
 ADD: INTEGER a, b, c
1. Read a & b
2. add a & b
3. store the sum of a & b to c
4. print c

 In C, exp(1) In C, exp(2)

void add(int a, int b){ int add(int a, int b){

int c; int c;
c=a+b; c=a+b;
printf(“%d”,c); return c;
} }
ANALYZING AN ALGORITHM
 Why is it important?
 Predict the feasibility requirement of your code
(algorithm).
 Usual requirements
 Execution time
 Memory space

 The complexity of the algorithm is determined in

terms of space and time.
 Space complexity ← execution time
 Time complexity ← memory space
SPACE COMPLEXITY
 Amount of computer memory required during
program execution.
 Instruction space – space required to store the
code.
 Fixed space requirement – input independent

 Variable space requirement – input dependent

 S = F(I) otherwise S = C
EXAMPLES
 Exp(1)
int main(){
printf(“KIIT UNIVERSITY”);
}

 Exp(2)
void bubble(int a[], int n){
int i,j;
for(i=0;i<n-1;i++){
for(j=0;j<n-1-i;j++)
if(a[j]>a[j+1])
swap (a[j],a[j+1])
}
}
TIME COMPLEXITY
 The amount of computer time that it needs to run
to completion.
 It is determined without considering the following
information-
 The machine we are executing,
 Its machine language instruction set,
 The time required by each machine instruction,
 The translation, a compiler/interpreter will make from
the source code to machine language.
EXAMPLES
 Exp(1)
x=x+1;

 Exp(2) Exp(4)
for(i=1 ; i<=n ; i++) int main(){
x=x+1; printf(“KIIT UNIVERSITY”);
}
 Exp(3)
for(i=1 ; i<=n ; i++)
for(j=1 ; j<=n ; j++)
x=x+1;
Our concern should be the order of magnitude/ growth of an algorithm
for an input n.
STEP COUNT
Instructions / code Step Count
x=x+1 constant, c
int main(){ ?
printf(“KIIT UNIVERSITY”);
return 0;
}

for(i=1 ; i<=n ; i++) n

x=x+1;
for(i=1 ; i<=n ; i++) n2
for(j=1 ; j<=n ; j++)
x=x+1;
EXAMPLE - 1
1. int i, f=1, n=5; -------------- 1 time
2. for(i=1 ; i<=n ; i++) -------------- n+1 times
3. f=f*i; -------------- n times
4. printf(“%d”,f); -------------- 1 time
T(n) = 1+(n+1)+n+1 = 2n+3 ≈ O(n)
if n=0 then T(n) = 1+1 ≈ O(1)

n i f=f*i , f=1
5 1 1*1=1
2 1*2=2
3 2*3=6
4 6*4=24
5 24*5=120
6
EXAMPLE - 2
1. int a=0,b=1,c, i, n=5; ------------- 1
2. printf(“%d %d”,a,b); ------------- 1
3. for(i=1 ; i<=n-2 ; i++ ){ ------------- (n-2)+1 = (n-1)
4. c=a+b; ------------- (n-2)
5. printf(“%d”,c); ------------- (n-2)
6. a=b; ------------- (n-2)
7. b=c; } ------------- (n-2)

T(n)= 1+1+(n-1)+2*(n-2) = 2+n-1+2n-4 = 3n-3 ≈ O(n)

i a b c=a+b
1 0 1 1
2 1 1 2
3 1 2 3
4
EXAMPLE - 3
1. int i, j, n=5, a[5]={12, 2, 51, 35, 7};
2. for(i=0;i<n-1;i++){
3. for(j=0;j<n-1-i;j++)
4. if(a[j]>a[j+1]){
5. a[j] = a[j] + a[j+1];
6. a[j+1] = a[j] - a[j+1];
7. a[j] = a[j] - a[j+1]; }
8. for(i=0 ; i<n ; i++)
9. printf(“%d”, a[i]);

T(n) = ?
ASYMPTOTIC NOTATIONS
 O (Big-oh) Notation
Consider a function f(n) which is non-negative for all
integers n=0. we say that f(n)is Big-oh g(n), which we
write f(n)=O(g(n)), if there exits an integer n0 and a
constant c>0 such that for all integers n=n0, f(n)=cg(n).

 In other words,
O(g(n))={f(n): ∃ positive constants c and n0 ∋ 0 ≤ f(n) ≤cg(n) ∀ 𝑛 ≥ 𝑛0 }
 O-notation to give an upper bound on a function to within a
constant factor.
EXAMPLE
 Two students, X & Y
# Events X Y TX ∼ TY
1 10th 70% 85% TX(1) ≺ TY(1)
2 (10+2)th 65% 78% TX(2) ≺ TY(2)
3 B.Tech 8.8 cgpa 8.7 cgpa TX(3) ≻ TY(3)
4 M.Tech GATE rank 120 GATE rank 1700 TX(4) ≻ TY(4)
5 Ph.D 5 publications 2 publications TX(5) ≻ TY(5)
6 Job 20 lakhs/p.a. 10 lakhs/p.a. TX(6) ≻ TY(6)

 Let, performance of X & Y are TX and TY respectively,

 Conclusion is for a large n, TX(n) ≻ TY(n)

 So, who is a better professional ?

 Ans. X
 When X outperforms Y and by what factor ?
 Ans. n0 and c
EXAMPLE
 Suppose C=1 then, f(n)=6n+135

f(n) = cn2
 6n +135 = cn2 = n2 (since c=1)
 0 = n2 - 6n -135
 0 = (n-15)(n+9)

Since (n+9) > 0 for all values n ≥ 0, then (n-15)=0

 n0=15

 Find n0 when c=2 & c=4.

NOTATIONS

Big-oh : 0≤ f(n) ≤ cg(n) Omega : 0 ≤ cg(n) ≤ f(n)

Theta : 0≤ 𝑐1𝑔 𝑛 ≤ f(n) ≤ c2 𝑔 𝑛 )

RATE OF GROWTH
𝒍𝒐𝒈𝟐 𝒏 n n𝒍𝒐𝒈𝟐𝒏 𝒏𝟐 𝒏𝟑 𝟐𝒏
0 1 0 1 1 2
1 2 2 4 8 4
2 4 8 16 64 16
3 8 24 64 512 256
4 16 64 256 4096 65536
TYPES OF TIME COMPLEXITY
 Best Case Time complexity
 Exp(1) – Linear search/Binary search

3 5 24 45 78

 Linear search, key=3

 a[0]==key, T(n)=O(1)

 Binary search, key=24

 m=(0+4)/2 = 2
 a[2]==key, T(n)=O(1)
TYPES OF TIME COMPLEXITY
 Worst Case Time complexity
 Exp(2) – Linear search/Binary search

3 5 24 45 78

 Linear search, key=78 / 80

 a[0]==key, T(n)=O(n) / O(n+1) ≈ O(n)

 Binary search, key=78 / 80

 m=(0+4)/2 = 2 …
 a[2]!=key,…
 finally T(n)=O(log 2 𝑛) ….(*)
TYPES OF TIME COMPLEXITY
 Average Case Time complexity
 Exp(3) – Linear search/Binary search
3 5 24 45 78
 Linear search, key=24
 a[0]==key, T(n)=O((n+1)/2) ≈ O(n)
 There are n cases that can occur, i.e. find at the first place,
the second place, the third place and so on up to the nth
place. If found at the ith place then i comparisons are
required. Hence the average number of comparisons over
these n cases is:
 average = (1+2+3.....+n)/n = (n+1)/2
 where the result was used that 1+2+3 ...+n is equal to
n(n+1)/2.
 Binary search, key=5
 m=(0+4)/2 = 2, m=(0+1)/2=0, m=(1+1)/2=1
 a[1]==key, finally T(n) ≈ O(log 2 𝑛) ….(*)
SORTING ALGORITHMS
Time Time Time Space
Data
Algorithm Complexity: Complexity: Complexity: Complexity:
Structure
Best Average Worst Worst

Quick Sort Array O(n log(n)) O(n log(n)) O(n2) O(log(n))

Merge sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(n)

Heap sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(1)

Smooth sort Array O(n) O(n log(n)) O(n log(n)) O(1)

Bubble sort Array O(n) O(n2) O(n2) O(1)

Insertion
Array O(n) O(n2) O(n2) O(1)
sort
Selection
Array O(n2) O(n2) O(n2) O(1)
sort