Algorithm Analysis

Algorithm Analysis
Three Parts,
Designing the algorithm Proving the correctness of the algorithm Analysis the algorithm

Example, Swapping of array contents

Method 1
int a1[n]; int a2[n]; for(int i=0;i<n;i++) { a2[i]=a1[(n-1)-i]; }

Analysis:
Size of the input array is n Size of the output array is n So the total space requirement is n+n=2n Time complexity is based on the number of assignment statements in the algorithm.
So the time complexity is n

Method 2
int a1[n]; int k=floor(n/2); for( int i=0; i<k; i++) { swap(&a1[i],&a1[(n-1)-i]); } swap(int *a, int *b) { int temp = *a; *a = *b; *b = temp; }

Method 2 Contd
Analysis Space Complexity
Size of the input array is n One temporary variable called temp So total space requirement is n+1

Time Complexity
Three assignments per swapping Totally n/2 swapping So time complexity is 3n/2.

Asymptotic Notations
Three standard notations are used,
Big Oh notaion (O) Big Omega notation (;) Big Theta notation (5

Big Oh notation
Big Oh notation: (Asymptotic upper bound)
f (n) = O (g (n)) If f (n) <= c*g (n) for all n n0, where c & n0 are constants > 0 O (g (n)): class of functions f (n) that grow no faster than g (n)
c*g(n) f(n)

n0

Example: T(n) = 2n + 5 is O(n). Why? 2n+5 <= 3n, for all n >= 5 T(n) = 5*n2 + 3*n + 15 is O(n2). Why? 5*n2 + 3*n + 15 <= 6*n2, for all n >= 6

; Notation: Asymptotic Lower Bound

T(n) = f(n) = ;(g(n)) if f(n) >= c*g(n) for all n > n0, where c and n0 are constants > 0
f(n) c*g(n)

n0

Example: T(n) = 2n + 5 is ;(n). Why? 2n+5 >= 2n, for all n > 0 T(n) = 5*n2 - 3*n is ;(n2). Why? 5*n2 - 3*n >= 4*n2, for all n >= 4

5 Notation: Asymptotic Tight Bound

T(n) = f(n) = 5(g(n)) if c1*g(n) <= f(n) <= c2*g(n) for all n > n0, where c1, c2 and n0 are constants > 0
c2*g(n) f(n) c1*g(n)

n0

Example: T(n) = 2n + 5 is 5(n). Why? 2n <= 2n+5 <= 3n, for all n >= 5 T(n) = 5*n2 - 3*n is 5(n2). Why? 4*n2 <= 5*n2 - 3*n <= 5*n2, for all n >= 4

Properties of Big Oh Notation

i. O (f(n)) + O (g(n)) = O (max { f(n), g(n)}) ii. f (n) = O(g(n)) and g (n) h (n) implies f (n) = O(h(n)) iii. Any function can be said as an order of itself. i.e., f(n) = O(f (n)) iv. Any constant value is equivalent to O (1). i.e., C = O (1) v. If limn {f (n) / g (n)} R>0 then f (n) (g (n)). vi. If limn {f (n) / g (n)} = 0 then f (n) O(g(n)) but f(n) not (g(n)). (g(n)) but f(n) vii. If limn {f (n) / g (n)} = + => f (n) not (g (n)).

Recurrence Relation
Types, Homogeneous Non Homogeneous Let f(n) is a time complexity of our algorithm for the input of size n. Then f(n) is recursively defined as,

f(n) = b1f(n-1) + b2f(n-2) + +bkf(n-k)

Associated Homogeneous relation is,

C0f(n) + C1f(n-1) + C2f(n-2) + +Ckf(n-k)=0, nk

Recurrence Relation
Solving Homogeneous Linear Recurrence Relation:
i. Write down the characteristic equation for the recurrence relation,

c0S(k) + c1S(k-1) + + cnS(k-n) = 0 c0an + c1an-1 + + cn = 0 Characteristic Equation

ii. Find all the roots of the characteristic equation. This root values are called as characteristic roots.

a1, a2, a3, ,an

iii. If a1 a2 a3 an i.e., all the roots are distinct. Then the general solution is,

S(k) = b1a1k + b2a2k + + bnank

iv.

v. vi.

If a1 = a2 and others are distinct, then the general solution is, S(k) = (b1+b2k)ak + b3a3k+ + bnank Substitute the initial condition in the general solution and find b1, b2, bn Substitute b1, b2, bn in the general solution to get the required solution.

Example: 1) Solve the recurrence relation S(k) + S(k-1) - 6S(k-2) = 0 and S(0) = -1, S(1)=8. 2) Solve the recurrence relation S(k) 8S(k-1) + 16S(k-2) = 0 initial conditions are, S(2) = 16 & S(3) = 80 3) Solve the recurrence relation S(k) 5S(k-1) + 6S(k-2) = 0, k 2 with initial conditions S(0) = 1, S(1) = 3

Pbm:1 Solution, Given,

S(k) + S(k-1) - 6S(k-2) = 0 & S(0) = -1, S(1)=8. ------ 1. The characteristic equation of 1. is, a2 + a 6 = 0 (a + 3) (a 2) = 0 => a = -3, a = 2 General solution is, S(k) = b1 (-3)k + b2 (2)k Initial conditions are, S(0) = -1, S(1)=8 S(0) = -3 b1 + 2 b2 = 8 ---------- 2. S(1) = b1 + b2 = -1 ---------- 3.

By solving 2. & 3. b1 = -2 b2 = 1 Therefore general solution is, S(k) = (-2)(-3)k + (1)(2)k

Recurrence Relation Contd

Solving Non Homogeneous Linear Recurrence Relation: i. To solve the recurrence relation S(k) + C1S(k-1) + + CnS(k-n) = f(k) ii. Step 1
Write the associated homogeneous relation and its general solution.

iii. Step 2
Obtain a particular solution of the recurrence relation by taking an educational guess at the form of particular solution.

Form of a particular solution 1. Constant q 2.Linear function of q and k q0 + q1k 3. An nth order polynomial q0 + q1k + q2k2 + + qnkn 4. An exponential function qak

Particular solution A constant d A linear form of d0 + d1k An nth degree polynomial d0 + d1k + d2k2 + + dnkn An exponential function dak

iv. Step 3 Substitute the guess from step 2 into the recurrence relation. If the guess is correct, it should help us to determine the unknown co efficient. If it is wrong, it should be apparent from the result of the substitution. Go to the step 2. v. Step 4 The general solution of the recurrence relation is the sum of homogeneous and particular solution. If no initial conditions are given, then the solution gets over. If n initial conditions are given, obtain n linear equations and solve the linear equations if possible to get required solution.

Example 1. Solve S(k) S(k-1) 6S(k-2) = -30 with initial conditions S(0) = 20, S(1) = -5 2. Solve S(k) 2S(k-1) + S(k-2) = 2 with initial conditions S(0) = 25, S(1) = 16 3. Solve the recurrence relation S(k) 3S(k-1) 4S(k-2) = 4k