Advanced Design and Analysis of Algorithms

CSC501 Dr. Hajira Jabeen

My Introduction
Ph.D. in August 2010

Expertise in
Artificial Intelligence Machine Learning Computational Intelligence Data Mining Classification Clustering

Evolutionary Computation
Swarm Intelligence

Book Publication
Genetic Programming: A Novel tool for Classification Issues and Advancements Hajira Jabeen and Abdul Rauf Baig Verlag, Germany

Journal Publications
(sum of Impact 9)
Jabeen, H and Baig, A R., "DepthLimited Crossover in Genetic

Programming for Classifier Evolution." Computers in Human Behaviour, Springer, 2010. (ISI Impact Factor 1.86)
Jabeen,

H and Baig, A. R., GPSO: Optimization of Genetic Programming Classifier Expressions for Binary Classification using Particle Swarm Optimization. International Journal of Innovative Computing, Information and Control, 2011. (ISI Impact Factor 2.93) Classification using Genetic Programming, Neurocomputing. (ISI Impact Factor 1.44)

Jabeen, H and Baig, A. R., Two-Stage Learning for Multi-Class

Jabeen, H and Baig, A. R., Two Layered Genetic Programming for

Mixed Variable Data Classification. Applied Soft Computing. (ISI Impact Factor 2.74)
Jabeen, H and Baig, A. R., Framework for Optimization of Genetic

Programming Evolved Arithmetic Classifier Expressions using Particle Swarm Optimization for Multi-Class Classification., Knowledge Based Systems. (ISI Impact Factor 1.57)

Journals
Jabeen, H and Baig, A R., "Review of Classification

using Genetic Programming." International Journal of Engineering Science and Technology, Feb 2010, Issue 2, Vol. 2. Jabeen, H and Baig, A R., "A Framework for Optimization of Genetic Programming Evolved Classifier Expressions." Lecture Notes in Computer Science, Springer, 2010. Jabeen, H and Baig, A R., "CLONAL-GP Framework for Artificial Immune System Inspired Genetic Programming for Classification." Lecture Notes in Computer Science, Springer, 2010. Jabeen, H and Baig, A. R., Multi-Class Classification using Genetic Programming. Lecture Notes in Computer Science, Springer 2010.

Conferences

Jalil, Z, Jabeen, H,Sponsor based Architecture for Resource Management in MultiAgent Systems, IADIS International Conference on Intelligent Systems and Agents 2007. Jabeen, H , Jalil, Z and Baig, A R., "Opposition Based Initialization in Particle Swarm Optimization (O-PSO), Montreal, Canada, 2009, Genetic and Evolutionary Computation Conference (GECCO 2009). Jabeen, H and Baig, A R., "DepthLimited Crossover in Genetic Programming for Classifier Evolution, Ulsan, South Korea, 2009, International Conference on Intelligent Computing (ICIC 2009). Jabeen, H and Baig, A R., "Particle Swarm Optimization Based Tuning of Genetic Program Evolved Classifier Expressions, Granada, Spain, 2010, International Workshop on Nature Inspired Cooperative Strategies for Optimization (NICSO 2010). Jabeen, H and Baig, A R., "Framework For Optimization Of Genetic Programming Evolved Classifier Expressions,Sansebastian, Spain,2010. Hybrid Artificial Intelligent Systems (HAIS 2010). Jabeen, H and Baig, A R., "CLONAL-GP Framework for Artificial Immune System Inspired Genetic Programming for Classification." Cardiff, UK, 2010. International Conference on Knowledge-Based and Intelligent Information & Engineering Systems (KES 2010). Jabeen, H and Baig, A. R., Multi-Class Classification using Genetic Programming. Changsha, China, 2010, International Conference on Intelligent Computing (ICIC 2010). Jabeen, H and Baig, A. R., Lazy Learning for Multi-Class Classification using Genetic Programming . Changsha, China, 2010, International Conference on Intelligent Computing (ICIC 2010). Imran, M, Jabeen, H, Ahmad, M, Abbas, Q, Bangyal, W and Abbas, Q Opposition based PSO and Mutation Operators (OPSO with Power Mutation), Shanghai, China, 2010, International Conference on Education Technology and Computer (ICETC 2010). Jalil, Z, Mirza,A, M, Jabeen, H, Word Length based Zero-Watermarking Algorithm for Tamper Detection in Text Documents, Chengdu, China, 2010, International Conference on Computer Engineering and Technology (ICCET 2010).

International Seminar
Data Classification using Genetic Programming at

School of Computer Science, Cardiff University, Sept, 2010.

International Presentations
ICIC 2009, Ulsan, South Korea, Presentation of the

accepted research paper DepthLimited Crossover in Genetic Programming for Classifier Evolution. KES 2010, Cardiff, UK, Presentation of the accepted research paper CLONAL-GP Framework for Artificial Immune System Inspired Genetic Programming for Classification. ICIC 2011, ZhengZhou, China, Presentation of the accepted research paper Lazy Learning for MultiClass Classification using Genetic Programming.

International Journal Reviewer

Knowledge Based Systems, (ISI Impact Factor

1.3), published by Springer.

International Conf. Reviewer

Invited reviewer for 4th IEEE International

Conference on Computer Science and Information Technology, IEEE-ICCSIT-2011, Chengdu, China.

Teaching
FAST-NU, COMSATS, FJWU, Iqra University

Introduction
Pre requisites
Introduction to Algorithms course Programming Mathematics

I will not teach you

How to program / debug Basic mathematics

Grading Policy
Quizzes
Assignments Midterm Exam Project Final

10% 5% 20 % 25% 40%

Quizzes
Unannounced

No makeup Quiz
Any miss will get 0

Assignments
Submit a HARDCOPY before the class.
No late submissions No email submissions Any excuse will NOT be entertained Any late will get 0

Must put headings on your submissions

Assignment number/project phase number Your Correct ID Your name

Project Target
Publish a Paper at the end of this course

At least know how to conduct research

Course Material
Will be provided as we proceed with the contents

Course Group http://groups.yahoo.com/group/ALG_IQRA/

Algorithm
A step by step procedure to solve a specific

problem in finite amount of time Input-> algorithm ->output Sorting problem Input a set of numbers Ouput list of sorted numbers

Types by Implementation
Recursive

Logical
Serial/parallel/distributed Deterministic/non deterministic Exact/Approximate

Types by Design
Brute force

Divide and Conquer

Dynamic Programming Greedy Linear Reduction Search Heuristic

Analysis of an Algorithm
Determine the running time of a program as a

function of its inputs Determine the total or maximum memory space needed for program data; Determine the total size of the program code; Determine whether the program correctly computes the desired result; Determine the complexity of the program--e.g., how easy is it to read, understand, and modify; and, Determine the robustness of the program--e.g., how well does it deal with unexpected or erroneous inputs?

Run time analysis

Run-time analysis is a theoretical classification

that estimates and anticipates the increase in

running time (or run-time) of an algorithm as its input size (usually denoted as n) increases.
The number of (machine) instructions which a

program executes during its running time is called its time complexity.

Empirical Analysis
Algorithms are independent on
Computer Language Operating system

It is difficult to analyze the running time of an

algorithm empirically.

Time Complexity
Take as an example a program that looks up a

specific entry in a sorted list of size n. Suppose this program were implemented on Computer A, a state-of-the-art machine, using a linear search algorithm, and on Computer B, a much slower machine, using a binary search algorithm. Benchmark testing on the two computers running their respective programs might look something like the following:

Order of growth
Landau symbols have mathematically precise

definitions. They have many uses, such as in the analysis of algorithms. These symbols are used to evaluate and to concisely describe the performance of an algorithm, in time and in space. O (called the Big Oh) (upper case greek letter THETA)

BIG Oh, Landau notation, asymptotic notation

The Big Oh is the upper bound of a function. In

the case of algorithm analysis, we use it to bound the worst-case running time, or the longest running time possible for any input of size n. We can say that the maximum running time of the algorithm is in the order of Big Oh.

Big Oh
1 get a positive integer from input

2 if n > 10
3 print "This might take a while..." 4 for i = 1 to n 5 for j = 1 to i 6 print i * j 7 print "Done!" Instructions 1,2,3,7 will be executed once. Evaluate execution of 4,5,6

 Mathematically speaking, O(n2) stands for a set of

functions, exactly for all those functions which, in the long run, do not grow faster than the function n2, that is for those functions for which the function n2 is an upper bound (apart from a constant factor.) To be precise, the following holds true: A function f is an element of the set O(n2) if there are a factor c and an integer number n0 such that for all n equal to or greater than this n0 the following holds f(n) cn2.

ln(n)

Big Oh Does Not Tell the Whole Story (operations on data)

Lower Bound
is also an order of growth but it is the opposite

of the Big Oh : it is the lower bound of a function. We can say that the minimum running time of the algorithm is in the order of .

Good news / Bad news

Hierarchy of complexities
Constant time // printing an input Logarithmic time // binary search Linear time // addition of input numbers Quadratic time //sorting Polynomial time // Exponential time // passwords Factorial time // TSP

Input Sizes
n= O(n2) O(2n) O(n!) 20 400 1048576 2.4 x 1018 40 1600 1099511627776 8.1 x 1047

Assume evaluating a solution takes 10-9 seconds

n= O(n2) O(2n) O(n!)

20 < 1 sec < 1 sec 77 yrs

40 < 1 sec 1,100 sec 25 x 1018 TRILLION yrs

Space Complexity Analysis

Less time AND Less memory

Exponential Memory Allocation