Lecture 1

Efficient Problem Solving

Efficiency Measures
• Performance of a solution
• Most of the software problems do not have a single best
solution
• Then how do we judge these solutions?
• The solutions are chosen based on performance measures
Performance Measures
• Time
• Quality
• Simplicity…
 Syllabus
UNIT I: Introduction:
• Asymptotic notations, Average, Best and Worst case analysis of algorithms for Time and Space complexity,
Amortized Analysis, Solving Recurrence Equations, Proof Techniques: by Contradiction, by Mathematical
Induction.
• Priority Queues: Heaps & Heap sort.
UNIT-II:Divide And Conquer And Greedy Strategy:
• Divide and Conquer: General Strategy, Exponentiation. Binary Search, Quick Sort and Merge Sort. Greedy
Method. General Strategy, Knapsack problem, Job sequencing with Deadlines, Optimal merge patterns,
Minimal Spanning Trees and Dijkstra's algorithm.
UNIT III:Dynamic Programming:
• General Strategy, Multistage graphs, OBST, 0/1 Knapsack, Traveling Salesperson Problem, Flow Shop
Scheduling.
UNIT IV:Backtracking & Branch And Bound:
• Backtracking: General Strategy, 8 Queen's problem, Graph Coloring, Hamiltonian Cycles, 0/1 Knapsack.
• Branch and Bound: General Strategy, 0/1 Knapsack, Traveling Salesperson Problem .
UNIT V:Parallel Algorithms:
• Computational Model, Basic Techniques and Algorithms (Complete Binary Tree, Pointer Doubling, Prefix
Computation), Selection, Merging, Sorting Networks, Parallel Sorting, Graph Problems (Alternate
Algorithm for Transitive Closure, All pairs shortest path)
UNIT VI:NP-Hard And NP-Complete Problems:
• Algorithms, Complexity-intractability, Non-Deterministic Polynomial time (NP) Decision problems, Cooks
Theorem.
• NP-Complete problems- Satisfiability problem, vertex cover problem.
• NP-Hard problems-graph, scheduling, code generation problems, Simplified NP Hard Problems.
 Books
• Text Books
– Bressard & Bratley
– Horowitz and Sahani

• Reference Books
– Cormen & Rivest
• http://mitpress.mit.edu/algorithms/
– Aho & Ullman
 Unit I
• Asymptotic notations (O,,)(1h)
• Average, Best and Worst case analysis of algorithms
for Time and Space complexity (1h)
• Solving Recurrence Equations(1 h)
• Proof Techniques: by Contradiction,
by Mathematical Induction. (1h)
• Priority Queues: Heaps & Heap sort. (1h)
• Amortized Analysis (1/2h)
 Introduction
• The word Algorithm is derived from the name
of the Persian mathematician and author
“Abu Jafar Mohammad ibn Musa al
Khowarizmi”
 What is an algorithm?
• The idea behind the computer program
• Stays the same independent of
– Which kind of hardware it is running on
– Which programming language it is written in
• Solves a well-specified problem in a general way
• Is specified by
– Describing the set of instances (input) it must work
on
– Describing the desired properties of the output
The definition of an algorithm
 What is an Algorithm?
• Finite Set of instructions that if followed
accomplish a particular task.

• Criteria's satisfied by an algo

 Input - Zero or more quantities externally supplied
 Output - At least 1 quantity produced
 Definiteness - Each instr. clear and unambiguous.
 Finiteness – For all cases algo terminates
 Effectiveness – Every instr. basic and simple to carry out
 Important Properties of Algorithms
• Correct
– always returns the desired output for all legal
instances of the problem.
• Unambiguous
• Precise
• Efficient
– Can be measured in terms of
• Time
• Space
– Time tends to be more important
 Expressing Algorithms
• English description
More easily More
expressed precise
• Pseudo-code

• High-level
programming
language

Adapted from http://www.cs.sunysb.edu/~algorith/lectures-good/node1.html

 Pseudocode
• High level description of algorithm

• Less detailed than program

• Does not cover design issues

• Written in English
 Pseudocode for Max of Array
Algorithm arrayMax(a,n)
Input array a of n integers
Output maximum element of a
max <- a[0]
for i <- 1 to n-1 do
if a[i]>max then
max <- a[i]
return max
 Life Cycle of Algorithm
• Design
– Divide and conquer
– Greedy
– Dynamic etc.
• Write
• Test
• Analyze
– space - time tradeoff
 Analyzing Algorithms
• Priori, posteriori analysis
– Before implementation, after implementation
• Problem size
– No of inputs (eg. search n elements)
– No of operations (eg. comparisons of n elements)
• Macro, micro analysis
• Time complexity, space complexity
 Recursive Algorithms
• Towers of Hanoi
Algorithm TowersOfHanoi(n,x,y,z)
// Move top n disks from tower x to tower y
{
if (n>=1) then
{
TowersOfHanoi(n-1,x,z,y);
write(“Move top disk from tower”,x,”to top of tower”,y);
TowersOfHanoi(n-1,z,y,x);
}
}
 Case Study
Problem
• Employee details are to be stored
• Data structures – Arrays, linked lists
Requirement
• fast access, given employee number
• easy of coding
• ease of testing
Summary
 What are algorithms?
• Finite set of instructions to accomplish a task.
• The properties of an algorithm are as follows:
– Input
– Output
– Finiteness
– Definiteness
– Effectiveness
EXTRA Material
 Improve performance of algorithm
• Better design
• Better table design
• Use of Index
• Client server issues
– Network speed, volume of data etc.
• Psychological issues
 Why study Algorithms?
• The Human Genome Project has the goals of
identifying all the 100,000 genes in human DNA,
determining the sequences of the 3 billion chemical
base pairs that make up human DNA, storing this
information in databases, and developing tools for
data analysis.
• Each of these steps requires sophisticated
algorithms. The solution of these biological problems
lie in the algorithm study, thereby enabling scientists
to accomplish tasks while using resources efficiently.
The savings are in time, both human and machine,
and in money, as more information can be extracted
from laboratory techniques.
 Why study Algorithms?
• The Internet enables people all around the world to
quickly access and retrieve large amounts of
information. In order to do so, clever algorithms are
employed to manage and manipulate this large
volume of data.

• Examples of problems which must be solved include

finding good routes on which the data will travel and
using a search engine to quickly find pages on which
particular information resides.
 Why study Algorithms?
• Electronic commerce (e-commerce) enables
goods and services to be negotiated and
exchanged electronically. The ability to keep
information such as credit card numbers,
passwords, and bank statements private is
essential if electronic commerce is to be used
widely. Public-key cryptography and digital
signatures are among the core technologies
used and are based on numerical algorithms
and number theory.
 Why study Algorithms?
• In manufacturing and other commercial settings, it is often important to
allocate scarce resources in the most beneficial way.

• An oil company may wish to know where to place its wells in order to
maximize its expected profit.

• A candidate for the presidency of the United States may want to

determine where to spend money buying campaign advertising in order to
maximize the chances of winning an election.

• An airline may wish to assign crews to flights in the least expensive way
possible, making sure that each flight is covered and that government
regulations regarding crew scheduling are met.

• An Internet service provider may wish to determine where to place

additional resources in order to serve its customers more effectively.
Some concrete problems :
• We are given a road map on which the distance between each pair of adjacent
intersections is marked, and our goal is to determine the shortest route from one
intersection to another.

• We are given a sequence A1, A2, ..., An of n matrices, and we wish to determine
their product A1 A2 An. Because matrix multiplication is associative, there are
several legal multiplication orders. The number of possible multiplication orders is
exponential in n, and so trying all possible orders may take a very long time.

• We are given an equation ax = b (mod n), where a, b, and n are integers, and we
wish to find all the integers x, modulo n, that satisfy the equation. There may be
zero, one, or more than one such solution. We can simply try x = 0, 1, ..., n - 1 in
order, but here is a more efficient method.

• We are given n points in the plane, and we wish to find the convex hull of these
points. The convex hull is the smallest convex polygon containing the points.
Intuitively, we can think of each point as being represented by a nail sticking out
from a board. The convex hull would be represented by a tight rubber band that
surrounds all the nails. Each nail around which the rubber band makes a turn is a
vertex of the convex hull. Any of the 2n subsets of the points might be the vertices
of the convex hull.
 Characteristics that are common to
many interesting algorithms.
1. There are many candidate solutions, most of
which are not what we want. Finding one that
we do want can present quite a challenge.
2. There are practical applications.