Growth of Functions

How does one measure algorithms

We can time how long it takes a computer

What if the computer is doing other things? And what happens if you get a faster computer?
A 3 Ghz Windows machine chip will run an algorithm at a different speed than a 3 Ghz Macintosh

So that idea didnt work out well

2

How does one measure algorithms

We can measure how many machine instructions an algorithm takes

Different CPUs will require different amount of machine instructions for the same algorithm

So that idea didnt work out well

How does one measure algorithms

We can loosely define a step as a single computer operation

A comparison, an assignment, etc. Regardless of how many machine instructions it translates into

This allows us to put algorithms into broad categories of efficientness

An efficient algorithm on a slow computer will always beat an inefficient algorithm on a fast computer
4

Bubble sort running time

The bubble step take (n2-n)/2 steps Lets say the bubble sort takes the following number of steps on specific CPUs: 2 Intel Pentium IV CPU: 58*(n -n)/2 2 Motorola CPU: 84.4*(n -2n)/2 2 Intel Pentium V CPU: 44*(n -n)/2

Notice that each has an n2 term

As n increases, the other terms will drop out

5

Bubble sort running time

This leaves us with:

Intel Pentium IV CPU: 29n2

Motorola CPU: 42.2n2 Intel Pentium V CPU: 22n2

As processors change, the constants will always change

The exponent on n will not

6

Thus, we cant care about the constants

An aside: inequalities
If you have a inequality you need to show:
x<y

You can replace the lesser side with something greater:

x+1 < y

If you can still show this to be true, then the original inequality is true

Consider showing that 15 < 20

You can replace 15 with 16, and then show that 16 < 20. Because 15 < 16, and 16 < 20, then 15 < 20
7

An aside: inequalities
If you have a inequality you need to show:
x<y

You can replace the greater side with something lesser:

x < y-1

If you can still show this to be true, then the original inequality is true

Consider showing that 15 < 20

You can replace 20 with 19, and then show that 15 < 19. Because 15 < 19, and 19 < 20, then 15 < 20
8

An aside: inequalities
What if you do such a replacement and cant show anything?

Then you cant say anything about the original inequality

Consider showing that 15 < 20

You can replace 20 with 10 But you cant show that 15 < 10 So you cant say anything one way or the other about the original inequality
9

Quick survey

a) b) c) d)

I felt I understand running times and inequality manipulation Very well With some review, Ill be good Not really Not at all

10

The 2002 Ig Nobel Prizes

Biology Physics

Interdisciplinary Chemistry Mathematics Literature

Peace

Hygiene Economics

Medicine

Courtship behavior of ostriches towards humans under farming conditions in Britain Demonstration of the exponential decay law using beer froth A comprehensive study of human belly button lint Creating a four-legged periodic table Estimation of the surface area of African elephants The effects of pre-existing inappropriate highlighting on reading comprehension For creating Bow-lingual, a computerized dog-to-human translation device For creating a washing machine for cats and dogs Enron et. al. for applying imaginary numbers to the business world Scrotal asymmetry in man in ancient sculpture

11

Review of last time

Searches

Linear: n steps Binary: log2 n steps Binary search is about as fast as you can get

Sorts

Bubble: n2 steps Insertion: n2 steps There are other, more efficient, sorting techniques
In principle, the fastest are heap sort, quick sort, and merge sort These each take take n * log2 n steps In practice, quick sort is the fastest, followed by merge sort
13

Big-Oh notation
Let b(x) be the bubble sort algorithm We say b(x) is O(n2)

This is read as b(x) is big-oh n2 This means that the input size increases, the running time of the bubble sort will increase proportional to the square of the input size
In other words, by some constant times n2

Let l(x) be the linear (or sequential) search algorithm We say l(x) is O(n)

Meaning the running time of the linear search increases directly proportional to the input size
14

Big-Oh notation
Consider: b(x) is O(n2)

That means that b(x)s running time is less than (or equal to) some constant times n2
That means that l(x)s running time is less than (or equal to) some constant times n

Consider: l(x) is O(n)

15

Big-Oh proofs
Show that f(x) = x2 + 2x + 1 is O(x2)

In other words, show that x2 + 2x + 1 c*x2

Where c is some constant For input size greater than some x

We know that 2x2 2x whenever x 1 And we know that x2 1 whenever x 1 So we replace 2x+1 with 3x2

We then end up with x2 + 3x2 = 4x2 This yields 4x2 c*x2

This, for input sizes 1 or greater, when the constant is 4 or greater, f(x) is O(x2) We could have chosen values for c and x that were different
16

Big-Oh proofs

17

Rosen, section 2.2, question 2(b)

Show that f(x) = x2 + 1000 is O(x2)

In other words, show that x2 + 1000 c*x2

We know that x2 > 1000 whenever x > 31

Thus, we replace 1000 with x2 This yields 2x2 c*x2

Thus, f(x) is O(x2) for all x > 31 when c 2

18

Rosen, section 2.2, question 1(a)

Show that f(x) = 3x+7 is O(x)

In other words, show that 3x+7 c*x

We know that x > 7 whenever x > 7

Duh! So we replace 7 with x This yields 4x c*x

Thus, f(x) is O(x) for all x > 7 when c 4

19

Quick survey

a) b) c) d)

I felt I understand (more or less) Big-Oh proofs Very well With some review, Ill be good Not really Not at all

20

Todays demotivators

21

A variant of the last question

Show that f(x) = 3x+7 is O(x2)

In other words, show that 3x+7 c*x2

We know that x > 7 whenever x > 7

Duh! So we replace 7 with x This yields 4x < c*x2 This will also be true for x > 7 when c 1

Thus, f(x) is O(x2) for all x > 7 when c 1

22

What that means

If a function is O(x)

Then it is also O(x2) And it is also O(x3)

Meaning a O(x) function will grow at a slower or equal to the rate x, x2, x3, etc.

23

Function growth rates

For input size n = 1000 O(1) O(log n) O(n) O(n log n) O(n2) O(n3) O(n4) O(nc) 2n n! nn 1 10 103 104 106 109 1012 103*c 10301 102568 103000

c is a consant
Many interesting problems fall into this category

24

Function growth rates

Logarithmic scale!

25

Integer factorization
Factoring a composite number into its component primes is O(2n) This, if we choose 2048 bit numbers (as in RSA keys), it takes 22048 steps

Thats about 10617 steps!

26

Formal Big-Oh definition

Let f and g be functions. We say that f(x) is O(g(x)) if there are constants c and k such that |f(x)| C |g(x)| whenever x > k

27

Formal Big-Oh definition

28

A note on Big-Oh notation

Assume that a function f(x) is O(g(x)) It is sometimes written as f(x) = O(g(x))

However, this is not a proper equality! Its really saying that |f(x)| C |g(x)|

In this class, f(x) is O(g(x))

we

will

write

it

as

29

The growth of combinations of functions

Ignore this part

30

Big-omega and Big-theta

Ignore this part

31

NP Completeness
Not in the textbook this is additional material A full discussion of NP completeness takes 3 hours for somebody who already has a CS degree

We are going to do the 15 minute version of it

Any term of the form nc, where c is a constant, is a polynomial

Thus, any function that is O(nc) is a polynomial-time function 2n, n!, nn are not polynomial functions
32

Satisfiability
Consider a Boolean expression of the form:
(x1 x2 x3) (x2 x3 x4) (x1 x4 x5) This is a conjunction of disjunctions

Is such an equation satisfiable?

In other words, can you assign truth values to all the xis such that the equation is true?

33

Satisfiability
If given a solution, it is easy to check if such a solution works

Plug in the values this can be done quickly, even by hand

However, there is no known efficient way to find such a solution

The only definitive way to do so is to try all possible values for the n Boolean variables That means this is O(2n)! Thus it is not a polynomial time function
NP stands for Not Polynomial

Cooks theorem (1971) states that SAT is NP-complete

There still may be an efficient way to solve it, though!

34

NP Completeness
There are hundreds of NP complete problems

It has been shown that if you can solve one of them efficiently, then you can solve them all Example: the traveling salesman problem
Given a number of cities and the costs of traveling from any city to any other city, what is the cheapest round-trip route that visits each city once and then returns to the starting city?

Not all algorithms that are O(2n) are NP complete

In particular, integer factorization (also O(2n)) is not thought to be NP complete

35

NP Completeness
It is widely believed that there is no efficient solution to NP complete problems

In other words, everybody has that belief

If you could solve an NP complete problem in polynomial time, you would be showing that P = NP If this were possible, it would be like proving that Newtons or Einsteins laws of physics were wrong

In summary:

NP complete problems are very difficult to solve, but easy to check the solutions of It is believed that there is no efficient way to solve them
36

Quick survey

a)
b) c) d)

I sorta kinda get the hang of NP completeness Very well With some review, Ill be good Not really Not at all

37

Star Wars: Episode III trailer

No, really!

38

Quick survey

a)
b) c) d)

I felt I understood the material in this slide set Very well With some review, Ill be good Not really Not at all

39

Quick survey

a)
b) c) d)

The pace of the lecture for this slide set was Fast About right A little slow Too slow

40

Quick survey

a) b) c) d)

How interesting was the material in this slide set? Be honest! Wow! That was SOOOOOO cool! Somewhat interesting Rather borting Zzzzzzzzzzz

41