Welcome to Intro.

To Theory of
Computation

1
 Theory of Computation
Objective
• This course is mainly about abstractions of what
we can compute.
• The treatment is mathematical, but the point of
view is that of Computer Science.
• Roughly speaking, the theory of computation can
be divided into three overlapping subareas such
as :
• formal languages and automata theory,
computability theory and complexity theory.
In this course:
• Fundamental capabilities and limitations of computers
• Mathematical properties of computer hardware and
software
• Relevance to practice, for example, in the design of
new programming languages, compilers, string
searching, pattern matching, computer security,
artificial intelligence, etc.
• Learn about problem solving through argue, prove,
express and abstract
• Assignments & problem sets after each unit
Academic Integrity
• You are encouraged to discuss the
assignments problems among yourselves.
• Each student must hand in her/his own
solution.
• no cheating or copying from any source

4
Grading
• Assignments: 25% (10%+15%)
• Midterm: 25%
• Final: 50%

5
Bibliography
Required: Michael Sipser: Introduction to the
Theory of Computation, Second Edition,
Thomson.

6
 Introduction to Computability
Theory
Lecture1: Finite Automata and
Regular Languages

7
 Origin
• Started in 1930’s by mathematicians and
logicians in an attempt to understand
computation
• Mainly, to find the answer that all
mathematical problems can be solved in a
systematic way
• These research led the computers we know
and use today
Introduction
Computer Science stems from two starting
points:
Mathematics: What can be computed?
And what cannot be computed?
Electrical Engineering: How can we build
computers?
Not in this course.

9
 The theory of computation
• 3 central areas
• Automata Theory
• Complexity Theory
• Computability Theory
 What is Automata Theory?
• Study of abstract computing devices, or “machines”
• Automaton = an abstract computing device
– Note: A “device” need not even be a physical hardware!
• A fundamental question in computer science:
– Find out what different models of machines can do and
cannot do
– Computability vs. Complexity

11
 Complexity Theory
• Deals with the problem of
– What makes some problem computationally hard
or others easy
– The theory deals with the aspects of problem
which are the root causes of difficulty
– One solution is to alter the problem (easy one or
approximate one)
 Example
Easy Problems (Efficiently solvable)
• Sorting a list of 1 million numbers
• Computing the fastest path from Quetta to
Karachi
Hard Problems
• Time table scheduling of all courses in a large
university
• Factoring a large integer number (say 250 digits)
into its prime factors
Introduction
Computability Theory deals with the profound
mathematical basis for Computer Science, yet
it has some interesting practical outcomes
that I will try to point out sometimes.
The question we will try to answer in this theory
is:
“What can be computed? What Cannot be
computed and where is the line between the
two?”

14
Computational Models
A Computational Model is a mathematical
object (Defined on paper) that enables us to
reason about computation and to study the
properties and limitations of computing.
We will deal with Three principal computational
models in increasing order of Computational
Power.

15
Computational Models
We will deal with three principal models of
computations:
1. Finite Automaton (in short FA).
recognizes Regular Languages .
2. Push Down Automaton (in short PDA).
recognizes Context Free Languages .
3. Turing Machines (in short TM).
recognizes Computable Languages .
16
Computation

CPU memory

17
temporary memory

input
CPU
output

Program memory

18
 Example: f ( x)  x 3

temporary memory

input
CPU
output
Program memory
compute xx
compute x x2
19
 f ( x)  x 3

temporary memory
input
x2
CPU
output
Program memory
compute xx
compute x x2
20
 temporary memory f ( x)  x 3
z  2*2  4
f ( x)  z * 2  8
input
x2
CPU
output
Program memory
compute xx
compute x x2
21
 temporary memory f ( x)  x 3
z  2*2  4
f ( x)  z * 2  8
input
x2
CPU
f ( x)  8
Program memory output
compute xx
compute x x2
22
 Automaton
temporary memory

Automaton
input
CPU
output

Program memory

23
 Automaton
temporary memory

Automaton
input

output
transition

state

24
 Different Kinds of Automata
Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

25
 Finite Automaton

temporary memory

input
Finite
Automaton
output

Example: Elevators, Vending Machines

(small computing power) 26
 Pushdown Automaton
Temp.
memory Stack Push, Pop

Pushdown input

Automaton
output

Example: Compilers for Programming Languages

(medium computing power) 27
 Turing Machine

Temp.
memory Random Access Memory

input
Turing
Machine
output

Examples: Any Algorithm

(highest computing power) 28
 Power of Automata
Simple More complex Hardest
problems problems problems

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
29
Turing Machine is the most powerful
computational model known

Question: Are there computational

problems that a Turing Machine
cannot solve?

Answer: Yes (unsolvable problems)

30
Finite Automata - A Short Example
• The control of a washing machine is a very
simple example of a finite automaton.
• The most simple washing machine accepts
quarters and operation does not start until at
least 3 quarters were inserted.
Control of a Simple Washing Machine

q0 25 q25 25 q50 25 q75

• Accepts quarters. 25

• Operation starts after at least 3 quarters were

inserted.
• Accepted words: 25,25,25; 25,25,25,25; …
Finite Automata - A Short Example
• The second washing machine accepts 50 cents
coins as well.

q0 25 q25 25 q50 25,50 q75

25,50
50 50
• Accepted words: 25,25,25; 25,50; …
Finite Automata - A Short Example
• The second washing machine accepts 50 cents
coins as well.
• The most complex washing machine accepts
$1 coins too. 100

q0 25 q25 25 q50 25,50,100 q75

50 50,100
25,50,100
Finite Automaton - An Example
0 q0 0,1
qs

1 q1 0,1

States: Q  {qs , q0 , q1}

Initial State: qs
Final State: q0

35
Finite Automaton – An Example
0 q0 0,1
qs

1 q1 0,1

Transition Function:  qs ,0  q0  qs ,1  q1

 q0 ,0   q0 ,1  q0  q1,0   q1,1  q1
,
Alphabet: 0,1. Note: Each state has all transitions .
,

Accepted words: 0,00,01,000,001,...

36
Finite Automaton – Formal Definition
A finite automaton is a 5-tupple Q, ,  , q0 , F 
where:
1. Qis a finite set called the states.
2. is a finite set called the alphabet.   
3.  : Q    Q is the transition function.
4. q0  Q is the start state, and
,
5. F  Q is the set of accept states.
,

37
Observations
1. Each state has a single transition for each
symbol in the alphabet.
2. Every FA has a computation for every finite
string over the alphabet.

38
Examples

1. M accepts all words (strings) ending with 1.

2

The language recognized by M , called

2

LM 2  satisfies: LM 2   w | w ends with 1 .

39
How to do it
1. Find some simple examples (short accepted and
rejected words)
2. Think what should each state “remember”
(represent).
3. Draw the states with a proper name.
4. Draw transitions that preserve the states’
“memory”.
5. Validate or correct.
6. Write a correctness argument.
The automaton
1 q1 1
qs
1 0
0 q0 0

Correctness Argument:
The FA’s states encode the last input bit and q1
is the only accepting state. The transition
function preserves the states encoding.
Examples

1. M accepts all words (strings) ending with 1.

2

LM 2   w | w ends with 1 .

2. M 3 accepts all words ending with 0.
2’. M 4 accepts all words ending with 0 and
the empty word .
This is the Complement Automaton of M 2 .

42
Deterministic Finite Automaton (DFA)
Input Tape
String
Output
“Accept”
Finite or
Automaton “Reject”

43
head
Initial Configuration
Input Tape

a b b a
Input String

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

Initial state

44
 Scanning the Input
a b b a

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

45
a b b a

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

46
 a b b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

47
 Input finished

a b b a

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

accept

48
 A Rejection Case

a b a
•
Input String
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

49
 a b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

50
 a b a
•
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

51
Input finished

a b a

a, b
reject

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

52
 Another Rejection Case
Tape is empty

( )
•Input Finished
a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4
reject

53
Language Accepted: L  abba 

a, b

q5
a a, b
b a b
q0 a q1 b q2 b q3 a q4

54
To accept a string:
all the input string is scanned
and the last state is accepting

To reject a string:
all the input string is scanned
and the last state is non-accepting

55
 Another Example
•L  , ab, abba
a, b

q5
b a a a, b
b
q0 a q1 b q2 b q3 a q4
Accept Accept Accept
state state state

56
 Empty Tape

( )
Input Finished
a, b

q5
b a a a, b
b
q0 a q1 b q2 b q3 a q4

accept

57
 Another Example

a a, b

q0 b q1 a, b q2
Accept trap state
state

58
a a b
Input String

a a, b

q0 b q1 a, b q2

59
a a b

a a, b

q0 b q1 a, b q2

60
a a b

a a, b

q0 b q1 a, b q2

61
Input finished

a a b

a a, b
accept

q0 b q1 a, b q2

62
 A rejection case

b a b
Input String

a a, b

q0 b q1 a, b q2

63
b a b

a a, b

q0 b q1 a, b q2

64
b a b

a a, b

q0 b q1 a, b q2

65
Input finished

b a b

a a, b

q0 b q1 a, b q2

reject

66
Language Accepted:
L  {a b : n  0}
n

a a, b

q0 b q1 a, b q2

67
Assignment

1. M 2 accepts all words (strings) ending with 1.

LM 2   w | w ends with 1 .

2. M 3 accepts all words ending with 0.

3. M 4 accepts all strings over alphabet a, b

that start and end with the same symbol .

68
Assignment (cont)

4. M 5 accepts all words of the form 0m1n

where m, n are integers and m, n  0 .

5. M 6 accepts all words in 0,1,00,01,10.

69
Languages
• Definition: A language is a set of strings over
some alphabet .
• Examples:
– L1  0,1,10,1110001
– L2  0m1n | n, m are positive integers 
bit strings whose binary 
– L3   
value is a multiple of 4 

70
 Language Accepted by DFA
Language of DFA :M

it is denoted asL M  and contains

all the strings accepted by M

We say that a language L

is accepted (or recognized)
by DFA M if L M   L
71
• For a DFA M  Q, ,  , q0 , F 
• Language accepted by M:
• 
L M   w   :  q0 ,w   F
* *

q0 w q q  F

72
• Language rejected by : M


LM   w   :  q0 ,w   F
* *

q0 w q q  F

73
 More DFA Examples
  {a , b }
a, b
a, b

q0 q0

L(M )  { } L (M )   *

Empty language All strings

74
   {a , b }

a, b

q0 a, b q0

L(M )  { }
Language of the empty string

75
   {a , b }
LM = { all strings with prefix ab}
a, b

q0 a q1 b q2

b a
accept

q3 a, b
76
 LM  = { all binary strings containing
• substring 001 }

0,1
1 0
1

 0 0 00 1 001
0
77
LM  = { all binary strings without
substring 001 }

1 0 0,1
1

 0 1
0 00 001

0
78
 
L(M )  awa : w  a , b 
*
 a
• b
b
q0 a q2 q3

b a
q4

a, b 79
 Regular Languages
• Definition:
• A language L is regular if there is
a DFA M that accepts it ( L(M )  L )

• The languages accepted by all DFAs

form the family of regular languages

80
Languages

• A language of an FA, M , LM  , is the set of

words (strings) that M accepts.

• If La  LM  we say that M recognizes La .

• If La is recognized by some finite

,
automaton A , La is a Regular Language.

81
Some Questions
Q1: How do you prove that a language La is regular?
A1: By presenting an FA, M , satisfying La  LM  .
Q2: Why is it important?
A2: Recognition of a regular language requires a
controller with bounded Memory.
Q3: How do you prove that a language La is not
regular?
A3: Hard! to be answered on Week3 of the course.
82
Wrap up
1. Motivated the course.
2. Defined Finite Automata.
3. Learned how to deal with construction of
automata and how to come up with a
correctness argument.

83
 Assignment
• Read and understand with examples the
concept of finite state transducer and two-
tape finite automaton.
• You should also write and submit a one or two
page report about these topics.