Automata Theory

 
 –  
2009 -2008

  .

www.uomustansiriyah.edu.iq
www.mustansiriya.com

1
 : ‫ﻣﻌﻠﻮﻣﺎت ﻣﻬﻤﺔ‬
Instructor: Hassan Kassim Mohammad

Objective
Theory of computation is the theoretical study of capabilities and limitations of
computers(Theoretical models of computation)
Develop skills of precise and formal reasoning in computer science.
REFERENCES:
1. Introduction to Computer Theory 2nd Edition
Daniel I. A. Cohen John Wiley & Sons, Inc 1997. ISBN 0-471-13772-3
2. Introduction to Automata Theory, Languages, and Computation, 2/E,
John E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman, Addison-Wesley 2001.
ISBN 0-201-44124-1.

Prerequisites
• Discrete Mathematics
• Introduction to Programming

Units: 3

Grading Policy
1st exam: 5% attendance: 5% homework & projects & Assignments & Quizzes: 5%
nd
2 exam: 5% attendance: 5% homework & projects & Assignments & Quizzes: 5%
Final exam: 70%

Notes
• Student must attend at least 80% of the total class sessions to pass the course.
• Any kind of cheating/plagiarism may result in an Fail grade in the course.
• No labs. But you should write some programs with any language you may know.
• There will be 30 lectures 100 minutes each.
(enforced by surprise quizzes – hence no makeup for a quiz).
• Late homework submissions will be penalized according to the formula: 100 –10*x2 where x: the number
of days of late submission.
• Cheating on homework #i will cause you to get zero grade for the homework.

Office Hours
Sunday 10:00 - 1:00 1:00 – 3:35 3:35 – 5:00
Monday 8:30 – 3:35 3:35 – 5:00
Wednesday 10:00 -1:00 1:00 – 2:15 2:15 – 5:00

Contact Information
Office: 3rd Floor, computer science dept. room no. 67
E-mail: world_nora@yahoo.com

2
Syllabus

1. Introduction, terminology, definitions

2. LANGUAGE
3. Introduction to Language
4. Regular Expressions
5. Finite Automata
6. Deterministic Finite Automaton (DFA)
7. NON Deterministic Finite Automaton (NFA)
8. Language Accepted by DFA
9. Language Accepted by NFA
10. FA with Epsilon moves
11. Construct regular expression from NFA
12. Regular Expression to NFA
13. Moore and Mealy machines
14. Pumping lemma for regular languages
15. Kleene Theorem
16. Regular Grammar
17. Myhill-Nerode Theorem Minimization of DFA
18. --------------------------EXAM
19. Context-free Languages
20. Pushdown Automata
21. CFG/CFL to NPDA
22. NPDA to CFG/CFL
23. CFG derivation trees Parsing
24. Chomsky normal form
25. Greibach normal form
26. Ambiguous CFL's
27. ---------------------------EXAM
28. TURING MACHINES TM
29. COMPUTABILITY and COMPLEXITY
30. Unsolvable Problems
31. Time Complexity
32. CYK algorithm for CFG's
33. CFL pumping lemma and properties
34. Halting Problem, language, construction
35. Church Turing Thesis
36. --------------------------FINAL EXAM

3
 1 ‫ ﻣﻠﺨﺺ ﻣﺤﺎﺿﺮة‬- ‫ﺣﺴﻦ ﻗﺎﺳﻢ‬.‫ أ‬-‫اﻟﻨﻈﺮﻳﺔ اﻻﺣﺘﺴﺎﺑﻴﺔ‬-‫ﻗﺴﻢ ﻋﻠﻮم اﻟﺤﺎﺳﺒﺎت‬-‫آﻠﻴﺔ اﻟﻌﻠﻮم‬-‫اﻟﺠﺎﻣﻌﺔ اﻟﻤﺴﺘﻨﺼﺮﻳﺔ‬

As a computer IT, you must study the following:

1- Automata and formal language.
Which answers - What are computers (Or what are models of computers)
2- Compatibility.
Which answers - What can be computed by computers?
3- Complexity.
Which answers - What can be efficiently computed?

Automaton & automata

In automata we will simulates parts of computers.
Or we will make mathematical models of computers
Automata are more powerful than any real computer because we can design any machine on
papers that can do everything we want.

Kinds of languages:

1- Talking language: (e.g.: English language.):

- It has alphabetic: ∑={a,b,c,….z , A,B,C,…..Z ,….}
- From these alphabetic we make sentences that belong to English language.
Now we want to know is this sentence is true or false so -We need grammar in English.

Example:
Ali is a good boy. (It is a sentence Є English language.)
It's semantic (meaning) & syntax (structure) is true according to English grammar.

Mona eats papers. (It is a sentence Є English language.)

It's semantic (meaning) is false but it's syntax (structure) is true according to English
grammar.

2- Programming language: (e.g.: c++ language.):

- It has alphabetic: ∑={a,b,c,….z , A,B,C,…..Z , ?, /, - ,\.}.
- From these alphabetic we make sentences that belong to programming language.
Now we want to know is this sentence is true or false so - We need a compiler to make sure
that syntax is true and we don't need to check the meaning because we talk to computer that
doesn't know any thing.

3- Formal language: (any language we want.)

- It has strings.
- From these strings we make sentences that belong to this formal language.
Now we want to know is this sentence is true or false so - We need rules.

4
Example:
Alphabetic:
∑={0,1}.
Sentences:
0000001.
1010101.
Rules:
Accept any sentence start with zero and refuse sentences that start with one.
So we accept: 0000001. As a sentence satisfy the rules.
And refuse: 1010101. As a sentence doesn't satisfy the rules.
so we can make our formal language but we must take in mind it's:
- ∑ alphabetic. (Arguments.)
- rules.
From which we can get the language elements and vice versa.

Example 1:
L = {yellow, black, red}. // this is a language elements.
The language alphabetic belongs to English.
Its rules: accept any element in this set and refuse others elements:
So we accept yellow. // as it satisfies the language rules.
And refuse gray. // as it doesn't satisfies the language rules.

Yes
input
No
Automaton

Example 2:
L = {x}. // this is a language element.
The language alphabetic belongs to English.
Its rules: set of all x over the alphabetic ∑.
So we accept x, xx, xxx, xxxx. // as it satisfies the language rules.

Example 3:
L = {x}. // this is a language element.
The language alphabetic belongs to English.
Its rules: set of all words over the alphabetic ∑ with odd length.
So we accept { x, xxx, xxxxx, xxxxxxx,……}. // as it satisfies the language rules.
And refuse: xx // because it is not odd length.

Note from this example:

We can write our language as following:
L= { x , x³, x5…..}. And we can write the rule description as mathematical expression:
Accept: { x^2n+1: n ≥0}.
Now let us see the empty word: λ
- It is a word of length zero.
5
 - Its notation is: λ or ε.
- Some language accept empty words others not.

1- Concatenation:

‫آﻞ اﻟﻜﻠﻤﺎت اﻟﺘﻲ ﻳﻜﻮن اﻟﺠﺰء اﻷول ﻣﻨﻬﺎ أول ﻋﻨﺼﺮ ﻓﻲ اﻟﻠﻐﺔ اﻷوﻟﻲ )اﻟﻤﺠﻤﻮﻋﺔ اﻷوﻟﻲ( واﻟﺠﺰء اﻟﺜﺎﻧﻲ ﻣﻨﻬﺎ أول ﻋﻨﺼﺮ‬
.(‫ﻓﻲ اﻟﻠﻐﺔ اﻟﺜﺎﻧﻴﺔ)اﻟﻤﺠﻤﻮﻋﺔ اﻟﺜﺎﻧﻴﺔ‬
- L1 concatenated L2 if:
{W| w= xy, x belongs to L1 & y belongs to L2}.
- Notation to the concatenation: . (The dot.):
So if L1 = {x, xxx}. //language (set) one.
And L2 = {xx}. // language (set) two.
So (L1.L2) means L1 concatenated L2 and it is equal = {xxx, xxxxx}. // according to the
definition of concatenation.

Examples on concatenations:
Ex1:
L1 = {a, b}.
L2 = {c, d}.
L1.L2 = {ac, ad, bC, bd}
Note: ab differ from ba.

Ex2:
∑= {x}.
L1 = {set of all odd words over ∑ with odd length}.
L1 = {set of all even words over ∑ with odd length}.
L1= {x, xxx, xxxxx, xxxxxxx……}.
L2= {λ, xx, xxxx, xxxxxx…}.
L1.L2 = {x, xxx, xxxxx, xxxxxxx…}.
Note:
.‫اﻟﺘﻜﺮار ﻏﻴﺮ ﻣﺴﻤﻮح داﺧﻞ اﻟﻤﺠﻤﻮﻋﺔ‬

Ex3:
L1 = {x, xxx}.
L2 = {xx}.
L1.L2 = {xxx, xxxxx}.
Some rules on concatenation:
λ.x = x
L1.L2 = {set of elements}

6
 2 ‫ ﻣﻠﺨﺺ ﻣﺤﺎﺿﺮة‬- ‫ﺣﺴﻦ ﻗﺎﺳﻢ‬.‫ أ‬-‫اﻟﻨﻈﺮﻳﺔ اﻻﺣﺘﺴﺎﺑﻴﺔ‬-‫ﻗﺴﻢ ﻋﻠﻮم اﻟﺤﺎﺳﺒﺎت‬-‫آﻠﻴﺔ اﻟﻌﻠﻮم‬-‫اﻟﺠﺎﻣﻌﺔ اﻟﻤﺴﺘﻨﺼﺮﻳﺔ‬

Regular expression (RE)

Regular languages are set of words, or strings, over an alphabet of characters valid for the language.

λ is the null string (there are no symbols in this string)

* is the set of all strings of length greater than or equal to 0

Example:
A = {a,b} // the alphabet is composed of a and b
a* = {λ, a,b,aa,ab,ba,bb,aaa,aab,…}
The symbol * is called the Kleene star.
∅ (empty set)
λ (empty string)
( ) delimiter ,
∪ + union (selection)
concatenation

∑ = {a, b, c, · · · }
The language does not include every possible combination of characters, so we need a way to define which
combinations are valid.
Forming Expressions

Given regular expressions x and y, x + y is a regular expression

representing the set of all strings in either x or y (set union)
x = {a, b}, y = {c, d}, x + y = {a, b, c, d}

Regular expressions give us a method for finding recognized words in strings, making them crucial for lexing
Regular expressions are good for describing lexemes (words) in a programming language:
Identifier = (a+b+· · ·+z)(a+b+· · ·+z+A+B+· · ·+Z+0+· · ·+9)_
Digit = (0 + · · · + 9)
Number = (1 + · · · + 9)Digit_ + −(1 + · · · + 9)Digit_
Keywords: if = if,while = while, · · ·
Mark Hills CS421 Lecture 9: Regular Expressions and Finite Automata
More Examples of Concatenation Operation •
Example 1
Let A={0,1}, W1 = 00110011, W2 = 00000
W1 • W2 = 0011001100000
W2 • W1 = 0000000110011
W1 • λ = W1 = 00110011
λ• W2 = W2 = 00000
x = {a, b}, y = {c, d}, xy = {ac, ad, bc, bd}

7
Examples of regular expressions
// NB. star-closure precedes concatenation; concatenation precedes union

Describe the language (what is the output of the following RE )

a ∪ b ∪ c =={a,b,c}
r=λ
r=a
r = r1 + r2
r = r1 r2
r = r1*
r = (r1)

λ == {λ}
λ* == {λ}
a == {a}
(a)2 == (a)(a) = {aa}
(a)* == {λ, a, aa, aaa, ….}
(a)+ == {a, aa, aaa, ….}
(a + a) == {a}
(a + b) == {a, b}
(a + b)2 == (a + b)(a + b) == {aa, ab, ba, bb}
(a + b + c) == {a, b, c}
(a + b)* == {λ, a, b, aa, bb, ab, ba, aaa, bbb, aab, bba, ….}
(abc) == {abc}
(λ + a) bc == {bc, abc}
ab* == {a, ab, abb, abbb, …}
(ab)* == {λ, ab, abab, ababab, …}
a + b* == {a, λ, b, bb, bbb, …}
a (a + b)* == {a, aa, ab, aaa, abb, aba, abaa, … }
(a + b)* a (a + b)* == {a, aaa, aab, baa, bab, …}
(a + λ)* == (a)* = {λ, a, aa, aaa, ….}

x* (a + b) + (a + b) = x* (a + b)
x* y + y = x* y
(x + λ)x* = x* (x + λ) = x*
(x + λ)( x + λ)* (x + λ) = x*

start with _ == a (a + b)*

end with _ == (a + b)* b
contains exactly 2 a's == (b)* a (b)* a (b)*
contains at least 2 a's == (a + b)* a (a + b)* a (a + b)*
contains 2 a's or 2 b's == [(b)* a (b)* a (b)*] + [(a)* b (a)* b (a)*)]
even no of a == [ (b)* a (b)* a (b)* ]*

8
Exercise

Ex. 1: Find a regular expression corresponding to the language of all strings over the alphabet { a, b } that
contain exactly three a's.

Ex. 2: Find a regular expression corresponding to the language of all strings over the alphabet { a, b } that end
with ab.

Ex. 3: Find a regular expression corresponding to the language of strings of even lengths over the alphabet of
{ b }.

Ex. 4: Find a regular expression corresponding to the language of all strings over the alphabet { a, b } that
contain exactly two successive a's. (‫)ﻣﺘﺘﺎﺑﻌﺔ‬

Ex. 4: Find a regular expression corresponding to the following expressions :

(λ)* =
(x)* + (λ) =
aa* b =
bba*a =
(a + b)* ba=

9
 3 ‫ ﻣﻠﺨﺺ ﻣﺤﺎﺿﺮة‬- ‫ﺣﺴﻦ ﻗﺎﺳﻢ‬.‫ أ‬-‫اﻟﻨﻈﺮﻳﺔ اﻻﺣﺘﺴﺎﺑﻴﺔ‬-‫ﻗﺴﻢ ﻋﻠﻮم اﻟﺤﺎﺳﺒﺎت‬-‫آﻠﻴﺔ اﻟﻌﻠﻮم‬-‫اﻟﺠﺎﻣﻌﺔ اﻟﻤﺴﺘﻨﺼﺮﻳﺔ‬

Finite Automata
The language can be defined by three forms regular
expression RE, transition graph TG, finite automata FA
A finite automaton can be thought of as the device
shown below consisting of a tape and a control circuit
which satisfy the following conditions:

1. The tape has the left end and extends to the right
without an end.
2. The tape is divide into squares in each of which a
symbol.
3. The tape has a read only head.
4. The head is always at the leftmost square at the
beginning of the operation.
5. The head moves to the right one square every time
it reads a symbol. It never moves to the left. When it sees no symbol, it stops and the automaton
terminates its operation.
6. There is a control determines the state of the automaton and also controls the movement of the head.

‫ﺷﻜﻞ رﺳﻮﻣﻲ ﻳﻤﺜﻞ ﺗﻨﻘﻼت ﺑﻴﻦ ﻣﺠﻤﻮﻋﺔ ﺣﺎﻻت ﺗﺴﻴﻄﺮ ﻋﻠﻴﻬﺎ ﻗﻮاﻋﺪ ﻋﻦ ﻃﺮﻳﻖ ﺷﺮﻳﻂ ﻣﻦ اﻟﻤﺪﺧﻼت‬

A DFA represents a finite state machine that recognizes a RE.

For example, the following FA: recognize (accept) string ab

A finite automaton consists of a finite set of states, a set of transitions (moves), one start state, and a set of
final states (accepting states). In addition, a DFA has a unique transition for every state-character
combination.
it is a set of states, and its “control” moves from state to state in response to external “inputs” .
A finite automaton, FA, provides the simplest model of a computing device. It has a central processor of finite
capacity and it is based on the concept of state.
finite state machine is a 5 tuple M = (Q, A, T, S ,F), where

o Q --set of states = {q0, q1, q2, ….}

o A -- set of input symbols ={a,b, …, 0, 1, …}
o T --set of transitions or rules
T

o S -- an initial state
o F -- the final state -- could be more than one final state

Designing (drawing) FA

State Start Final Transition

with numbers or any name - or small arrow + or double circle (only one input or symbol on the edge)
a,b allowed means (a or b)

loop

10
Example1: Q = { 0, 1, 2 }, ∑= { a }, A = { 1 }, the initial state is 0 and T is shown in the following table.

State (q) Input (a)

0 1
1 2
2 2

trace string aaaa on this machine (if it is accepted or not)

the string is accepted if and only if we reached final state and the string is finished then

Example2: Q = { 0, 1, 2 }, A= { a, b }, F = { 1 }, the initial state is 0 and T is shown in the following table.

State (q) Input (a) Input (b)

0 1 2
1 2 2
2 2 2

Example 3: Q = { 0, 1 }, A= { a, b }, f = { 0 }, the initial state is 0 and T is shown in the following table.

State (q) Input (a) Input (b)

0 0 1
1 1 1

Transition diagram:

TG has many inputs on the edge ab FA has only one input on the edge a

Deterministic Finite Automata DFA and Non Deterministic Finite Automata NFA
DFA: different input from state to different states NFA: one input from state to different states

11
Language accepted by FA
string is accepted by a DFA if and only if the DFA starting at the initial state and ends in an accepting state
after reading the string. lets start with simplest FA.

Q)) Find if the string aababa accepted or not on the above FA

12
Examples of languages accepted by FA
FA RE
λ

aa

a+ = aa*

a*

a+b

(a+b)*

a*b

b(a+b)*

(a+b)*b

13
a(a+b)*b

(a+b)* b(a+b)*

ab(a+b)*

a*babb*

(aa)*ba

contains 3 a's b*ab*ab*ab*

contains even number of a =

(b*ab*ab*)+

14
Converting a Regular Expression into a Finite Automaton

RE FA
λ

aa

a+ = aa*

NFA
a*

a+b

(a+b)*

a*b

b(a+b)*

NFA
(a+b)*b

NFA

15
a(a+b)*b

NFA
(a+b)* b(a+b)*

NFA
ab(a+b)*

NFA
a*babb*

NFA
(aa)*ba

contains 3 a's b*ab*ab*ab*

contains even number of a = (b*ab*ab*)+

dividable by 3

all bit strings that begin with 0 and end with 1

all bit strings whose number of 0's is a multiple of

5

all bit strings with more 1's than 0's

all bit strings with no consecutive 1's

16
Converting NFA into DFA

: ‫ﺗﻮﺟﺪ ﺛﻼﺛﺔ ﺧﻄﻮات ﻟﻠﺘﺤﻮﻳﻞ‬

‫أوﻻ ﻋﻤﻞ ﺟﺪول اﻟﺘﻨﻘﻼت‬

‫ﺛﺎﻧﻴﺎ رﺳﻢ اﻟﺸﻜﻞ اﻟﺠﺪﻳﺪ‬

‫ﺛﺎﻟﺜﺎ إزاﻟﺔ اﻟﺤﺎﻻت اﻟﺰاﺋﺪة اﻟﺘﻲ ﻻ ﻳﻤﻜﻦ اﻟﻮﺻﻮل إﻟﻴﻬﺎ‬

1)) transition table

state a b
q0 q1q2 q0
q1 q2 q3
q2 q2q3 q2
q3 q3 -
q1q2 q2q3 q2q3
q2q3 q2q3 q3

Any state contains final mark it as a final state

2)) draw the new FA from transition table

3)) remove unreachable states (marked by dashed circle – state q1 and state q3 ) because we can not reach it.

---------------------------------------------------------------------------------------------------------------------------------

HW convert the following NFA into DFA

17