CMSC 451 Lecture Notes on DFAs, NFAs, Regular Expressions

CMSC 451 Selected Lecture Notes
http://www.cs.umbc.edu/~squire/cs451_lect.html
CS451 Selected Lecture Notes

This is one big WEB page, used for printing
Note that this le is plain text and .gif les. The plain text spells out Greek letters because they can not be typed.
These are not intended to be complete lecture notes. Complicated figures or tables or formulas are included here in case they were not clear or not copied correctly in class. Information from the language definitions, automata definitions, computability definitions and class definitions is not duplicated here. Lecture numbers correspond to the syllabus numbering.
Contents
Lecture 1 Fast Summary Part 1 Lecture 2 Fast Summary Part 2 Lecture 3 DFA and regular expressions Lecture 4 Nondeterministic Finite Automata NFA Lecture 5 NFA with epsilon moves Lecture 6 regular expression to NFA Lecture 7 NFA to regular expression, Moore, Mealy Lecture 8 Pumping Lemma for regular languages Review basics of proofs Lecture 9 Intersection of two languages, closure Lecture 10 Decision Algorithms Lecture 11 Quiz 1 Lecture 12 Myhill-Nerode minimization Lecture 13 Formal Grammars, CFG Lecture 14 Context Free Grammar derivation trees Lecture 15 CFG 'simplication' algorithm Lecture 16 Chomsky Normal Form Lecture 17 Greibach Normal Form Lecture 18 Inherently ambiguous CFL's, Project Lecture 19 Quiz 2 Lecture 20 Push Down Automata Lecture 21 CFG/CFL to NPDA Lecture 22 NPDA to CFG/CFL Lecture 23 Turing Machine Model Lecture 24 CYK algorithm for CFG's Lecture 25 Pumping Lemma for Context Free Languages
1 of 66
05/21/2011 09:31 AM
Lecture 25a CFL closure properties Lecture 26 The Halting Problem Lecture 27 Church Turing Thesis Lecture 28 Review Lecture 29 Final Exam Other Links
Lecture 1 Fast Summary Part 1

A fast summary of what will be covered in this course (part 1) For reference, read through the language definitions. automata definitions and
Lecture 2 Fast Summary Part 2

A fast summary of what will be covered in this course (part 2) For reference, read through the computability definitions and complexity class definitions
Lecture 3 DFA and regular expressions

Example of a Deterministic Finite Automata, DFA
Machine Definition M = (Q, Sigma, delta, q0, F) Q = { q0, q1, q2, q3, q4 } the set of states (finite) Sigma = { 0, 1 } the input string alphabet (finite) delta the state transition table - below q0 = q0 the starting state F = { q2, q4 } the set of final states (accepting when in this state and no more input) inputs | 0 | 1 | ---------+---------+---------+ q0 | q3 | q1 | q1 | q1 | q2 | states q2 | q2 | q2 | q3 | q4 | q3 | q4 | q4 | q4 | ^ ^ ^ | | | | +---------+-- every transition must have a state +-- every state must be listed
delta
2 of 66
05/21/2011 09:31 AM
An exactly equivalent diagram description for the machine M. Each circle is a unique state. The machine is in exactly one state and stays in that state until an input arrives. Connection lines with arrows represent a state transition from the present state to the next state for the input symbol(s) by the line. L(M) is the notation for a Formal Language defined by a machine M. Some of the shortest strings in L(M) = { 00, 11, 000, 001, 010, 101, 110, 111, 0000, 0001, 0010, 0011, 0100, 0101, 0110, 1001, ... } In words, L is the set of strings over { 0, 1} that contain at least two 0's starting with 0, or that contain at least two 1's starting with 1. Every 00 11 000 001 010 011 0110 input q0 q3 q0 q1 q0 q3 q0 q3 q0 q3 q0 q3 q0 q3 sequence goes through a sequence of states, for example q4 q2 q4 q4 q4 q4 q3 q4 q3 q3 q3 q3 q4
More information on DFA
Definition of a Regular Expression -----------------A regular expression may be the null string, r = epsilon A regular expression may be an element of the input alphabet, sigma, r = a A regular expression may be the union of two regular expressions, r = r1 + r2 A regular expression may be the concatenation (no symbol) of two regular expressions,
3 of 66
05/21/2011 09:31 AM

r = r1 r2
A regular expression may be the Kleene closure (star) of a regular expression r = r1* (the asterisk should be a superscript, but this is plain text) A regular expression may be a regular expression in parenthesis r = (r1) Nothing is a regular expression unless it is constructed with only the rules given above. The language represented or generated by a regular expression is a Regular Language, denoted L(r). The regular expression for the machine M above is r = (1(0*)1(0+1)*)+(0(1*)0(0+1)*) Later we will give an algorithm for generating a regular expression from a machine definition. For simple DFA's, start with each accepting state and work back to the start state writing the regular expression. The union of these regular expressions is the regular expression for the machine. For every DFA there is a regular language and for every regular language there is a regular expression. Thus a DFA can be converted to a regular expression and a regular expression can be converted to a DFA.
4 of 66
05/21/2011 09:31 AM
Given a DFA and are accepted by consuming to do available to do
one or more strings, determine if the string(s) the DFA. This may be error prone and time by hand. Fortunately, there is a program this for you. do the following:
On irix.gl.umbc.edu
ln -s /afs/umbc.edu/users/s/q/squire/pub/dfa dfa cp /afs/umbc.edu/users/s/q/squire/pub/download/ab_b.dfa . dfa < ab_b.dfa # or dfa < ab_b.dfa > ab_a.out On linux.gl.umbc.edu do the following:
ln -s /afs/umbc.edu/users/s/q/squire/pub/linux/dfa dfa cp /afs/umbc.edu/users/s/q/squire/pub/download/ab_b.dfa .
5 of 66
05/21/2011 09:31 AM

dfa < ab_b.dfa # or dfa < ab_b.dfa > ab_a.out
Full information is available at Simulators The source code for the family of simulators is available.
Lecture 4 Nondeterministic Finite Automata, NFA

Important! nondeterministic has nothing to do with random, nondeterministic implies parallelism. The difference between a DFA and a NFA is that the state transition table, delta, for a DFA has exactly one target state but for a NFA has a set, possibly empty (phi), of target states. Example of a NFA, Nondeterministic Finite Automata given by a) Machine Definition M = (Q, sigma, delta, q0, F) b) Equivalent regular expression c) Equivalent state transition diagram and example tree of states for input string 0100011 and an equivalent DFA, Deterministic Finite Automata for the first 3 states. a) Machine Definition M = (Q, sigma, delta, q0, F)
Q = { q0, q1, q2, q3, q4 } the set of states sigma = { 0, 1 } the input string alphabet delta the state transition table q0 = q0 the starting state F = { q2, q4 } the set of final states (accepting when in this state and no more input) inputs delta | 0 | 1 | ---------+---------+---------+ q0 | {q0,q3} | {q0,q1} | q1 | phi | {q2} | states q2 | {q2} | {q2} | q3 | {q4} | phi | q4 | {q4} | {q4} | ^ ^ ^ | | | | +---------+-- a set of states, phi means empty set +-- every state must be listed b) The equivalent regular expression is (0+1)*(00+11)(0+1)* This NFA represents the language L = all strings over {0,1} that have at least two consecutive 0's or 1's.
6 of 66
05/21/2011 09:31 AM
c) Equivalent
NFA
state transition diagram.
Note that state q3 does not go anywhere for an input of 1. We use the terminology that the path "dies" if in q3 getting an input 1. The tree of states this NFA is in for the input 0100011 input q0 / \ q3 q0 dies / \ q1 q0 dies / \ q3 q0 / / \ q4 q3 q0 / / / \ q4 q4 q3 q0 / / dies / \ q4 q4 q1 q0 / / / / \ q4 q4 q2 q1 q0 ^ ^ ^ | | | accepting paths in NFA tree 0 1 0 0 0 1 1
Construct a DFA equivalent to the NFA above using just the first three rows of delta (for brevity, consider q3 and q4 do not exists).
7 of 66
05/21/2011 09:31 AM
The DFA machine is M' = (Q', sigma, delta', q0', F') The set of states is Q' = 2**Q, the power set of Q = { phi, {q0}, {q1}, {q2}, {q0,q1}, {q0,q2}, {q1,q2}, {q0,q1,q2} } Note: read the eight elements of the set Q' as names of states of M' use [ ] in place of { } if you prefer. sigma is the same sigma = { 0, 1} The state transition table delta' is given below The starting state is set containing only q0 q0' = {q0} The set of final states is a set of sets F' = { {q2}, {q0,q2}, {q1,q2}, {q0,q1,q2} } Note: Include any group that contains a final state of the DFA.
sigma delta' | 0 | 1 | ----------+-------------+-------------+ phi | phi | phi | {q0} | {q0} | {q0,q1} | states {q1} | phi | {q2} | Q' {q2} | {q2} | {q2} | {q0,q1} | {q0} | {q0,q1,q2} | {q0,q2} | {q0,q2} | {q0,q1,q2} | {q1,q2} | {q2} | {q2} | {q0,q1,q2} | {q0,q2} | {q0,q1,q2} |
never reached never reached never reached
never reached
Note: Some of the states in the DFA may be unreachable yet must be specified. Later we will use Myhill minimization.
DFA (not minimized) equivalent to lower branch of NFA above. The delta' was constructed directly from delta. Using the notation f'({q0},0) = f(q0,0) to mean: in delta' in state {q0} with input 0 goes to the state shown in delta with input 0. Take the union of all such states. Further notation, phi is the empty set so phi union the set A is just the set A.
8 of 66
05/21/2011 09:31 AM

Some samples: f'({q0,q1},0) f'({q0,q1},1) f'({q0,q2},0) f'({q0,q2},1) = = = = f(q0,0) f(q0,1) f(q0,0) f(q0,1) union union union union
f(q1,0) f(q1,1) f(q2,0) f(q2,1) = = = = {q0} {q0,q1,q2} {q0,q2} {q0,q1,q2}
The sequence of states is unique for a DFA, so for the same input as above 0100011 the sequence of states is {q0} 0 {q0} 1 {q0,q1} 0 {q0} 0 {q0} 0 {q0} 1 {q0,q1} 1 {q0,q1,q2} This sequence does not have any states involving q3 or q4 because just a part of the above NFA was converted to a DFA. This DFA does not accept the string 00 whereas the NFA above does accept 00.
Given a NFA and are accepted by consuming to do available to do
one or more strings, determine if the string(s) the NFA. This may be error prone and time by hand. Fortunately, there is a program this for you. do the following:
On irix.gl.umbc.edu
ln -s /afs/umbc.edu/users/s/q/squire/pub/nfa nfa cp /afs/umbc.edu/users/s/q/squire/pub/download/fig2_7.nfa . nfa < fig2_7.nfa # or nfa < fig2_7.nfa > fig2_7.out On linux.gl.umbc.edu do the following:
ln -s /afs/umbc.edu/users/s/q/squire/pub/linux/nfa nfa cp /afs/umbc.edu/users/s/q/squire/pub/download/fig2_7.nfa . nfa < fig2_7.nfa # or nfa < fig2_7.nfa > fig2_7.out Full information is available at Simulators The source code for the family of simulators is available.
Lecture 5 NFA with epsilon moves

Definition and example of a NFA with epsilon transitions. Remember, epsilon is the zero length string, so it can be any where in the input string, front, back, between any symbols. There is a conversion algorithm from a NFA with epsilon transitions to a NFA without epsilon transitions. Consider the NFA-epsilon move machine M = { Q, sigma, delta, q0, F} Q = { q0, q1, q2 } sigma = { a, b, c } and epsilon moves q0 = q0 F = { q2 } sigma plus epsilon
9 of 66
05/21/2011 09:31 AM

delta | a | b | c |epsilon ------+------+------+------+------q0 | {q0} | phi | phi | {q1} ------+------+------+------+------q1 | phi | {q2} | phi | {q2} ------+------+------+------+------q2 | phi | phi | {q2} | phi ------+------+------+------+-------
The language accepted by the above NFA with epsilon moves is the set of strings over {a,b,c} including the null string and all strings with any number of a's followed by any number of b's followed by any number of c's. ("any number" includes zero) Now convert the NFA with epsilon moves to a NFA M = ( Q', sigma, delta', q0', F') First determine the states of the new machine, Q' = the epsilon closure of the states in the NFA with epsilon moves. There will be the same number of states but the names can be constructed by writing the state name as the set of states in the epsilon closure. The epsilon closure is the initial state and all states that can be reached by one or more epsilon moves. Thus q0 in the NFA-epsilon becomes {q0,q1,q2} because the machine can move from q0 to q1 by an epsilon move, then check q1 and find that it can move from q1 to q2 by an epsilon move. q1 in the NFA-epsilon becomes {q1,q2} because the machine can move from q1 to q2 by an epsilon move. q2 in the NFA-epsilon becomes {q2} just to keep the notation the same. q2 can go nowhere except q2, that is what phi means, on an epsilon move. We do not show the epsilon transition of a state to itself here, but, beware, we will take into account the state to itself epsilon transition when converting NFA's to regular expressions.
10 of 66
05/21/2011 09:31 AM
The initial state of our new machine is {q0,q1,q2} the epsilon closure of q0 The final state(s) of our new machine is the new state(s) that contain a state symbol that was a final state in the original machine. The new machine accepts the same language as the old machine, thus same sigma. So far we have for out new NFA Q' = { {q0,q1,q2}, {q1,q2}, {q2} } or renamed sigma = { a, b, c } F' = { {q0,q1,q2}, {q1,q2}, {q2} } or renamed q0 = {q0,q1,q2} or renamed inputs delta' | a | b | c ------------+--------------+--------------+-------------qx or {q0,q1,q2} | | | ------------+--------------+--------------+-------------qy or {q1,q2} | | | ------------+--------------+--------------+-------------qz or {q2} | | | ------------+--------------+--------------+-------------Now we fill in the transitions. Remember that a NFA has transition entries that are sets. Further, the names in the transition entry sets must be only the state names from Q'. Very carefully consider each old machine transitions in the first row. You can ignore any "phi" entries and ignore the "epsilon" column. In the old machine delta(q0,a)=q0 thus in the new machine delta'({q0,q1,q2},a)={q0,q1,q2} this is just because the new machine accepts the same language as the old machine and must at least have the the same transitions for the new state names. inputs delta' | a | b | c ------------+--------------+--------------+-------------qx or {q0,q1,q2} | {{q0,q1,q2}} | | ------------+--------------+--------------+-------------qy or {q1,q2} | | | ------------+--------------+--------------+-------------qz or {q2} | | | ------------+--------------+--------------+-------------No more entries go under input a in the first row because old delta(q1,a)=phi, delta(q2,a)=phi Now consider the input b in the first row, delta(q0,b)=phi, delta(q1,b)={q2} and delta(q2,b)=phi. The reason we considered q0, q1 and q2 in the old machine was because out new state
{ qx, qy, qz } { qx, qy, qz } qx
11 of 66
05/21/2011 09:31 AM
has symbols q0, q1 and q2 in the new state name from the epsilon closure. Since q1 is in {q0,q1,q2} and delta(q1,b)=q1 then delta'({q0,q1,q2},b)={q1,q2}. WHY {q1,q2} ?, because {q1,q2} is the new machines name for the old machines name q1. Just compare the zeroth column of delta to delta'. So we have
inputs delta' | a | b | c ------------+--------------+--------------+-------------qx or {q0,q1,q2} | {{q0,q1,q2}} | {{q1,q2}} | ------------+--------------+--------------+-------------qy or {q1,q2} | | | ------------+--------------+--------------+-------------qz or {q2} | | | ------------+--------------+--------------+-------------Now, because our new qx state has a symbol q2 in its name and delta(q2,c)=q2 is in the old machine, the new name for the old q2, which is qz or {q2} is put into the input c transition in row 1.
inputs delta' | a | b | c ------------+--------------+--------------+-------------qx or {q0,q1,q2} | {{q0,q1,q2}} | {{q1,q2}} | {{q2}} or qz ------------+--------------+--------------+-------------qy or {q1,q2} | | | ------------+--------------+--------------+-------------qz or {q2} | | | ------------+--------------+--------------+-------------Now, tediously, move on to row two, ... . You are considering all transitions in the old machine, delta, for all old machine state symbols in the name of the new machines states. Fine the old machine state that results from an input and translate the old machine state to the corresponding new machine state name and put the new machine state name in the set in delta'. Below are the "long new state names" and the renamed state names in delta'.
inputs delta' | a | b | c ------------+--------------+--------------+-------------qx or {q0,q1,q2} | {{q0,q1,q2}} | {{q1,q2}} | {{q2}} or {qz} ------------+--------------+--------------+-------------qy or {q1,q2} | phi | {{q1,q2}} | {{q2}} or {qz} ------------+--------------+--------------+-------------qz or {q2} | phi | phi | {{q2}} or {qz} ------------+--------------+--------------+--------------
12 of 66
05/21/2011 09:31 AM

inputs delta' | a | b | c ---+------+------+----/ qx | {qx} | {qy} | {qz} / ---+------+------+----Q' qy | phi | {qy} | {qz} \ ---+------+------+----\ qz | phi | phi | {qz} ---+------+------+-----
<-- input alphabet sigma
The figure above labeled NFA shows this state transition table. It seems rather trivial to add the column for epsilon transitions, but we will make good use of this in converting regular expressions to machines. regular-expression -> NFA-epsilon -> NFA -> DFA.
Lecture 6 Construction: machine from regular expression

Given a regular expression there is an associated regular language L(r). Since there is a finite automata for every regular language, there is a machine, M, for every regular expression such that L(M) = L(r). The constructive proof provides an algorithm for constructing a machine, M, from a regular expression r. The six constructions below correspond to the cases: 1) The entire regular expression is the null string, i.e. r = epsilon 2) The entire regular expression is empty, i.e. L=phi L={epsilon}
r = phi
3) An element of the input alphabet, sigma, is in the regular expression r = a where a is an element of sigma. 4) Two regular expressions are joined by the union operator, + r1 + r2 5) Two regular expressions are joined by concatenation (no symbol) r1 r2 6) A regular expression has the Kleene closure (star) applied to it r* The construction proceeds by using 1) or 2) if either applies. The construction first converts all symbols in the regular expression using construction 3). Then working from inside outward, left to right at the same scope, apply the one construction that applies from 4) 5) or 6).
13 of 66
05/21/2011 09:31 AM
Note: add one arrow head to figure 6) going into the top of the second circle. The result is a NFA with epsilon moves. to a NFA without epsilon moves. Further get a DFA. All these machines have the regular expression from which they were This NFA can then be converted conversion can be performed to same language as the constructed.
The construction covers all possible cases that can occur in any regular expression. Because of the generality there are many more states generated than are necessary. The unnecessary states are joined by epsilon transitions. Very careful compression may be performed. For example, the fragment regular expression aba would be a e b e a
14 of 66
05/21/2011 09:31 AM

q0 ---> q1 ---> q2 ---> q3 ---> q4 ---> q5 with e
used for epsilon, this can be trivially reduced to
a b a q0 ---> q1 ---> q2 ---> q3 A careful reduction of unnecessary states requires use of the Myhill-Nerode Theorem of section 3.4 in 1st Ed. or section 4.4 in 2nd Ed. This will provide a DFA that has the minimum number of states. Within a renaming of the states and reordering of the delta, state transition table, all minimum machines of a DFA are identical. Conversion of a NFA to a regular expression was started in this lecture and finished in the next lecture. The notes are in lecture 7. Example: r = (0+1)* (00+11) (0+1)* Solution: find the primary operator(s) that are concatenation or union. In this case, the two outermost are concatenation, giving, crudely: //---------------\ /----------------\\ /-----------------\ -->|| <> M((0+1)*) <> |->| <> M((00+11)) <> ||->| <> M((0+1)*) <<>> | \\---------------/ \----------------// \-----------------/ There is exactly one start "-->" and exactly one final state "<<>>" The unlabeled arrows should be labeled with epsilon. Now recursively decompose each internal regular expression.
Lecture 7 Convert NFA to regular expression

Conversion algorithm from a NFA to a regular expression. Start with the transition table for the NFA with the following state naming conventions: the first state is 1 or q1 or s1 which is the starting state. states are numbered consecutively, 1, 2, 3, ... n The transition table is a typical NFA where the table entries are sets of states and phi the empty set is allowed. The set F of final states must be known. We call the variable r a regular expression. We can talk about r being the regular expression with i,j subscripts ij Note r is just a (possibly) different regular expression from r 12 53 Because we need multiple columns in a table we are going to build, we also use a superscript in the naming of regular expression. 1 r 12 3 r 64 r 1k k k-1 r ij
are just names of different regular expressions 2
15 of 66
05/21/2011 09:31 AM

We are going to build a table with n
rows and n+1 columns labeled
| k=0 | k=1 | k=2 | ... | k=n ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | n r | r | r | r | ... | r 11 | 11 | 11 | 11 | | 11 ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | n r | r | r | r | ... | r 12 | 12 | 12 | 12 | | 12 ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | n r | r | r | r | ... | r 13 | 13 | 13 | 13 | | 13 ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | r | r | r | r | ... | 21 | 21 | 21 | 21 | | ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | r | r | r | r | ... | 22 | 22 | 22 | 22 | | ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | r | r | r | r | ... | 23 | 23 | 23 | 23 | | ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | r | r | r | r | ... | 31 | 31 | 31 | 31 | | ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | r | r | r | r | ... | 32 | 32 | 32 | 32 | | ----+--------+-------+-------+-----+-----| 0 | 1 | 2 | | r | r | r | r | ... | 33 | 33 | 33 | 33 | |
Only build column n for 1,final state The final regular expression is then the union, +, of the column n
^ |-
2 Note n rows,
all pairs of numbers from 1 to n
Now, build the table entries for the k=0 column:
/ 0 / +{ x | delta(q ,x) = q } r = / i j ij \ \ +{ x | delta(q ,x) = q } + epsilon \ i j
i /= j
i = j
16 of 66
05/21/2011 09:31 AM
where delta is the transition table function, x is some symbol from sigma the q's are states 0 r could be phi, epsilon, a, 0+1, or a+b+d+epsilon ij notice there are no Kleene Star or concatenation in this column
Next, build the k=1 column: 1 r = ij 0 r i1 ( r 0 * 0 ) r + 11 1j 0 r ij note: all items are from the previous column
Next, build the k=2 column: 2 r = ij 1 r i2 ( r 1 * 1 ) r + 22 2j 1 r ij note: all items are from the previous column
Then, build the rest of the k=k columns: k r = ij k-1 r ik ( r kk k-1 ) * r kj k-1 + k-1 r ij
note: all items are from previous column
Finally, for final states p, q, r n r + 1p n r + 1q r 1r n
the regular expression is
Note that this is from a constructive proof that every NFA has a language for which there is a corresponding regular expression.
Some minimization rules for regular expressions These can be applied at every step.
Note: phi is the empty set epsilon is the zero length string 0, 1, a, b, c, are symbols in sigma x is a variable or regular expression ( ... )( ... ) is concatenation ( ... ) + ( ... ) is union ( ... )* is the Kleene Closure = Kleene Star (phi)(x) = (x)(phi) = phi
17 of 66
05/21/2011 09:31 AM
(epsilon)(x) = (x)(epsilon) = x (phi) + (x) = (x) + (phi) = x x + x = x
(epsilon)* = (epsilon)(epsilon) = epsilon (x)* + (epsilon) = (x)* = x* (x + epsilon)* = x* x* (a+b) + (a+b) = x* (a+b) x* y + y = x* y (x + epsilon)x* = x* (x + epsilon) = x* (x+epsilon)(x+epsilon)* (x+epsilon) = x*
Now for an example: Given M=(Q, sigma, delta, q0, F) as delta | a | b | c --------+------+------+----q1 | {q2} | {q2} | {q1} --------+------+------+----q2 | phi | phi | phi --------+------+------+----Q sigma q0 F = = = = { q1, q2} { a, b, c } q1 { q2}
| k=0 | k=1 (using e for epsilon) -----+-------------+-----------------------------------r | c + epsilon | (c+e)(c+e)* (c+e) + (c+e) = c* 11 | | -----+-------------+-----------------------------------r | a + b | (c+e)(c+e)* (a+b) + (a+b) = c* (a+b) 12 | | -----+-------------+-----------------------------------r | phi | phi (c+e)* (c+e) + phi = phi 21 | | -----+-------------+-----------------------------------r | epsilon | phi (c+e)* (a+b) + e = e 22 | | -----+-------------+------------------------------------
| k=0 | k=1 | k=2 (using e for epsilon) -----+-------------+----------+------------------------r | c + epsilon | c* | 11 | | | -----+-------------+----------+------------------------r | a + b | c* (a+b) | c* (a+b)(e)* (e) + c* (a+b)
only final
18 of 66
05/21/2011 09:31 AM
state
12 | | | -----+-------------+----------+------------------------r | phi | phi | 21 | | | -----+-------------+----------+------------------------r | epsilon | e | 22 | | | -----+-------------+----------+------------------------the final regular expression minimizes to Additional topics include Mealy Machines Moore Machines c* (a+b) and
Lecture 8 Pumping Lemma for Regular Languages

Review of Basics of Proofs. The Pumping Lemma is generally used to prove a language is not regular. If a DFA, NFA or NFA-epsilon machine can be constructed to exactly accept a language, then the language is a Regular Language. If a regular expression can be constructed to exactly generate the strings in a language, then the language is regular. If a regular grammar can be constructed to exactly generate the strings in a language, then the language is regular. To prove a language is not regular requires a specific definition of the language and the use of the Pumping Lemma for Regular Languages. A note about proofs using the Pumping Lemma: Given: Formal statements A and B. A implies B. If you can prove B is false, then you have proved A is false. For the Pumping Lemma, the statement "A" is "L is a Regular Language", The statement "B" is a statement from the Predicate Calculus. (This is a plain text file that uses words for the upside down A that reads 'for all' and the backwards E that reads 'there exists') Formal statement of the Pumping Lemma: L is a Regular Language implies (there exists n)(for all z)[z in L and |z|>=n implies {(there exists u,v,w)(z = uvw and |uv|<=n and |v|>=1 and i (for all i>=0)(uv w is in L) )}] The two commonest ways to use the Pumping Lemma to prove a language is NOT regular are:
19 of 66
05/21/2011 09:31 AM
a) show that there is no possible n for the (there exists n), this is usually accomplished by showing a contradiction such as (n+1)(n+1) < n*n+n b) show there is no way to partition z into u, v and w such that i uv w is in L, typically for a value i=0 or i=2. Be sure to cover all cases by argument or enumerating cases. Note: The pumping lemma only applies to languages (sets of strings) with infinite cardinality. A DFA can be constructed for any finite set of strings. Use the regular expression to NFA 'union' construction. n n Notation: the string having n a's followed by n b's is a b which is reduced to one line by writing a^n b^n Languages that are not regular: L = { a^n b^n n<0 } L = { a^f1(n) b^f2(n) n0 } for any function f(n)< k*n+c for all constants k and c L = { a^(n*n) n>0 } also applies to n a prime(n log n), 2^n, n! L = { a^n b^k n>0 k>n } can not save count of a's to check b's k>n L = { a^n b^k+n n>0 k>1 } same language as above Languages that are regular: L = { a^n n>=0 } this is just r = a* L = { a^n b^k n>0 k>0 } no relation between n and k, r = a a* b b* L = { a^(37*n+511) n>0 } 511 states in series, 37 states in loop
Lecture 9 Intersection and other closures

A class of languages is simply a set of languages with some property that determines if a language is in the class (set). The class of languages called regular languages is the set of all languages that are regular. Remember, a language is regular if it is accepted by some DFA, NFA, NFA-epsilon, regular expression or regular grammar. A single language is a set of strings over a finite alphabet and is therefor countable. A regular language may have an infinite number of strings. The strings of a regular language can be enumerated, written down for length 0, length 1, length 2 and so forth. But, the class of regular languages is the size of a power set of any given regular language. This comes from the fact that regular languages are closed under union. From mathematics we know a power set of a countable set is not countable. A set that is not countable, like the real numbers, can not be enumerated. A class of languages is closed under some operation, op, when for any two languages in the class, say L1 and L2, L1 op L2 is in the class. This is the definition of "closed."
20 of 66
05/21/2011 09:31 AM
Summary. Regular languages are closed under operations: concatenation, union, intersection, complementation, difference, reversal, Kleene star, substitution, homomorphism and any finite combination of these operations. All of these operations are "effective" because there is an algorithm to construct the resulting language. Simple constructions include complementation by interchanging final states and non final states in a DFA. Concatenation, union and Kleene star are constructed using the corresponding regular expression to NFA technique. Intersection is constructed using DeMorgan's theorem and difference is constructed using L - M = L intersect complement M. Reversal is constructed from a DFA using final states as starting states and the starting state as the final state, reversing the direction of all transition arcs. We have seen that regular languages are closed under union because the "+" is regular expressions yields a union operator for regular languages. Similarly, for DFA machines, L(M1) union L(M2) is L(M1 union M2) by the construction in lecture 6. Thus we know the set of regular languages is closed under union. In symbolic terms, L1 and L2 regular languages and L3 = L1 union L2 implies L3 is a regular language. The complement of a language is defined in terms of set difference __ from sigma star. L1 = sigma * - L1. The language L1 bar, written as L1 with a bar over it, has all strings from sigma star except the strings in L1. Sigma star is all possible strings over the alphabet sigma. It turns out a DFA for a language can be made a DFA for the complement language by changing all final states to not final states and visa versa. (Warning! This is not true for NFA's) Thus regular languages are closed under complementation. Given union and complementation, by DeMorgan's Theorem, L1 and L2 regular languages and L3 = L1 intersect L2 implies ______________ __ __ L3 is a regular language. L3 = ( L1 union L2) . The construction of a DFA, M3, such that L(M3) = L(M1) intersect L(M2) is given by: Let M1 = (Q1, sigma, delta1, q1, F1) Let M2 = (Q2, sigma, delta2, q2, F2) Let S1 x S2 mean the cross product of sets S1 and S2, all ordered pairs, Then M3 = (Q1 x Q2, sigma, delta3, [q1,q2], F1 x F2) where [q1,q2] is an ordered pair from Q1 x Q2, delta3 is constructed from delta3([x1,x2],a) = [delta1(x1,a), delta2(x2,a)] for all a in sigma and all [x1,x2] in Q1 x Q2. We choose to say the same alphabet is used in both machines, but this works in general by picking the alphabet of M3 to be the intersection
21 of 66
05/21/2011 09:31 AM
of the alphabets of M1 and m2 and using all 'a' from this set.
Regular set properties: One way to show that an operation on two regular languages produces a regular language is to construct a machine that performs the operation. Remember, the machines have to be DFA, NFA or NFA-epsilon type machines because these machines have corresponding regular languages. Consider two machines M1 and M2 for languages L1(M1) and L2(M2). To show that L1 intersect L2 = L3 and L3 is a regular language we construct a machine M3 and show by induction that M3 only accepts strings that are in both L1 and L2. M1 = (Q1, Sigma, delta1, q1, F1) with the usual definitions M2 = (Q2, Sigma, delta2, q2, F2) with the usual definitions Now construct M3 = (Q3, Sigma, delta3, q3, F3) defined as Q3 = Q1 x Q2 set cross product q3 = [q1,q2] q3 is an element of Q3, the notation means an ordered pair Sigma = Sigma = Sigma we choose to use the same alphabet, else some fix up is required F3 = F1 x F2 set cross product delta3 is constructed from delta3([qi,qj],x) = [delta1(qi,x),delta2(qj,x)] as you might expect, this is most easily performed on a DFA The language L3(M3) is shown to be the intersection of L1(M1) and L2(M2) by induction on the length of the input string. For example: M1: Q1 = {q0, q1}, Sigma = {a, b}, delta1, q1=q0, F1 = {q1} M2: Q2 = {q3, q4, q5}, Sigma = {a, b}, delta2, q2=q3, F2 = {q4,q5} delta1 | a | b | ---+----+----+ q0 | q0 | q1 | ---+----+----+ q1 | q1 | q1 | ---+----+----+ delta2 | a | b | -----+----+----+ q3 | q3 | q4 | -----+----+----+ q4 | q5 | q3 | -----+----+----+ q5 | q5 | q5 | -----+----+----+
M3 now is constructed as Q3 = Q1 x Q2 = {[q0,q3], [q0,q4], [q0,q5], [q1,q3], [q1,q4], [q1,q5]} F3 = F1 x F2 = {[q1,q4], [q1,q5]} initial state q3 = [q0,q3] Sigma = technically Sigma_1 intersect Sigma_2 delta3 is constructed from delta3([qi,qj],x) = [delta1(qi,x),delta2(qj,x)] delta3 | a | b | ---------+---------+---------+ [q0,q3] | [q0,q3] | [q1,q4] |
this is a DFA when both
22 of 66
05/21/2011 09:31 AM

---------+---------+---------+ [q0,q4] | [q0,q5] | [q1,q3] | ---------+---------+---------+ [q0,q5] | [q0,q5] | [q1,q5] | ---------+---------+---------+ [q1,q3] | [q1,q3] | [q1,q4] | ---------+---------+---------+ [q1,q4] | [q1,q5] | [q1,q3] | ---------+---------+---------+ [q1,q5] | [q1,q5] | [q1,q5] | ---------+---------+---------+
M1 and M2 are DFA's
As we have seen before there may be unreachable states. In this example [q0,q4] and [q0,q5] are unreachable. It is possible for the intersection of L1 and L2 to be empty, thus a final state will never be reachable. Coming soon, the Myhill-Nerode theorem and minimization to eliminate useless states.
Lecture 10 Decision algorithms and review

Decision algorithms for regular sets: Remember: An algorithm must always terminate to be called an algorithm! Basically, an algorithm needs to have four properties 1) It must be written as a finite number of unambiguous steps 2) For every possible input, only a finite number of steps will be performed, and the algorithm will produce a result 3) The same, correct, result will be produced for the same input 4) Each step must have properties 1) 2) and 3) Remember: A regular language is just a set of strings over a finite alphabet. Every regular set can be represented by an regular expression and by a minimized finite automata, a DFA. We choose to use DFA's, represented by the usual M=(Q, Sigma, delta, q0, F) There are countably many DFA's yet every DFA we look at has a finite description. We write down the set of states Q, the alphabet Sigma, the transition table delta, the initial state q0 and the final states F. Thus we can analyze every DFA and even simulate them. Theorem: The regular set accepted by DFA's with n states: 1) the set is non empty if and only if the DFA accepts at least one string of length less than n. Just try less than |Sigma|^|Q| strings (finite) 2) the set is infinite if and only if the DFA accepts at least one string of length k, n <= k < 2n. By pumping lemma, just more to try (finite) Rather obviously the algorithm proceeds by trying the null string first. The null string is either accepted or rejected in a finite time. Then try all strings of length 1, i.e. each character in Sigma. Then try all strings of length 2, 3, ... , k. Every try results in an accept or reject in a finite time. Thus we say there is an algorithm to decide if any regular set
23 of 66
05/21/2011 09:31 AM

represented by a DFA is a) empty, b) finite and
c) infinite.
The practical application is painful! e.g. Given a regular expression, convert it to a NFA, convert NFA to DFA, use Myhill-Nerode to get a minimized DFA. Now we know the number of states, n, and the alphabet Sigma. Now run the tests given in the Theorem above. An example of a program that has not been proven to terminate is terminates.c with output terminates.out We will cover a program, called the halting problem, that no Turing machine can determine if it will terminate. Review for the quiz. (See homework WEB page.)
Lecture 11 Quiz 1
Closed book. Multiple choice. Covers lectures, reading assignments and homework. See details on Homework WEB page here
Lecture 12 Myhill-Nerode Minimization

Myhill-Nerode theorem and minimization to eliminate useless states. The Myhill-Nerode Theorem says the following three statements are equivalent: 1) The set L, a subset of Sigma star, is accepted by a DFA. (We know this means L is a regular language.) 2) L is the union of some of the equivalence classes of a right invariant(with respect to concatenation) equivalence relation of finite index. 3) Let equivalence relation RL be defined by: xRLy if and only if for all z in Sigma star, xz is in L exactly when yz is in L. Then RL is of finite index. The notation RL means an equivalence relation R over the language L. The notation RM means an equivalence relation R over a machine M. We know for every regular language L there is a machine M that exactly accepts the strings in L. Think of an equivalence relation as being true or false for a specific pair of strings x and y. Thus xRy is true for some set of pairs x and y. We will use a relation R such that xRy <=> yRx x has a relation to y if and only if y has the same relation to x. This is known as symmetric. xRy and yRz implies xRz. This is known as transitive. xRx is true. This is known as reflexive.
24 of 66
05/21/2011 09:31 AM
Our RL is defined xRLy <=> for all z in Sigma star (xz in L <=> yz in L) Our RM is defined xRMy <=> xzRMyz for all z in Sigma star. In other words delta(q0,xz) = delta(delta(q0,x),z)= delta(delta(q0,y),z) = delta(q0,yz) for x, y and z strings in Sigma star. RM divides the set Sigma star into equivalence classes, one class for each state reachable in M from the starting state q0. To get RL from this we have to consider only the Final reachable states of M. From this theorem comes the provable statement that there is a smallest, fewest number of states, DFA for every regular language. The labeling of the states is not important, thus the machines are the same within an isomorphism. (iso=constant, morph=change) Now for the algorithm that takes a DFA, we know how to reduce a NFA or NFA-epsilon to a DFA, and produces a minimum state DFA. -3) Start with a machine M = (Q, Sigma, delta, q0, F) as usual -2) Remove from Q, F and delete all states that can not be reached from q0. Remember a DFA is a directed graph with states as nodes. Thus use a depth first search to mark all the reachable states. The unreachable states, if any, are then eliminated and the algorithm proceeds. -1) Build a two dimensional matrix labeling the right side q0, q1, ... running down and denote this as the "p" first subscript. Label the top as q0, q1, ... and denote this as the "q" second subscript 0) Put dashes in the major diagonal and the lower triangular part of the matrix (everything below the diagonal). we will always use the upper triangular part because xRMy = yRMx is symmetric. We will also use (p,q) to index into the matrix with the subscript of the state called "p" always less than the subscript of the state called "q". We can have one of three things in a matrix location where there is no dash. An X indicates a distinct state from our initialization in step 1). A link indicates a list of matrix locations (pi,qj), (pk,ql), ... that will get an x if this matrix location ever gets an x. At the end, we will label all empty matrix locations with a O. (Like tic-tac-toe) The "O" locations mean the p and q are equivalent and will be the same state in the minimum machine. (This is like {p,q} when we converted a NFA to a DFA. and is the transitive closure just like in NFA to DFA.) NOW FINALLY WE ARE READY for 1st Ed. Page 70, Figure 3.8 or 2nd Ed. Page 159. 1) For p in F and q in Q-F put an "X" in the matrix at (p,q) This is the initialization step. Do not write over dashes. These matrix locations will never change. An X or x at (p,q) in the matrix means states p and q are distinct in the minimum machine. If (p,q) has a dash, put the X in (q,p) 2) BIG LOOP TO END For every pair of distinct states (p,q) in F X F do 3) through 7)
25 of 66
05/21/2011 09:31 AM
and for every pair of distinct states (p,q) in (Q-F) x (Q-F) do 3) through 7) (Actually we will always have the index of p < index of q and p never equals q so we have fewer checks to make.) 3) 4) 5) If for any input symbol 'a', (r,s) has an X or x then put an x at (p,q) Check (s,r) if (r,s) has a dash. r=delta(p,a) and s=delta(q,a) Also, if a list exists for (p,q) then mark all (pi,qj) in the list with an x. Do it for (qj,pi) if (pi,qj) has a dash. You do not have to write another x if one is there already. 6) 7) If the (r,s) matrix location does not have an X or x, start a list or add to the list (r,s). Of course, do not do this if r = s, of if (r,s) is already on the list. Change (r,s) to (s,r) if the subscript of the state r is larger than the subscript of the state s END BIG LOOP Now for an example, non trivial, where there is a reduction. M = (Q, Sigma, delta, q0, F} and we have run a depth first search to eliminate states from Q, F and delta that can not be reached from q0.
26 of 66
05/21/2011 09:31 AM
Q = {q0, q1, q2, q3, q4, q5, q6, q7, q8} Sigma = {a, b} q0 = q0 F = {q2, q3, q5, q6} delta | a | b | ----+----+----+ q0 | q1 | q4 | ----+----+----+ q1 | q2 | q3 | ----+----+----+ q2 | q7 | q8 | ----+----+----+ q3 | q8 | q7 | ----+----+----+ q4 | q5 | q6 | ----+----+----+ q5 | q7 | q8 | ----+----+----+ q6 | q7 | q8 | ----+----+----+ q7 | q7 | q7 | ----+----+----+ q8 | q8 | q8 | ----+----+----+ note Q-F = {q0, q1, q4, q7, q8} We use an ordered F x F = {(q2, (q2, (q2, (q3, (q3, (q5, q3), q5), q6), q5), q6), q6)} q1), q4), q7), q8), q4), q7), q8), q7), q8), q8)}
We use an ordered (Q-F) x (Q-F) = {(q0, (q0, (q0, (q0, (q1, (q1, (q1, (q4, (q4, (q7,
Now, build the matrix labeling the "p" rows q0, q1, ... and labeling the "q" columns q0, q1, ... and put in dashes on the diagonal and below the diagonal
q0 q1 q2 q3 q4 q5 q6 q7 q8
q0 q1 q2 q3 q4 q5 q6 q7 q8 +---+---+---+---+---+---+---+---+---+ | - | | | | | | | | | +---+---+---+---+---+---+---+---+---+ | - | - | | | | | | | | +---+---+---+---+---+---+---+---+---+ | - | - | - | | | | | | | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | | | | | | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | | | | | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | - | | | | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | - | - | | | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | - | - | - | | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | - | - | - | - | +---+---+---+---+---+---+---+---+---+
Now fill in for step 1) (p,q) such that p in F and q in (Q-F) { (q2, q0), (q2, q1), (q2, q4), (q2, q7), (q2, q8),
27 of 66
05/21/2011 09:31 AM
(q3, q0), (q3, q1), (q3, q4), (q3, q7), (q3, q8), (q5, q0), (q5, q1), (q5, q4), (q5, q7), (q5, q8), (q6, q0), (q6, q1), (q6, q4), (q6, q7), (q6, q8)}
q0 q1 q2 q3 q4 q5 q6 q7 q8
q0 q1 q2 q3 q4 q5 q6 q7 q8 +---+---+---+---+---+---+---+---+---+ | - | | X | X | | X | X | | | +---+---+---+---+---+---+---+---+---+ | - | - | X | X | | X | X | | | +---+---+---+---+---+---+---+---+---+ | - | - | - | | X | | | X | X | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | X | | | X | X | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | X | X | | | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | - | | X | X | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | - | - | X | X | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | - | - | - | | +---+---+---+---+---+---+---+---+---+ | - | - | - | - | - | - | - | - | - | +---+---+---+---+---+---+---+---+---+
Now fill in more x's by checking all the cases in step 2) and apply steps 3) 4) 5) 6) and 7). Finish by filling in blank matrix locations with "O". For example (r,s) = (delta(p=q0,a), delta(q=q1,a)) so r=q1 and s= q2 Note that (q1,q2) has an X, thus (q0, q1) gets an "x" Another from F x F (r,s) = (delta(p=q4,b), delta(q=q5,b)) so r=q6 and s=q8 thus since (q6, q8) has an X then (p,q) = (q4,q5) gets an "x" It depends on the order of the choice of (p, q) in step 2) whether a (p, q) gets added to a list in a cell or gets an "x". Another (r,s) = (delta(p,a), delta(q,a)) where p=q0 and q=q8 then s = delta(q0,a) = q1 and r = delta(q8,a) = q8 but (q1, q8) is blank. Thus start a list in (q1, q8) and put (q0, q8) in this list. [ This is what 7) says: put (p, q) on the list for (delta(p,a), delta(q,a)) and for our case the variable "a" happens to be the symbol "a".] Eventually (q1, q8) will get an "x" and the list, including (q0, q8) will get an "x'. Performing the tedious task results in the matrix:
q0 q1 q2 q3 q4 q5 q6 q7 q8 +---+---+---+---+---+---+---+---+---+ q0 | - | x | X | X | x | X | X | x | x | +---+---+---+---+---+---+---+---+---+ q1 | - | - | X | X | O | X | X | x | x |
The "O" at (q1, q4) means {q1, q4}
28 of 66
05/21/2011 09:31 AM

+---+---+---+---+---+---+---+---+---+ q2 | - | - | - | O | X | O | O | X | X | +---+---+---+---+---+---+---+---+---+ q3 | - | - | - | - | X | O | O | X | X | +---+---+---+---+---+---+---+---+---+ q4 | - | - | - | - | - | X | X | x | x | +---+---+---+---+---+---+---+---+---+ q5 | - | - | - | - | - | - | O | X | X | +---+---+---+---+---+---+---+---+---+ q6 | - | - | - | - | - | - | - | X | X | +---+---+---+---+---+---+---+---+---+ q7 | - | - | - | - | - | - | - | - | O | +---+---+---+---+---+---+---+---+---+ q8 | - | - | - | - | - | - | - | - | - | +---+---+---+---+---+---+---+---+---+
is a state in the minimum machine
The "O" for (q2, q3), (q2, q5) and (q2, q6) means they are one state {q2, q3, q5, q6} in the minimum machine. many other "O" just confirm this.
The "O" in (q7, q8) means {q7, q8} is one state in the minimum machine
The resulting minimum machine is M' = (Q', Sigma, delta', q0', F') with Q' = { {q0}, {q1,q4}, {q2,q3,q5,q6}, {q7,q8} } four states F' = { {q2,q3,q5,q6} } only one final state q0' = q0 delta' | a | b | ----------+----------------+------------------+ {q0} | {q1,q4} | {q1,q4} | ----------+----------------+------------------+ {q1,q4} | {q2,q3,q5,q6} | {q2,q3,q5,q6} | ----------+----------------+------------------+ {q2,q3,q5,q6} | {q7,q8} | {q7,q8} | ----------+----------------+------------------+ {q7,q8} | {q7,q8} | {q7,q8} | ----------+----------------+------------------+
Note: Fill in the first column of states first. Check that every state occurs in some set and in only one set. Since this is a DFA the next columns must use exactly the state names found in the first column. e.g. q0 with input "a" goes to q1, but q1 is now {q1,q4} Use the same technique as was used to convert a NFA to a DFA, but in this case the result is always a DFA even though the states have the strange looking names that appear to be sets, but are just names of the states in the DFA.
29 of 66
05/21/2011 09:31 AM
It is possible for the entire matrix to be "X" or "x" at the end. In this case the DFA started with the minimum number of states. At the heart of the algorithm is the following: The sets Q-F and F are disjoint, thus the pairs of states (Q-F) X (F) are distinguishable, marked X. For the pairs of states (p,q) and (r,s) where r=delta(p,a) and s=delta(q,a) if p is distinguishable from q, then r is distinguishable from s, thus mark (r,s) with an x. If you do not wish to do minimizations by hand, on irix.gl.umbc.edu use ln -s /afs/umbc.edu/users/s/q/squire/pub/myhill
myhill
or get the C++ source code from /afs/umbc.edu/users/s/q/squire/pub/download/myhill.cpp instruction on use are here
Lecture 13 Context Free Grammars, CFG

Grammars that have the same languages as DFA's grammar is defined as G = (V, T, P, S) where is a set of variables. We usually use capital letters for variables. is a set of terminal symbols. This is the same as Sigma for a machine. is a list of productions (rules) of the form: variable -> concatenation of variables and terminals S is the starting variable. S is in V. A string z is accepted by a grammar G if some sequence of rules from P can be applied to z with a result that is exactly the variable S. We say that L(G) is the language generated (accepted) by the grammar G. A V T P
30 of 66
05/21/2011 09:31 AM
To start, we restrict the productions P to be of the form A -> w w is a concatenation of terminal symbols B -> wC w is a concatenation of terminal symbols A, B and C are variables in V and thus get a grammar that generates (accepts) a regular language. Suppose Q = { S Sigma = q0 = S F = { S we are given a machine M = (Q, Sigma, delta, q0, F) with } { 0, 1 } } delta | 0 | 1 | ---+---+---+ S | S | S | ---+---+---+
this looks strange because we would normally use q0 is place of S The regular expression for M is (0+1)*
We can write the corresponding grammar for this machine as G = (V, T, P, S) where V = { S } the set of states in the machine T = { 0, 1 } same as Sigma for the machine P = S -> epsilon | 0S | 1S S = S the q0 state from the machine
the construction of the rules for P is directly from M's delta If delta has an entry from state S with input symbol 0 go to state S, the rule is S -> 0S. If delta has an entry from state S with input symbol 1 go to state S, the rule is S -> 1S. There is a rule generated for every entry in delta. delta(qi,a) = qj yields a rule qi -> a qj An additional rule is generated for each final state, i.e. S -> epsilon (An optional encoding is to generate an extra rule for every transition to a final state: delta(qi,a) = any final state, qi -> a with this option, if the start state is a final state, the production S -> epsilon is still required. ) See g_reg.g file for worked example. The shorthand notation S -> epsilon | 0S | 1S is the same as writing the three rules. Read "|" as "or". Grammars can be more powerful (read accept a larger class of languages) than finite state machines (DFA's NFA's NFA-epsilon regular expressions). i i For example the language L = { 0 1 | i=0, 1, 2, ... } is not a regular language. Yet, this language has a simple grammar S -> epsilon | 0S1
31 of 66
05/21/2011 09:31 AM
Note that this grammar violates the restriction needed to make the grammars language a regular language, i.e. rules can only have terminal symbols and then one variable. This rule has a terminal after the variable. A G V T P S grammar for matching parenthesis might be = (V, T, P, S) = { S } = { ( , ) } = S -> epsilon | (S) | SS = S
We can check this be rewriting an input string ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ) ( ) ( ( ) ) ) ) ( ) ( ) ( S ) ) ) ( ) ( ) S ) ) ( ) S S ) ) ( ) S ) ) S S ) ) S ) ) S ) S
S S S S S S S S
-> -> -> -> -> -> -> ->
(S) where the inside S is epsilon (S) (S) where the inside S is epsilon SS (S) where the inside S is epsilon SS (S) (S)
Thus the string ((()()(()))) is accepted by G because the rewriting produced exactly S, the start variable. More examples of constructing grammars from language descriptions: Construct a CFG for non empty Palindromes over T = { 0, 1 } The strings in this language read the same forward and backward. G = ( V, T, P, S) T = { 0, 1 }, V = S, S = S, P is below: S -> 0 | 1 | 00 | 11 | 0S0 | 1S1 We started the construction with S -> 0 and S -> 1 the shortest strings in the language. S -> 0S0 is a palindrome with a zero added to either end S -> 1S1 is a palindrome with a one added to either end But, we needed S -> 00 and S -> 11 to get the even length palindromes started. "Non empty" means there can be no rule S -> epsilon. n n Construct the grammar for the language L = { a b n>0 } G = ( V, T, P, S ) T = { a, b } V = { S } S = S P is: S -> ab | aSb Because n>0 there can be no S -> epsilon The shortest string in the language is ab a's have to be on the front, b's have to be on the back. When either an "a" or a "b" is added the other must be added in order to keep the count the same. Thus S -> aSb. The toughest decision is when to stop adding rules. In this case start "generating" strings in the language S -> ab ab for n=1 S -> aSb aabb for n=2 S -> aaSbb aaabbb for n=3 etc.
32 of 66
05/21/2011 09:31 AM

Thus, no more rules needed.
"Generating" the strings in a language defined by a grammar is also called "derivation" of the strings in a language.
Lecture 14, CFG Derivation Trees

Given a grammar with the usual representation G = (V, T, P, S) with variables V, terminal symbols T, set of productions P and the start symbol from V called S. A derivation tree is constructed with 1) each tree vertex is a variable or terminal or epsilon 2) the root vertex is S 3) interior vertices are from V, leaf vertices are from T or epsilon 4) an interior vertex A has children, in order, left to right, X1, X2, ... , Xk when there is a production in P of the form A -> X1 X2 ... Xk 5) a leaf can be epsilon only when there is a production A -> epsilon and the leafs parent can have only this child. Watch out! A grammar may have an unbounded number of derivation trees. It just depends on which production is expanded at each vertex. For any valid derivation tree, reading the leafs from left to right gives one string in the language defined by the grammar. There may be many derivation trees for a single string in the language. If the grammar is a CFG then a leftmost derivation tree exists for every string in the corresponding CFL. There may be more than one leftmost derivation trees for some string. See example below and ((()())()) example in previous lecture. If the grammar is a CFG then a rightmost derivation tree exists for every string in the corresponding CFL. There may be more than one rightmost derivation tree for some string. The grammar is called "ambiguous" if the leftmost (rightmost) derivation tree is not unique for every string in the language defined by the grammar. The leftmost and rightmost derivations are usually distinct but might be the same. Given a grammar and a string in the language represented by the grammar, a leftmost derivation tree is constructed bottom up by finding a production in the grammar that has the leftmost character of the string (possibly more than one may have to be tried) and building the tree towards the root. Then work on the second character of the string. After much trial and error, you should get a derivation tree with a root S. We will get to the CYK algorithm that does the parsing in a few lectures. Examples: Construct a grammar for L = { x 0^n y 1^n z n>0 }
33 of 66
05/21/2011 09:31 AM
Recognize that 0^n y 1^n is a base language, say B B -> y | 0B1 (The base y, the recursion 0B1 ) Then, the language is completed S -> xBz using the prefix, base language and suffix. (Note that x, y and z could be any strings not involving n) G = ( V, T, P, S ) where V = { B, S } T = { x, y, z, 0, 1 } S = S P = S -> xBz B -> y | 0B1 * Now construct an arbitrary derivation for S => x00y11z G A derivation always starts with the start variable, S. The "=>", "*" and "G" stand for "derivation", "any number of steps", and "over the grammar G" respectively. The intermediate terms, called sentential form, may contain variable and terminal symbols. Any variable, say B, can be replaced by the right side of any production of the form B -> <right side> A leftmost derivation always replaces the leftmost variable in the sentential form. (In general there are many possible replacements, the process is nondeterministic.) One possible derivation using the grammar above is S => xBz => x0B1z => x00B11z => x00y11z The derivation must obviously stop when the sentential form has only terminal symbols. (No more substitutions possible.) The final string is in the language of the grammar. But, this is a very poor way to generate all strings in the grammar! A "derivation tree" sometimes called a "parse tree" uses the rules above: start with the starting symbol, expand the tree by creating branches using any right side of a starting symbol rule, etc. S / | \ / | \ / | \ / | \ / | \ x B z / | \ / | \ / | \ / | \ 0 B 1 / | \ / | \ 0 B 1
34 of 66
05/21/2011 09:31 AM

| y Derivation ends x 0 0 y 1 1 z
with all leaves
terminal symbols, a string in the language generated by the grammar. More examples of grammars are: G(L) for L = { x a^n y b^k z k > n > 0 } note that there must be more b's than a's B -> aybb | aBb | Bb G = ( V, T, P, S ) where V = { B, S } T = { a, b, x, y, z } S = S P = S -> xBz B -> aybb | aBb | Bb Incremental changes for "n > k > 0" Incremental changes for "n >= k >= 0"
thus
B -> aayb | aBb | aB B -> y | aBb | aB
Independent exponents do not cause a problem when nested equivalent to nesting parenthesis. G(L) for L = { aî b^j c^j dî e^k f^k i>=0, j>=0, k>=0 } | | | | | | | +---+ | +---+ +-----------+ G = ( V, T , P, S ) V = { I, J, K, S } T = { a, b, c, d, e, f } P = S -> IK I -> J | aId J -> epsilon | bJc K -> epsilon | eKf
S = S
G(L) for L = { aî b^j c^k | any unbounded relation such as i=j=k>0, 0<i<k<j } the G(L) can not be a context free grammar. Try it. This will be intuitively seen in the push down automata and provable with the pumping lemma for context free languages.
What is a leftmost derivation trees for some string? It is a process that looks at the string left to right and runs the productions backwards. Here is an example, time starts at top and moves down. Given G I E S = (V, T, P, S) V={S, E, I} T={a, b, c} S=S P= -> a | b | c -> I | E+E | E*E -> E (a subset of grammar from book) a + b * c I E S I E [E + E]
Given a string
derived but not used
35 of 66
05/21/2011 09:31 AM

E S I E * E] E S
derived but not used
[E
done! Have S and no more input.
Left derivation tree, just turn upside down, delet unused. S | E / | \ / | \ / | \ E * E / | \ | E + E I | | | I I c | | a b Check: Read leafs left to right, must be initial string, all in T. Interior nodes must be variables, all in V. Every vertical connection must be tracable to a production.
Lecture 15 CFG simplication algorithm

The goal here is to take an arbitrary Context Free Grammar G = (V, T, P, S) and perform transformations on the grammar that preserve the language generated by the grammar but reach a specific format for the productions. Overview: Step Step Step Step Step Step 1a) 1b) 2) 3) 4) 5) Eliminate useless variables that can not become terminals Eliminate useless variables that can not be reached Eliminate epsilon productions Eliminate unit productions Make productions Chomsky Normal Form Make productions Greibach Normal Form
The CYK parsing uses Chomsky Normal Form as input The CFG to NPDA uses Greibach Normal Form as input Details: one step at a time
1a) Eliminate useless variables that can not become terminals See 1st Ed. book p88, Lemma 4.1, figure 4.7 2nd Ed. section 7.1 Basically: Build the set NEWV from productions of the form V -> w where V is a variable and w is one or more terminals.
36 of 66
05/21/2011 09:31 AM

Insert V into the set Then iterate over the in w as a terminal if variables that can be
NEWV. productions, now accepting any variable it is in NEWV. Thus NEWV is all the reduced to all terminals.
Now, all productions containing a variable not in NEWV can be thrown away. Thus T is unchanged, S is unchanged, V=NEWV and P may become the same or smaller. The new grammar G=(V,T,P,S) represents the same language. 1b) Eliminate useless variables that can not be reached from S See 1st Ed. book p89, Lemma 4.2, 2nd Ed. book 7.1. Set V'=S, T'=phi, mark all production as unused. Iterate repeatedly through all productions until no change in V' or T'. For any production A -> w, with A in V' insert the terminals from w into the set T' and insert the variables form w into the set V' and mark the production as used. Now, delete all productions from P that are marked unused. V=V', T=T', S is unchanged. The new grammar G=(V,T,P,S) represents the same language.
2)
Eliminate epsilon productions. See 1st Ed. book p90, Theorem 4.3, 2nd Ed. book 7.1 This is complex. If the language of the grammar contains the null string, epsilon, then in principle remove epsilon from the grammar, eliminate epsilon productions, then put S -> epsilon back into the grammar later. The new grammar G=(V,T,P,S) represents the same language except the new language does not contain epsilon.
3)
Eliminate unit productions. See 1st Ed. book p91, Theorem 4.4, 2nd Ed. 7.1 Iterate through productions finding A -> B type "unit productions". Delete this production from P. Make a copy of all productions B -> gamma, replacing B with A. Be careful of A -> B, B -> C, C -> D type cases, there needs to be copies of B -> gamma, C -> gamma, D -> gamma for A. Delete duplicate productions. (sort and remove adjacent duplicate) The new grammar G=(V,T,P,S) represents the same language.
Briefly, some pseudo code for the above steps. Step 1a) The set V' = phi loop through the productions, P, to find: A -> w where w is all terminals union V' with A n := 0 while n /= |V'|
37 of 66
05/21/2011 09:31 AM
n := |V'| loop through productions to find: A -> alpha where alpha is only terminals and variables in V' union V' with A end while Eliminate := V - V' loop through productions delete any production containing a variable in Eliminate, V := V' Step 1b) The set V' = {S} The set T' = phi n := 0 while n /= |V'| + |T'| n := |V'| + |T'| loop through productions to find: A -> alpha where A in V' union V' with variables in alpha union T' with terminals in alpha end while loop through productions delete any production containing anything outside V' T' and epsilon V := V' T := T' Step 2) The set N = phi n := -1 while n /= |N| n = |N| loop through productions to find: A -> epsilon union N with A delete production A -> alpha where no terminals in alpha and all variables in alpha are in N union N with A delete production
end while if S in N set null string accepted loop through productions A -> alpha where at least one variable in alpha in N generate rules A -> alpha' where alpha' is all combinations of eliminating the variables in N Step 3) P' := all non unit productions ( not A -> B ) U := all unit productions loop through productions in U, |U| times, to find: A -> A ignore this A -> B loop through productions in P' copy/substitute B -> gamma to A -> gamma in P'
38 of 66
05/21/2011 09:31 AM
P := P' eliminate duplicate productions (e.g. sort and check i+i against i) See link to "Turing machines and parsers." The CYKP, CYK parser, has the above steps coded in C++ and with "verbose 3" in the grammar file, most of the simplification is printed. Of possible interest is a test case g_elim.g input data to cykp and output g_elim.out
Lecture 16 Chomsky Normal Form

Chomsky Normal Form is used by the CYK algorithm to determine if a string is accepted by a Context Free Grammar. Step 4) in the overall grammar "simplification" process is to convert the grammar to Chomsky Normal Form. Productions can be one of two formats A -> a or A -> BC The right side of the production is either exactly one terminal symbol or exactly two variables. The grammars must have the "simplification" steps 1), 2) and 3) out of the way, that is 1) No useless variables, 2) No nullable variables and 3) no unit productions.
Step 4) of "simplification" is the following algorithm: 'length' refers to the number of variables plus terminal symbols on the right side of a production. Loop through the productions For each production with length greater than 1 do Replace each terminal symbol with a new variable and add a production new variable -> terminal symbol. Loop through the productions For each production with length grater than 2 do Replace two rightmost variables with a new variable and add a production new variable -> two rightmost variables. (Repeat - either on a production or loop until no replacements.) Now the grammar, as represented by the productions, is in Chomsky Normal Form. proceed with CYK. An optimization is possible but not required, for any two productions with the same right side, delete the second production and replace all occurrences of the second productions left variable with the left variable of the first production in all productions.
Example grammar:
39 of 66
05/21/2011 09:31 AM
G = (V, T, P, S) S S A A A -> -> -> -> -> aAS a SbA SS ba
V={S,A} T={a,b} S=S
First loop through productions (Check n>1) S -> aAS becomes S -> a stays A -> SbA becomes A -> SS A -> ba stays becomes S B S A C A A D E -> -> -> -> -> -> -> -> -> BAS a a SCA b SS DE b a (B is the next unused variable name)
(C is the next unused variable name)
Second loop through productions (Check n>2) S -> BAS becomes B -> a stays S -> a stays A -> SCA becomes C A A D E -> -> -> -> -> b SS DE b a stays stays stays stays stays S F B S A G C A A D E -> -> -> -> -> -> -> -> -> -> -> BF AS a a SG CA b SS DE b a (F is the next unused variable)
Optimization is possible, B -> a, S -> a, E -> a can be replaced by the single production S -> a (just to keep 'S') and all occurrences of 'B' and 'E' get replaced by 'S'. Similarly D -> b can be deleted, keeping the C -> b production and substituting 'C' for 'D'. Giving the reduced Chomsky Normal Form: S F S A G C A A -> -> -> -> -> -> -> -> SF AS a CG CA b SS CS
For a computer generated reduction, a different naming convention
40 of 66
05/21/2011 09:31 AM
was chosen (to aid in debugging). First, a terminal symbol "a" was replaced the prefixing "T_", thus "a" becomes "T_a". Once a substitution is made, that substitution is remembered so that there will be at most |T| rules generated of the form T_a -> a When there are more that two variables on the right hand side of a production, the new production is named "C_" concatenated with the last two variables separated by an underscore. This provides an easy reduction if the same two variables are replaced more than once. The productions will be sorted and duplicates will sort together and can be detected and eliminated quickly. (In order to be completely safe, this algorithm requires that no underscores were used in the initial grammar.) An example uses g_elim.out shown in the last lecture. Extracted and cleaned up, here is the Chomsky portion.
after eliminate, sorted productions: A -> a B -> a S -> A B S -> a A S -> a S a S a S -> a S a a S -> a a S a S -> a a a Chomsky Chomsky A -> B -> S -> S -> S -> S -> S -> S -> T_a -> 1, replace terminal with variable part 1, sorted productions: a a A B T_a A T_a S T_a S T_a T_a S T_a T_a T_a T_a S T_a T_a T_a T_a a
Chomsky 2, new production for each pair over two Chomsky Part 2 generated productions C_ST_a -> S T_a C_T_aS -> T_a S C_ST_a -> S T_a C_T_aT_a -> T_a T_a C_ST_a -> S T_a C_ST_a -> S T_a C_T_aS -> T_a S C_T_aT_a -> T_a T_a after Chomsky, sorted productions: A -> a
41 of 66
05/21/2011 09:31 AM

B C_ST_a C_T_aS C_T_aT_a S S S S S T_a -> -> -> -> -> -> -> -> -> -> a S T_a T_a A T_a T_a T_a T_a a
T_a S T_a B A C_ST_a C_T_aS C_T_aT_a
after Chomsky, Variables: A B C_ST_a C_T_aS C_T_aT_a
T_a
Terminals unchanged, S unchanged. This simple structure for the productions makes possible an efficient parser.
Lecture 17 Greibach Normal Form

Greibach Normal Form of a CFG has all productions of the form A ->aV Where 'A' is a variable, 'a' is exactly one terminal and 'V' is a string of none or more variables. Every CFG can be rewritten in Greibach Normal Form. This is step 5 in the sequence of "simplification" steps for CFG's. Greibach Normal Form will be used to construct a PushDown Automata that recognizes the language generated by a Context Free Grammar. Starting with a grammar: G = ( V, T, P, S) 1a) Eliminate useless variables that can not become terminals 1b) Eliminate useless variables that can not be reached 2) Eliminate epsilon productions 3) Eliminate unit productions 4) Convert productions to Chomsky Normal Form 5) Convert productions to Greibach Normal Form using algorithm below: Re-label all variables such that the names are A1, A2, ... , Am . The notation A(j) refers to the variable named Aj . New variables may be created with names B1, B2, ... referred to as B(j) . Productions may be created and/or removed (a mechanical implementation may use coloring to track processed, added and deleted productions.) Step 1 (repeat until no productions are added, m-1 times at most) begin (m is the number of variables, can change on each repeat) for k := 1 to m do begin for j := 1 to k-1 do for each production of the form A(k) -> A(j) alpha do begin for all productions A(j) -> beta do
42 of 66
05/21/2011 09:31 AM
add production A(k) -> beta alpha remove production A(k) -> A(j) alpha end; end; for each production of form A(k) -> A(k) alpha do begin add production b(k) -> alpha and B(k) -> alpha B(k) remove production A(k) -> A(k) alpha end; for each production A(k) -> beta where beta does not begin A(K) do add production A(k) -> beta B(k) end; Step 2 (sort productions and delete any duplicates ) Remove A -> A beta by creating B -> alpha B Substitute so all reules become Greibach with starting terminal. For neatness, sort productions and delete any duplicates see book: 2nd Ed. page 271, section 7.1, Exercise 7.1.11 construction Example (in file format, input given to program output file greibach.pda) cykp,
// g1.g a test grammar // G = (V, T, P, S) V={S,A} T={a,b} S=S // start S terminal a b ; verbose 7 // causes every step of ever loop to be printed // runs a very long time ! S -> a A S ; S -> a ; A -> S b A ; A -> S S ; A -> b a ; enddef abaa
// greibach.pda , edited, from start S terminal a ; terminal b ; variable variable variable variable variable A A ; C_AS ; C_T_bA ; S ; T_a ; a C_AS C_T_bA ;
cykp < g1.g > g1_cykp.out
->
43 of 66
05/21/2011 09:31 AM

A A A A C_AS C_AS C_AS C_AS C_AS C_T_bA S S T_a enddef -> -> -> -> -> -> -> -> -> -> -> -> -> a C_AS S ; a C_T_bA ; a S ; b T_a ; a C_AS C_T_bA a C_AS S S ; a C_T_bA S ; a S S ; b T_a S ; b A ; a C_AS ; a ; a ;
S ;
Note that an optional optimization could delete the last rule, and replace T_a by S.
Lecture 18 Inherently ambiguous CFL's, project

See project description
A CFL that is inherently ambiguous is one for which no unambiguous CFG can exists. i i j j i j j i L= {a b c d | i,j>0} union {a b c d | i,j>0} is such a language i i j j The productions for a b c d could be 1) S -> I J 2) I -> a b 3) I -> a I b 4) J -> c d 5) J -> c J d i j j i The productions for a b c d could be (using K instead of J) 6) S -> a K d 7) S -> a S d 8) K -> b c 9) K -> b K c Now consider the case i = j. This is a string generated by both grammars and thus will have two rightmost derivations. (one involving I and J, and one involving K) Thus ambiguous and not modifiable to make it unambiguous. An ambiguous grammar is a grammar for a language where at least one string in the language has two parse trees. This is equivalent to saying some string has more than one leftmost derivation or more than one rightmost derivation.
44 of 66
05/21/2011 09:31 AM
Lecture 19 Quiz 2
Similar to Quiz 1 Closed book. Multiple choice questions based on lectures, reading assignments and homework. Details on Homework 7-10 page here
Lecture 20 Push Down Automata, PDA

Push Down Automata, PDA, are a way to represent the language class called Context Free Languages, CFL, covered above. By themselves PDA's are not very important but the hierarchy of Finite State Machines with corresponding Regular Languages, PDA's with corresponding CFL's and Turing Machines with corresponding Recursively Enumerable Sets (Languages), is an important concept. The definition of a Push Down Automata is: M = (Q, Sigma, Gamma, delta, q0, Z0, F) where Q = a finite set of states including q0 Sigma = a finite alphabet of input symbols (on the input tape) Gamma = a finite set of push down stack symbols including Z0 delta = a group of nondeterministic transitions mapping Q x (Sigma union {epsilon}) x Gamma to finite sets of Q x Gamma star q0 = the initial state, an element of Q, possibly the only state Z0 = the initial stack contents, an element of gamma, possibly the only stack symbol F = the set of final, accepting, states but may be empty for a PDA "accepting on an empty stack" Unlike finite automata, the delta is not presented in tabular form. The table would be too wide. Delta is a list of, nondeterministic, transitions of the form: delta(q,a,A) = { (qi,gammai), (qj,gammaj), ...} where q is the current state, a is the input tape symbol being read, an element of Sigma union {epsilon} A is the top of the stack being read, The ordered pairs (q sub i, gamma sub i) are respectively the next state and the string of symbols to be written onto the stack. The machine is nondeterministic, meaning that all the pairs are executed causing a branching tree of PDA configurations. Just like the branching tree for nondeterministic finite automata except additional copies of the pushdown stack are also created at each branch. The operation of the PDA is to begin in state q0, read the symbol on the input tape or read epsilon. If a symbol is read, the read head moves to the right and can never reverse to read that symbol again. The top of the stack is read by popping off the symbol. Now, having a state, an input symbol and a stack symbol a delta transition is performed. If there is no delta transition defined with these three values the machine halts. If there is a delta transition with the (q,a,A) then all pairs of (state,gamma) are performed. The gamma represents a
45 of 66
05/21/2011 09:31 AM
sequence of push down stack symbols and are pushed right to left onto the stack. If gamma is epsilon, no symbols are pushed onto the stack. Then the machine goes to the next state, q. When the machine halts a decision is made to accept or reject the input. If the last, rightmost, input symbol has not been read then reject. If the machine is in a final state accept. If the set of final states is empty, Phi, and the only symbol on the stack is Z0, then accept. (This is the "accept on empty stack" case)
Now, using pictures we show the machines for FSM, PDA and TM +-------------------------+----------------- DFA, NFA, NFA epsilon | input string | accepts Regular Languages +-------------------------+----------------^ read, move right | | +-----+ | | |--> accept +--+ FSM | M = ( Q, Sigma, delta, q0, F) | |--> reject +-----+ +-------------------------+----------------- Push Down Automata | input string |Z0 stack accepts Context Free Languages +-------------------------+----------------^ read, move right ^ read and write (push and pop) | | +-----------------------+ | +-----+ | | |--> accept +--+ FSM | M = ( Q, Sigma, Gamma, delta, q0, Z0, F) | |--> reject +-----+ +-------------------------+----------------- Turing Machine | input string |BBBBBBBB ... accepts Recursively Enumerable +-------------------------+----------------- Languages ^ read and write, move left and right | | +-----+ | | |--> accept +--+ FSM | M = ( Q, Sigma, Gamma, delta, q0, B, F) | |--> reject +-----+
An example of a language that requires the PDA to be a NPDA, Nondeterministic Push Down Automata, is L = { w wr | w in Sigma and wr is w written backwards }
Lecture 21 CFG/CFL to NPDA

46 of 66 05/21/2011 09:31 AM
Given a Context Free Grammar, CFG, in Greibach Normal Form, A -> aB1B2B3... Construct an NPDA machine that accepts the same language as that grammar. We are given G = ( V, T, P, S ) and must now construct M = ( Q, Sigma, Gamma, delta, q0, Z0, F) Q = {q} the one and only state! Sigma = T Gamma = V delta is shown below q0 = q Z0 = S F = Phi NPDA will "accept on empty stack" Now, for each production in P (individual production, no vertical bars) A -> a gamma constructs | | | | | +-------------+ | | | +-----------+ | gamma is a possibly empty, sequence of | | | symbols from Gamma +---+ | | | | | V V V delta(q, a, A) = {(q, gamma), more may come from other productions} If gamma is empty, use epsilon as in (q, epsilon) Finished! An example: Given production C -> 0 C T C -> 0 T T S -> 0 C S -> 0 T T -> 1 Resulting delta delta(q, 0, C) = delta(q, 0, C) = delta(q, 0, S) = delta(q, 0, S) = delta(q, 1, T) = # 1 2 3 4 5
(q, (q, (q, (q, (q,
CT) TT) C) T) #e)
NPDA running with input tape 0011 and initial stack S 0011 |S| ^ | | delta #3 and #4 both apply, resulting in two stacks and advancing input 0011 |C| |T| ^ | | | | delta #1 and #2 apply to the first stack, the second stack dies, advancing 0011 |C| |T| dies ^ |T| |T| | | | | delta #5 applies to the second stack, the first stack dies, advancing 0011 dies |T| ^ | | delta #5 applies, the stack becomes empty and advance to beyond input 0011 | | ^ | |
47 of 66
05/21/2011 09:31 AM

accept input string.
Another conversion algorithm that uses more states, three to be exact, is somewhat simpler. We are given G = ( V, T, P, S ) and must now construct M = ( Q, Sigma, Gamma, delta, q0, Z0, F) Q = {q0, q1, q2} Sigma = T Gamma = V union {z} where z not in V (an extra symbol) delta = Q x (Sigma union epsilon) x Gamma -> Q x Gamma q0 = q0 Z0 = z (to get the NPDA started) F = q2 the final accepting state
shown below
Two predefined transitions, the start and the accept are: delta(q0, epsilon, z) = { (q1, Sz) } delta(q1, epsilon, z) = { (q2, z) } For every rule in P , Greibach normal form, A -> aU generate delta(q1, a, A) = { (q1, U) } union all other sets from (q1, a, A) Note: The empty string must be removed to create Greibach normal form, thus a grammar that initially accepted the empty string needs the extra transition delta(q0, epsilon, z) = { (q2, z) } The conversion is proved to simulate a leftmost derivation.
Lecture 22 NPDA to CFG/CFL

Given a NPDA M = ( Q, Sigma, Gamma, delta, q0, Z0, Phi) and Sigma intersection Gamma = Phi, Construct a CFG G = ( V, T, P, S ) Set T = Sigma S = S V = { S } union { [q,A,p] | q and p in Q and A in Gamma } This can be a big set! q is every state with A every Gamma with p every state. The cardinality of V, |V|=|Q|x|Gamma|x|Q| Note that the symbology [q,A,p] is just a variable name. (the states in the NPDA are renamed q0, q1, ... if necessary) Construct the productions in two stages, the S -> , then the [q,A,p] -> S -> [q0,Z0,qi] for every qi in Q (including q0) |Q| of these productions
[qi,A,qm+1] -> a[qj,B1,q2][q2,B2,q3][q3,B3,q4]...[qm,Bm,qm+1] is created for each q2, q3, q4, ..., qm+1 in Q for each a in Sigma union {epsilon} for each A,B1,B2,B3,...,Bm in Gamma such that there is a delta of the form
48 of 66
05/21/2011 09:31 AM
delta(qi,a,A) = { ...,(qj,B1B2B3...Bm), ...} Note three degenerate cases: delta(qi,a,A)=phi makes no productions
delta(qi,a,A)={(qj,epsilon)} makes [qi,A,qj] -> a delta(qi,epsilon,A)={(qj,epsilon)} makes [qi,A,qj] -> epsilon
The general case: Pictorially, given delta(qi,a,A)= (qj,B1B2) generate the set | | | | | | for qk being every state, +-------------+ | | | | | while qm+1 is every state | +---------------+ | | | | | | | | | | | +--+ | | | | | | +-------+ | | | | | | +-------+ | | | | | | | V V V V V V [qi,A,qm+1] -> a[qj,B1,qk][qk,B2,qm+1] | | ^ ^ | | | | | +---+ | +--------------------------+
The book suggests to follow the chain of states starting with the right sides of the S -> productions, then the new right sides of the [q,a,p] -> productions. The correct grammar is built generating all productions. Then the "simplification" can be applied to eliminate useless variables, eliminate nullable variables, eliminate unit productions, convert to Chomsky Normal Form, convert to Greibach Normal Form. WOW! Now we have the Greibach Normal Form of a NPDA with any number of states and we can convert this to a NPDA with just one state by the construction in the previous lecture. The important concept is that the constructions CFG to NPDA and NPDA to CFG provably keep the same language being accepted. Well, to be technical, The language generated by the CFG is exactly the language accepted by the NPDA. Fixing up any technical details like renaming Gamma symbols if Gamma intersection Sigma not empty and accepting or rejecting the null string appropriately. The reverse of the example in the previous lecture. |Q|=1 makes it easy. Given: NPDA = (Q, Sigma, Gamma, delta, q0, Z0, F) Q={q} Sigma={0,1} Gamma={C,S,T} q0=q Z0=S F=Phi delta(q, 0, C) = (q, CT) delta(q, 0, C) = (q, TT) delta(q, 0, S) = (q, C) delta(q, 0, S) = (q, T)
49 of 66
05/21/2011 09:31 AM

delta(q, 1, T) = (q, epsilon)
Build: G = (V, T, P , S) V = { S, qCq, qSq, qTq } four variable names T = Sigma = {0, 1} (dropping the previous punctuation [,,]) S = Z0 = S S-> productions S -> qSq Note! qSq is a single variable just dropped the [,, ] symbols other productions: delta(q, 0, C) = (q, CT) | | | | || +-----+ | | +----+ || |+----------+ |+------+| || | || +-+ || | || | qCq -> 0 qCq qTq | | | | | +---+ | +--------------------+
was
C -> 0 C T
continue using same method on each delta: qCq -> 0 qTq qTq qSq -> 0 qCq qSq -> 0 qTq qTq -> 1 (epsilon becomes nothing) Now, if you prefer, rename the variables to single letters (assuming you have a big enough alphabet) For this simple example qCq becomes just C, qSq becomes just S and qTq becomes just T, thus the productions become: C -> 0 C T C -> 0 T T S -> 0 C S -> 0 T T -> 1 This grammar is Greibach normal form for L(G)={0^n 1^n | n<0} Now, working an example for another NPDA for this same language: M = ( Q, Sigma, Gamma, delta, q0, Z0, Phi) q0, q1 } = { 0, 1 } = { Z0, V0, V1 } = (q0,0,Z0) = (q0,V0) pop Z0 write V0 (for zero) (q0,0,V0) = (q0,V0V0) add another V0 to stack (q0,1,V0) = (q1,epsilon) pop a V0 for a one (q1,1,V0) = (q1,epsilon) pop a V0 for each 1 accept on empty stack Build: G = (V, T, P , S) V = { S, see below in productions } NPDA Q = { Sigma Gamma delta
50 of 66
05/21/2011 09:31 AM

T = Sigma = {0, 1} S = Z0 = S S-> productions S -> [q0,Z0,q0] S -> [q0,Z0,q1]
(one for each state)
P1 P2
delta productions (q0,0,Z0) = (q0,V0)
(one for each state) P3
[q0,Z0,q0] -> 0 [q0,V0,q0] | | +---------------+ [q0,Z0,q1] -> 0 [q0,V0,q1] | | +---------------+ (q0,0,V0) = (q0,V0V0)
P4
(two combinations of two states)
[q0,V0,q0] -> 0 [q0,V0,q0] [q0,V0,q0] | | | | | +----+ | +--------------------------+ [q0,V0,q0] -> 0 [q0,V0,q1] [q1,V0,q0] | | | | | +----+ | +--------------------------+ [q0,V0,q1] -> 0 [q0,V0,q0] [q0,V0,q1] | | | | | +----+ | +--------------------------+ [q0,V0,q1] -> 0 [q0,V0,q1] [q1,V0,q1] | | | | | +----+ | +--------------------------+ (q0,1,V0) = (q1,epsilon) [q0,V0,q1] -> 1 (q1,1,V0) = (q1,epsilon) [q1,V0,q1] -> 1 A a brief check, consider the string from the derivation P2, P4, P9 that produces the string 01 P2, P4, P8, P9, P10 that produces the string 0011
P5
P6
P7
P8
P9
P10
51 of 66
05/21/2011 09:31 AM
Lecture 23 Turing Machine Model

M = ( Q, Sigma, Gamma, delta, q0, B, F)
Q = finite set of states including q0 Sigma = finite set of input symbols not including B Gamma = finite set of tape symbols including Sigma and B delta = transitions mapping Q x Gamma to Q x Gamma x {L,R} q0 = initial state B = blank tape symbol, initially on all tape not used for input F = set of final states +-------------------------+----------------| input string |BBBBB ... accepts Recursively Enumerable Languages +-------------------------+----------------^ read and write, move left and right | | +-----+ | | |--> accept +--+ FSM | | |--> reject +-----+
+-------------------------+----------------| input and output string |BBBBB ... computes partial recursive functions +-------------------------+----------------^ read and write, move left and right | | +-----+ | | | +--+ FSM |--> done (a delta [q,a]->[empty], but may never happen ) | | +-----+ delta is a table or list of the form: [qi, ai] -> [qj, aj, L] or [qi, ai] -> [qj, aj, R] (optional [qi, ai] -> [qj, aj, N]) qi is the present state ai is the symbol under the read/write head qj aj L R N is is is is is the next state written to the tape at the present position move the read/write head left one position after the write move the read/write head right one position after the write optional no movement of the tape.
It is generally a pain to "program" a Turing machine. You have to convert the algorithm into the list of delta transitions. The fallback is to describe in English the steps that should be performed. The amount of detail needed depends on the reader of the algorithm accepting that there is an obvious way for the Turing machine to perform your steps.
52 of 66
05/21/2011 09:31 AM
There are a lot of possible Turing machines and a useful technique is to code Turing machines as binary integers. A trivial coding is to use the 8 bit ASCII for each character in the written description of a Turing machine concatenated into one long bit stream. Having encoded a specific Turing machine as a binary integer, we can talk about TMi as the Turing machine encoded as the number "i". It turns out that the set of all Turing machines is countable and enumerable. Now we can construct a Universal Turing Machine, UTM, that takes an encoded Turing machine on its input tape followed by normal Turing machine input data on that same input tape. The Universal Turing Machine first reads the description of the Turing machine on the input tape and uses this description to simulate the Turing machines actions on the following input data. Of course a UTM is a TM and can thus be encoded as a binary integer, so a UTM can read a UTM from the input tape, read a TM from the input tape, then read the input data from the input tape and proceed to simulate the UTM that is simulating the TM. Etc. Etc. Etc. In a future lecture we will make use of the fact that a UTM can be represented as an integer and can thus also be the input data on the input tape. For an example of programming a Turing Machine see Turing Machine simulator Basically, any algorithm can be coded in a high order language, or coded in assembly language, or coded as a Turing Machine program, or built out of digital logic. A Turing Machine program is a bit-by-bit description of an algorithm. Each program step, as in assembly language, is one simple operation. But at a much lower level than any assembly language. A Turing Machine program step is a 'delta' entry [qi, ai] -> [qj, aj, move] When in state qi, if the then transition to state moving one tape position L for left, R for right, read head sees tape symbol ai, qj, writing symbol aj to the tape and according to 'move' which can be N for no move.
For computer input to a TM simulator, the five items are just written with white space as a separator and an optional sixth field that is a comment. Special character pairs #b are used for one blank. ## is used for epsilon, nothing written to tape. A sample computer input for an algorithm to add unary strings is:
53 of 66
05/21/2011 09:31 AM
// add.tm add unary strings of zeros // input tape 000...00 000..00 blank separated, blank at end // output tape 000...000 sum of zeros on input tape start s0 halt s9 // use halt rather than 'final' when computing a result limit 20 s0 s0 s1 s1 s2 0 #b 0 #b 0 s0 s1 s1 s2 s9 ## 0 ## ## #b R R R L R skip over initial 0's write 0 over blank keep moving detect at end blank extra 0 delta delta delta delta delta 1 2 3 4 5
tape 000#b00#b
should end with five zeros on tape
The simulation starts with the TM in state s0 and read head on the first character. +----------|000 00 ---- tape +----------^ | | read/write head position TM in state s0 Thus, delta 1 applies. The new state is the same s0, nothing is written to the tape and the head is moved one place right.
+----------|000 00 ---- tape +----------^ | | read/write head position TM in state s0 Following the steps, delta 2 writes a zero over the blank and goes to state s1. When in state s1 and a zero is read from the tape, delta 3, stay in state s1 and move one space right. When in state s1 and a blank is read from the tape, delta 4, go to state s2 and back up one space (now over last zero). When in state s2 and a zero is read from the tape, delta 5, go to the final state, s9, and write a blank over the zero. Machine stops when no delta applies. If in a final state when machine stops, the algorithm (program) finished successfully.
54 of 66
05/21/2011 09:31 AM
+----------|00000 ---- tape +----------^ | | read/write head position TM in state s9 What would happen if there was a '1' on the tape with the above TM program? The machine would stop in a non-final state which indicates no answer was computed (like segfault).
Lecture 24 CYK algorithm for CFG's

Use the "simplification" steps to get to a Chomsky Normal Form.
Given a CFG grammar G in Chomsky Normal Form and a string x of length n Group the productions of G into two sets { A | A -> a } target is a terminal { A | A -> BC } target is exactly two variables
V is a two dimensional matrix. Each element of the matrix is a set. The set may be empty, denoted phi, or the set may contain one or more variables from the grammar G. V can be n by n yet only part is used. x[i] represents the i th character of the input string x Parse x using G's productions for i in 1 .. n V[i,1] = { A | A -> x[i] } for j in 2..n for i in 1 .. n-j+1 { V[i,j] = phi for k in 1 .. j-1 V[i,j] = V[i,j] union { A | A -> BC where B in V[i,k] and C in V[i+k,j-k]} } if S in V[1,n] then x is in CFL defined by G. In order to build a derivation tree, a parse tree, you need to extend the CYK algorithm to record (variable, production number, from a index, from B index, from C index) in V[i,j]. V[i,j] is now a set of five tuples. Then find one of the (S, production number, from a, from B, from C) entries in V[1,n] and build the derivation tree starting at the root.
55 of 66
05/21/2011 09:31 AM
Notes: The parse is ambiguous if there is more than one (S,...) in V[1,n] Multiple levels of the tree may be built while working back V[*,k] to V[*,k-1] and there may be more than one choice at any level if the parse is ambiguous.
Example: given a string x = baaba given grammar productions A -> a B -> b C -> a S S A B C -> -> -> -> -> AB BC BA CC AB i 1(b) 1 2 j 3 4 5 phi phi B S,A,C B B S,A 2(a) A,C B 3(a) A,C S,C 4(b) B S,A 5(a) A,C
V[i,j]
S,A,C ^ |_ accept
Derivation tree B b
S C A a A a B C B b C a
This can be a practical parsing algorithm. But, not for large input. If you consider a computer language, each token is treated as a terminal symbol. Typically punctuation and reserved words are unique terminal symbols while all numeric constants may be grouped as one terminal symbol and all user names may be grouped as another terminal symbol. The size problem is that for n tokens, the V matrix is 1/2 n^2 times the average number of CFG variables in each cell. The running time is O(n^3) with a small multiplicative constant. Thus, a 1000 token input might take 10 megabytes of RAM and execute in about one second. But this would typically be only a 250 line input, much smaller than many source files. For computer languages the LALR1 and recursive descent parsers are widely used. For working small problems, given a CFG find if it generates a specific string, use the available program cykp
56 of 66
05/21/2011 09:31 AM
Using the 'cykp' program on the sample grammar, trimming some, the result was lect24.out The input was lect24.g
Lecture 25 Pumping Lemma for Context Free Languages

The Pumping Lemma is generally used to prove a language is not context free. If a PDA machine can be constructed to exactly accept a language, then the language is a Context Free Language. If a Context Free Grammar can be constructed to exactly generate the strings in a language, then the language is Context Free. To prove a language is not context free requires a specific definition of the language and the use of the Pumping Lemma for Context Free Languages. A note about proofs using the Pumping Lemma: Given: Formal statements A and B. A implies B. If you can prove B is false, then you have proved A is false. A B | A implies B -----+-----------F F | T F T | T (you can prove anything to be true with a false premise) T F | F T T | T For the Pumping Lemma, the statement "A" is "L is a Context Free Language", The statement "B" is a statement from the Predicate Calculus. (This is a plain text file that uses words for the upside down A that reads 'for all' and the backwards E that reads 'there exists') Formal statement of the Pumping Lemma: L is a Context Free Language implies (there exists n)(for all z)[z in L and |z|>=n implies {(there exists u,v,w,x,y)(z = uvwxy and |vwx|<=n and |vx|>=1 and i i (for all i>=0)(uv wx y is in L) )}] The two commonest ways to use the Pumping Lemma to prove a language is NOT context free are: a) show that there is no possible n for the (there exists n), this is usually accomplished by showing a contradiction such as (n+1)(n+1) < n*n+n b) show there is no way to partition some z into u,v,w,x,y such that
57 of 66
05/21/2011 09:31 AM
i i uv wx y is in L, typically for a value i=0 or i=2. Be sure to cover all cases by argument or enumerating cases. [This gives a contradiction to the (for all z) clause.]
Some Context free languages and corresponding grammars: i i L={a b | i>=1} 2i 3i L={a b | i>=1}
S->ab | aSb
S-> aabbb | aaSbbb
[any pair of constants for 2,3]
i j i L={a b c | i,j>=1} i i j L={a b c | i,j>=1} i i j j L={a b a b | i,j>=1}
S->aBc | aSc
B->b | bB
S->DC
D->ab | aDb
C->c | cC
S->CC
C->ab | aCb
i i L={ua wb y | i>=1 and u,w,y any fixed strings}
S->uDy
D->awb | aDb
R R L={ww | w in Sigma star and w is w written backwards}
S->xx | xSx for all x in Sigma
Some languages that are NOT Context Free Languages L={ww | w in Sigma star} i i i L={a b c | i>=1} i j k L={a b c | k > j > i >=1} f(i) i L={a b | i>=1}
where f(i)=i**2 f(i) is the ith prime, f(i) not bounded by a constant times i meaning f(i) is not linear
i j i j L={a b c d | i, j>=1}
Review of basics of proofs:

You may be proving a lemma, a theorem, a corollary, etc A proof is based on:
58 of 66
05/21/2011 09:31 AM

definition(s) axioms postulates rules of inference
(typical normal logic and mathematics)
To be accepted as "true" or "valid" Recognized people in the field need to agree your definitions are reasonable axioms, postulates, ... are reasonable rules of inference are reasonable and correctly applied "True" and "Valid" are human intuitive judgments but can be based on solid reasoning as presented in a proof.
Types of proofs include: Direct proof (typical in Euclidean plane geometry proofs) Write down line by line provable statements, (e.g. definition, axiom, statement that follows from applying the axiom to the definition, statement that follows from applying a rule of inference from prior lines, etc.) Proof by contradiction: Given definitions, axioms, rules of inference Assume Statement_A use proof technique to derive a contradiction (e.g. prove not Statement_A or prove Statement_B = not Statement_B, like 1 = 2 or n > 2n) Proof by induction (on Natural numbers) Given a statement based on, say n, where n ranges over natural numbers Prove the statement for n=0 or n=1 a) Prove the statement for n+1 assuming the statement true for n b) Prove the statement for n+1 assuming the statement true for n in 1..n Prove two sets A and B are equal, prove part 1, A is a subset of B prove part 2, B is a subset of A Prove two machines M1 and M2 are equal, prove part 1 that machine M1 can simulate machine M2 prove part 2 that machine M2 can simulate machine M1 Limits on proofs: Godel incompleteness theorem: a) Any formal system with enough power to handle arithmetic will have true theorems that are unprovable in the formal system. (He proved it with Turing machines.) b) Adding axioms to the system in order to be able to prove all the "true" (valid) theorems will make the system "inconsistent." Inconsistent means a theorem can be proved that is not accepted as "true" (valid). c) Technically, any formal system with enough power to do arithmetic is either incomplete or inconsistent.
59 of 66
05/21/2011 09:31 AM
Lecture 25a CFL closure properties

Given two CFG's G1 = (V1, T1, P1, S1) for language L1(G1) G2 = (V2, T2, P2, S2) for language L2(G2)
Rename variables until V1 intersect V2 is Phi. We can easily get CFG's for the following languages: L1 union L2 = L3(G3) G3 = (V1 union V2 union {S3}, T1 union T2, P1+P2+P3, S3) S3 -> S1 | S2
L1 concatenated L2 = L4(G4) G4 = (V1 union V2 union {S4}, T1 union T2, P1+P2+P4, S4) S4 -> S1S2
L1 star = L5(G5) G5 = (V1 union V2 union {S5}, T1 union T2, P1+P2+P5, S5) S5 -> S5S1 | epsilon L2 substituted for terminal "a" in L1 = L6(G6) G6 = (V1 union V2, T1-{a} union T2, P6+P2, S1) P6 is P1 with every occurrence of "a" replaced with S2. Notice that L1 intersect L2 may not be a CFG. i i j i j i Example: L1={a b c | i,j>0} L2={a b c | i,j>0} are CFG's i i i but L1 intersect L2 = {a b c | i>0} which is not a CFG. The complement of L1 may not be a CFG. The difference of two CFG's may not be a CFG. The intersection of a Context Free Language with a Regular Language is a Context Free Language. As a supplement, the following shows how to take a CFG and possibly use 'yacc' or 'bison' to build a parser for the grammar. The steps are to create a file xxx.y that includes the grammar, and a file that is a main program that runs the parser. An example grammar is coded in acfg.y and a sample main program is coded in yymain.c One possible set of commands to run the sample is: bison acfg.y gcc yymain.c acfg.tab.c a.out
60 of 66
05/21/2011 09:31 AM

abaa
The output from this run shows the input being read and the rules being applied: read a read b read a A -> ba read a read end of line \n converted to 0 S -> a S -> aAS accepted A grammar that may be of more interest is a grammar for a calculator. Simple statement such as a=2 b=3 a+b that prints the answer 5 can be coded as a CFG. The following example uses bison format and has a simple lexical analysis built in for single letter variables. calc.y and a sample main program yymain.c One possible set of commands to run calc is: bison calc.y gcc -o calc yymain.c calc.tab.c calc a=2 b=3 a+b You should get the result 5 printed. See the grammar to find out what other operations are available
Lecture 26 The Halting Problem

The "Halting Problem" that no one will ever a Turing machine that halt (stop, exit) for is a very strong, provably correct, statement be able to write a computer program or design can determine if an arbitrary program will a given input.
This is NOT saying that some programs or some Turing machines can not be analyzed to determine that they, for example, always halt. The Halting Problem says that no computer program or Turing machine can determine if ALL computer programs or Turing machines will halt or not halt on ALL inputs. To prove the Halting Problem is unsolvable we will construct one program and one input for which there is no computer program or Turing machine that can correctly determine if it halts or does not halt.
61 of 66
05/21/2011 09:31 AM
We will use very powerful mathematical concepts and do the proofs for both a computer program and a Turing machine. The mathematical concepts we need are: Proof by contradiction. Assume a statement is true, show that the assumption leads to a contradiction. Thus the statement is proved false. Self referral. Have a computer program or a Turing machine operate on itself, well, a copy of itself, as input data. Specifically we will use diagonalization, taking the enumeration of Turing machines and using TMi as input to TMi. Logical negation. Take a black box that accepts input and outputs true or false, put that black box in a bigger black box that switches the output so it is false or true respectively. The simplest demonstration of how to use these mathematical concepts to get an unsolvable problem is to write on the front and back of a piece of paper "The statement on the back of this paper is false." Starting on side 1, you could choose "True" and thus deduce side 2 is "False". But staring on side 2, which is exactly the same as side 1, you get that side 2 is "True" and side 1 is "False." Since side 1, and side 2, can be both "True" and "False" there is a contradiction. The problem of determining if sides 1 and 2 are "True" of "False" is unsolvable.
The Halting Problem for a programming language. We will use the "C" programming language, yet any language will work. Assumption: There exists a way to write a function named Halts such that: int Halts(char * P, char * I) { /* code that reads the source code for a "C" program, P, determines that P is a legal program, then determines if P eventually halts (or exits) when P reads the input string I, and finally sets a variable "halt" to 1 if P halts on input I, else sets "halt" to 0 */ return halt; } Construct a program called Diagonal.c as follows: int main() { char I[100000000]; /* make as big as you want or use malloc */ read_a_C_program_into( I ); if ( Halts(I,I) ) { while(1){} } /* loop forever, means does not halt */ else return 1; } Compile and link Diagonal.c into the executable program Diagonal. Now execute Diagonal < Diagonal.c
62 of 66
05/21/2011 09:31 AM
Consider two mutually exclusive cases: Case 1: Halts(I,I) returns a value 1. This means, by the definition of Halts, that Diagonal.c halts when given the input Diagonal.c. BUT! we are running Diagonal.c (having been compiled and linked) and so we see that Halts(I,I) returns a value 1 causes the "if" statement to be true and the "while(1){}" statement to be executed, which never halts, thus our executing Diagonal.c does NOT halt. This is a contradiction because this case says that Diagonal.c does halt when given input Diagonal.c. Well, try the other case. Case 2: Halts(I,I) returns a value 0. This means, by the definition of halts, that Diagonal.c does NOT halt when given the input Diagonal.c. BUT! we are running Diagonal.c (having been compiled and linked) and so we see that Halts(I,I) returns a value 0 causes the "else" to be executed and the main function halts (stops, exits). This is a contradiction because this case says that Diagonal.c does NOT halt when given input Diagonal.c. There are no other cases, Halts can only return 1 or 0. Thus what must be wrong is our assumption "there exists a way to write a function named Halts..." A example of a program that has not been proven to halt on all numbers is terminates.c some cases terminates.out Every Turing machine can be represented as a unique binary number. Any method of encoding could be used. We can assume ASCII encoding of sample s 0 space ... R as 01110011 00110000 00100000 ... 01010010 just a big binary integer.
The Halting Problem for Turing machines. Assumption: There exists a Turing machine, TMh, such that: When the input tape contains the encoding of a Turing machine, TMj followed by input data k, TMh accepts if TMj halts with input k and TMh rejects if TMj is not a Turing machine or TMj does not halt with input k. Note that TMh always halts and either accepts or rejects. Pictorially TMh is:
+---------------------------| encoded TMj B k BBBBB ... +---------------------------^ read and write, move left and right | | +-----+ | | |--> accept +--+ FSM | always halts | |--> reject +-----+
63 of 66
05/21/2011 09:31 AM
We now use the machine TMh to construct another Turing machine TMi. We take the Finite State Machine, FSM, from TMh and 1) make none of its states be final states 2) add a non final state ql that on all inputs goes to ql 3) add a final state qf that is the accepting state Pictorially TMi is:
+------------------------------------------| encoded TMj B k BBBBB ... +------------------------------------------^ read and write, move left and right | | +----------------------------------+ | | __ | | | / \ 0,1 | | | +-| ql |--+ | | | +-----+ | \___/ | | | | | |--> accept-+ ^ | | +--+-+ FSM | |_____| | | | TMh |--> reject-+ _ | | +-----+ | // \\ | | +-||qf ||------|--> accept | \\_// | +----------------------------------+
may not halt
We now have Turing machine TMi operate on a tape that has TMi as the input machine and TMi as the input data.
+------------------------------------------| encoded TMi B encoded TMi BBBBB ... +------------------------------------------^ read and write, move left and right | | +----------------------------------+ | | __ | | | / \ 0,1 | | | +-| ql |--+ | | | +-----+ | \___/ | | | | | |--> accept-+ ^ | | +--+-+ FSM | |_____| | | | |--> reject-+ _ | | +-----+ | // \\ | | +-||qf ||------|--> accept | \\_// | +----------------------------------+
may not halt
Consider two mutually exclusive cases: Case 1: The FSM accepts thus TMi enters the state ql. This means, by the definition of TMh that TMi halts with input TMi. BUT! we are running TMi on input TMi with input TMi
64 of 66
05/21/2011 09:31 AM
and so we see that the FSM accepting causes TMi to loop forever thus NOT halting. This is a contradiction because this case says that TMi does halt when given input TMi with input TMi. Well, try the other case. Case 2: The FSM rejects thus TMi enters the state qf. This means, by the definition of TMh that TMi does NOT halt with input TMi. BUT! we are running TMi on input TMi with input TMi and so we see that the FSM rejecting cause TMi to accept and halt. This is a contradiction because this case says that TMi does NOT halt when given input TMi with input TMi. There are no other cases, FSM either accepts or rejects. Thus what must be wrong is our assumption "there exists a Turing machine, TMh, such that..." QED. Thus we have proved that no Turing machine TMh can ever be created that can be given the encoding of any Turing machine, TMj, and any input, k, and always determine if TMj halts on input k.
Lecture 27 Church Turing Thesis

This is a mathematically unprovable belief that a reasonable intuitive definition of "computable" is equivalent to the list provably equivalent formal models of computation: Turing machines Lambda Calculus Post Formal Systems Partial Recursive Functions Unrestricted Grammars Recursively Enumerable Languages (decision problems)
and intuitively what is computable by a computer program written in any reasonable programming language.
Lecture 28 Review
Review previous lectures
65 of 66
05/21/2011 09:31 AM
Lecture 29 Final Exam

Closed book. Covers all lectures, all reading assignments, all homework and project. Multiple choice and short answer questions. 1) go over the Quiz 1 and Quiz 2 the final exam will include some of these questions 2) You may find it helpful to go over the Lecture Notes but not the details of constructions 3) Understand what classes of languages go with what machines (automata) and grammars and regular expressions 4) Understand the statement of the Pumping Lemma for Context Free Languages 5) Understand the Halting Problem and why it is not computable 6) Understand the Church Turing Hypothesis
Other links
Syllabus homework details HW1..HW6, Quiz 1 homework details HW7..HW10, Quiz 2 and Final Lecture notes, one per page Lecture notes, one big page Project description Turing machine and other simulators and parsers Downloadable source and executables CMSC 451 Course Page Formal Language Denitions Automata Denitions Computability Denitions Language Class Denitions
Go to top
Last updated 3/25/04
66 of 66
05/21/2011 09:31 AM

CMSC 451 Lecture Notes on DFAs, NFAs, Regular Expressions

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CMSC 451 Lecture Notes on DFAs, NFAs, Regular Expressions

Uploaded by

Copyright:

Available Formats

CMSC 451 Selected Lecture Notes

CS451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

Lecture 1 Fast Summary Part 1

Lecture 2 Fast Summary Part 2

Lecture 3 DFA and regular expressions

CMSC 451 Selected Lecture Notes

More information on DFA

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

Given a DFA and are accepted by consuming to do available to do

ln -s /afs/umbc.edu/users/s/q/squire/pub/linux/dfa dfa cp /afs/umbc.edu/users/s/q/squire/pub/download/ab_b.dfa .

CMSC 451 Selected Lecture Notes

Lecture 4 Nondeterministic Finite Automata, NFA

CMSC 451 Selected Lecture Notes

state transition diagram.

CMSC 451 Selected Lecture Notes

never reached never reached never reached

CMSC 451 Selected Lecture Notes

Given a NFA and are accepted by consuming to do available to do

Lecture 5 NFA with epsilon moves

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

{ qx, qy, qz } { qx, qy, qz } qx

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

<-- input alphabet sigma

Lecture 6 Construction: machine from regular expression

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

used for epsilon, this can be trivially reduced to

Lecture 7 Convert NFA to regular expression

are just names of different regular expressions 2

CMSC 451 Selected Lecture Notes

all pairs of numbers from 1 to n

Now, build the table entries for the k=0 column:

/ 0 / +{ x | delta(q ,x) = q } r = / i j ij \ \ +{ x | delta(q ,x) = q } + epsilon \ i j

CMSC 451 Selected Lecture Notes

note: all items are from previous column

Finally, for final states p, q, r n r + 1p n r + 1q r 1r n

the regular expression is

CMSC 451 Selected Lecture Notes

(epsilon)(x) = (x)(epsilon) = x (phi) + (x) = (x) + (phi) = x x + x = x

CMSC 451 Selected Lecture Notes

Lecture 8 Pumping Lemma for Regular Languages

CMSC 451 Selected Lecture Notes

Lecture 9 Intersection and other closures

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

this is a DFA when both

CMSC 451 Selected Lecture Notes

Lecture 10 Decision algorithms and review

CMSC 451 Selected Lecture Notes

Lecture 12 Myhill-Nerode Minimization

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

The "O" at (q1, q4) means {q1, q4}

CMSC 451 Selected Lecture Notes

CMSC 451 Selected Lecture Notes

Lecture 13 Context Free Grammars, CFG