You are on page 1of 18

Dr A. Alvi Ch.

4 Decidability

Ch4 Decidability
 It may seem “obvious” that every function can be computed/every problem can be
solved algorithmically, given sufficient time.
o However, this turns out to be false: there exist functions that cannot be
computed within any finite number of steps!!
o This chapter discusses algorithmic unsolvability.
 Why study undecidability/unsolvability?
o To know which problems cannot be solved exactly and hence try to develop
algorithms that don’t give the exact solution, but a good-enough approximate
solution.
o This is also of deep philosophical interest (to some of us? ). Gives us a
perspective on computation. Insaan knows of the limits of his/her artifacts.
 We begin by studying some problems solvable by algorithms, or in other words,
decidable languages.
o This order of studying helps us appreciate better the unsolvable problems.
o The decidable problems discussed here have practical applications.
 First, we study decidable problems concerning regular languages
o What are regular languages? Languages accepted by finite automata.
o Applications of finite automata:
http://www.cs.odu.edu/~toida/nerzic/390teched/regular/fa/fa-applications.html
Examples of systems that can be modeled by FA are: control circuits of computers,
computer network communication protocols, lexical analysers for compilers etc.
o We can decide whether
 a finite automaton will accept a string
 the language of a finite automaton is empty
 two finite automata are equivalent
 We choose to represent various computational problems by languages.
o Doing so is convenient because we have already set up terminology for dealing
with languages.
o For example, the acceptance problem for DFAs of testing whether a particular
DFA accepts a given string can be expressed as a language, ADFA. This

Page 1 of 18
Dr A. Alvi Ch.4 Decidability

language contains the encodings of all DFAs together with strings that the
DFAs accept. Let

o Similarly, other computational problems can be posed as testing membership in


a language
 Showing that a language is decidable = Showing that the
computational problem is decidable
 Theorem 4.1: Now we show that ADFA is decidable:
o This nice feature of Proof Idea, where, without being too formal, the book
outlines the main idea of a proof, before formally proving it.

o Proof:
 The input <B,w> is a representation of a DFA B together with a
string w.
 One reasonable representation of B is simply a list of its five
components, Q, ∑, , qo, and F.
 When M receives its input, M first determines whether it properly
represents a DFA B and a string w. If not, M rejects.
 Then M carries out the simulation directly. It keeps track of B's
current state and B's current position in the input w by writing this
information down on its tape.
 Initially, B's current state is qo and B's current input position is the
leftmost symbol of w.
 The states and position are updated according to the specified
transition function . When M finishes processing the last symbol
of w, M accepts the input if B is in an accepting state; M rejects the
input if B is in a nonaccepting state.

Page 2 of 18
Dr A. Alvi Ch.4 Decidability

 Theorem 4.2: Now do the same for NFA’s: ANFA is a decidable language

o Try it on your own using a TM N


o We can use a different technique just to demonstrate a new concept which will
be used repeatedly in future:
 Idea: Use the TM M in the previous theorem as a helper machine
to the TM N that decides ANFA
o Proof: M works on a DFA, so N first converts the NFA into a DFA (use
procedure of Theorem 1.39).
o Pass the DFA as input to M

 Try yourself now whether a regular expression generates a given string.

o
Theorem 4.3: AREX is a decidable language

 DFA = NFA = Regular expression for decidability purposes, as a TM can convert


from one encoding to another.

Page 3 of 18
Dr A. Alvi Ch.4 Decidability

 Emptiness Testing: Now we test whether a finite automaton accepts any strings at all,
rather than a particular string (as in previous examples). Let

 Proof: We use a TM T that uses a marking algorithm as in Ex. 3.23 (which students
were assigned to read; the one about graph)

Question: How is transition happening? What’s the input?


Answer: You only need to go through the states of the automaton using the
transition function, which caters to all legal input to the machine.

HW: Do Theorem 4.5 yourself (Equivalence of two DFAs)


HW: Do Theorem 4.7 yourself (Whether a string is generated by a CFG)

Page 4 of 18
Dr A. Alvi Ch.4 Decidability

 CFG = PDA as they can be inter-converted. So everything said about CFGs is also
true about PDAs.

Page 5 of 18
Dr A. Alvi Ch.4 Decidability

 Theorem 4.8: Emptiness testing for the language of a CFG (i.e. is the language
generated by a CFG empty?)
o What’s a CFG?
o Let

o Sort of a backtracking algorithm where we start from the terminal symbols


and work our way back to start variable by marking symbols.

 Theorem 4.9: Every context-free language is decidable. [by a TM, of course]


o Let A be a CFL.
o Not a good idea to try to simulate PDA for A on a TM. Why?
o Because the PDA might go on forever on some branches of computation. So not
a decider.
o Use the machine S in Theorem 4.7 that decides ACFG.

Page 6 of 18
Dr A. Alvi Ch.4 Decidability

 We have now arrived at the following classification of languages:

Page 7 of 18
Dr A. Alvi Ch.4 Decidability

The Halting Problem


 One of the philosophically most important theorems of computation theory
o You may believe that we can solve any problem using a sufficiently powerful
algorithm. NOT SO!
o There’s this problem (and many others) that cannot be solved by any
algorithm/computer.
o This is not a very difficult, made-up problem by Computer Scientists that
nobody in the real world will ever encounter in her daily life.
o It’s a very common problem that we’d like to solve:
 Given a program and data for the program, we want to ask if the
program will ever finish computation and halt?
 For example, in pseudocode, the program
while True: continue
does not halt; rather, it goes on forever in an infinite loop.
 On the other hand, the program
print "Hello World!"
halts very quickly.
 Described more precisely:
“Does there exist an algorithm which, given two bit strings, the
first representing a program and the second representing data,
determines whether or not the program run on the data will stop?”
 Another way to look at this is:
You are given a program along with a mathematical specification
of what the program is supposed to do (e.g. calculate the GPA of
students). You need to verify that the program correctly
works/does what it is supposed to do. CANNOT BE DONE!
Software verification as a general problem is impossible by
computer
o Common, every-day problems are unsolvable
 Does a program sort an array of integers?
 Both program and specification are precise mathematical objects
 Proving program = = specification should be just like proving that
triangle1 = = triangle2 …

Page 8 of 18
Dr A. Alvi Ch.4 Decidability

 NO!
o Many such unsolvable problems
o Aim: For you to develop a feel for unsolvable problems and to be able to prove
their unsolvability.
 The difficulty in the halting problem lies in the requirement that the decision
procedure must work for all programs and inputs.
 There are programs (interpreters) that simulate the execution of whatever source code
they are given. Such programs can demonstrate that a program does halt if this is the
case: the interpreter itself will eventually halt its simulation, which shows that the
original program halted. However, an interpreter will not halt if its input program
does not halt, so this approach cannot solve the halting problem as stated. It does
not successfully answer "doesn't halt" for programs that do not halt.
 While Turing's proof shows that there can be no general method or algorithm to
determine whether algorithms halt, individual instances of that problem may very
well be susceptible to attack.
o Given a specific algorithm, one can often show that it must halt for any input,
and in fact computer scientists often do just that as part of a correctness proof.
But each proof has to be developed specifically for the algorithm at hand;
there is no mechanical, general way to determine whether algorithms on a
Turing machine halt (think of the interpreter analogy).
o However, there are some heuristics that can be used in an automated fashion to
attempt to construct a proof, which succeed frequently on typical programs.
This field of research is known as automated termination analysis.

Page 9 of 18
Dr A. Alvi Ch.4 Decidability

 Problem: Whether a given TM M accepts a given input string. Call this ATM, just as
we used ADFA and ACFG.
o While ADFA and ACFG are decidable, ATM is not!
o Let

 Theorem 4.11: ATM is undecidable.


o Before showing this, we show that ATM is Turing-recognizable.
o The following TM U recognizes ATM

U is the universal Turing machine proposed by Turing that can simulate any other
Turing machine from the description of that machine.
 Diagonalization Method: Used to prove the Halting Problem.
o A technique discovered by mathematician Georg Cantor in 1873 to count
infinite sets.
o Sounds weird, but even infinite sets are classified as countable or uncountable,
depending on if we can find a way to enumerate/count the elements of an
infinite set.
o If we can pair the elements of one set with those of another, then we can
compare their sizes.
o Cantor thought of a nice way of doing this mapping/pairing using diagonals of a
matrix. Can be done for infinite sets.
o Good link:
http://platonicrealms.com/minitexts/Infinity-You-Cant-Get-There-From-Here/
o Hand-counting example.

Page 10 of 18
Dr A. Alvi Ch.4 Decidability

o Note that no element of set A can remain unmapped because a function maps
every element in the domain (set A) to an element in the range (set B).

o So the set of natural numbers is used to count the members of another set.

Page 11 of 18
Dr A. Alvi Ch.4 Decidability

o Now we prove an even more weird correspondence. That of rational numbers


and of N.

o If we let be the set of positive rational numbers, Q seems


to be much larger than N.
o Yet these two sets are the same size according to our definition.
o We give a correspondence with N to show that Q is countable.
o One easy way to do so is to list all the elements of Q. Then we pair the first
element on the list with the number 1 from N, the second element on the list
with the number 2 from N, and so on.
o We must ensure that every member of Q appears only once on the list.
o Form an infinite matrix containing all the rational numbers.
o Number 𝑖/𝑗 occurs in 𝑖 th row and 𝑗th column.

o We are trying to count/enumerate the set of rational numbers. If we can show a


correspondence with the set of natural numbers, we are done.

Page 12 of 18
Dr A. Alvi Ch.4 Decidability

o You might think now that we can set up such correspondences with N for any
infinite set.
o This isn’t so. There are uncountable sets that are too big.

o We prove by contradiction by assuming a correspondence f exists between N


and R, but then show that a certain real number x has no corresponding
natural number.

Page 13 of 18
Dr A. Alvi Ch.4 Decidability

o x can be any real number. We choose its integer part to be zero for no special
reason.

Page 14 of 18
Dr A. Alvi Ch.4 Decidability

o BUT – we assumed we had a complete list. This is a contradiction. Therefore,


our assumption that we could make a countable list of the real numbers is false!
The real numbers have a higher order of infinity than the natural numbers, i.e.,
they are cardinally greater.
o It is natural to ask, “Well, why not just add the new number to the list?” Indeed,
we could do so. However, this fails to address the fundamental point of the
argument: we assumed we had a complete list of real numbers, and then showed
that this assumption cannot be true. It is the existence of this contradiction
which forces the conclusion that the real numbers aren't countable. And of
course, even if we added our new x to the list, we could use the same process to
create infinitely more. There's just no way to create a completed “match-up”
between the sets.
o Concise proof here:
http://mathonline.wikidot.com/proof-that-the-set-of-real-numbers-is-
uncountable
o The preceding theorem has an important application to the theory of
computation. It shows that some languages are not decidable or even Turing-
recognizable, for the reason that there are uncountably many languages yet only
countably many Turing machines.
o Because each Turing machine can recognize a single language and there are
more languages than Turing machines, some languages are not recognized
by any Turing machine.
 To show that the set of all Turing machines is countable we first
observe that the set of all strings ∑* is countable, for any alphabet
∑. With only finitely many strings of each length, we may form a
list of ∑* by writing down all strings of length 0, length 1, length
2, and so on.
 The set of all Turing machines is countable because each Turing
machine M has an encoding into a string (M).
 If we simply omit those strings that are not legal encodings
of Turing machines, we can obtain a list of all Turing
machines.
 Corollary 4. 18
Some languages are not Turing-recognizable.
HW: Please do yourself, class!

Page 15 of 18
Dr A. Alvi Ch.4 Decidability

 Ready to prove now:

Page 16 of 18
Dr A. Alvi Ch.4 Decidability

Page 17 of 18
Dr A. Alvi Ch.4 Decidability

 A Turing-unrecognizable language
o We showed that ATM is not decidable but Turing-recognizable at least.
o Some languages not even that!
o Complement of a language is all the strings not in the language.
o A language is called co-Turing-recognizable if it’s the complement of a
Turing-recognizable language.
o We need the following theorem:
o

o HW: Do the proof yourself, class, of the above.

Page 18 of 18

You might also like