FOR!'{!

\L LANGUAGES: ORI':;Ins LA2JD DIFECTIOIJS

S.A. GREIBACH

Department of System Science

University of California, Los AnB;eles, CA 90024

1. IntIDduction and appropriate programming techniques were devised.

At the same time, similar methods of syntax speci-
My purpose is to survey the orlglns of the
fication and programming techniques were used in
theory of formal languages and automata through
the definition and implementation of Problem Orien-
1964 and to indicate some of the main directions
ted Programming Languages. In the early 60s, all
which the :-;ttldy of the sub4ect has takc::>n since tIlen.
of these various threads were brought together as
In the discussion of origins, I shall concentrate
it was recognized, and then formally proved, that
on those I know best: the work on mathematical
all these models defined the same class of languages,
linguistics and automatic translation of the 50s
namely, the family of context-free (CF) phrase
and early 60s, which led me into the field. I had
structure languages.
hoped to trace the developments since 1964 thIDugh
the papers presented at the IEEE Symposia on the
TI1e rest of this paper is divided into three
FOill1dations of Computer Science (initia11v, Switch-
main sections chronologically: prior to 1956; 1956
ing Circuit Theory and Logical Design; later,
to 1964; 1965 to the present. The year 1956 saw
Switching and Automata TI1eory) and the companion,
the publication of the tentative definition of
parvenu, AC11 Symposia on the Theory of Computing.
phrase structure grammar [Chomsky, 1956], though the
This good intention was impossible to sustain, par-
Chomsky hierarchy of languages and ITBchines did not
ticularly as I moved fIDm the exuberant 60s into
appear in print until 1959 [Chomsky ,1959 J. The
the grim 7 Os. LDng gone are the days in which ab-
year 1964 is a watershed: the Chomsky hierarchy of
stracts of one's summer research and the theses of
languages, grarmnars and machines was completed.
one's best students automatically appeared at the
After that, formal language theory as a discipline
next Conference! Still I shall give pride of place
diverged from mathe:rn::ltical or computational linguis-
to such cont1'ibutions where possible.
tics. Most (though not all) of the subsequent de-
velopments were inspired by ideas related to com-
There (:il'e at least five (not altogether dis-
puter rather than natural languages and to comput-
tinct) sources foY' the ideas developed in fonnal
ing and programning. Thus, from approximately 1965
language theory. They Eire:
on, one can regard formal languaF\e theory as a
elogic and recursive function theory branch (originally the main one) 8f theoretical
computer science.
•s\vitching cil'cuit trleory c]IKl

emodeling of biological systems.,
developm::ntal systems and
eITBthematical and computational linguistics
·computer prograrmning and the of ALGOL
and other Problem Qr)iented lrl.nguages
Recursive function theory is ;:3urveyed elsewhel'C in
these Proceedings so I shall mention relevant ideas
only in passing. The emphasis of this paper is on
formal languages, so I shall descrilR only the l'e-
lated parts of the development of finite automata
theory, skimping 01' skipping purely machine 01' sys-
tem oriented topics.
Phrase structure gpammars wen=? originally cle-
scribed by Chomsky [l959a] as a forrna.lization of
the Irrrrnediate Constituent (IC) analysis used by lin-
tWists ~"'1 describing thp morpholog:/- aDd syntax' of
natural languages. The work on :rn::lchine translation
at various institutions used various
theories as bases for automatic syntactic analysis
t Th · .
lS paper was supported ill part by the National
Science Foundation under Grants NSF MCS-78-04725
and NSF MCS-79-01439.
Past 1964, time and space allow me to touch
on only a few themes and mv references become a
very sparse subset of the available literature. I
shall, not unnaturally, dwell most on the direction~;
in which my oVJTl research went.
. Before Ctlomsky: 1936-1955
As evel'y computer science student should kr10v],
the Chomsky hierarchy in its standard formulation
consists of foul' classes of languages, each class
p1'operly containing the next: recursively enumer-
able, context-sensitive, context-free, regular.
These classes can be defined by placing an increas-
ingly strict set of restrictions on general rewrit-
ing systems or equivalently by considering the
languages accepted by nondeterministic machines
with an increasingly restricted type of data struc-
ture (or Horking tape): 'I\.lring maChL1J.e, linear
bounded automaton, pushdown store automaton, finite
state automaton. 1he formal proof of equivalence
of grammars and IIBchines was completed in 1964,
but some of the definitions are much older.

CH ]471-2/79/0000-0066$00.75 (~~) 1979 IEEE 66

 The types of machines and grarrnnars used to
define the class of recursively enumerable languages
were defined and studied in the 30s and 40s by
Turing [1936J and Post [1943,1947J, respectively.
Many of the notions I survey here are now presented
in quite different formulations from those in the
original papers. By contrast, Turing's definition
of a "Turing TlBchine" , its jusTi fi_c:at-ion and the
construction of the first univer'lsa1 Turing machine,
are little different from the notation we use now.
The -halting problem is not explicitly discussed; on
the other hand, the tabular description of machines
(frequently with added documentation or comments!)
is more convenient than some I have seen in recent
books. "No attempt has yet been JIBde to show that
the "computable" numbers include all nlIDlbers which
VJould naturally be regarded as computable. All (1r-
guments that can be given are bound to be, funrla-
mentally, appeals to intuition and, for this reason,
rather unsatisfactory ITk1thematically. The real
question at issue is, 'What are the possible pro-
cesses that can be carried out in computing a num-
ber?' "[Turing, 1936, p 249J.t
The generation of sets by rewrltlng systems,
called production systems, ~,vas formalized by Post
[1943J who observed that the sets so generated are
recursively enumerable sets and vice versa. The
particular type of production system used by Chom-
sky was defined and termed "semi-Thue" by Post
[1947J who gives the natural mapping from Turing
machines (Post uses quadruples rather than Turing's
quintuples) to semi-Thue systems. In 1936, Post-
gave, independently, an informal description of com-
putat ion quite similar· to Turing's and declared
that proving enough formulations equivalent to re-
cursiveness would change Church's thesis "not :~o
much to a definition or to an axiom but to a natu-
pal law" [Post, 1936,p 105 J.
Thus, the grammars (or generating systems)
and machines at the "top" of the Chomsky hierarchy
were known, and known to be equivalent, by 1947.
Another paper by Post from this periOd later pro-
vided the key tool used in proving problems in for-
ITBI language theory undecidable: the Correspondence
Problem [Post, 1946]. (It is nlITIored that this
paper was inspired by World War II experience in
cryptography! )
At the "tJOttom" of the hierarchy ~ the f 2..rH-
l
ily of regular languages first defined ?y Kleene
[lQSl and 1956J. Kleene's formulation lS based on
the paper of McCulloch and Pitts [1943J on the
logical analysis of nervous activity in which cer-
tain asslllTlptions regarding nerve nete:: were formu-
lated. In Kleene's paper, finite automata are
represented by nerve nets. Instead of words or
strings, the paper discusses tables of input pat-
terns tagged as initial or noninitial. A table
+ note the date of Wells' paper! -- is diagrammed
IAccoroing to Professor I. J. Good, the first "Tur-
ing Tl12lchine" was a data-processor, nicknamed the
Bombe, used by ULTAA during World War II to crack
the Enigma machine [Lewin, ULTRA Goes to ~~r: The
Secret 8tory~ Hutc11inson,1978,p 58J. SO Turing
machines are very practical!
67
represents an ... al with an at t~e 1 if
an it is initial and, if noninitial, represents
.. . ~ ... al with the location of an in time
unspecified. An event is represented by a set
of tables. The concatenation EF of events E
and F is the set of tables ~ where X lS a table
In r which is not initial and Y is in F (so in
Clu'rent no-tation is reLlIIv IT). The class of reg-
ular events is defined as the closure of the unit
events ane1 the empty set under) union, concatenation
and an operation we can call E·':r ~ the correspon-
dence between rep;ular events aJicl finite automata is
established.
The context-free phrase :=:;tructure grammar is
(~ne attempt to fnrmcilize the notion of Irrunedidtc
Consti tupnt erC) anal vsic~, an approach to the de-
;~cription of the morphology and syntax of natural
lan,P:uage advocated in the 40s ancI early 50s by
!\rrerican linguists such as Bloch, Harris and Wells.
Bloch explains, "In analyzin,P~ a given sentence, we
first 5501(Jte the irrunc'diate constl tuents of the
sentence as a whole, th('n the (~onstituents of each
constituent, and so nn to the ultimate constituents
-- at c,-very step choosing our constituents in such
a way that the total number of different construc-
tions will reJYBin as SITk'ill as possible" [Bloch,
1946J. And again, "When a word contains three or
more morphemes,Lt is usually necessary to analyze
j t into two and only two IMMEDIl\TE CONTITUilITS OJ
either or both of which may be susceptible of fur-
ther anaJysis" [Bloch .-md Trager, 1942J.
Various attempts were rrade at formalizing
this procedure. In "Frum 1"1orpheme to Utterance.,"
Harris first apologizes for introducing "methods
as mathematical as the one proposed here." His
method consists of f+t'st defining substitution
classes of morphemes ,or sequences of morphemes,
\,ihich can appear in the same context or envir'on-
ment in the lan~;udgE). are fonned; an
equation Be =A me,3J1S -rha t a morpheme of class B
followed by arnorpheme of class C can be substi-
tuted for d mornheme of class A. The utterance
is divided into mc~rphemes vlhich are placed into
morpheme c L::Jsses. Then -the equations are used to
make repeated sub~~tittJtion until (one hopes) the
~tlhole uttercJ1ce is grouped into a sentence type;
i.e . ., a morpheme class which corresponds to a
v,Jhole sentence rHarris ,1946J.
i\nc)ther' attempt vJa~3 made by \~ells to "re-
place by a unified systeTI1c-=ttic theory the hetero-
anc] incomplete methods hitherto offered
dt:Jtennining immediate constituents" [\JJells,
~~47~p 31]. A sentence is ,Jivided into 2ICs
(three or more are allowed by Wells only when no
jivision intu Lwo rcs HB.k:es sense), each Ie i-l'l.tc
ICs and .so nn down. As ,3Jl example, the sentence
"the king of England opened Parliament" -- please
++
Bloch and Trager [1942J define a morpheme as a
linguistic form which cannot be divided into small-
er mean in f2;fu1 parts; e.g., "opened" contains two
ITDrphemes, "open" and "-ed".
 -I-
by ':"Jells: I n
the king III of
open III ed Parliament
which in tree notation would be: " tem that is capable of generating formal languages
the king of England
In these formulations, IC analysis was meant
as a method for describing the structure of a stY'ing
known to be a sentence and not a method for genera-
ting sentences nor distinguishing which strings.are
sentences. Wells says, "The task of IC-analysls
is the task not of descY'ibing what utterances occur
but of describing, after these utterances have been
given, what their constituents ~" [Wells, 1947~
p 100J. Harris recognized that hls formulae could
be used in either direction since he warned, "there
are further limitations of selection arrDng the IYDr-
phemes, so that not all the sequences provided by
the formulae occur" [Harris, 1946;p 178J.
In his formulation, Bar-Hillel was already
concerned with "situations in which a completely
mechanical procedure is required for discovering
the syntactic structure of a giv~n string .. SUc~ a
situation arises, for instance, lD connectlon wlth
the problem of mechanized translat~on" [Bar-I?-llel,
1953;p 47J. Bar-Hillel proposed hls categor&al
grammars as a combination ~f.the.met~od~ dev~l~ped
by the Polish logician KaSlffilr AJduklewlcz [ Dle
svntaktische Konnexitat;' Studia philosophica~ 1
(1935) 1-27J with the methods of IC analysis.
Words and strings are assigned to categories which
can be basic or operator. When an operator cate-
gory forms out of left arguments of the categories
u l '··· ,urn and right arguments wl.' ... ,wn ' a
string belonging to the category v, it is said to
belong to the category
v
z
(u l )··· (um)!WIJ··· [wnJ
Then the basic rule of derivation is a cancellatiorl
rule:
cancels to (derives)
For example, if John belongs to category
n
to LnT ' and sleeps
to ---cnT
Poor John sleeps as
t The older among us may recognize traditional
methods of diagrarruning sentences, here and later,
but I do not have a nice scholarly reference for it!
s s
LnT n Tn) to nTn) to
1111 England ,
Thus, the categorial grammars of Bar-Hillel
are perhaps the first example of a mathematical sys-
and which was inspired by work on machine transla-
tion. This became a stronger theme in the late 50s
and early 60s.
3. The Formative Years: 1956-1964
The phrase structure model appears in Chom-
sky's"Three Models for the Description of Lmguage"
open ed Parliament [1956J. The three models defined in that paper do
not correspond at all to the classes of the Chomsky
hierarchy. They are theories for the explanation
of linguistic phenomena rather than precis~ ~the­
matical models. These three models are: FlDlte
State Markov Processes; Phrase Structure (as a for-
malization of IC Analysis); Transformational Gram-
mar. The finite state grammars defined here
actually generate only a proper subset of the reg-
ular languages. Under phrase structure, Chomsky
defines a [L: ,FJ grammar where F is a finite set
of rules of the form uXv ~ uyv, u ,X, v and y
over a finite alphabet Vp, X is a single symbol,
Y is nonempty and I is a finite set of initial
strings over Vp. A grammar gives rise to two
different languages. One, the derived language, is
the set of all strings derivable from the axioms of
L ; i.e., the sentential form language. (So one of
the ideas used in L-systems appears right here!)
The other, the terminal language, is the set of all
strings which are in the derived language and are
terminated; i.e., to which no further rules apply.
Chomsky notes that "in every interesting case
there will be a terminal vocabulary VT .•• every
terminal string is a string in VT and no symbol
of VT is rewritten in any of the rules of F.
In such a case, we can interpret the terminal
strings as constituting the language unde~ anc:lysis
(with VT as its vocabulary) and the derlvatlons
of the strings as py'Oviding their phrase structure"
[Chomsky ,1956;p 117J. Chomsky shows that there are
terminal languages which are not derivable and that
the families of derivable and of finite state lan-
guages are incomparable. The distinction betwe:n
what later became context-free and context-senSl-
tive grammars is not clear here; in fact, the
example of a language not terminal ~ {VIfW I w in
{a b}+}) is obviously context-sensltlve. In
tr~sformational grammar, transforrrations operate
on the derivations of a string produced by a phrase
structure gramnar. A transformat ion is built from
v
elementary transformations akin ~o pJ:rase structure
rules all of which must be applled m parallel;
this is similar to derivations in tabled context-
n , poor sensitive L-systems.
s
,one can ana 1yze The 1956 paper did stress the idea of ge~era­
tive grammar: not just diagramning utterances ill a
language but actually pruviding a mechanism for
generating all and only the sentences of the lan-
guage.
The Chomsky hierarchy, more or less in its
present form, appeared in "On Certain Formal Prop-
68
perties of Grammars" [Chomsky ,1959aJ; the defini-
tions appear in a 1958 Quarterly Progress Report of
the MIT Research laboratory of ElectrDnics [Chom-
sky,1958; the QPR's ar€ an important source of in-
formation on early WJrk on formal language theoryJ.
Here the set of symbols is formally divided into
VN (nonterminals) and VT (terminals); a rule must
have at least one nonterminal on the left hand side;
the terminal language is now the set of strings
over VT generated from a designated initial sym-
bol. The four types of gramrrars and languages are
defined here:
Type 0 (no restriction)
Type 1 uAv-+uyv , A in
VN ' Y nonempty
Type 2 A-+y , y nonempty
Type 3 A-+aB or A-+a, A
VN ' a in VT
lD
The identification of Type a languages with recur-
sively enumerable sets is noted as well as the fact
that Type 1 languages form a proper subclass of the
family of recursive sets. It is shown that
{anbmanbmcc I n,m ~ l} is Type 1 but not Type 2,
using not a pumping leJllTIB but an argument on the
copying power of Type 2 grammars; it is noted that
there are Type 2 languages which are not Type 3 .
Thus, there is a proper hierarchy of both gramrrar
and language types.
The connection between Type 3 languages and
finite state languages was noted. Indeed, Chomsky
and Miller [1958J basically define a finite state
grammar as a particular type of finite state auto-
maton and make no distinction between the concepts.
Their "finite state grarrrrn::lr" has a finite set of
states, a designated initial state So and, for
each state Si' a finite number of transitions
(arrows) to other states, each labeled either with
a symbol or with the empty word. A word is gener-
ated by this grarrrrnar if it takes the grarrrrrar from
So back to So without passing through So en
route. Among other things, it is shown that the
gramrrar can be made W1arnbiguous (and indeed alrrDst
deterministic) in the sense that empty word trans-
itions lead only into So and, for each Si and
symbol a, there is at most one arrow labeled as
leaving Si; the proof is essentially the same as
the construction of Rabin and Scott [1959]. It is
also shown that the finite state languages form a
Boolean algebra and a characterization akin to that
of regular expressions is provided.
The family of finite state or regular lan-
guages was also studied from the point of view
derived from considering switching circuit theory
and logical design and inspired by Kleene' s paper
[1951,1956J. Myhill proved that a one letter set
is regular if and only if it is ultimately peri-
odic and that a set is regular if and only if it
is the union of some of the equivalence classes of
a congruence relation of finite index [Myhill,1957J.
Copi, Elgot and Wright [1958J cleaned up Kleene's
formulation of nets, events and regular expressions.
McNaughton and YarrBda [19601 prDved the so-called
Kleene-Myhill theorem using state graphs for auto-
mata and the standard regular expression language,
69
including the empty word. Ott and Feinstein [1961J
converted regular expressions (using the now stan-
dard notation) into Improper State Diagrams (a type
of nondeterministic automata allowing transitions
on empty input) and Improper State Diagrams into
Proper State Diagrams (deterministic machines).
The most quoted paper on finite state lan-
guage is probably Rabin and Scott [1959J. Here,
we have perhaps the first appearance of the stan-
dard definition of nondeterministic finite state
acceptor and the classic conversion into determin-
istic autorrata, although, as noted above, the idea
and the result already appear in Chomsky and Miller
[1958J. Regular languages are characterized in
terms of congruence relations of finite index
(credited to Myhill [1957J) and in terms of right
invariant equivalence relations of finite index
(credited to Nerode [1958J). Some of the closure
properties are established: reversal, the Boolean
operations. Decision algorithms are provjded for
the emptiness, finiteness and equivalence problems.
Two-way and twJ-tape deterministic finite autorrata
are defined and a reduction of two-way to one-way
machines given (in the sarre issue of the journal,
Shepherson [1959J gives a simpler construction).
For languages accepted by deterministic two-tape
automata, it is shown that projections are regular,
emptiness and finiteness are decidable, and there
is closure under complementation but not intersec-
tion or union. A noteworthy development is the
first (?) use of the Post Correspondence Problem,
in this case to show the W1decidability of the
emptiness of intersection problem for deterministic
two-tape automata and then the undecidability of
emptiness for twJ-tape two-way machines.
The theory of context-free languages was be-
ing developed during the sarre period. I have not
been able to pinpoint the initial appearance of
the term "context-free" for the Type 2 case; per-
haps some reader can help. It is not used by
Chomsky in 1958 or 1959, nor by Bar-Hillel, Perles
and Shamir [1961] nor Ginsburg and Rice [1962].
It is however used by Scheinberg [1960], who notes
that context-free graJIll1B.rS are a special case of a
semi-Thue system and shows that the family of con-
text-free languages is closed under union but not
intersection or complementation; he uses pumping
type arguments to show that {anbnan I n ~ l} is not
context-free. The term "context-free" is also
used by Chomsky in "On the Notion 'Rule of Gram-
mar'" [1961].
Important parts of the theory of context-
free languages were presented in 1961 by B3.r-
Hillel, Perles and Shamir and by Parikh. BPS
call a Type 2 language a "simple phrase struc-
ture language" (SPL) . Among other things, that
paper established that the class of SPLs is effec-
tively closed W1der reversal, union, concatenation,
star closure and SUbstitution, and that erasing
and renaming rules can be effectively eliminated.
The famous uvwxy-theorem or pumping lemma (later
strengthened by Ogden [1968J) appears. The r€duc-
tion process for Type 2 gramnars appears and is
used to show the decidability of emptiness and
infiniteness. The cross-product construction
is introduc ed to Sf'101r} the closure of SPL tinder in-
tersection with FAL (Finite AutorncJ.ton Language =
fini te state language). The Post Correspondence
Problem is used 3S the key tool to establish the
undecidablility of a nu~er of problems including:
emptiness of intersection; the universe problem;
whether an SPL is an FAL; inclusion; equivalence,
even equivalence to an FAL.
In an NIT QPR [Parikh,196lJ, which did not
appear in journal form until 1966 but was still
widely influential, Parikh established two key l'e-
suIts: the existence of inherently ambiguous CF
languages (the language {anbmakbr I n = k or m = r,
n,m,k,r ~ I} cannot be generated by any unambiguous
CF gr~); the .=ommutative map of a CF language
lS seffilllnear.
\~ile the theory of context-free languages
was being developed, systems with the power of
Type 2 grammars were being used (knowingly or un-
knowingly) for the specification and automatic syn-
tactic analysis of both natural and ar~ificial
(i. e., programning) languages.
Nany of the systems for automatic syntactic
analysis used explicitly or implicitly the concept
of a pushdo~m store. The origin of the concept of
pushdown storage is not clear. t10st writers attri-
bute it to Burks, Warren and v~ight [1954J who for-
malized the parenthesis-free notation of Lukasie-
wicz [1951J and used it to describe a "simple and
easily mechanizable process of truth-table~ computa-
t~on"; the idea of a pushdown store appears impli-
Clt in the description of their logical machine.
It was used more explicitly by Newell and Shaw in
"Programning the Logic 'Theory Machine ll [1957J.
Their basic data structure is a list vll10se elempnts
can be lists. ~~ item in a list is given bv a lo-
cation word, half of which points to Cis the address
of) the element and the other half of which points
to the location word of the next item on the list.
To keep track of the available space, their system
uses an available-space list which is used as a
pushdown store. "vlhenever space is reouired for
building up a new list, this~ is obtained by using
words from the front of the available-space list
and, whenever information is erased and the words
that held it become available for use elsewhere,
these words are added to the front of the available-
space list" [Newell and Shaw, 1957 J.
The term "last-in-first-out" storage was used
explicitly by Sarrelson and B3.uer [1960J who pro-
posed the use of LIFO storage as an aide in the
translation of ALGOL (say) formulae into ffi3.chine in-
structions. They say "all incoming (source) infor-
mation given by symbols which cannot immediately be
evaluated is introduced, in the sequence of first
a.ppearance, into a special storage called 'symbols
cellar' , where at any given time onlv the element
introduced last (the highest level of the cellar)
is of immediate consequence and need be available.
Each new symbol of source information is, in turn,
compared to the momentarily highest cellar symbol.
These two symbols in conjunction determine the
evaluation Cif.possible) of the highest cellar sym-
bol by generatlon of an appropriate machine instruc-
70
tion a.1'1d the new state of the cell?.Y' by eliminating
the old highest symbol and/or introducing the new
symbol as the case may be ... translation of formula
p::ograrns into ~achine operations, with the excep-
tlon of recurslve address calculation, can be done
sequentially, without storing of the formula pro-
gram, as a pure input process."
The mechanical translation programs at Har-
vard and MIT used the idea of a pushdown store.
At MIT, Yngve in "A Model and a Hypothesis for
Language Structure" [1961J starts by assuming a
phrase structure (Type 2) grammar for English and
constructs a pushdown store transducer to generate
English sentences using the phr'ase structure gram-
mar. He adds a depth hypothesis, limiting the num-
ber of symbols to be stored in the pushdown store
which reduces it in power to a finite state trans~
ducer. On the other hand, in order to handle dis-
continuous constituents, he adds rules of the form
(e.g.) A-~B ... C which allow uDAv , say, to
become uBDCv instead of uDBCv, etc.
Rhodes at the National Bureau of Standards
[Rhodes, 1961, the published version of a 1959
reportJ originated for the automatic syntactic
analysis of Russian a rrethod later to be called
"predictive analysis" fThe idea is that each word
when analyzed may make predictions as to the occur-
r~nc~ of future words ir'.. the sentence. IThese pre-
dlctlons are treated in a roughly LIFO fashion
~this ~s not explicitly stated in the Rhodes paper);
J_teratlons are allowed if the analysis cannot be
completed or if there are alternative analyses.
At Harvard, the ideas of Burks, Warren and
\Nright and of Rhodes were developed into the Mul-
tiple-Path Predictive Analyzer for English and for
;<J.Jssian. An early report by Bossert [1960J dis-
cusses the implementation of predictive analysis
and explicitly deals with "prediction pools" which
are essentially pushdown stores. The thesis of
Sherry [1961J discusses the predictive syntactic
analyzer ~or Russian and hypothesizes for languages
an essentlal formula model. In this model, words
are functors or variables. A variable has meas-
~Jre -1 while a degree n fW1ctor has measure n;
measures are additive. A formula is essentiaZ if'
and only if it has measure 0 cmd all initial sub-
strings have nonnegative measure. Other condi tion~;
are added, to requin? variables to be in special
c~asses or in special orders, etc. rThere are algo-
rlthms to test whether a formula is essential. Two
interesting points are that the algoritruns are
given in Iverson Notation (the precursor of APL)
~d that.their total correctness is formally veri-
Iled, USlng so-called "6M - theorems" which say
that, if the algorithm reaches a well-formed sub-
formula (called 6M), then that subfonmula
will be correctly handled and afterwards will leave
no trace on the data st-ructure (a pushdown store);
such theorems d.re a curious prefiguration of the
Floyd-Manna verification procedure. (Incidentally,
Floyd [196lJ gave an inductive inference method of
proving properties of the sets of sentences gener-
able from the nonterminals in a CF grammar.)
Oettinger [1961J specifically used the term
"pushdown store" for predictive analysis and linked
it to "LIFO" storage. "The Rhodes method of 'pre-
dictive' syntactic analysis is based on the obser-
vation that in scanning through a Russian sentence
from left to right it is possible, on the one hand,
to make predictions about the syntactic str~ctures
to be rret further to the right and, on the other
hand, to determine the syntactic role of the wond
cUYTently being scanned by testing what previously
made predictions it fulfills. The predictions are
stored in a linear array called the 'prediction
pool' which behaves approximately as a pushdown
store .... Predictive analysis yields a description
of the syntactic structure of a sentence in terms
consonant ... with old-fashioned parsing, immediate
constituent theory ... or phrase structure theory ....
It remains to analyze the exact relation of the
predictive ID2thod to these theories" [Oettinger,
1961,p 105J.
The predictive syntactic analysis method was
essentially nondeterministic since different matches
of prediction and word could be made and result in
different prediction strings. Thus the methcxl was
implemented by following; different paths at the
sar.-e ti.rre, hence the term "multiple-path syntactic
analyzer" [Kuno, Oettinger,1962]. I just came
across a very old paper of mine, "Iterations for
the Syntactic Analyzer" [Greibach,196J], which dis-
cusses the problem and tries (not too successfully)
to recast it in terms of threading paths through a
maze, and shows that in a few very special cases
the number of iterations would be linear in the
length of the input. We were very much aware of
the problem of exponential blow-up in the number
of iterations (or paths), though we felt that this
did not happen in "real" natural languages; I do
not think we suspected that a polynomial parsing
algorithm was possible.
A somewhat different model was used at RAND,
based on the idea of tree diagrams of sentences
[Hays,196lJ. In these dependency systems~ the
nodes of a dependency tree correspond to terminal
symbols (word or word classes) of the sentence
whose structure it represents, in contrast to the
phrase structure tree, where terminal symbols are
associated only with the bottom node on each branch.
One occurrence is independent, the origin of the
tree. Every other occurrence except the origin
depends on one and only one other occurrence; every
occurrence except the origin depends, directly or
indirectly, on the origin. A dependency tree is
drawn in such a way that its nodes are arranged
horizontally from left to right in sentence word
order. It is also made to obey the restriction of
projectivity [Lecerf and Ihm,1960J so that no two
branches of the tree cross each other and vertical
projection lines extending from each node onto a
horizontal line drawn below the tree do not inter-
sect any of the branches of the tree. (A similar
idea using "predicts" instead of "depends on" could
be used for predictive analysis [Greibach,1962,
1963aJ. )
Immediate Constituent analysis was also used.
Sakai [1962J proposed an algorithm for parsing
sentences based on IC analysis with strict binary
division. 1m extensive IC grarrunar for English was
71
developed at RAND by Robinson [1962J for use with
a parsing algorithm attributed to Cocke [Hays,1962J.
This algorithm consisted of generating for a given
sentence, all two-word constitutes. All consti-
tutes of length n are generated using as constitu-
ents the constitutes of smaller length previously
obtained (including individual words, constitutes
of length 1), until all constitutes of the sen-
tence length aI'e obtained. The program had the
very important feature of processing a given phrase
of a given type at most once. Thus, Cocke's algo-
rithm essentially showed that context-free lan-
guages can be parsed in polynomial t -J.; i)~ Cindeed,
OCn 3 )) although that was not immedLD' :,'recognized.
Independently, computer prograrrmers devised
a syntax for the specification of the l~ference
language for ArhOL. As Backus explained at the
1959 ICIP Conference, "There must exist a precise
description of those sequences of symbols which
constitute legal IAL prograITls .... for every legal
program there must be a precise description of its
'meaning', the process or transformation which it
describes ... . Heretofore there has existed no formal
descr'iption of a machine-independent language (other
than that provided implicitly by a complete trans-
lating program) which has met either of the two
requirements above .... Only the description of legal
programs has been completed, however .... In the de-
scription of IAL syntax which follows, we shall
need some metalinguistic conventions for charac-
terizing various strings of symbols" [Backus, 1959 J .
The "metalinguistic convention" is not defined for-
mally but given by an example of formulae in what
is now called BNF (foY' Backus Naur Form), possible
values of the variable defined and the cormnent
".. . the formula above gives a recursive rule for
the formation of the values of the variable ... "
The ATbOL 60 report used BNT to specify the
syntax of the prDposed International Algorithmic
Language (IAL) [Naur,1060]. The original syntax
of ALGOL turned out to be ambiguous, an embarras-
sing situation, and some of those involved wondered
about the possibility of automatically detecting
such ambiguities. Cantor [1962] proved that this
is impossible -- the ambiguity problem for Backus
systems is undecidable. (His example of an ambigu-
ous ALGOL statement is: "if B /\ C , then for
I: = 1 step 1 until N do if D v E ,then
A[I]: = 0 else K: =K+l;K: =K-l" , which under one
interpretation will have the effect of leaving K
lITlchanged when 1 CB /\ C) holds and under the other
will change K to K-l when 1 (B /\ C) holds.)
Cantor used not the Post Correspondence Problem
but the existence of an undecidable Post normal
system.
IrDns [1961J discussed "A Syntax Directed
Compiler for ALGOL 60", which was to be "a compil-
ing system which essentially separates the func-
tions of defining the language and translating it
into another". His paper uses the syntax of ALGOL
60 and extends it to allow specification of meaning
(in terms of the target language) as well as of
form.
 Thus, the same importai'l.t ideas emerged for .
the automatic translation of both natural and artl-
ficial languages:
-separating syntax and semantics
-using a generative grammar to specify the set
of all and only legal sentences (programs)
-analyzing the syntax of the sentence (program)
and then using the analysis to drive the
tra'l~~ c+ion (compilation)
FurtherITDre., roughly similar methods were used to
specify th2 syntax and one key prograrrrrning tech-
nique -- the pushdO'wn or LIFO store -- was used for
at least part of the analysis or translation [Oet-
tinger,196l; Samelson and Bauer,1960J.
Among the systems for generatio~ or,analysis
proposed by linguists and computer SClentlsts were
the following:
-IC analysis
-categorial grammar
-Type 2 (CF) phrase structure grarnma
·BNF (the syntax of ALGOL 60)
-dependency systems
·predictive analysis
·pushdown store acceptor
These various systems were gradually shown
to have the same power for language specification.
This was not unexpected; in a paper not published
in full until 1964, Gross [1962J hypothesized that
all the well-known models of language used in the
fields of rrechanical translation and infor'TIBtion
retrieval would be the same in power, but gave no
full proofs. In his 1959 paper, Chomsky had al-
ready shown that Type 2 grarrtrIBrS are no more pow~r­
ful, in weak generative capacity, than IC analysls
(in the sense of strict binary division and no con-
text) by showing that all CF languages could be
generated by rules of the kind A -+ b (b terminal;
in IC analysis, this cO:rY€sponds to "word b ,be-
longs to class A") and A -+ Be (B, C nonterrrunals;
this corresponds to "a constitute of class B fol-
lowed by a constitute of class C can compose a
constitute of class A"); this is the Chomsky
Normal Form.
Bar-Hillel, Gaifman and Shamir [1960J proved
that "simple [i.e. Type 2J phrase-st:r:ucture gram-
IIBrS as defilled by Chomsky, are equlvalent to
vari;us types of categorial grammars, as discussed
by Ajdukiewicz and Par-Hillel, which are thereby
shown to be equiValent among themselves." In the
process, the authors show that Type 2 gramrIB.rS can
be converted to a restricted form of category sys-
tem: vocabulary V, simple categories ill a set T,
operator categories of the forms [A\B] or
[A\[A\BJJ for A,B,C in T, an assignment func-
tion f such that, for each a in V, f(a) is a
finite nonempty set of simple and operator catego-
72
ries, canceling rules A[A\BJ to B and
A[A\[B\CJJ to [B\CJ and an initial category. A
string al ... a n o~er V is gene~ated if and only
if there are Ai ill f(ai) ,Is l s n such that
Al .. ·An cancels to S. This. fonnulation o~ cate-
gory system uses a slightly dlfferent notatlon from
that originally proposed by Bar-Hillel [1953J.
Gaifman [1961; revised and published in 1965J used
the equivalence of categorial and Type 2 grarrrnars
to establish the equivalence of those two systems
with a certain formulation of dependency systems.
In his formulation, roughly speaking, a filllction
f assigns words to categories in such a way that
some word is assigned to each category and there
are a finite set of rules of the form X(Yl' ... 'Yr
. ': Zl' ... ,Zs) rreaning that if X is in f(a), then
a as a word of category X can be expanded into
(Y ,b ) ... (Y ,b ) (X ,a) (Zl ,c ) ... CZ s ,c s ) which can be
1 1 r r l
seen to correspond to context-free rules of the
form X -+ Y ... Y a Zl ... Z or be diagrammed
1 r s
X in f(a) , b. in F(Y.), c. in f(Z.) , where
l l ] ]
, i' Zj are categories and a, b,l ,c.] are words
X Y
(terminals). (Cf Greibach [1962,1963].)
Ginsburg and Rice [1962J connected the syntax
of AlliOL 60 with Type 2 phrase structure grammars
by considering as AlliOL-like the langua~e~ def~­
able as one of the coordinates of the rm.nJlIB.l f lxed
point of an n-tuple standard function; this is the
basis for the power series approach to CF languages.
This paper also define? the family of seq~ential
languages, considered l ts closure propertles an?
proved that it properly inc~udes the.class of fl-
nite state and is properly lncluded In the class of
Type 2 languages.
The equivalence of nondeterministic pushdown
store acceptors and context-free grammars was in-
dependently established by Chomsky and Schutzen-
berger [Chomsky, 1962; Chomsky and Schutzenberger ,
1963] and Evey [1963J. In a 1962 QFR, Chomsky de-
fined nondeterministic pushdown store automata
with e-rules; these machines accept if, at the
first return to the initial state, the pushdown
store is empty and the whole input has been pro-
cessed. The paper shows that, for a pushdown store
automa..ton M, there is a finite state transducer T
and a regular set R such that L(M) =T(K n R) ,
where K is a one-sided cancellation language, and
thus L(M) is context-free via the closure of the
family of CF languages under intersection.with
regular sets [Bar-Hillel, Perles,and Sh~r,1961J
and under finite state transductlons [Glnsburg and
Rose,1963; draft available earlierJ. The paper
also characterizes CF languages as h(K n R) for
h a horromorphism and claims that one can take as
K the two-sided cancellation language; however,
I have never been able to understand that proof,
nor the one in [Schutzenberger,1963J.
In his thesis, Evey [1963J used a different
approach. He defined a "hierarchy!! of six sets of
machines (with finite state control and up to two
pushdown stores, possibly restricted to counters
or to 1 or 0 stores) and partitioned them into ac-
ceptors and transducers and into deterministic and
nondeterministic. He showed that the
class of languages accepted by nondeterministic
acceptors of a given type is the same as the set of
output languages (ranges) of deterministic trans-
ducers of the same type. Thus, the class of lan-
guages accepted by nondeterministic pushdown store
a~ceptors (pdas) is the class of images of sets
L:~' under pushdown store transduction. He showed
that the latter is contained in the family of con-
text-free languages through consideration of spe-
cial types of bracketed languages and what he called
"pushdown store gramrrars". A pushdown store gram-
IIBr is a Type 0 grammar such that there is a finite
upper bound on the distance between the first non-
terminal and the first available nonterminal (i.e.,
available for application of a rule) in any derived
string. I t is easy to convert pdas to pushdown
store grammars; the difficult part is to show that
pushdown store graJflffi3.rs generate CF languages
(without of course appeallllg to the equivalence of
CF graJI1l'IBrS and pdas!). Note that the important
part of the proof of the equivalence between CF
graJI1l'IBrS and pdas is the conversion of pdas to
grammars ; it was apparent to us all that a CF gram-
mar could be parsed using a nondeterministic pda
with e-rules once the definition was at hand.
Evey also established that left-to-right or right-
to-left derivations suffice for CF grammars and
that discontinuous rules of the type advocated by
Yngve [1961J do not increase the power of a CF
gramrrar; the latter fact was also observed by Mat-
thews [1963J.
In my thesis [Greibach,1963aJ, I completed
this set of results by establishing that the method
of predictive analysis can handle all Type 2 lan-
guages. Rules in a predictive analysis grammar are
considered to be of the form (P,c) I Pl·· .Pk where
P and Pj are predictions and c is a syntactic
word class. The argument pair (P,c) indicates
that word class c may fulfill a prediction P
(thus initiating a syntactic structure correspond-
ing to P) if P is on top of the prediction pool
(pushdown store); then P is removed from the top
of the pool and replaced by PI . · . F\ , a string of
new predictions that have to be fulfilled in the
indicated order (PI first, I\ last). There were
various exceptions permitted in the actual Harvard
syntactic analysis program, but we need not con-
sider them here; rrost of them did not affect the
theoretical power of the system. In this formula-
tion, such a rule corresponds to a phrase structure
rule P ~ cPl· · · Pk ,where P,PI'. · . ,Pk are non-
terminal symbols and c is a terminal symbol.
Thus, to prove that predictive analysis could han-
dle all CF languages, it suffices to show that
any Type 2 grarrnar can be converted to one with
rules of the forms: P ~ cPl ..• Pk or P ~ c ; i. e. ,
the so-called Greibach Nornal Form. I also con-
sidered various types of predictive and dependency
73
systems. Notice that a rule in GNF can be con-
sidered a special case of a rule in a dependency
system (or, perhaps, Gaifman normal form!) where
the one terminal symbol per rule can appear any-
where in the right hand side of the rule; hence,
Gaifrnan's theorem can be considered to be a corol-
lary of mine. I did not realize until long after-
wards that, using the closure of the family of CF
languages under reversal, it is not hard to obtain
a GNF grammar from a restricted categorial system
and vice versa. Incidentally, another restricted
type of Type 2 grammar, the operator grammar (no
two nonterminals adjacent), proposed at this time
by Floyd [1963j in connection with tne use of pre-
cedence grarrnnars tor parsing prograrrnning languages,
can be shown to suffice for CF languages, using
GNF [Greibach,1965J. In my thesis, I also showed
ambiguity undecidable in a very restricted case:
m-inimal linear graJnmars, meaning only one nontermi-
nal X and rules of the form X~ uXv, X-+ u., u
and v strings of terminals. Since the usual for-
mulation of the Post Correspondence Problem did
not work, I proved a variant undecidable and used
that for the reduction [Greibach,1963bJ; this tech-
nique has been used by others.
The top of the hierarchy had already been
recognized, as I have observed earlier. However,
different formulations of Turing machines were ob-
tained. Counter machines were vaguely mentioned
by Chomsky [1959J. Minsky [196lJ showed that in a
certain sense a machine with two counters can sim-
ulate a Turing machine. A very elegant and intui-
tive direct proof (without the encodings used by
Minsky for input and output) was provided by Fisch-
er in the first FOCS paper we cite [Fischer,1963;
the full paper appeared in 1966J. He showed that
two pushdown stores can simulate a Turing tape,
two counters can simulate a pushdown store, and
two counters can simulate any number of counters.
Thus, at this stage, we have a graJTU'TB.Y' and a
machine for: Type 0, Type 2 and Type 3. We also
have various other characterizations in each case
(such as the regular expression characterization
for regular languages). Only the Type 1 languages
remain to be characterized by machine. This was
the last case completed. The type of machine need-
ed was fundamentally different from those in the
other cases. What was required was not a restric-
tion on the access of the machine to its work tape"
but on the amount of tape allowed. We should note
that both time and tape restrictions on Turing
machines were defined in the early 1960s.
Yamada [196lJ first formalized the concept of
realtime computation and established the existence
of fill1ctions not realtime computable. Rabin [1964J
provided what is now the standard definition of the
class of languages recognizable in realtime by
k-tape or mu1titape deterministic Turing machines
and showed that two tapes are JIDre powerful than
one tape in realtime. In the 1964 meeting of the
Syrnposiwn on Switching Circuit Theory and Logical
Design, Hartmmis and Stearns [1964J presented the
classification of numbers, functions and sequences
by time compZexity on a multitape Turing machine;
their hierarchy results can be applied to the class-
ification of formal languages by their
of recognition. In his thesis, Cole [
ered computation in realtime by iterative
finite state machines.
The classifaction of sets by space complexity
of recognition was initiated by Myhill in a' widely
cited but never published report [Myhill,1960J. He
defined a deterministic linear bounded automaton
(lbaJ as a two-way deterministic finite state auto-
maton with overprinting. He showed that rudiment-
ary sets can be defined by deterministic lba and
posed the question of whether this is a proper con-
tainment, a question which is still open today.
The paper also gave an informal definition of ma-
chines with storage restricted by an arbitrary
function f such that fen) ~ n everywhere.
Landweber [1963J showed that the class of
languages generated by Type 1 phrase strUCTLrr'e lan-
guages (the class of context-sensitive, CS , lan-
guages) is not enlarged by allowing endmarkers, is
closed under the operation of intersection and in-
cludes the class of languages accepted by deter-
ministic lbas.
I recall that, when I was wrltlng my thesis
in 1963, it was an lllteresting open question whe-
ther lbas corresponded to Type 1 gramnars as pdas
to Type 2 gramrrers. With hindsight, it is hard to
see why this was a difficult problem at all. Part
of the reason was that we were not really used to
nondeterministic machines -- to think nondetermin-
istically -- and Myhill's paper defined only deter-
ministic lbas. It was known that deterministic
and nondeterministic finite automata had the same
power [Chomsky and Miller,1958; Rabin and Scott,
1959J and -- as noted, for example, in the thesis
of Evey [1963J -- that deterministic and nondeter-
ministic Turing acceptors had the same power. It
was suspected and shortly proved mdependently by
various people [Cole,1964; Haines,1965; Ginsburg
and Greibach,1966, based on a 1964 draft] that de-
terministic pushdown automata did not accept all
CF languages and so were of lesser power than non-
deterministic pdas. But lbas?
Kuroda [1964J completed the Chomsky hier'ar-
chy of languages and ma.chines when he defined non-
detePministic lbas and showed that each context-
sensitive (1\JPe 1) language can be accepted by a
nondeterministic lba. He also gave a proof that
the language accepted by a nondeterministic Iba
is context-sensitive. Further, he showed that the
class of languages accepted by deterministic lbas
is a Boolean algebra containing the class of con-
text-free languages. He stated two important prob-
lems which rernain open today: Is the class of
context-sensitive languages closed under complemen-
tation? and, Is the class of nondeterministic lba
languages equal to the class of deterministic lba
languages? (i.e.; is NSPACE(n) = DSPACE(n)? the
"LBA. problem").
Thus, the Chomsky hierarchy was completed and
the stage set for most of the developments of formal
74
language theory. A good survey of formal language
theory in its relation to mathematical linguistics,
the syntactic analysis of natural languages, meth-
ods for machine translation,and information retrie-
val, can be found in the notes for the 1964 Harvard
Summer School Course on Language Data Processing
[Oettinger,1964J. Although the divergence was not
rapid nor complete, that represents the last time
when formal language theory could really be consid-
ered part of mathematical linguistics or, alter-
natively, the start of the disciplme as a part of
theoretical computer science.
4. Directions after 1964
From 1964 on, formal language theory devel-
oped as a branch of theoretical computer science.
The take-off point was the study of the t;rammars,
machines and languages of the Chomsky hierarchy:
restrictions and extensions of context-free gram-
mars and pushdown automata; restrictions on the
computing ability and resources of Turing machines;
abstractions of the properties of various classes
of languages; the need for better algorithms for
the recognition of languages; consideration of the
practical use of context-free grammars in parsing
and compiling.
Exactly which topics are classified under
"formal language theory" and the relative emphasis
placed on them is debatable: different books on
the subject have different selections. Time and
space do not allow me to cover all possible topics
nor to cover anyone topic at sufficient length.
I claim an author's prerogative to dewll most on
the Subjects I know best and the part of the theo-
ry to which I contributed.
The directions that I shall consider briefly
are:
-Restrictions on Context-Free Grammars and PDAs
-Extensions of Context-Free Grammars and PDAs
-Unifying Frameworks
-Complexity Ouestions
Those four topics are not disjoint and the overlap
and interaction between these areas (and others)
has been a fruitful source of techniques and re-
sults. Some directions I shall not survey (although
I have been connected with a few) include: algebraic
theory of autorra.ta and languages; probabilistic lan-
guages; fuzzy languages; 2-dimensional languages;
semantics (including schematology).
5: Below the Context-Free:
Restrictions on Context-Free Grammars and PDAs
Various types of restrictions have been
placed on context-free grammars and on pushdown
store autorra.ta (pda). Many of the restrictions do
not affect the power of the devices. For example,
we have seen that restricting the productions of a
context-free grarmnar to Chomsky Normal Form or to
Greibach NOrITBl Form or to operator form still per-
mits the generation of all context-free languages.
(That is, modulo the empty word which I shall de-
sisnate e .)
Chomsky's definition of a pda allowed what I
have called "e-rules": transitions of the machine
which do not advance the input tape. Eliminating
e-rules, thus restricting the pda to operate in
"quasirealtime" , does not limit the power of a pda,
even if at the same time the number of states is
restricted to one [Greibach,1963aJ. Restricting a
pda to deterministic operation does make a differ-
ence, however. This is the first restriction we
shall consider' and, from a pr'actical viewpoint,
the most important. Restricting the number of non-
terminals in a rule to two makes no difference, as
we saw, but restricting the number of nonterminals
in each intermediate step of a derivation does; we
shall consider restrictions of that sort and the
corresponding types of pdas. The third restric-
tion we shall consider is not directly on the gram-
rrBrS or ma.chines but on the form of words in a lan-
guage.
Deterministic Context-Free Languages
The basic properties of deterministic push-
down store automata (dpda) ';Jere st"udied by Schutz-
enberger [1963J, ~insburg and Greibach [1965J, and
Haines [1965J under different formulations. The
langpage accepted by a deterministic pushdown store
automaton is called a cieterminist1:c context-free
language (dcfl)~ every context-free languap;e (cfl)
can be generated as the range of a deterministic
pushdoWfl store transducer [Evey, 1963 J . The pal~n­
drome or mirror image language {wx Iw in {a, b }~~ ,
x is the reversal of w} is a cfl which is not
deterministic (Cole[1964J, Ginsburr and Greibach
[1965J), so the family of deterministic context-
fl'ee languages (DCFL) is a proper subfamily of
the family of context-free languages (CFL).
Fischer [1963J claimed that DCFL is l1 easi-
ly" sho\VI1 to be closed tmder complementat~ion. How-
ever, there is one difficulty. The standard def-
LYlitions of a dpda allow e~rules and hence the
possibility that the rrBchine can "loop" and never
read all of its input tape. Eliminating e-r1ules
produces the class of realtime pda languages
which Ginsburg and C;reibach [1965 J showed to be a
proper subfamily of DCFL; their counterexample
can be recognized by a realtime Turing machine
(TI1) with one working tape, but Rosenberg [1967bJ
pr'Ovided an examp1e of a dcfl not realtime defin-
able by any multitape 1M. However, it is possible
to clLTinate loopLYlg in deterroinistic pdas nnrl
thus DCFL is closed under complementation [Schutz-
enberger,1963; Haines ,1965; Ginsburg and Greibach,
1965 J. Ivlor€ significantly, this also shows that
dpdas can be ffi3.de to oper-ate in linear time. An-
other important fact is that a dpda can be effec-
tivelv converted to an eqllivalent unambiguous con-
text-free grammar (i.e., one left-to-right deriva-
tion for each word in the language) [Haines,1965;
C;insburg and Greiffich, 1~165 J.
These two properties -- lineae time recog-
nltlon and unambiguity -- naturally directed atten-
tion to deterministic context-free languages but
it was the introduction of LR(k) grammars b~l
Y~luth [1965J which established their import~lce for
the syntax of progrilllllning languages. The LR(k)
75
restriction on context-free grammars is different
in natuy'€ from those we consider below. In a right-
to-left derivation ... U::9V . .. , the nontermina1
expanded in the step u =?v and the production
used in that step identify a handle of v. Roughly
spea~ing, a grammar is LR(k) (i.e., left-to-right
translatable) if the handle of a right-to-left
sentential form is always uniquely determined by
the string (of terminals and nonterminals) to its
left and the next k (terminal) symbols to its
right. The language generated by an LR(k) grammar
is a dcfl while every defl has some LR(l) gram-
ffi3.r. For k fixed, it is decidable whether a con-
text-free grammar G is LR(k) but it is undecid-
able whether there is an integer k such that G
is LR(k) ; for the latter proof, Knuth defined
and used another Valliant of the Post Correspondence
Problem. Knuth [1965J discussed Pallsing procedures
for LR(k) gra.rrDTla.l"'s and the use of LR(k) grarrrrnars
for All;OL. Since that paper, various authors,
such as DeRemer [1969,1971J have considered the use
of subclasses of LR(k) grarrrrnars for parsing pro-
grammin~ languages.
A different grammatical characterization of
DCFL was given by Harrison and Havel [1972;1973J.
They defined the class of (realtime) strict deter-
ministic grammars that generates precisely the
class of languages accepted by empty store by
(realtime) dpdas which block on empty store. It
is decidable whether a grammar is strict determin-
istic. Addin,Q, an endrnarker to a dcfl makes it
strict deterministic so, for most purposes, the
theories of deterministic and strict deterministic
context-free languages coincide.
Precedence languages were introduced by Floyd
[1963J while Wirth and Weber [1966J proposed an
extension in their discussion of Euler. The rela-
tionship between precedence and deterministic con-
text-free languages was initially not quite clear.
M. Fischer [1969aJ showed that the class of Floyd
operator precedence languages is properly contained
in the class of backwards deterministic Wirth-Weber
precedence, which in turn is properly contained in
DCFL. However, every cf1 can be generated by
some Wirth-VJeber pD2cedence grarrrrmr. On the other
hand, Graham [1970J observed that "if one takes
the definitions of extended precedence to be those
sUfSgested by Wirth and Weber in the descr'iptive
portions of their paper, rather than the formal
rlefin-it--jons presented, then all the classes of
extended precedence languages which are nontrivial
extensions of simple precedence are equivalent and
are equivalent to the class of deterministic lan-
guages and to the class of bounded right context
languages. "
Ginsburg and Greibach [1965J established
most of the basic closure and decision properties
of DCFL. They showed that inclusion is Ufldecid-
able as are the emptiness of intersection (L n L' =
v?) and the identification problems (does a con-
text-free grammar G generate a deterministic
context-free language?). The equivalence problem
and most of the identification (or subclass con-
tainment) problems remain open. Stearns [1967J
showed that it is decidable whether a dcfl is
regular.
 The equivalence problem has been shown decid-
able for various subclasses of deterministic lan-
guages. A parenthesis grammar has rules of the
form A -+ (u) where u does not contain n (T1 or
TI ) " . McNaughton [1967J defined parenthesis gram-
mars, showed that they generate deterministic
context-free languages and that inclusion and equi-
valence are decidable. Knuth [1967J showed that it
is decidable whether an arbitrary context-free gram-
mar generates a parenthesis language. Usually,
equivalence is shown decidable by showing inclusion
decidable. The first known exception is the class
of simple languages (generated by grammars in Grei-
ba.ch Nonnal Form such that a pair (Z ,b) of a
nonterminal Z and terminal b appears in at most
one rule Z -+ bu) for which Korenjak and Hopcroft
[1966J showed equivalence decidable, while Friedman
[1976J showed inclusion undecidable. Rosenkrantz
and Stearns [1969J showed equivalence decidable for
the LL(k) grarrrrnars of Lewis and Stearns [1966 J
(which are stronger than simple gramnars but weaker
than LR(k) ). Interest in the equivalence problem
was reawakened by Valiant [1974J who provided pow-
erful new techniques and by Friedman [1974J who
showed the problem to be logically equivalent to
the strong equivalence problem for monadic recur-
sion schemes. Beeri [1975J improved Valiant's
algorithm for deciding equivalence of finite turn
dpdas . Recent1y it has been shown that Valiant's
algorithm for nonsingular languages is also valid
for realtime strict deterministic languages and
that "Ll = L2" is decidable for L l realtin1e
strict deterministic and L2 merely deterministic
[Oyamaguchi,Honda and Inagaki,1979J; this may be
the strongest result to date.
Linear-like Languages: Nonterminal Restrictions
Chomsky and Schutzenberger [1963J defined
linear context-free grammars by productions whose
right hand side contain at most one nonterminal and,
by adding arbitrary productions which could be used
only initially, extended the concept to metalinear
gramnars. Greibach [1966J showed that, for a sym-
bol c not in L, lcL is linear if and only if
L is regular if and only if (lc)~': is metalillear;
thus, metalinear graJI1l1B.rS are stronger than linear
but weaker than context-free.
Sentential forms of lillear graJ11I1BT'S can con-
tain at most one nonterminal. Banerji [1963J gen-
eralized this concept to nonterminal bounded gram-
:rn:.:ITS where there is a uniform upper bound on the
number of occurrences of nonterminals ill sentential
forms. Ginsburg and Spanier [1966bJ provided a
different but equivalent grammatical characteriza-
tion for the languages generated by such grammars
and called them ultralineup. They also established
a JIBchine characterization via fin'tete-turn (also
called reversal-bounded) pdas: there is a uniform
upper bound on the number of times a pda can turn
from pushing (increasing the store) to popping (de-
creasing the store). This bound corresponds to the
bound on the number of occurrences of nonterminals
for- the equivalent grc3JTlffi3.r. A proper hierarchy of
classes, all properly included in CFL, is defined
by varying this bound. Thus, linear languages are
those accepted by single-turn pdas. Rosenberg
76
Q967al provided different ITBchine realizations of
the linear context-free languages using two-tape
finite state acceptors; this enabled him to prove
the unQecidability of the universe problem
("L = L~:") .
Instead of restricting the occurrence of non-
terminals in all derivations, one can require that
there is a round k such that, for each word w
generated by the grarrrrnar, there is a derivation of
w in which no sentential form contains more than
k occurrences of nonterminals. Ginsburg and
Spanier D-968aJ called such grammars derivation
bounded and proved that they generate the closure
of the linear context-free languages under substi-
tution. Nivat [1967J and Yntema [1967J studied
this very illteresting family under different for-
mulations and proved that it is indeed a proper
subfamily of CFL. Walljasper [1974J defined a
family intermediate between ultralinear and deriva-
tion bounded by placing a uniform upper bound on
the occurr€nces of nonterminals only on left-to-
right derivations.
As mentioned, restricting the number of
states in a nondeterministic pda does not limit
its power (depending on the precise definition of
acceptance). Two pushdown store symbols likewise
suffice to define all cfls. However, restricting
the store to one symbol plus an endmarker yields
the family of one counter languages, which is a
proper subfamily. Greibach [1967J showed that a
two-way infinite hierarchy of context-free lan-
guages could be obtained by combining the finite
turn restriction with a generalization of the
counter restriction (the pushdown store contents
must belong to a fixed bounded regular set).
Bounded Languages
A language L is bounded if there are words
wI, ... ,wn such that L ~ wl*"'wn * The study
of bounded context-free languages was very fruit-
ful for they have elegant algebraic characteriza-
tions which allowed precise proofs of difficult
results on inherent ambiguity. A bounded language
L ~ wl*"'wn * is context-free if and only if its
Parikh mapping (i. e., the set { (iI' .. · ,in) I
WI i 1 . .. wn i n '}
ill L ) .lS the f illlte
.. .
unlon 0
f
linear sets each with stratified periods [Ginsburg
and Spanier,1966 a ], it is an unambiguous context-
free language if and only if its Parikh mapping is
the finite union of disjoint linear sets each with
independent stratified periods [Ginsburg and Ull-
ian,1966J. Inclusion, equivalence and ambiguity
are decidable for context-free grammars which gen-
erate bounded languages [Ginsburg and Spanier ,1964;
Ginsburg and Ullian,1966J. Ginsburg and Ullian
[1966J used the characterization of unambiguous
bounded context-free languages to prove that it is
undecidable whether a context-free grammar gener-
ates an inherently ambiguous language (one for
which there is no equivalent unambiguous context-
free grammar).
 6. Between Context-Free and Contex-Sensitive:
Extensions of Context-Free Grammars and Pdas
The concepts of a pushdown store automaton
and a context-free grammar have been extended in
various ways. Usually such systems yield a family
lying between context-free and context-sensitive.
When the input tape access was changed -- e. g., two-
way input heads -- this did not necessarily hold;
however, I take the liberty of surveying the differ-
ent cases together.
Stack like
A pushdown store automaton can compute only
on the top of its work tape, adding, removing or
changing symbols, and cannot visit the interior with-
out erasing the symbols above it. A stack automaton
-- introduced by Ginsburg, Greibach and Harrison
[1966,1967J -- can operate either in pushdown mode
on top of the store or in reading mode in the in-
terior of the store; when it visits the interior of
the store, it can read but not write; i.e., it can-
not change the contents of the interior of the store.
A nonerasing stack automaton can push but not pop
(erase) while in pushdown mode. A checking automa-
ton [Greibach,1968aJ is a nonerasing stack automaton
which once in reading mode can never return to push-
down ITDde~ essentially all it can do is write a word
on its working tape and then move back and forth
checking its working tape against its input tape.
A nested stack automaton [Aho,1967J can act in push-
down mode, in reading mode or in stack creation
mode. In stack creation mode, it can create a new
nested stack below the currently scanned symbol; in
pushdown mode, it operates on the top of the main
stack or of one of the nested interior stacks; in
reading mode, it can move its head within one of
the stacks without writing or out of the bottom but
never out of the top of one of the nested stacks.
It is the last mentioned restriction that keeps the
nested stack automaton from having the power of a
Turing machine.
The hierarchy of languages accepted by the
corresponding one-way nondeterministic devices can
be roughly diagrammed:
CHECKING t NONERASING STACK\
7- STACK ~ NESTED STACK
PUSHDOWtJ STORE / of productions such that, at time j in a deriva-
f CSL .
The families of context-free and of nonerasing stack
languages are incomparable. The families ot lan-
guages accepted by the one-way nondeterministic
(deterministic) devices of all these types (check-
ing, nonerasing stack, pushdown store, stack,
nested stack) have most of the same closure and de-
cision properties. An interesting exception is the
substitution operato~: the family of (nonerasing)
stack languages is not closed under substitution
while the other three are~ the substitution closure
of the family of stack languages is properly in-
cluded in the family of nested stack languages
[Greibach,1969J. These observations strongly Ln-
fluenced the development of AFL theory and its
variations.
77
fi~ltiple Tapes and Heads
We have already seen variations in the access
to the input tape. Rabin and Scott [1959J consid-
ered finite automata with two-way input (i. e., the
reading head can move left or right on its input)
and with two input tapes. Rosenberg considered
finite automata with multiple input tapes [1964J or
with many reading heads on one input tape [1965J.
Two interesting problems were left open by these
papers: are k+ 1 heads better than k for a fi-
nite state acceptor reading its input tape in one
direction only? is equivalence decidable for deter-
ministic finite state acceptors with n ~ 2 one-way
input tapes? The first question was recently set-
tled by Yao and Rivest [1976J. The second question
remains open for n ~ 3; it is easy to see that
for n = 2 and other variations <e.g., nondeterminism
or two-way head motion or multiple heads on one
tape) it is undecidable. Bird [1973J proved that
equivalence is decidable for deterministic one-way
two-tape finite state automata while Beeri [1975J
provided an algorithm with an exponential time
bound; recently, Friedman and Greibach [1979J showed
that the question is polynomially decidable. For
one-way input tapes, the projection of the language
accepted by an n-tape finite state machine on one
of its tapes is regular. M. Fischer [1969J showed
that the family of context-sensitive languages (CSL)
can be characterized by the projections on one tape
of languages accepted by two-way two-tape nondeter-
ministic finite state automata. Gray, Harrison and
Ibarva [1967J considered pda with a two-way input
tape while Ginsburg, Greibach and Harrison [1967J
defined two-way stack automata.
Extensions of Context-Free Grammars
The power of a context-free grammar can be
extended either by r€stricting the use of its pro-
ductions or by adding different types of productions.
Salomaa [1973J calls the first method regulated re-
writing. In a matrix grammar -- introduced in [Ab-
raham,1965J -- rules are organized into sequences
called matrices. A matrix rule [PI' .. ' ,PtJ is
applied to a string u l by first using PI to
derive u2 from ul".·' and finally using Pt
to derive Ut+l from Ut. In a time-varying gram-
mar [Salomaa,1970J, there is an ultimately periodic
sequence PI,". ,Pj, ... of subsets of the set P
tion, only productions from Pj can be used. In a
programmed grammar [Rosenkrantz,1967J, the produc-
tions are labeled and each Droduction is associated
to two sets of production l~bels, a success field
and a failure field. If a production can be ap-
plied, it is applied leftmost and the next produc-
tion must be selected from the success field; if a
production cannot be applied, the next production
must be selected from the failure field. Arbitrary
context-free programmed grarrrrnars (cfpgs) generate
all r.e. sets while nonerasing cfpgs generate a
family properly between CFL and CSL [Rosen-
krantz,1967J.
Altman and Banerji [1965J introduced the no-
tion of the string of production labels associated
with a Cleft-to-right) derivation. Ginsburg and
Spanier [l968l:i) defined aontroz, sets on grarrmaP8: a
gramnar G generates with cOntrol set R the set
of words w for which it has 1eft-to-right deriva-
tions with associated production ·labe1 strings in
R. Not surprisingly, rostof these devices ;do not
affect the power of:'''gI\31IIMrs with Type 0, 1 or 3
productions but do'"affeGt those with Type 2 ·(con-
text-free) produCtions. Further, there are· close
relationships 'arrong' 'these diff~t systems with
or witheut 1eftlrost· restrictions arid, with or talith-
out·· appearance cheCking (which allOWS 'Certain .
specified productions to be skipPed whetr' their ap-
plication ·is 'iJripossible)~. A good survey apPears
in· .SalonE.a "'[1973] . . systems, rules are applied in parallel -- eVery
Rewriting is regulated in a slightlY,differ-
ent fashion in' scattered 'Context \granm!rs' [GreibaCh
and Hopcroft,f969]. Rules are grOuped into sequeh"-
ceS .as fu: matrix .'~. buT· they' IID.1st J:>e applied
in ~le1 and'in order' from left: to right ; .a non-
erasing restriction is applied. ,The family· o,f
scatt~d ~ccmtext l~guages has thepropertiea "of :
the' fmly of' context-sensi1;lve languages, which
is one':reason it has. so>'far i~en impossible to
'prove ~~ ebnt~t. If, a sequence' [Al 0+ wl' .",
···.,An~Wn].:' 'car;' be :ap~lied'?J11~ when theAl are.
all the· nontermmals iri the 'strmg, one ohtams' the
abso·Z,utely paraZ,Z,-el graJ11!7laPB of Raj1ich [1971].
These .generate an interesting sUbset of one-way
checking autonatbn languages: there is a uniform
upPer bOtmd on', the number oftimes"-the checking
automaton can visit any··tape sq~. ' language, T for "tabled~'r n"for deterministic~ P
The family. of nested stack aut~ta' qorres-
ponds to tM:> quite diffenmt 'extensions of .context-
free gramnars: the inde~ea ~'of Ah6 [1967]
and the or macro grc3IIlIErS of M. Fischer [1968],
both inspired by Constructs iri prog:remnrlng lan-'
guages .. '. In. both cases, context-free productions
are aUgTriented by allowing arguments. 'in' the case
of ',nacre ~s, 'each"nonte~l haS ~t
places Occupied by: '~~ings; a¥e A(xi~:·· · ,'.Xn'>'
... v I'eplaces a str:J:ng A(ul'·· ·,Un) by . v' whe!'e
each 'Xi . in v fs' ~placed by':' ui. The 01 '(oUt~
side-in)' restriction rreans that the nIle 'is' appli-.c-
able only if A is outeIm)s1:; i'-e., nOt :ih -an' ar-
gunent place of·'any nbnterm:i.rli.U; this pr'Oduces 'the
class· of . INDEXED or nested stack languages ... ···'lhe
IO ·(inside~out) restriction neans that the rule 'is·
applicable only if A is innerm:>st; i.e. ,the 'Ui
are' terminal strings. The families of 01 and 10
languages are incomparable.
As: we have seen', the syntax of ALGOL 60 was
one of ."the inspirations'· :for'the study of context..;"
free langUage's. Double-level· 'context-free gr'am-
nars,·or ·W~granuars, were, proposed for"the ·syntax
of ., AI.mL 6'8 ~ One level' cif the· :'grc:nmer produces
strings bfcharacterswmch naY"'~Ee,·tise~ fu1:he, non-
terminals of' the serond level, thus possibly pro-
~.: ~~j~~=iogi~f~te:r:
ulations apectr" in [Sirifzoff',1967] and··[Chastellier
and CO~rauer,19691. Since' W-grairJnars'pt'Oduce-
all r.e. sets, various restrictions on their"gener-
ative power have been considered (cf.,[Greibach,
1974]).
78
L-Systems
'"IrsYstems are defined -by .restrictions on the
application of productions but have generated such
a large body ,of 'literature that theY~'must be con-
sidered to fOrm a ,s~parate topic. , 'l'heywere intro-
duced bytJndermeyer [1968J to capttlrecertain .
features of .devel~al systen'B, 'and thus fonn
the'qnly post~ehansky.topicwe sha1.i examine not
inspi.redprllnarily by cariPuter science concepts.
As we have seen, the basic ideas' 'Iof L-systems
were prefigured in Chomsky's 1959 paper. In .1-
symbol in a string is simultaneously :rewritten --
and one considers the derived 'langUClge' (no .distinc-
tion 'between tenninals and"noritermi.:rials) as well as
perhaps the 'te~l l~e. . In fact, "terminal'
language" !"eally divide'S into two ideas: eit~ the
adutt language of strings ·whichcanilot be rewrit1:en
into other strings or ·the e~tended ·~e of
strings over a "partiCUlar designated vocabulary.
If the· rules are context·free' rules, we have OL
systems (or L-systems without interactIons). In
tab Z,edL-systems, productions are grouped into sets
called tables and" during a rewriting step ea9h· sym-
bol must be rewritten using a Proouciion fran the
same table. In a detel'l1linistic system, only one '.
'rule fran a table is applicable· toa symbol, while
in a prtopagating .system, .niles arenonerasing.
Systems are designed mnem:mically: 'E· for extendeCl
for' propagating. 'From a"fonnallanguage"iewpo~t,
the family of ~L languages is perllaps'thenost
interesting, 'Since it .has 'the mst attractive prop-
erties. "
Rozenber~ ~d Ibucet [1971] established the
basic propert1.es of OL languages. Rozei1berg
j
[1973a,b] considered TOL and EroL languages.
Blattner [1973] showed that eqUivalende is tmde";'<
cidable for OL systems, while &1lik and Fri~'
[19771 showed that it is decidable' for 'OOL sys-
tems (which was for a long time an important open
question). Checking stack pushdown automata, a
machine m:xie1 -far m'OL·· languages, were defined
by van 'I2euwen ['1976J. T!'a.nslational techniques
for dealing with families of OL languages (and
in ,particular, establishing the appropriate ~la­
tionships anong the various families) are given
in Skyum [1975] and Ehrenfeucht and Rozenberg
[1975]. Surveys of the sub1ect apPear in the l:x:>oks
by Rozenberg and Salornaa [1974] and Hernan and
Rozenberg [1975].
Trees
Tree diagrams were introduced' early in the
study of context-free languages (and, in one sense,
they preceded fonnal l~e theory).·. Transform-
were
ations on trees stUdied in connection' with
bdtli the~syntax of natural ianguages and the .seman-
tics of pr'ogranming' languages.· . ,, . "
lewis and' Steams [195'6] fonnalized. the idea
of syntax ~:iIected tpan~ticers (S!YI') ·and· di~eus~
the ~atrt.q~stioh bf when' a . (sinple) SDT can
be iiriplenented bYa determ:i.riistic'oo';':1.ine pushpoWJ:l
store transducer and obtained, aJIDng other things,
formal generalizations of the translational algo-
rithms of Oettinger [1961J and Sherry [1960J. Aho
and Ullman CL969al generalized sm to GSDT and
compared them to tree walking automata.
The tree transducers studied by Baker [1973J
and Engelfriet [1975J can be considered as parallel
rewriting devices opel'ating either fYDffi the bottom
up or from the top down. The compositon of n
bottom-up Ctop-down) transductions can be realized
by the composition of n+l top-down (bottom-up)
transductions. Letting Dn be the family of tree
languages obtained from the recognizable sets of
trees by the composition of n top-down transduc-
tions, and similarly Un for bottom-up, we have
. .. Dn s; Un +1 s; Dn+1 ... The interesting question
remained: is this hierarchy proper?
This question was recently settled in the
affirmative. Engelfriet, Rozenberg and Slutzki
[1978J offer a "systematic treatment of the rela-
tion$hips between parallel rewriting systems Ctop-
down tree transducer, £TOL system) and two-way
machines C2-way gsm, tree-walking automaton,
checking stack automaton) restricting the copy-
ing power of these devices the iteration of non-
deterministic top-down tree transducers, of non-
deterministic 2-way gsm's and of control on ETOL
systems each gives rise to a proper hierarchy."
This beautiful result is a good advertisement for
ETOL systems and for the translational techniques
related to AFL theory.
7. Complexity Questions
Complexity Classes
The classification of functions, and by im-
plication languages, by computation time on a mul-
titape Turing machine was introduced by Hartmanis
and Stearns [1964J and the existence of a hierarchy
was derronstrated. The classification of languages
by the space required for recognition on an off-
line multitape Turing machine and the corresponding
hierarchy was presented by Stearns, Hartmanis and
lewis [1965J who showed that there are minimum
boW1ding fW1ctions in the sense that, e. g., for
offline Turing machines, if the space bound grows
"much slower than" loglogn, the machine is no more
powerful than a finite state automaton.
Various techniques were used for both nega-
tive and positive results. For establiShing nega-
tive results Ci.e., L cannot be accepted by a
machine of type X within bound Y), the most
common tools are diagonalization Ce.g., to estab-
lish the basic time and space hierarchies) and in-
formation theoretic arguments on the amount of in-
fonnation needed Ce. £;., in showing that certain
languages are not realtime definable).
Two more techniques were introduced in 1965:
Hennie presented the notion of "crossing sequences"
and used it to show that, in linear time, a single-
tape Turing machine is no ITDre powerful than a
finite state acceptor. Ruby and Fischer intro-
duced translational Cpadding) techniques for improv-
ing the time hierarchy in certain cases.
79
For positive results, simulation techniques
are useful. Hartmanis et al. [1964,1965J used data
compression techniques to establish linear speedup
theorems: a bound of kfCn) on time or space
yields no more than a bound of fen) (k > 1) except
for linear versus realtime. Fischer, Meyer and
Rosenberg [1967J introduced the idea of "tape fold-
ing" in order to show that, in realtime, multiple
heads on one working tape can be siJIlulated by sev-
eral working tapes with one head apiece. This idea
was combined with the notions of "guessiJ1g" plus
some results from AFL theory to show that, for
nondeterministic multitape Turing machines, linear
time is no better than quasirealtime and two Turing
tapes Cor three pushdown stores) suffice [Book,
Greibach, 1969J. This contrasts with the determin-
istic realtime case: there is a dcfl which can
be accepted in linear but not in real time [Rosen-
berg,1967bJ and k+l tapes are always better than
k [Aanderaa, 197L~ J.
New light was shed on the long-standing LEA
problem in 1969. Savitch showed that NSPACECSCn))
~DSPACECCSCn))2) for SCn) logn;t this is
still the best general simulation result known.
Ftlrther, there is an upward translational result:
if :CSPACE(S(n)) = NSPACE(SCn)) for any Sen) 2
log n, equality holds for all lar8er space bounds.
Book and Wegbreit [1971J and Ibarra [1972J
used Savitch's simulation result, the deterministic
space hierarchy results and translational tech-
niques to establish a nondeterministic space hier-
archy in certain cases Ce.g., NSPACECnr ) 7 NSPACE(n~
for 1 s; r < S ). Cook [1972 J established a corres-
ponding nondeterministic time hierarchy. Various
hierarchy results were refined and sharpened by
Seiferas et al. [1973J.
Interesting connections were established be-
tween these classes of formal languages defined by
the complexity of the recognition problem and
classes defined in other ways. The first such re-
sult, of course, was the connection between CSL
and NSPACECn) Ci.e., the class of languages ac-
cepted by nondeterministic Ibas). Hopcroft and
Ullman Q967bJ showed that the classes of languages
accepted by nonerasing stack automata with a two-
way input tape can be characterized by space bound-
ed Turing machines: DSPACECnlogn) CNSPACECn 2 ))
is the class of languages accepted by 2-way non-
erasing deterministic CnonJe terrninistic) stack
automata. vJhat about s;eneral two-way stack auto-
mata? Cook [1969J showed that the same bounds
apply if the Turing machine is also allowed an
1mbounded pushdown store; such a ITBchine is called
an auriliary pda. Let APDA(SCn)) be the class
of languages accepted by offline machines with an
unbounded pushdown store and an SCn)-bounded
Turing tape. Then APDACnlogn) CAPDACn 2 )) is
the class of languages accepted by deterministic
tIn this notation, SPACE indicates a bound on work-
ing tape space and TIME on computation time, the
function in parentheses is the bound and (N)D in-
dicates computation on an offline Cnon)determin-
istic multitape Turing machine.
 Cnondeterministic) two-way stacK automata. Slllce
l\pDACSCn)) = Uc DTlIv1EC2 cS (n)) , stack automata lan-
guages can also be characterized by time oounded
complexity classes.
Another important question concerns time-
space tradeoffs. Clearly, XTlliECSCn)) ~ XSPACE(S(n)),
but is space rrore powerful than time? Hopcroft and
Ullman 0968~ provided some evidence while Hopcroft,
Paul and Valiant [1975J settled the question in the
affirmative: DTIME(S(n)JogS(n)) ~ DSPACE(S(n)) ,
so in particular there is a context-sensitive lan-
guage which cannot be recognized in linear time on
a deterministic Turing machine.
Cook [1971J introduced the famous F vs. ~w
question: can every language recognizable nondeter-
ministically in polynomial time (i.e., every lan-
guage in NP) be also recognized deterministic-
ally in polynomial time Ci.e., be in P)? Cook
(and then Karp [1972J) showed that this question
was equivalent in a certain sense to many famous
open problems. "It is shown that any recognition
problem solved by a polynomial time-bounded nonde-
terministic Turing machine can be 'reduced' to the
problem of deciding whether a given propositional
formula is a tautology. Here' reduced' means,
roughly speaking, that the first problem can be
solved deterministically in polynomial time pro-
vided an oracle is available for solving the sec-
ond " [Cook,197IJ.
A nondeterministic machine in the usual def-
inition accepts if some path in its computation
tree is "good". Wotschke [1975J considered machines
which accept only if all paths are good, as well as
other variations on the method of acceptance.
Kozen [1976J,and independently Chandra and Stock-
meyer [1976J, developed the idea of alternating
existential and universal quantification in the
definition of acceptance by a nondeterministic ma-
chine and showed that this bridged the time and
space hierarchie s; e. g., ATTI1E (log n) =DTll1E (poly)
and ATTI1E(poly) = DSPACE(poly) (where A indi-
cates an alternating machine and poly indicates
general polynomial bounds).
Time and Space Bounds for Language Recognition
As I have indicated, back in the early 1960s
we were very much concerned with whether the poten-
tially exponential blow-up in many of the algorithms
proposed for recognizing context-free languages was
necessary. Actually, the algorithm of Cocke al-
ready provided a negative answer, but this was not
recognized. Kasami [1965J and Younger [1966J in-
dependently gave algorithms which they proved re-
quired only time kn 3 (i.e., O(n 3 ) ) ; this gen-
eral method is now called the Cocke-Kasami-Younger
algorithm (cf. Harrison [1978J for a good exposi-
tion). La.ter, Valiant [1975J cormected the problem
of context-free language recognition with computa-
tions with Boolean matrices to yield a theoretically
faster (O(n 2 . S1 )) algorithm.
So polynomial time suffices. What about
space? Lewis, Stearns and Hartmanis [1965J showed
that CFL ~ DSPACE«logn)2) ; their construction
80
\vas the basis for Savitch' s simUlation. Recently,
Cook [1979J showed that all deterministic context-
free languages can be recognized deterministically
simultaneously in n 3 time and Clog n) 2 time.
How hard is the problem? A:re these upper
bounds tight? Lewis, Stearns and Hartmanis [1965J
showed that there are cfls recognizable offline
in loglog n space but also that some, for example
{anlfl I n 2 I} ,require log n space. mes log
space suffice for deterministic recognition? Sud-
borough [1973J showed that this problem is in a
sense equivalent to the Iba problem: CFL
~ DSPACE (log n) if and only if DSPACE (S(n)) =
NSPACE(S(n)) for all Sen) 2 log n .
Cook and Karp showed that the P vs NP ques-
tion could be reduced to the question of whether
one of a large (and ever-growing!) number of lan-
guages (problems) can be recognized (solved) in
deterministic polynomial time. The complexity of
the recognition problem for CFL can be reduced
to the complexity of one context-free language ~,
the "hardest" context-free language, a "nondeter-
ministic" variation of a Dyck set [Greibach,1973J.
Every cfl is an inverse homomorphic image of
either La or La - {e}; hence, the time (space)
complexity of recognition (parsing) of 4J is the
realizable least upper bOW1d for time (space) com-
plexity of recognition (parsing) of context-free
languages. A corresponding characterization for
DCFL does not exist [Greibach,1973J. However,
Sudborough showed that there is a "hardest" dcfl
1..0' in a different sense: OCFL ~ DSPACE(logn)
iff DSPACE(logn) is equal to the class of lan-
guages recognized by deterministic polynomial time
bounded log n tape-boW1ded auxiliary pdas (Which
is the class of languages log n-reducible to dcfl
[Arora and Sudborough,1976J).
8. Unifying Framework
The same closure properties appeared again and
again as we examined various restrictions of, and
extensions of, pushdown store automata. Certain
decision properties also recurred: membership was
usually decidable for machines with a two-way tape
and emptiness and member/ship in the one-way case,
while emptiness was usually undecidable in the two-
way case. Not only the results reappeared; the
constructions used in the proofs were almost iden-
tical and did not appear to depend on the particu-
lar devices W1der construction. Thus, in the
surrnner of 1966, attempts were made to find a uni-
fying framework for these observations; the results
were presented at the 1967 Symposium (1967 was a
very good year for forrnal language theory!) in the
papers by Hopcroft and Ullman and by Ginsburg and
Greibach.
AFLs~ AFAs and Balloons
A balloon automaton [Hopcroft and Ullman,1967aJ
has a finite state control (deterministic or non-
deterministic), an input tape (one-way or two-way)
and an abstract memory called the balloon. Balloon
control fW1ctions change the configuration of the
balloon while a balloon information fW1ction allows
the machine to receive only a finite aIIlOW1t of in-
formation from the balloon by mapping the set of
balloon configurations into a finite set. The
finite state control knows the state of the control,
the input and the value of the balloon information
function and changes state, moves the input tape
and selects the balloon control fW1ction. A class
of Balloon automata (BA) is closed if it contains
all allowable machines with balloon information and
balloon control functions constant (so, defines all
regular sets) and if, whenever it contains machines
MI and M2 , it contains any other allowable machine
whose balloon control fW1ctions are taken from
those of MI and M2 and whose balloon informa-
tion function makes only distinctions made by MI
and M2. All types of stack automata form closed
classes of balloon autOl'TBta. Various closure prop-
erties hold for an arbitrary closed class of bal-
loon automata. IDEA: inverse gsm, intersection
with regular; INBA: inverse gsm, W1ion, concaten-
ation, Kleene star, intersection with regular, gsm,
quotient with regular; 2DBA : reversal, inverse
gsm, intersection~ 2NBA: same as 2DBA plus W1ion.
In developing AFL theory, we first consid-
ered the desired closure properties and then looked
at the corresponding class of machines. An AFL is
a nontrivial family of languages closed W1der W1ion,
concatenation, Kleene plus, nonerasing horromorphism
and intersection with regular sets; a full AFL is
an AFL closed under arbitrary homomorphism (i. e. ,
erasing) [Ginsburg and Greibach,1967J. vve knew
that something interesting was happening when we
showed that a full AFL is always closed W1der
finite transductions and quotient with regular sets
and that, to obtain a full AFL from a family of
languages, it suffices first to take finite trans-
ductions and then to substitute them into regular
sets. An AFA schema. is determined by a type of
storage structure, an information function on the
storage, a set of instructions and a function that
maps a storage configuration and an instruction
into a new storage configuration, subject to a few
conditions. An AFA associates the corresponding
schema to the family of all possible machines with
finite state control acting on storage of the ap-
propriate type and limited to receiving (as for
balloon automata) only a finite amount of inforrn:i-
tion from the storage. Using these definitions,
we were able to show that full AFLs are the natu-
ral characterization for the family of languages
accepted by any reasonable class of one-way non-
deterministic ITBchines (AFAs or lNBAs) and
similarly AFLs corresponded to one-way quasi-
realtime (nondeterministic, W1iform upper bound on
the number of e-rules in a row) acceptors.
Thus, we now had a natural framework for
studying one-way nondeterministic acceptors. Other
types of ITBchines were also considered. An AFDL
is a nontrivial family of languages closed W1der
inverse gsm, marked W1ion and star and removal
of endmarkers [Chandler,1969J. A nontrivial class
of languages is defined by a class of 2DBA. if and
only if it is closed W1der marked ill1ion, IIB.rked
star and inverse two-way gsm mappings [Aho,Ull-
ffi3l1, 1969bJ.
What happens if the acceptor is allowed to
have multiple storage tapes? One-way nondetermin-
81
istic acceptors with n tapes, tape i of Scheme
i, define precisely the class of homomorphic images
of the intersection of n languages, the i th lan-
guage definable by a realtime acceptor of Scheme i;
limitations on the computation time of the acceptor
correspond to limitations on the homomorphism [Grei-
bach and Ginsburg,1972; work done in 1968J. This
generalizes the observation that every r.e. set can
be obtained as the intersection of two dcfls [Gins-
burg, Greibach and Harrison, 1966 J; a corresponding
result just for quasirealtime multipushdown store
automata was used to show the NTll1E(n) is con-
tained in the family of scattered context languages
[Greibach and Hopcroft,1969J. Similar results hold
for transducers, thus generalizing Evey's results
on acceptors vs. transducers; if the multiple tapes
are nested, the appropriate operator is substitu-
tion rather than intersection.
Combining the work of Chomsky and Schutzen-
berger [1963J with my normal form theorem, one
characterizes CFL as the family of sets
h(g-l (K2) n R) ,where K2 is the one-sided Dyck
set on two letters, R is regular, h and g are
homomorphisms with h nonerasing. The natural
generalization of this result lies in the concept
of a principal AfL or trio. A trio is a family
of languages closed W1der nonerasing homomorphism,
inverse homomorphism and intersection with regular
sets; a full trio is a trio closed ill1der erasing.
A (fu ll) sem1·,.4.FL is a (full) trio closed W1der
union. If .Cf is the least (full) AFL containg L,
it is a (full) principal AFL with (full) genera-
tor L; principal (full) trios (semiAFLs) are sim-
ilarly defined. If a storage (AFA) schema is
finitely encoded -- e.g., pdas with only two
storage symbols -- the corresponding family of one-
way nondeterministic acceptors defines a full prin-
cipal AFL and every full principal AFL can be
so defined [Ginsburg and Greibach,1970J. In the
multitape case, one takes as generator the shuffle
of the generators corresponding to the individual
tapes. So, for example, the characterization of
the r.e. sets by two COill1ter machines implies that
a full generator for the r.e. sets is the shuffle
of {Onl nc In ~ l}~': and {anJ:1d In ~ l}~~.
Applications: Structure of Special Families
of Languages
The concept of AFA and AFL inspired the two-
way classification of CFLs found in [Greibach,1967J.
Boasson [1971J applied similar algebraic techniques
to the study of one cOW1ter languages while Gold-
stine [1972J considered AFLs generated by boill1ded
languages.
To define CFL, it suffices to take pda that
have just two storage symbols and no e-rules and
do not reinitialize the store . Results in Ginsburg,
Goldstine and Greibach [1972J and in Greibach [1970,
1972J show that this is not accidental. If L is
a full generator of CFL, then every cfl can be
obtained from L u {e} by nonerasing a-transducer
i.rna.ges. Hence, for any reasonable machine defini-
tion of CFL, there is an equivalent subfamily of
finitely encoded quasirealtime acceptors which do
not reinitialize the store. Boasson and Nivat
[1976J showed that every parenthesis language is
either cierivEttion-bow-tded
CFL.
Additional operations can be added to define
particular families. For a substitution f, the
iteration of f on L is the language obtained
by repe~te91y applying f to L; f is nested
if b lS In feb) for each symbol. A superAFL
(hyperAFLJ is a full AFL closed under nested
iterated substitution (iterated substitution).
This allows us to link L-systems neatly to the
Chomsky hierarchy after all: the family of regular
languages is the least full AFL, CFL is the least
superAFL and the family of £1OL languages is the
least hyperAFL (cf, Rozenberg and Salomaa [1974J,
pp 250-301).
Applications: Translational Techniques
and Hierarchies
Here is an example of a translational or
bridge theorem: for c a sym]x)l not in L, LcL
is regular if and only if L is regular. Thus,
from the existence of a language L that is lin-
ear but not regular, one can deduce the existence
of a language LcL that is metalinear but not lin-
ear. AFL theory is a good framework for such
results. I used them in [Greibach,1967J to obtain
the hierarchy results and in [Greibach,1968a,1969J
to show that the family of (one-way) stack lan-
guages is not closed under sUbstitution and its
substitution closure is properly contained in
INDEXED.
These results can be abstracted elegani~ly
some cases. Here is an example. Let Sub (~)l ,~ )
be {feLl) I Ll in :/1 and f an '~2-substitu­
tion} and T(Ll ,L 2 ) = {blwl···bnwn I bl ... b
n
L , b a sym]x)l, Wi lD L2 }· If Ll and L2
l i
are over disjoint alphabets and SRI and !£) are
full serniAFLs, then T(L ,L ) in Sub(~l"~)
l 2
implies that either L is in 5£1 or L lS
l 2
~2' Now this allows one to conclude that, if
is a full semiAFL not closed under substitution,
then iterating substitutions on .<f yjplds an in-
finite hierarchy and the closure of ~ tmder sub-
stitution is not a full principal AFL. As an im-
mediate corollary: CFL properly contains the
substitution closure of the linear cfl and the
one counter languages, a result that is tedious
to obtain otherwise. Various translational theo-
rems and their applications appear in [Greibach,
1970,1972J.
The most beautiful example of the use of
this type of translational techniques is, as indi-
cated arove, Engelfriet's tree transducer hierarchy
theorem [Engelfriet, Rozenberg and Slutzki,1978J.
Applications: Complexity Classes
Many of the complexity classes mentioned
above (such as NP) are AFLs and AFL theory
can often shed light on some of the problems men-
tioned and assist in simulation proofs. A good
\:;xample is the proof that IJ'TIME(n) can:bE> defined
by quasirealtime TI'ls \tJith one checking stack and
one pda (or three pdas) [Book EiJld Greibach, 196 9 ] .
Here, the fact that every language in NTll1E(n) is
the nonerasing homomorphic image of the intersection
of cfls and that the families of languages in-
volved are closed under nonerasing homomorphism
demonstrates that it suffices to show that one can
define all intersections of cfls by the desired
type of machine. It also shows that NTll1E(n) is
a principal ArL. A similar result was shown for
the family BNP of quasirealtime reversal bounded
multipushdown store machines: three single-turn
pushdown stores suffice so every L in BNP is
the nonerasing homomorphic image of the intersec-
tion of three linear cfls [Book, Nivat and Pater-
son ,1974 J.
~vegbreit [1969J showed that NSPACE(n) is a
principal AFL. These results were used by Book,
Greibach and Wegbreit, and Ibarra [1970J to show
that, under various general conditions, time and
space complexity classes form AFLs or principal
AFLs.
Two interesting applications of this technique
are the results that P =NP iff P is closed un-
der nonerasing homomorphism iff NTll1E(n) ~ P
[Book,1972J and the construction of various complex-
ity classes from the regular sets via homomorphic
replication: the family of r.e. languages (respec-
tively BNP, NP) is the least family of languages
containing the regular sets and closed under inter-
section and arbitrary (respectively, linear-erasing
in
and polynomial-erasing) homomorphic replication
[Book,1977J.
Applications: General Reduction Theorems
in
Results of this nature say that, under general
circumstances, a certain property is undecidable
or rW-hard, etc., or that undecidability of one
problem implies undecidability of another. For
example, Hartmanis [1967J showed that, for a mono-
In
tonically increasing function Sen) ~ log n , if a
.~ property A dichotomizes DSPACE(S(n)) but not
Ule f arnily of regular' languages, then A is unde-
cidable for S(n)-space bounded Turing machines.
If a family of languages ff is effectively
closed under union and concatenation by regular
sets and has an undecidable universe problem and
A is any property defined on !L ,which is true
for all reguJ_ar sets, false for at least one lan-
guage in .P, and is preserved by inverse gsm,
union with {e} and intersection with regular sets,
then A is undecidable for 5f [Greibach,1968bJ.
Hopcroft and Ullman D.968aJ showed that the
emptiness and membership problems for various
classes of balloon automata are related: for ex-
ample, emptiness is decidable for IDEA if
and only iff membership is decidable for 2NEA.
Hopcroft [1969J showed that "L = R" is undecidable
for L context-free and R any fixed unbounded
regular set; Hartmanis and Hopcroft [1970J showed
that containment, equivalence and the universe
problem are undecidable for any family of recursive
sets containing the AFL generated by {anlfl I n ~ l}.
82
Grammar Forms
The development of AFL theory was initially
machine oriented. The attempt to complete the
usual triple of languages, machines and grlarrrrnars
via grammar forms has initiated a large body of lit-
eratl~e, which it is too early to assess. The
basic idea is to use a graTIlfIlC.lr as a template and
consider which grLlJIlllUrs can b2 obtained from it in
certain nat~a1 ways. A sample r1esul t : the f am-
ily of one counter 1anVlages does not correspond
to a grammatical family of this type; this agrees
with one's intuition. Basic results can be found
in Cremers and Gin~:;buY'g [1974,1975].
9. Conclusion
He:. have seen how formal lal'1guLlgc theory pe-
ceived much of its impetus from the machine trans-
lation projects of the 50s and early 60s and asso-
ciated work in mathematical linguistics, how the
systems used had the same matheTTBtical model as
teclmiques used in specifying and translating prob-
lem oriented prograrruning languages (notably, AII;OL) ,
how the Chomsky hierarchy of languages, machines
and grarrmars was developed and completed and how
the subject became part of theoretical computer
science around 1964. We examined a few of the di-
rections formal language theory has taken since
1964, in general paralleling the development of the
rest of theoretical computer science, inspired by
the same questions (parsing and tr anslating, algo-
1
rithms and computational complexity, etc.)
What next? Some of the famous (or infamous)
problems posed at the start remain open (dpua
equivalence, the Iba problem) and still contri-
bute to the development of the subject. Subsequent
work has raised nev.l questions" some of which have
been solved (DOL equivalence), some not (P vs. NP).
At tirr~s, there has been a tendency to regal~ at
least part of the subj ect as pari~ of pure mathernat-
ic~) and to treat it in this way. I think that is
a bad idea (although SOm2 of my best theorems are
mathematical). I hope that I have shown that for-
mal language theory is an integral and important
part of theoretical computer science, and should
reJTI.ain so.
BIBLIOGRAPHY
Items are ordered first clu-'Onologically by
year and then alphabetically by first author. A
~itation [Squiggle,1984aJ means that the reference
is the first item by Squiggle listed for 1984.
Some of the papers listed b210w may not be cited
(I hope not vice versa).
For FOCS and SIGACT papers~ I have tried to
list the item under the Symposium date and include
iTl the listing other incarnations of the paper'; I
probably have missed several. I have referenced
each Proceedings in full only for the first list-
ing; thereafter using the abbreviation "ACM Symp"
or "IEEE Symp", as appropriate. Other papers may
be listed under the date of appearance of either
the progress report" thesis, conference or pub-
lished version, not in a consistent fashion. I
83
have alsoirlcluded a few of the influential books
on fonnal languages wid related topics.
A general apology is due. Time and space did
not me to list all or even most of the im-
portant papers, so I have chosen instead to offend
everyone by omitting at least one significant con-
tribution of each author (including myself).
.7936
Post, [.L., Finite combinatory processes - Formula-
tion 1., ~lo?Arriat of Symbolic Logic., 1 (1936 )103-105.
Turing, A.r·1., On computable numbers, with an appli-
cai=ion to the Entscheidurlgsproblem, Proceedings
of the London Mathemat'ieaZ Society., Ser.2, vol.42
(1936-37) 230-265; Correction, ibid., 43 (1937)
544-546.
1942
C.L., Outl1:ne of L1~ngu1.:stiG
Society of America at the
h7averlv Press, 1942.
1943
HcCu110ch, H.S. and Pitts, E. A logical calculus
of the ideas irmna.nent in nervous activity, BuZZ-
(.IF MaJ}zcrna i ~>'(J 7 5 (1943) 115-133.
Post, E. L., For'TnaJ reductions of the ,o;eneral com-
binatorial decision problem, American dOUl)nal of
Mathematics., 65 (1943) 197-215.
Zg46
Bloch, B., Studies in colloquial Japarlese ,II,
Language., 22 (1946) 200-2 1t8.
Harris, Z., Fr'om morpheme 1.:0 utterance., Language.,
22 (1946) 161-183.
Post., E.L." A variant of a recursively unsolvable
problenl, BulZ.Am.Math.Soe . ., .52 (194·6) 26 L+-268.
Post., E.L., ReCtlrsivc unsolvability of a problem
of 111ue, ~I. Symbo Z·z:c~ LO(Jic., 12 (1947) 1-11.
\0ells., R.:=., Immediate constituents, Language., 23
(1947) ()1-ll7.
195]
Harris ~., Me thods in 8tnu:t ura l Lingui-s tics.,
'I
IJniv.Chicago Press, 1951.
}'=leene, S. c. Rep:t\?sentation of events in nerve
nct~~ and . .ta, ProJect Rand Res. Memn.
3utoTYl.3
R~L?04_, Santa Bonica, CA (Dec. IS ,1951); published
in Automata S't;udieD (Sharmon, C.D. and i1cCapthy,
lJ., eds) fJr"'incton Uni v . P:cess, Princeton, NJ
(195FJ) pp 3-42.
Lukasie\tJicz, IT • ., /irisro+lp'8 Sylto[J'Z:stic from the
Standpo1>z t of ModeP'r2 Porma l Logic., Oxford Univ .
Press, 1951.
1952
Harris, Z., Discourse analysis, Language., 28 (1952)
1-30.
1953
Bar-Hillel, Y., A quasi-arithmetical notation for
syntactic description, Language., 29 (1953)
47-58.
1J54
VSlS elf a
notation"
tion~ (1954) 53-57.
1.956
Chomsky, N., Three models for the description of
language, 1956 Symp. TheoY'L! ~ MIT~
September 1956; published IRE £1'ans. I!zfo Y'-
mation Theory, IT-2 (1956) 113-124.
Shannon, C. E. and HCCill)thy" J. (eds.), AutomcIta
Studies~ Princeton Univ.Press, Princeton, NLT,1956.
1957
Chomsky, 1J., Si:l'uctures ~ Houton G Co. ,
'S-Gravenhat:e, 1957.
Harris, Z., Co-occurrence and transformation in
linguistic structure., Language~ 33 (1957)283-340.
t"1vhill" lJ., Finite automata and the reDresentation
of events., 1i,4DC Tech.Rept. ~ 52-624 (1957) 112-137.
1'Jewel1 'I A. 1 Emd Sha,,'1., lJ. C. ., Programming the logic
theory machine 1 Pl)OC. lies tern d oint Computer Conf.
Los Angetes~ e4-, Feb.7957~ pp. 230-240.
1958
Chomsky, A.r,J., Some properties of phrase structure
grammars., MIT Res.Lab. of EZectponics~ .Prog.
Rept.-, 49 (April 1958) 108-111.
Chomsky, A.lT. and I'·1iller, G.A., Finite state lan-
guac:;es, Information (5 Control~ 1 (1958) 91-112.
Copi, I. 1'1., El£ot, C. and Vlright, \J. B., Realization
of events by logical nets, d.Association Computing
Machinery.' 5 (1958) 181-196.
Davis, H., Computabi and Unso lvaln~ l i ty.' r-1cGraw-
Hill, NY, 1958.
Nerode, A., Lineae autoInCiton transformation., Ppoc.
Am8 Y1 iean !vJczth. Soc1:ety-, g (1958) 541-544.
1959
Backus, J.W . ., The syntax and semantics of the pro-
posed international alGebraic language of the
ZtJr10h At:M-('~ t:onfpP?!lcP) Dy'(~C-. Tnt? ('onf.
Inform. Processing.' !JNESCO PaYil's" 195.0, pp 125-132.
Chomsky, 1'J., On certain forma.l properties of V'am-
mars, Inform.Contr.~ (1959) 137-167.
Chomsky, N., A note on phrase-structure grammar's"
Inform.Contr . .' 2 (1959) 393-395.
Rabin, M. O. and Scott, D. '\ Finite automata and
their decision Droblems, IBM J.Reseapch Devel-
opment.' 3 (1959) 114-125.
Shepherdson , J. C. ,The reduction of two-way autoInClta
to one-way automata, IBM J.Res.& Dev.~ 3 (1959)
198-200.
1960
Bar- Hille1, Y., Gaifman, H. and Shamir., E., On
categorial and phrase structure gramrrB.l's, Bu l Z•
Res.Council Israel~ 9 (1960) 155-166.
Bossert, W., The implementation of predictive anal-
ysis, Rept. No. NSF-4~ Math.Linguistics and Auto-
matic Translation.' Harvard Comput.Lab. ,June 1960.
:L:.-,ljaur;hton, R. and Yamada, H., Pep:ular expressions
and state p:raphs for automata" IHE-P(;EC.' 9 (1960)
39-47.
Ilyhill, J, Linear bounded automata, Tv'ADC Tech. Note
6-.165 (1960).
lJaur,P. (ed.), Report on the algorithmic languaGe
ALGOL 60, CACM 3 (1960) 299-314.
y
Samelson, K. and Bauer, F.L., Sequential formula
translation, CACM.' 3 (1960) 76-83.
Scheinberg, S. ., Note on the .Boolean properties of
context-free languages, Inform.Contr . .'3 (1960)
372-375.
Sherry, M. E., Syntactic Analusis i.n Automatic Trans-
lation, Ph.D. thesis, Harvard, August 1960, as
Rept. No. NSF-5.' Math.Ling.Autom.Transl . .'Harvard
Computation Laboratory, Cambridge, Mass.
1961
~ Bar-Hillel, Y., Perles, M. and Shamir, E., On for-
mal properties of simple phrase sturucture gram-
mars., Z.Phonetik.Sprach.Komm . .' 14 (1961) 143-179.
Chomsky, N., On the notion "Rule of Grammar 11 , in
StYiucture of Language and Its Mathematical As-
pects.' Proc.Sym.Appl.Math.' 12 (1961) 6-24, Amer.
t1ath. Soc.
Floyd" R.\'J., A note on mathematical induction on
phrase structure grammars, Inform.Contr . .' 4
(1961) 353-358.
Hays, D. G., Groupins; and dependency theories, in
Proc.Natl.Symp. Machine Translation (H.P. Ed-
mundson, ed.) (1961) 258-66; based on RM'ID Memo.
P-19190, Feb. 1960.
Gaifman, H., Dependency systems and phrase struc-
ture systems, RM'ID MeITD P-2315 (1961) j published
in Inform.ContYi . .' 8 (1965) 304-337.
Greibach, S., Iterations for the syntactic analy-
zer, Rept. No. NSF-6.' Math.Ling.Auto.Trans . .'
Earvard Comput.Lab., Cambridge, Mass. June 1961.
Irons., E.T . ., A syntax directed compiler for ALGOL
60., CACM.' 4 (1961) 51-54.
Hinsky, ti. L . ., Recursive unsolvabilitv of Post's
Drob1em of "Tag\! and other topics in the theory
of Turing machmes, Ann.Math. ~ 74 (1961) 437-354.
OettinBer, A.~., Automatic syntactic analysis and
the Dushdo~ store, in Struc.Lang.Math.Aspects.'
PYioc.Symp.Appl.Math . .' 12 (1961) 104-129, Amer.
Math. Soc.
Ott., G. and Feinstein, n.H., Design of sequential
machines from their regular expressions, JACM,
8 (1961) 585-600.
Parikh, R. J ., Language-generating devices, MIT
Res.Lab.Electr . .' QPR # 60 (1961) 199-122~ pub-
lished as "On context-free languages", JACM.' 13
(1966) 570-581.
Rhodes, I., A new approach to the mechanical syn-
tactic analysis of Russian, Mech.1~ans. .' 6
(1961) 33-50; publ.version of 1959 NBS Report.
84
Sakai, I. , Syntax in lli1iversal translation., Proc.
1961 Intl.Conf.Mach.Trans.Langs.App~Lang.Anal.~
Her Majesty's Stationery Office, London, 1962.
Yamada, H., Real-time computation and recursive
functions not real-time computable, IRE Trans.
Comput.~ EC-10 (1961) 753-760.
Yngve, V., A model and a hypothesis for language
structure, Proc.Amer.Phil.Soc.~ 104 (1961) 444-466.
1962
Land\veber'l P. S. 'l Three thcor ems on phrase struc-
Greibach, S., Inverses of Phrase Structure Genera-
tors~ Ph.D. thesis, Harvard Univ., June 1963~
also Rent. NSF-II, Math. L1~nq.Autom. Transl. -' Har-
vard Comput. Lab., Cambrirl.fYe·, Mass. Publ. as 'l "The
undecidability of +-he anmig;uity problem for mini-
mal linear grarnrnars;l In.form.Contr.~ 6 (1963) 119-
125: "ForJn3.1 parsin,a.: systems,n CACM~ 7 (1964)
499-50L~, (md "A new nOrrrB1-form theorem for con-
tehi=-free phrase :=ctructtwe p;rammars," ~TACM-, .7,8
(1965) L+2-52.
1

Cantor, D.G., On the ambiguity problem of Backus

ture gr\liTlJTBrS of Type 1, Trz~form.Contxl':J h (1963)
systems, JACM~ 9 (1962) 477-479.
131-1~~E, .
Chomsky, N., Context-free grammars and pushdown
1\latthews'l G. H. ,Discontinuity and as'ymmetry in phrase
storage, MIT-Res.Lab.El.~ QPR # 65 (1962) 187-194.
stcuc ture rrciJl1ITli.lrS, Info /ltn. ('ern tl)., B (19tl3) 137-
Ginsburg, S., and Rice, H. G., Two families of lan- 11-+6.
guages related to ALGOL, JACM~ 9 (1962) 350-371.
n.p. ~ On context-free languaGes
Greibach, S., Forma.l systems in linguistics., RAND autoITBta, Infollm. C()'rztY'. ~ 6 (196])
Doc. 9143-PR~ RN~ Corp., SM, CA, Sept 1962.
Gross, M., On the equivalence of models of lan~uage 1964
used in the fields of mechanical translation and
B.3.r-I1il1el., Y. (ed.), L(JJL(!UO~]e lInd Inf"ormat1:on~
information retrieval, preliminary draft 1962 ~l
/\rldison qp~;lt=~v., 'l rt3.S,C;. 'l 19tJL~.
publ. Inform.Stor.Retr.~ 2 (1964) L~3-57.
Hays, D.G., Automatic lan~uage data processing, in
ro 1(=', S. n. ~ Fir)u.! -1j1'irnc 7il7 I i;('YlOt1:VP
/1pY'ays F/n'z'te,-,S~/otp
,'I 'l Ph.D. thesis,
Computer Applications in the Behavioral Sciences,
Hcll'varci Univ., Au,gus t 1 (jb4: (-]] so P,ept. no. BL-36 'l
Prentice-Hall,lJJ, 1962 (Chapter 7).
Theor-y of 5'71,-i reJu'n(l, Harvard Comnut. Lab, Cambridge.
Klli1o, S. and Oettinger, A.G., MUltinle-path syntac-
Cinsburg, S. and Spanier., ~.I1., Rotmded ALGOL-like
tic analyzer, in Information Processin9~ 19B2:J
langua,q:es., Trans.Am.Math. Soc. ~ 113 (1964) 333-:368.
North Holland Publ. Co . ., ~lT., 1963.
I Iartmanis 'l lJ. <.=md Stearns., 1\. T~., Computational com-
Robinson, J., Preliminary codes and rules for the
plexitv of recursive sequence., Proe.IEEE Symp.
automatic parsing of English, RAND Memo RM-3339-
~:;Zl'. r-'ire.,'h. ,~Lo9' Des. ~ PI-'-z'nceton~ NrI~ Nov. 1964~
PR~ ~JD Corp., Sa.nta t10nica, CA, lJec. 1962.
PI! 82-CJlJ: also see, I10n the computational com-
1.963 plexitv of algorithms, 11 TJ)ans. Am. ,l'vlath. Soc. -,117
(1965) 285-306.
Banerji, R.B., Phrase-structure lanEUages, finite
machines and charmel capacity, Inform.Contr., 6 Kurcxia, S. Y., Classes of languar-,es and linear-
(1963) 153-162. bOlli1ded automata, Inform. Corztjl. ~ 7 (196 L+) 207-223.
Chomsky, J'J., Formal properties of grarrunars, in Hand- r1oore, I.F. (ed.), Mzchines: Selected
hook of Mathematical Psychology~ vol II (Luce, R. PapCl)S, Addison \vesley., ReadinG'l Mass., 1964.
D., Bush, R.R. and Galanter, E . ., eds) \'Ji1e'} E
'l /\. C. O. (eel.)., No Les fe)}) the Harvar'd
Sons, NY, 1963 (Chapter 12).
Summer- School Program on Language Data Processing,
Chomsky, J' J. and Schut zenberger, M. P., TI1e algebraic Harvard Comput. La.b. , Aup;u~;t 19b Ji.
theory of context-free languages, in Computer PYIO-
Eahin'l H. 0 . ., Eea1tirne ., IsY'cze l rT. Math. ~
gramming and Formal SYDtems (Braffort., P. and 1 (1974) 203-211.
Hirschberg, D., eds) Horth-Holland, nT., 1963.
Rosenberg., A.l, . ., On n-tape finite state acceptors,
Evey, R.J., The Theory and Applications of Pushdown
IEEE Symp. ~ .76-81:, taken from Nonwriting Ex-
Store Machines~ Ph.D. thesis., Harvard Univ . ., May
fi?!'7,~1'n)!8 OT 4u.torn.ato-> Ph.D. thesis., Har-
1~b3~ also Rept.No. NS}'-lU, Math.L1:ng.Jlutom.'f'1-ar?s.
vard Un iv., AtU;ust 19 G5 ~ also see Fischer, P. C.
Harvard Comput. Lab. 'l Cambridge. 'l J'1ass.
,_Hill lljll'fJ" A.L., one-way nonwr'itiIlg
Fischer'l P. C., On computability by certain classe~; aut riFk3ta, (1968) 88-101.
of restricted Turing machines, Proc.IPEE Symp.
1965
Circuit and Logical Design~ Chi-
cag()~ Sept 1963~ pp. : publ. as PTurins; /~l)raham., S. ~ Some (Juestions of nhl"nse structure
machines with restricted memory access" 'l InfoYlm. ,craI'lmars, ('ornp. LlYig. ~ 4 (1965") 61- 70.
ContI')., 9 (i966) 364-379.
~ E. ar}~ Bclrl(.=:r·
1
., ~) • ., Some T)roblems of f inite
Floyd, R.\'I., Syntactic analysis and operator pre- , In TOl-m. Contr. ~ 8 (1965) 351-363.
cedence, ~IACl4~ 10 (1963) 316-333.
, S. and Creibach, S., Deterministic con-
Ginsburg, S. and Rose, G.F., Operations which pre- text free langua~es., Proc.IEEE Symp.Sw.Circ.Th.
serve definabi1ity in languages, JACM:J 10 (1963) 8 Log.Des.~ Univ. Michigan~ Ann Arbor~ Oct.lg65~
29-47. pp 203-220; publ. in Inform.Contr.~9 (1966)620-648.
Ginsburg, S. and Spanier, E.H., Quotients of context- Haines, L.G., Generation and recognition of formal
free languages, JACM~ 10 (1963) 487-493. laJ?guages~ Ph.D. thesis, June 1965, MIT ,Cambridge.

85
Hennie, sequences and off-IDle
Till·ing rnachin.e computations., IEEE 8ynrp -' PP .168-
172 ~ pub1. as POne-tape off-line Till'"'ing machine
computations, Inform.Contr." 8 (1965) 553-578.
KasaJI1i, T . ., lm efficient recognition and syntax-
analysis algorithm for context-free languages,
Rept. No. 2-, Univ. Hau)aii Contr. # AF19(628)-
4379, 1965.
Knuth, D.E., On the translation of languages from
left to rip;ht, Inform.Cont-r.-, 8 (1965) 607-639.
leHis, F. M., Stearns., R. E. and Hartrnanis, J.,
~Iernory bounds for recognition of context-free
illld context-sensitive la~guages~ IEEE Symp.-,
pp.191-202.
Rosenberg, A.L., On multi-head finite automata,
IEEE Synp.~ pp.221-228; publ. in IBM J.Res.&
Dev.-, 10 (1966) 388-394.
Ruby, S.S. and Fischer, P.C., TranslaL~iona] methoj2,
and computational complexity, .IEEE Symp.-, pp.
173-178.
Stearns., R.S . ., Hartmanis., J. and Lewis, P.N., Hiep-
archies of memory limited computations., IEEE
Symp.-, pp.l79-190.
1966
Ginsburg, S., The Mathematical Theory of Context-
Free Languages-, McGraw-Hill., 1966.
Ginsburg, S. ., Greibach, S.A. and Harrison, M.A.,
One-way stack automata - Extended abstract, P-roc,
IEEE Symp. Switching and Automata Theory~ Berke-
Zey-, CA-, Oct. 1966~ pp.47-52~ publ. as "One-way
stack automata," ~TACM-,14 (1967) 389-418.
Ginsburg, S. and Spanier, E.B"! Semigroups, Pres-
burger formulas and languages" Pacific rI.Math.,
16 (1966) 285-29b.
Ginsburg, S. and Spanier, I.H., Finite-tt1rll push-
down automata, SIAM rJ. Cont-roZ~ 4 (1966) 429-453.
ellisburg, S. and Ul1ian, lJ. , Ambiguity irt context-
free lan~J)lages, JACM, 1.3 (1966) 62-89.
Greibach, S., The unsolvability of the recognition
of linear context-free languages., rJACM-, 13 (1966)
582-S87.
Korenjak., A.J-. and Hopcroft, ,J.E., Simple deter-
ministic languages, IEEE Symp.~ pp. 36-46.
Lewis, P.M. and Stearns, R.E., Syntax directed
transduction, IEEE Symp.~ pp.2l-35; publ. in
rIACM-, 15 (1968) 465-488.
Wirth, N. and Weber, H., EUl.EF., a generalization
of PlGOL and its formal definition CACM-, .9
'I
(1966) 13-25 and 88-89.
Younger, D.H., Context-free language processing in
time n 3 _, IEEE Symp. -' pp.7-20; publ. as "Recog-
nition and parsing of context-free lc3Il.guages in
time n 3 -' Inform. Contr. -' .10 (1967) J89-208.
196?
Aha, A. V., Indexed grarnrnar)s - 1m extens ion of
context-free grammars, Proe.IEEE Symp. S7.Jitch-
ing and Automata Theory~ Austin TX~ Oct.1967~
pp.21-31; publ.in JACM-, 15 (1968) 647-671; also as
TfNested stack automata, If ~TACM~ 16 (1969) 383-406.
86
Fischer, .C. and Meyer . A.P., Rea.] time counter
4"1chines - Preliminary version, TEEE Symp. -' pp.
~. 48 -15 4; an e>"'Panded version publ. a.s: Fischer,
P.C., Meyer', A.F.. and Rosenberg, A.L. "Collilter
'I
lTBchines and counter languages," Math.Syst.Th.~2
(1968) 265-283.
fischer, P.C., Meyer, A.R. and Rosenberg, A.L.,
Turing machll1es with several read-write heads,
IEEE Symp.-, pp.117-127; publ.as "Real-time sim-
ulation of multihead tape units.," JACM~ 19
(1972) 590-607.
Ginsburg, S. and Greibach, S., Abstract families
of languages, IEEE Symp . ., pp.128-245; publ. in
Studies in Abstract Families of Languages., (Gins-
burg, S., Greibach, S. and Hopcroft, J., eds),
Mem.Am.Math.Soc . ., No. 87 (1969) 1-32.
C:;insburg, S., Greibach, S.A. and Harrison, M.A.,
Stack automata and compiling, JACM (1967) 172-201.
(-;ray, J.N., Harrison, M.A. and Ibarra, O.H., Two-
way pushdown automata, Inform.Contr . ., 11 (1967)
30-70.
Greibach, S., An infinite hierarchy of context-free
languages, IEEE Symp.-, pp.32-366; publ. in JACM.,
16 (1969) 91-106.
Hartmanis, J., The complexity of ~~decidab1e prob-
lems in automata theory, IEEE Symp.-, pp.112-116;
publ. in JACM~ 16 (1969) 160-167.
HopcYDft, LT. E. and Ullman, J. D., An approach to a
wlified theory of automata, IEEE Symp. -' pp.140-
147~ publ. in Bell Syst.Tech.J.~ 46 (1967)
1793-1829.
Hopcroft, J. E. and Ullman, lJ. D. ., Nonerasing stack
automata, I':J.Comput.Syst.Sci.~ 1 (1967) 166-186.
Kasarni, T., A note on computing time for recog-
ni tion ot languages generated by linear gram-
mars., Infolm.C~ntr.-, 10 (1967) 209-214.
Knuth~ D.E., A characterization of parenthesis
languages, Inform.Contr.-, 11 (1967) 269-289.
McNaughton, P'O' Parenthesis gramnars, (JACM-, 14
(1967) 490-500.
Nivat, M., Transductions des langages de Chomsky-,
Ph.D. thesis, Grenoble Univ., 1967; pUbl. in
Ann.Inst.Fourier-, Grenoble., 18 (1968) 339-455.
Rogers, rI . ., Theory of Recursive Functions and
Effective Computability, McGraw-Hill, 1967.
Rosenberg, A. L., A machine realization of the
1L"'1ear context-free languages., InfoYWl. Cont-r. ~
10 (1967) 175-188.
Eosenberg, A.L., Peal-time definable languages,
~TACM, 14 ( 1967) 645 - 662 .
Fosenkrantz, D.lT., Prograrrnned grarrrrnars - A ne""
device for gener)ating formal languages, IEEE
8ymp.-, pp.14-20; publ. as "Programmed grammars
and classes of formal languages, fT JACM~ 16
(1969) 107-131.
Sintzoff, M., Existence of Vru"'1 Wijngaarden's
syntax for every' recursively erll.nnerable set,
Ann.Soc.Sci. de B~~xelles~ 2 (1967) 115-118.
Stearns, R. E . ., A regularity test for pushdown
machines, Inform.Contr.-, 11 (1967) 323-340.
1968
Fischer, M.J., Grammars with macro-like produc-
tions 3 Proc.IEEE.Symp.Switching and Automata
TheorY3 SchenectadY3 NY3 Oct.1968 3 pp.131-142
- Abstract of Ph.D. thesis, Harvard Univ., May
1968; also Rept. No. NSF-22 3 Math. Ling. Autom.
Trans l. ~ Harvard Computation Laboratory.
Ginsburg, S. and Hopcroft, J. E., Two-way balloon
automata and AFL, IEEE Symp.~ pp.292-297~ publ.
in JACM~ 17 (1970) 3-13.
Ginsburg, S. and Spanier, E.H., terivation-bounded
languages, IEEE Syrrrp. ~ pp. 306-314 ~ publ. in JCSS~
2 (1968) 228-250.
Ginsburg, S. and Spanier, E.H., Control sets on
grammars, Math.Syst.Th. , 2 (1968) 159-178.
Greibach, S., Checking automata and one-way stack
languages, IEEE Symp.~ pp.287-291; publ. in
J.Comput.Syst.Sci.~ 3 (1969) 196-217.
Greibach, S., A note on undecidable properties of
formal languages, Math.Syst.Th. ~ 2 (1968) 1-6.
Hopcroft, J. E. and Ullman, J. D., Decidable and un-
decidable questions about automata, JACM~ 15
(1968) 317-324.
Hopcroft, J. E. and Ullman, J. D., Relations between
time and tape complexities, JACM~ 15 (1968)
414-427.
Lindenmeyer, A., Mathematical models for cellular
interactions in development, I and II, J.Theor-
etical Biology~ 18 (1968) 280-315.
Ogden, W., A helpful result for proving inherent
ambiguity, Math.Syst.Th.~ 2 (1968) 191-194.
1969
Aho, A.V. and Ullman, J.D., Translations on a con-
text-free gramnar, Proc.ACM Symp. Theory of Com-
puting~ Marina del Rey~ CA~ May 1969, pp.93-112;
publ.in Inform.Contr.~ 19 (1971) 439-475.
Aho, A.V. and Ullman, J.D., A characterization of
two-way deterministic classes of languages, Proc.
IEEE Symp.Switching and Automata Theory~ Water-
l003 Ont.~ Oct. 1969~ pp.149-156; publ. in
J.Comput.Syst.Sci.~ 4 (1970) 523-538.
Book,R. V. and Greibach, S.A., Quasi-realtime lan-
guages - Extended abstract, ACM Symp . ., pp.15-18;
publ. as "Quasirealti.m2 languages," Math. Syst.
Th.~ 4 (1970) 97-111.
Chandler, W.J., Abstract families of deterministic
languages, ACM Symp. ~ pp. 21-30.
Chastellier, G. de and Colmerauer, A., W-grarrrrna.r,
Proc.ACM Natl.Conf.~ (1969) pp.5l1-Sl8.
Cook, S.A., Variations on pushdown maChines, ACM
SYrrrp.~ pp.229-232; publ. as "Characterizations
of pushdown ITBchines in terms of time-bounded
computers," JACM, 18 (1971) 4-18.
teRemer, F. L., Generating parsers for BNF gram-
mars, Proc. 1969 Spring Joint Computer Conf.~
AFIPS Press, Montvale, NJ, 1969, pp.793-799.
Fischer, M.J., Some properties of precedence lan-
guages, ACM Symp.~ pp.18l-l90.
87
Fischer, M.J., Two characterizationsof the context-
sensitive languages, IEEE Symp.~ pp.149-l56.
Ginsburg, S., Greibach, S. and Hopcroft, J., (eds.)
Studies in Abstract Families of Languages~ Mem.
Am.Math.Soc. No. 87~ 1969.
Greibach, S. and Hopcroft, J.E., Scattered context
grammars, J.Comput.Syst.Sci.~ 3 (1969) 233-247.
Greibach, S., Full AFLs and nested iterated substi-
tution, IEEE Syrrrp.~ pp.222-230~ publ. in Inform.
Contr.~ 16 (1970) 7-35.
Hopcroft, J.E., On the equivalence and containment
problems for context-free languages, Math.Syst.Th.~
3 (1969) 119-124.
Hopcroft, J.E. and Ullman, J.D., Formal Languages
and Their Relation to Automata~ Addison-Wesley,1969.
Ogden, W., Intercalation theorems for stacK lan-
guages, ACM Symp.~ pp.3l-42.
Rosenkrantz, D.J. and Stearns, R.E., Properties of
deterministic top down grarrrrnars, ACM Symp. pp. 3
165-180; publ. in Inform.Contr.~ 17 (1970) 226-255.
Rounds, W.C., Context-free grammars on trees, ACM
Symp. ~ pp . 143-148 ~ publ. as "Mapping and grarrrrnars
on trees," Math.Syst.Th.~ 4 (1970) 257-287.
Savitch, W.J., Deterministic simulation of non-
detenninistic Turing machines, ACM Symp. ~ pp.
247-248~ publ. as "Relationships between non-
deterministic and deterministic tape complexi-
ties." J.Comput.Syst.Sci.~ 4 (1970) 177-192.
van Wijngaarden, A. (ed.), Report on the algorith-
mic language ALGOL 68, Numerische Mathmatik~ 14
(1969) 79-218.
1970
Book, R.V., Greibach, S.A. and Wegbreit, B., Tape-
and time-1:xJunded Turing acceptors and AFLs, Proc.
ACM Symp. Theory of Computing~ Northampton 3Mass . .,
May 1970, pp.92-99; publ. in JCSS~ 5 (1971) 606-
621~ also see Book, R.V., Greibach, S., Ibarra ,0.
and Wegbreit, B., Tape-bounded Turing acceptors
and principal AFLs, JCSS~ 5 (1971) 622-625.
Early, J., An efficient context-free parsing algo-
rithm, CACM~ 13 (1970) 94-102.
Ginsburg, S. and Greibach, S., Principal AFLs ,
JCSS~ 4 (1970) 308-338.
Graham, S., Extended precedence languages, bounded
right context languages and deterministic lan-
guage s (extended abstract), Proe. IEEE Syrrrp.
Switching and Automata TheorY3 Santa Monica~ CA 3
Oct. 1970~ pp.175-180; abstract of Precedence
Languages and Bounded Right-Context Languages~
Ph.D. thesis, Stanford Univ., 1971; publ. as
"On bounded right-context languages and grarrrrnars,"
SIAM J.Comput.~ 3 (1974) 224-254.
Greibach, S.A., Chains of full AFLs, Math.Syst.
Th.~ 4 (1970) 231-242.
Hartmanis, J. and Hopcroft, J. E., What makes some
language theory problems undecidable , J. Comput.
Syst.Sci.~ 4 (1970) 368-376.
Salomaa, A., Periodically time-variant context-free
grammars, Inform.Contr., 17 (1970) 294-311.
1971
Boasson , L., till iteration theorem for one-counter
languages, Proc.ACM Symp. Theory of Computing~
Shaker Hgt8.~ Ohio~ pp.116-120~ pub1. as "Two
iteration theorems for sorre families of lan-
guages, JCSS~ 7 (1973) 583-596; also see Cones
rationnels et familIes agreables de langages
- Application au langage a compteur~ Ph.D. thesis,
Univ. Paris, 1971.
Book, R. V. and Wegbreit, B., A note on AFLs and
bounded erasing, Inform.Contr.~ 19 (1971) 18-29.
Cook, S.A., The complexity of theorem-proving pro-
cedures, ACM Symp.~ pp.15l-l58.
Cook, S.A., Linear time simulation of deterministic
two-way pushdown automata, Proc.IFIP Congress'71~
North-Holland, Pmsterdam, #TA-2, pp.172-l79.
DeRemer, r.L., Simple LR(k) grammars, CACM~
(1971) 454-460.
Meyer, A.R. and Fischer, M.J., Economy of descrip-
~ion by automata, grarrrrnars and formal systems,
Proc.IEEE Symp.Switching and Automata Theory~
East Lansing~ Mich.~ Oct. 1971~ pp.188-l91.
Rajlich, V., Absolutely parallel gramnars and two
way deterministic finite-state transducers, ACM
Symp.~ pp.132-l37~ publ. in JCSS (1972) 324-342.
Rozenberg, G. and Doucet, P.G., On OL-languages,
Inform.Contr.~ 19 (1971) 302-318.
1972
Aho, A.V. and Ullman, J.D., The Theory of Poysing,
Translation, and Compiling, Prentice-Hall: Vol.I:
Parsing, 1972; Vol. II: Compiling~ 1973.
B3k:er, B. and Pook, R. V. , Reversal-bounded multi-
pushdown machines, Proc.IEEE Symp. switching
and Automata Theory, Univ.Maryland, Oct.1972~
pp.207-211; publ. in JCSS, 8 (1974) 315-332.
Book, R. V., On languages accepted in polynomial
time, SIAM J.Comput., 1 (1972) 281-287.
Cook, S.A., A hierarchy for nondeterministic time
complexity, Proc.ACM Symp. Theory of Computing~
Denver~ CO, May 1972, pp.187-192.
Ginsburg, S., Greibach, S. and Goldstine, J., Uni-
formly erasable AFLs, ACM Symp., pp.207-214;
publ. in JCSS, 10 (1975) 165-182.
Goldstine, J. , Substitution and bounded languages,
JCSS, 6 (1972) 9-29.
Greibach, S., Syntactic operators on full semi-
AFLs, JCSS, 6 (1972) 30-76.
Greibach, S. and Ginsburg, S., Multitape MAs,
JACM, 19 (1972) 193-221.
Harrison, M.A. and Havel, I.M., Real-time strict
detenninistic languages, SIAM J. Comput., 1
(1972) 333-349.
Ibarra, 0., A'note concerning nondeterministic
tape complexities, JACM, 19 (1972) 608-612.
Karp, R.M., Reducibility aIIDng combinatorial prob-
lems, in Complexity of Computations (Miller,
R.E. and Thatcher, J.W., eds) Plenum Press, NY,
1971, pp.85-l04.
Lindenmeyer, A. and Rozenberg, G., Developmental
systems and languages, ACM Symp., pp.2l4-221;
publ. in part as "Developmental systems with
locally catenative formulas," Acta Informatica,
2 (1973) 214-248.
Meyer, A.R. and Stockmeyer, L.J., The equivalence
problem for regular expressions with squaring re-
quires exponential space, IEEE Symp.,pp.125-129.
1973
Baker, B., Tree transductions and families of tree
languages, Proc.ACM Symp.Theory of Computing,
Austin, TX, May 1973~ pp.200-206; Ph.D. thesis,
Harvard Univ., 1973.
Bird, M., The equivalence problem for deterministic
two-tape automata, JCSS, 7 (1973) 218-236.
Blattner, M., The unsolvability of the equality
14
problem for sentential forms of context-free
grammars, JCSS, 7 (1973) 463-468.
Greibach, S .A., Jump PDAs, deterministic context-
free languages, principal AFDLs and polynomial
time recognition, ACM Symp., pp.20-29; publ. as
"The hardest context-free language," SIAM J. Com-
put.~ 2 (1973) 304-310, and as "Jump PDAs and
hierarchies of deterministic context-free lan-
guages," SIAM J.Comput., 3 (1974) 111-127.
Harrison, M.A. and Havel, I.M., Strict determinis-
tic gramnars, JCSS~ 7 (1973) 237-277.
Hunt, H., On the time and tape complexity of lan-
guages, ACM Symp., pp.10-19.
Rounds, W... Complexity of recognition in inter-
mediate-level languages, Froc. IEEE Symp. Switch-
ing and Automata Theory, Univ.lowa~ Oct. 1973,
pp. 145-158.
Rozenberg, G., TOL systems and languages, Inform.
Contr. 3 23 (1973) 357-381.
Rozenberg, G., Extensions of tabled OL-systems and
languages, Int.J.Comput.Inf.Sci.~ 7 (1973) 463-468.
Salornaa, A., Formal Languages, Academic Press, 1973.
Seiferas, J.I., Fischer, M.J. and Meyer, A.R.,
Refinements of the nondeterministic time and
space hierarchies, IEEE Syrrrp.~ pp.130-l37; publ.
as: Seiferas, J., "Techniques for separating space
complexity c1asses;'JCSS, 14 (1977) 73-99; and:
Seiferas, J., "Relating refined space complexity
c1asses,"JCSS~ 14 (1977) 100-129; also see Sei-
feras, J.I., Fischer, M.J. and Meyer, A.R., "Sep-
arating deterministic time complexity classes,"
JACM~ 24 (1978) 146-167.
Sudborough, I.H., On tape-rounded complexity classes
and multihead finite automata, IEEE Symp. -' pp.138-
144; publ. in JCSS~ 10 (1975) 62-76.
1974
Aanderaa, S.O., On k-tape versus (k-l)-tape real time
computation, in Complexity of Computation rSIAM-AMS
Proc.~ vol.7) (Karp, R.M.,ed.) Amer.Math.Soc, Prov-
idence, RI, 1974, pp. 75-96.
Pillo, A. V., Hopcroft, J .E. amd Ullman, J.D., The
Design and Analysis of Computer Algorithms,
Addison-Wesley, 1974.
88
Eook, F..V., Nivat, M. and Paterson, M, Intersec-
tions of linear context-free languages and re-
versal botmded multipushdown machines, Proc.
ACM Symp.Theory of Computing~ Seattle~ Wash.~
May 1974~ pp.290-292~ publ. as "Reversal-
bounded acceptors and intersections of linear
languages;' SIAM J.Comput.~ 3 (1974) 283-295.
Cremers, A. and Ginsburg, S., Characterization
of context-free grarrnnatical families, Proc.
IEEE Symp. Switching and Automata Theory ~ Neu)
Orleans~ LA~ Oct. 1974~ pp.199-204; see ver-
sion pool as: Cremers, A., Ginsburg, S., and
Spanier, E.H., "The structure of context-free
grammatical families," JCSS~ 15 (1977) 262-279.
Friedman, E. P., Relationships between monadic
recursion schemes and deterministic context-
free languages, IEEE Symp.~ pp.43-51; publ.
as "Equivalence problems for deterministic
context-free languages and rronadic recursion
schemes,"JCSS~ 14 (1977) 344-359.
Greibach, S.A., Sone restrictions on W-gramnars,
ACM Symp.~ pp.256-265; publ. in Intl.J.Com-
puter Inform.Sci.~ 3 (1974) 289-327.
Rozenberg, G. and Salomaa, A., L Systems~ Lect.
Notes in Comput.Sci., No. 15, Springer-Verlag
1974.
Valiant, L.G., The decidability of equivalence
for determinist~finite-turnpushdown auto-
mata, ACM Symp.~ pp.27-32; part of Decision
Procedures for Families of Deterministic
Pushdown Automata~ Ph.D. thesis, Univ.Warwick,
UK, July 1973~ publ. in Inform. Contr. ~ 25
(1975) 123-133.
Walljasper, S.J., Left-derivation rounded lan-
guages, JCSS~ 8 (1974) 1-7.
1975
Beeri, C., An improvement on Valiant's decision
procedure for equivalence of deterministic
finite-turn pushdown automata, Froc. IEEE Symp.
Berkeley~ CA~ Oct. 1975, pp.128-l34; pOOl.in
Theor.Comput.Sci.~ 3 (1976) 305-320.
Cremers, A. and Ginsburg, S., Context-free gram-
mar forms, JCSS~ 11 (1975) 86-117.
Ehrenfeucht, A. and Rozenberg, G., On the (com-
binatorial) structure of L languages without
interactions (extended abstract), Froc.ACM
Symp.Theory of Computing~ Albuquerque~ NM~
May 1975~ pp.137-144.
Engelfriet, J., Bottom-up and top-down tree
transformations - A comparison, Math.Syst.Th.~
9 (1975) 198-231.
Ginsburg, S., Algebraic and Automata-Theoretic
Properties of Formal Languages~ North-Holland,
1975.
Ginsburg, S. and Lynch, N., Comparative com-
plexity of graJIUIBr forms, ACM Symp. pp. 153-
158; see "Size complexity in context-free
grarrrrnar forms," JACM~ 23 (1976) 582-598 and
"Derivation complexity in context-free gram-
mar forms," SIAM J.Comput.~ 6 (1977) 123-138.
Herman, G.T. and Rozenberg, G., Developmental
Systems and Languages~ North-Holland, 1975.
89
Hopcroft, J., Paul, W. and Valiant, L., On time
versus space and related problems, IEEE Symp.~
pp.57-64.
Hunt, H. and Rangel, LT.L., Decidability of equi-
valence, containment, intersection and separ-
ability of context-free languages, IEEE Symp.
pp.144-150.
Hunt, H.B. and Szymanski, T.G., On the complexi-
ty of grarrmar and related problems, ACM Symp. ~
pp.54-65~ publ. as "Complexity metatheorems
for context-free grammar problems," JCSS~ 13
(1976) 318-334.
Perrault, C.R., Intercalation theorems for tree
transducer languages, ACM Symp.~ pp.126-136;
publ. in JCSS~ 13 (1976) 318-334.
Wotschke, D., Degree-languages, polynomial time
recognition and the L&\ problem, ACM Symp~pp.
145-152; publ. as "Degree-languages, a new
concept of acceptance," JCSS~ 14 (1977) 187-
209. (Part of Boolean Operators on Languages~
Ph.D. thesis, UCLA, 1974.)
Valiant, L., General context free recognition in
less than cubic time, JCSS~ 10 (1975) 308-315.
1976
Arora, A. and Sudborough, I. H., On languages
log-tape reducible to context-free languages,
Proc. 1976 Conf.Information Science & Systems~
BQltimore~ MD~ pp.27-32.
Boasson , L. and Nivat, M., Parenthesis genera-
tors, Proc. IEEE Symp.Foundations of Computer
Science~ Houston~ TX~ Oct. 1976~ pp.67-70.
Chandra, A.K. and Stockmeyer, L.J., Alternation,
IEEE Symp. ~ pp. 98-108.
Friedman, I.P., The inclusion problem for simple
languages, Th.Comput.Sci.~ 1 (1976) 297-316.
Kozen, D., On parallelism in Turing rna.chines, IEEE
Symp.~ pp.89-97.
Sudborough, I.H., On deterministic context-free
languages, Imlltihead automata and the power of
an auxiliary pushdown store, Proc. ACM Symp.
Theory Computing~ Hershey~ PA~ May 1976~pp.14l-l48.
van Leeuwen, J., Variations of a new machine model,
IEEE Symp.~ pp.228-235.
Yao, A. and Rivest, R.L., K+l heads are better
than K, IEEE Symp.~ pp.67-70.
1977
Book, R. V., Language representation theorems: How
to generate the R.E. sets from the regular sets. ,
Proc.IEEE Sympo.Foundation of Computer Science~
Providence~ RI~ Nov. 1977~ pp.58-61; pub1. as
"Simple representations of certain classes of
languages," JACM~ 25 (1978) 23-31.
CUlik, K., II and Fris, I., The decidability of
the equivalence problem for DOL systems, In-
form.Contr.~ 35 (1977) 20-39.
Kintala, C.M.R. and Fischer, P.C., Computations
with a restricted number of nondeterministic
steps (extended abstract), Proc.ACM Symp. Theory
of Computing~ Bo¥lder~ CO~ M9Y 197?~pp.178-~85;
publ. as "Real-tl.JTl.e computatlons WlTn restrlcted
 nondeterrriIlism, l1ath. Syst. ~:'h . .,72 (1979)
219-232.
1978
Engelfriet, J. and Rozenberg, G., Equality
languages, fixed point languages and repre-
sentatiors of the recursively enumerable lan-
guages, Proc.IEEE Symp. on Foundations of
Computer Science~ Ann Arbor~ Mich., Oct.
1978~ pp.123-126.

Engelfriet, J., Rozenbe.rg, G. and Slutzki, G.,

Tree transducers, L systems and two-way
machines, Proc.ACM Symp. Theory of Comput-
ing~ San Diego~ CA~ pp.66-74.

Harrison, M.A., Introduction to Formal Lan-

guage Theory~ Addison-Wesley, 1978.
1979
Cook, S.A., Deterwinistic CFLs are accepted
simultaneously in polynomial time and log
squared space, Proc.ACM Symp.Theory of Com-
puting~ Atlanta~ GA, May 1979~ pp.338-345.

Friedman, E.P. and Greibach, S.A., A poly-

nomial algorithm for deciding the equiv-
alence problem for 2-tape deterministic
finite state acceptors, submitted for
publication.
Oyama.guchi, M., Honda, N. and Inagaki, Y.,
The equivalence problem for realtime strict
deterministic languages, submitted for pub-
lication.

90