Formal Language Lec1

Tsung-Yi Ho
tyho@cs.nthu.edu.tw
http://theta.cs.nthu.edu.tw
Department of Computer Science
National Tsing Hua University
Hsinchu, Taiwan
1 Ch 1 Introduction to Formal Language (Spring 2017)

Outline
1 Mathematical Preliminaries and Notation
2 Three Basic Concepts
3 Some Applications (omit)

Mathematical Preliminaries
Sets
Functions
Relations
Graphs
Proof Techniques

SETS
A set is a collection of elements
A = {1, 2, 3}
B = {train, bus, bicycle, airplane}
We write
1 A 1 is an element of the set A
ship B ship is not an element of the set B

Set Representations
C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, , k } finite set
S = { 2, 4, 6, } infinite set
S = { j : j > 0, and j = 2k for some k>0 }

Explicit
notation
S = { j : j is nonnegative and even }
A = { 1, 2, 3, 4, 5 }
U
6 A
2 3 8
1
7 4 5
9
10
Universal Set: all possible elements
U = { 1 , , 10 }
Set Operations
A = { 1, 2, 3 } B = { 2, 3, 4, 5 }
A B
Union (U)
2 4
1
A U B = { 1, 2, 3, 4, 5 } 3 5
Intersection
(U)
U
A B = { 2, 3 } 2
3
Difference (-)
A-B={1}
1
B - A = { 4, 5 }
Venn diagrams
Complement
Universal set = {1, , 7}
A = { 1, 2, 3 } A = { 4, 5, 6, 7}
4
A
A 3 6
1
2
5 7
A=A
{ even integers } = { odd integers }
Integers
1 odd
even
6 5
2
0
4
3 7

DeMorgans Laws
U
AUB=A B
U
A B=AUB

Empty, Null Set:
={} The set with no elements
SU =S
U
S = = Universal Set
S - =S
-S=

Subset
A = { 1, 2, 3 } B = { 1, 2, 3, 4, 5 }
A B
U
Subset: If every element of A is also an element of B
Proper Subset: If A B, but B contains an element not in A
U
B
A B
U

Disjoint Sets
A = { 1, 2, 3 } B = { 5, 6}
U
A B=
A B

Set Cardinality
For finite sets
A = { 2, 5, 7 }
|A| = 3 (set size)

Powersets
A powerset is a set of sets
S = { a, b, c }
Powerset of S = the set of all the subsets of S
Example 1.1
2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: if S is finite, then |2S| = 2|S| ( 8 = 23 )

Cartesian Product
Example 1.2
A = { 2, 4 } B = { 2, 3, 5, 6 }
A X B = { (2, 2), (2, 3), (2, 5), (2, 6),

(4, 2), (4, 3), (4, 5), (4, 6) }
|A X B| = |A| |B|
Note that the order in which the elements of a pair are written matters
The pair (4, 2) is in A X B, but (2, 4) is not

Partition
A set can be divided by separating it into a number
of subsets. Suppose that S1, S2, , Sn are subsets
of a given set S and that the following holds:
1. The subset S1, S2, , Sn are mutually disjoint;

2. S1 U S2 U U Sn = S;
3. None of the Si is empty.
Then S1, S2, , Sn is called a partition of S.

FUNCTIONS
Rules that assign to elements of one set a unique element of another set
domain range
4 A B
f(1) = a a
1
2 b
3 c
5
f:AB
If A = domain
then f is a total function
otherwise f is a partial function
Asymptotic Analysis
20 Ch 1
O(n) + O(n) = 2O(n) ? Introduction to Formal Language (Spring 2017)
Asymptotic Analysis

Asymptotic Analysis
Example 1.3
f(n) = 2n2 + 3n,
g(n) = n3,
h(n) = 10n2 + 100.
f(n) = O(g(n)),
g(n) = (h(n))
f(n) = (h(n))
RELATIONS
Relations are more general than functions:
In a function, each element of the domains has exactly one associated
element in the range;
In a relation, there may be several such elements in the range.
R = {(x1, y1), (x2, y2), (x3, y3), }
xi R yi
e. g. if R = >: 2 > 1, 3 > 2, 3 > 1

Equivalence Relations ()
Reflexive: xRx
Symmetric: xRy yRx
Transitive: x R y and y R z xRz
Example: R =
x=x
x=y y=x
x = y and y = z x=z

Example 1.4
On the set of nonnegative integers, we can define a relation
xy
If and only if
x mod 3 = y mod 3.
Then 2 5, 12 0, and 0 36.

Clearly this is an equivalence relation, as it satisfies
reflexivity, symmetry, and transitivity.

GRAPHS
A directed graph (digraph)
e
b
node
a d
c
Nodes (Vertices)
V = { a, b, c, d, e }
Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }

Labeled Graph
2
6 e
b 2
1 3
a 6 d
5
c

Walk
e
b
a d
Walk is a sequence of adjacent edges

(e, d), (d, c), (c, a)

Path
e
b
a d
Path: a walk where no edge is repeated

Simple path: no node is repeated

Cycle
base e
b
3
a 1 d
2
c
Cycle: a walk from a node (base) to itself

Simple cycle: only the base node is repeated

Loop
An edge from a vertex to itself
(v1, v3), (v3, v2) is a simple path from v1 to v2

(v1, v3), (v3, v3), (v3, v1) is a cycle (not simple one)
There is a loop on vertex v3

Euler Tour
8 base
7 e
b 1
4 6
a 5 2 d
3
c
A cycle that contains each edge once

Hamiltonian Cycle
5 base
e
b 1
4
a 2 d
3
c
A simple cycle that contains all nodes

Finding All Simple Paths
e
b
a d
c
origin

Step 1
e
b
a d
c
origin
(c, a)
(c, e)

Step 2
e
b
a d
(c, a) c
origin
(c, a), (a, b)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
Step 3
e
b
a d
(c, a) c
origin
(c, a), (a, b)
(c, a), (a, b), (b, e)
(c, e)
(c, e), (e, b)
37
(c,Che),
1
(e, d) Introduction to Formal Language (Spring 2017)
Step 4
e
b
a d
(c, a)
(c, a), (a, b) c
origin
(c, a), (a, b), (b, e)
(c, a), (a, b), (b, e), (e,d)
(c, e)
(c, e), (e, b)
(c, e), (e, d) Introduction to Formal Language (Spring 2017)
38 Ch 1
Trees
root
parent
leaf
child
Trees have no cycles

root
Level 0
Level 1
leaf Height 3
Level 2
Level 3

Proof Techniques
Direct/Constructive Proof
Proof by Induction
Proof by Contradiction

If X, then Y
Assume X is true, show directly that Y is true.
(e.g. X = it rains, Y = sidewalk will wet)


Induction
We have statements P1, P2, P3,
If we know
for some b that P1, P2, , Pb are true
for any k >= b that
P1, P2, , Pk imply Pk+1
Then
Every Pi is true

Proof by Induction
Inductive basis
Find P1, P2, , Pb which are true
Inductive hypothesis
Lets assume P1, P2, , Pk are true,
for any k >= b
Inductive step
Show that Pk+1 is true
Example 1.5
Theorem: A binary tree of height n has at most 2n
leaves.
Proof by induction:
let L(i) be the maximum number of
leaves of any subtree at height i

We want to show: L(i) <= 2i
Inductive basis
L(0) = 1 (the root node)
Inductive hypothesis
Lets assume L(i) <= 2i for all i = 0, 1, , k
Induction step
we need to show that L(k + 1) <= 2k+1

Induction Step
height
k
k+1
From Inductive hypothesis: L(k) <= 2k

Induction Step
height
k L(k) <= 2k
k+1
L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1
(we add at most two nodes for every leaf of level k)

Remark
Recursion is another thing
Example of recursive function:
f(n) = f(n-1) + f(n-2)
f(0) = 1, f(1) = 1

Example 1.6
A set l1, l2, , ln of mutually intersecting straight
lines divides the plane into a number of separated
regions
1 line 2 regions, 2 lines 4 regions, 3 lines 7 regions
l1 l2
l3
Solve it
recursively!!

Example 1.6
ln Let A(n) denote the number of
l3 regions generated by n lines
l1 l2
1 1 1 1 1 A(n + 1) = A(n) + n + 1, n = 1, 2, ,
ln+1
A(1) = 2, A(2) = 4, A(3) = 7, A(4) = 11

n(n + 1)
A(n) = +1
2
then
n(n + 1) (n + 1)(n + 2)
A(n + 1) = +1+ n +1 = +1
2 2

Proof by Contradiction
We want to prove that a statement P is true
we assume that P is false

then we arrive at an incorrect conclusion
therefore, statement P must be true

Example 1.7
Theorem: 2 is not rational
Proof:
Assume by contradiction that it is rational
2 = n/m
n and m have no common factors
We will show that this is impossible

2 = n/m 2 m2 = n2
n is even
Therefore, n2 is even
n=2k
m is even
2 m2 = 4k2 m2 = 2k2
m=2p
Thus, m and n have common factor 2
Contradiction! Introduction to Formal Language (Spring 2017)

55 Ch 1
Outline
1 Mathematical Preliminaries and Notation
2 Three Basic Concepts
3 Some Applications

Three Basic Concepts
Languages
Grammars
Automata (will discuss in Chap. 2)

A language is a set of strings
String: A sequence of symbols from the alphabet
Examples: cat, dog, house,
Defined over an alphabet:

S = {a, b, c,!, z}

Alphabets and Strings
S = {a, b}
a
ab u = ab
abba v = bbbaaa
baba w = abba
aaabbbaabab
String Operations
w = a1a2 !an abba
v = b1b2 !bm bbbaaa
Concatenation
wv = a1a2 !anb1b2 !bm abbabbbaaa

w = a1a2 !an ababaaabbb
Reverse
R
w = an !a2a1 bbbaaababa

String Length
w = a1a2 !an
Length: w =n
Examples: abba = 4
aa = 2
a =1
Length of Concatenation
uv = u + v
Example: u = aab, u = 3
v = abaab, v = 5
uv = aababaab = 8
uv = u + v = 3 + 5 = 8
Empty String
A string with no letters: l
Observations: l =0
lw = wl = w
labba = abbal = abba

Substring
Substring of string:
a subsequence of consecutive characters
String Substring
abbab ab
abbab abba
abbab b
abbab bbab
Prefix and Suffix
abbab
Prefixes Suffixes
l abbab w = uv
a bbab prefix
ab bab suffix
abb ab
abba b
abbab l
Another Operation
n
w = ww
$
!#%!
"w
n
Example: (abba ) = abbaabba

2
0
Definition: w =l
(abba ) = l
0

The * Operation
S * : the set of all possible strings from
alphabet S
S = {a, b}
S* = {l , a, b, aa, ab, ba, bb, aaa, aab,!}

The + Operation
+ : the set of all possible strings from
S
alphabet S except l
S = {a, b}
S* = {l , a, b, aa, ab, ba, bb, aaa, aab,!}
+
S = S * -l
+
S = {a, b, aa, ab, ba, bb, aaa, aab,!}
Languages
A language is any subset of S*
Example: S = {a, b}
S* = {l , a, b, aa, ab, ba, bb, aaa,!}
Languages: {l }
(Finite)
{a, aa, aab}
70
{l , abba, baba, aa, ab, aaaaaa}
Introduction to Formal Language (Spring 2017)
Ch 1
Note that:
Sets = { } {l}
Set size {} = = 0
Set size {l} = 1

String length l =0
Another Example
n n
An infinite language L = {a b : n 0}
l
ab
L abb L
aabb
aaaaabbbbb
Operations on Languages
The usual set operations
{a, ab, aaaa} " {bb, ab} = {a, ab, bb, aaaa}
{a, ab, aaaa} ! {bb, ab} = {ab}
{a, ab, aaaa} - {bb, ab} = {a, aaaa}
Complement: L = S * -L
{a, ba} = {l , b, aa, ab, bb, aaa,!}

Reverse
R R
Definition: L = {w : w L}
Examples: {ab, aab, baba} = {ba, baa, abab}

R
n n
L = {a b : n 0}
R n n
L = {b a : n 0}
Concatenation
Definition: L1L2 = {xy : x L1, y L2 }
Example: {a, ab, ba}{b, aa}
= {ab, aaa, abb, abaa, bab, baaa}

Another Operation
n
Definition: L =# $!
LL" L
n
{a, b} = {a, b}{a, b}{a, b} =

3
{aaa, aab, aba, abb, baa, bab, bba, bbb}

0
Special case: L = {l}
0
{a , bba , aaa } = {l}
More Examples
n n
L = {a b : n 0}
2 n n m m
L = {a b a b : n, m 0}
2
aabbaaabbb L
Note that n and m in the above are unrelated
Star-Closure (Kleene *)
0 1 2
Definition: L* = L " L " L !
Example:
l ,
a, bb,

{a, bb}* =
aa , abb, bba , bbbb ,
aaa, aabb, abba, abbbb,!
Positive Closure
+ 1 2
Definition: L = L " L "!
= L * -{l}
a, bb,
+
{a, bb} = aa, abb, bba, bbbb,
aaa, aabb, abba, abbbb,!

Grammars
Wikipedia says:
Languages can be described as a system
of symbols and the grammars (rules) by
which the symbols are manipulated
Grammar is the study of rules governing
the use of language.

Grammars
Think back to your days of learning English

Rules for constructing a simple sentence
Sentence = noun phrase + verb phrase
Noun phrase =
Name (Joe)
Article + noun (the car)
Verb Phrase =
Verb (runs)
Verb + prepositional phrase
Prepositional Phrase =
Preposition + noun phrase (from the car)

Grammars
Look at the sentence. Is this grammatically correct?
Joe runs from the car.
Sentence = noun phrase + verb phrase
= noun + verb phrase
= Name + verb phrase
= Joe + verb phrase
= Joe + verb + prepositional phrase
= Joe + verb + preposition + noun phrase
= Joe + verb + from + noun phrase
= Joe + verb + from + article +noun
= Joe + verb + from + article +noun
= Joe + verb + from + the + car
= Joe + runs + from + the + car
Valid sentence!
Definition 1.1
A grammar G is defined as a 4-tuple:

G = (V, T, S, P)
Ex:
where G = ({S}, {a, b}, S, P)
V is a finite set of variables P: S aSb,
S
T is a finite set of terminals
S V, called start variable
P is a finite set of production rules

G = (V, T, S, P)
Grammars
Lets formalize this a bit:

x (V, T)+
Production rules (x y) where y (V, T)*
They specify how the grammar transforms one string into another
We say that can be derived from in one step:
A is a production rule
= 1 A 2
= 1 2

We write * if can be derived from (or say
derives ) in zero or more steps.

Definition 1.2
Let G = (V, T, S, P) be a grammar. Then the set

*
L(G) = {w T*: S w}
is the language generated by G
If w L(G), then the sequence

S w1 w2 wn w
is a derivation of the sentence w.
S, w1, w2, , wn are called sentential forms

G = (V, T, S, P)
Example 1.11
Consider the grammar:

G = ({S}, {a, b}, S, P) L(G)?
With P given by L(G) = {anbn : n0}
S aSb,
S
Then
S aSb aaSbb aabb,
So we can write
* aabb
S
String aabb is a sentence in the language generated by G
86 Ch 1
aaSbb is a sentential form Introduction to Formal Language (Spring 2017)
G = (V, T, S, P)
Example 1.12
Find a grammar that generates

L = {anbn+1 : n0}
Previous example
G = ({S}, {a, b}, S, P) with P: S aSb, S
All we need to do is generate an extra b
G = ({S, A}, {a, b}, S, P), with productions
S Ab,
A aAb,
A

G = (V, T, S, P)
Example 1.13
Consider the grammar:
G = ({S}, {a, b}, S, P) Take = {a, b}, and let na(w) and nb(w)
With P given by denote the number of as and bs in the
S SS, string w
L(G)?
S ,
S aSb, L = {w: na(w) = nb(w)}
S bSa,
Does this grammar indeed generate the language?

Proof by induction!!
Assume that all w L with |w| 2n can be derived with
G
For n =1, trivial
G = (V, T, S, P)
Example 1.13
S SS, Take = {a, b}, and let na(w) and nb(w)
S , denote the number of as and bs in the
S aSb, string w
S bSa,
L = {w: na(w) = nb(w)}
Assume that all w L with |w| 2n can be derived with
G
Take any w L of length 2n+2.
If w = aw1b, then w1 is in L, and |w1| = 2n. By assumption,
S* w1
Then
S aSb * aw1b = w (so is bSa)
Else
S SS * w S * w Introduction
1w2 = w
89 Ch 1 to Formal Language (Spring 2017)
1
Equivalent of Grammars
Two grammars G1 and G2 are equivalent if

they generate the same language
(L(G1) = L(G2))
Example 1.14
G1 = ({S}, {a, b}, S, P1) with P1:
S aSb, S
G2 = ({S, A}, {a, b}, S, P2) with P2:
S aAb| , A aAb|

Homework I
1.1
20, 29, 42
1.2
14, 19, 22

Formal Language Lec1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Formal Language Lec1

Uploaded by

Copyright:

Available Formats

Tsung-Yi Ho

1 Ch 1 Introduction to Formal Language (Spring 2017)

1 Mathematical Preliminaries and Notation

2 Three Basic Concepts

3 Some Applications (omit)

3 Ch 1 Introduction to Formal Language (Spring 2017)

4 Ch 1 Introduction to Formal Language (Spring 2017)

ship B ship is not an element of the set B

5 Ch 1 Introduction to Formal Language (Spring 2017)

S = { j : j > 0, and j = 2k for some k>0 }

Universal Set: all possible elements

10 Ch 1 Introduction to Formal Language (Spring 2017)

11 Ch 1 Introduction to Formal Language (Spring 2017)

={} The set with no elements

12 Ch 1 Introduction to Formal Language (Spring 2017)

13 Ch 1 Introduction to Formal Language (Spring 2017)

14 Ch 1 Introduction to Formal Language (Spring 2017)

|A| = 3 (set size)

15 Ch 1 Introduction to Formal Language (Spring 2017)

Powerset of S = the set of all the subsets of S

Observation: if S is finite, then |2S| = 2|S| ( 8 = 23 )

16 Ch 1 Introduction to Formal Language (Spring 2017)

A X B = { (2, 2), (2, 3), (2, 5), (2, 6),

The pair (4, 2) is in A X B, but (2, 4) is not

17 Ch 1 Introduction to Formal Language (Spring 2017)

1. The subset S1, S2, , Sn are mutually disjoint;

Then S1, S2, , Sn is called a partition of S.

18 Ch 1 Introduction to Formal Language (Spring 2017)

21 Ch 1 Introduction to Formal Language (Spring 2017)

R = {(x1, y1), (x2, y2), (x3, y3), }

e. g. if R = >: 2 > 1, 3 > 2, 3 > 1

24 Ch 1 Introduction to Formal Language (Spring 2017)

Then 2 5, 12 0, and 0 36.

25 Ch 1 Introduction to Formal Language (Spring 2017)

26 Ch 1 Introduction to Formal Language (Spring 2017)

27 Ch 1 Introduction to Formal Language (Spring 2017)

Walk is a sequence of adjacent edges

28 Ch 1 Introduction to Formal Language (Spring 2017)

Path: a walk where no edge is repeated

29 Ch 1 Introduction to Formal Language (Spring 2017)

Cycle: a walk from a node (base) to itself

30 Ch 1 Introduction to Formal Language (Spring 2017)

(v1, v3), (v3, v2) is a simple path from v1 to v2

31 Ch 1 Introduction to Formal Language (Spring 2017)

A cycle that contains each edge once

32 Ch 1 Introduction to Formal Language (Spring 2017)

A simple cycle that contains all nodes

33 Ch 1 Introduction to Formal Language (Spring 2017)

34 Ch 1 Introduction to Formal Language (Spring 2017)

35 Ch 1 Introduction to Formal Language (Spring 2017)

Trees have no cycles

39 Ch 1 Introduction to Formal Language (Spring 2017)

40 Ch 1 Introduction to Formal Language (Spring 2017)

41 Ch 1 Introduction to Formal Language (Spring 2017)

42 Ch 1 Introduction to Formal Language (Spring 2017)

43 Ch 1 Introduction to Formal Language (Spring 2017)

We have statements P1, P2, P3,

44 Ch 1 Introduction to Formal Language (Spring 2017)

46 Ch 1 Introduction to Formal Language (Spring 2017)

47 Ch 1 Introduction to Formal Language (Spring 2017)

From Inductive hypothesis: L(k) <= 2k

48 Ch 1 Introduction to Formal Language (Spring 2017)

L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1