Natural Language Understanding

What is understanding ? Inference about speakers goals and assumptions and context of interaction NLU program requires Large amount of knowledge Reason effectively with knowledge
1

Natural Language Understanding (Contd..)

Why NLU is AI ? Intelligent behaviour

Perception

Communication

through sight, hearing, touch smell, taste and generating words. Ability to communicate effectively is an intelligent behaviour
2

Natural Language Understanding (Contd..)

NLU is used in
Question answering Database queries Automatic translation system Story understanding

NLU problem is Complex

Shall I compare thee to a summers day thou art more lovely and more temperate Rough winds do share the darling buds of may, and summers lease what all too short a date Shakespeare

NLU problem is Complex (Contd..)

Dictionary meaning of words is not sufficient inference requires - Metaphor -Application of name or description to an object for which it is not directly applicable Eg. A camel is a ship of desert Variety is a spice of life. - Analogies Eg As clear as crystal - Background knowledge
5

Levels of Analysis for Natural language understanding (NLU)

1. 2. Prosody : - Deals with the rhythm of language - Importance in poetry or religious chants. Phonology : - Phonome is the smallest unit of sound - Relates sounds to words we recognize - Importance in speech recognition and generation Morphology : - Knowledge relating to word constructions from basic units. - Morpheme is the smallest unit of meaning - Construction of friendly from friend.
6

3.

Levels of Analysis for NLU (Contd..)

4. Syntactic : - Knowledge related to forming grammatically correct sentences. 5. Semantic : - Knowledge concerned with meaning of words and phrases and sentences. 6. Pragmatic : - Relates to use of sentences in different contexts. Ex : Do you know what time it is ? Yes is an inappropriate answer.
7

Levels of Analysis for NLU (Contd..)

7. World Knowledge : - Knowledge of the physical world - role of goals and intentions in communication. - Background knowledge - essential to understand full meaning of a text.

Levels of Analysis for NLU (Contd..)

Natural Language Understanding

Text or written language Understanding

Speech understanding

- More complex - Often corrupted by noise

Stages of NLU
INPUT : MOTHER PATTED CHILD
PARSING PARSE TREE : NOUN PHARSE NOUN MOTHER PATTED CHILD VERB SENTENCE VERB PHRASE NOUN PHARSE NOUN

SEMANTIC INTERPRETATION

Next 10

Stages of NLU (Contd..)

SEMANTIC INTERPRETATION INTERNAL REPRESENTATION PERSON : MOTHER AGENT PAT INSTRUMENT PERSON : CHILD OBJECT HAND

CONTEST/ WORLD KNOWLEDGE INTERPRETATION

Next
11

Stages of NLU (Contd..)

CONTEXT/WORLD KNOWLEDGE INTERPRETATION EXPANDED REPRESENTATION :

EXPERIENCER

LOVE

OBJECT

PERSON : MOTHER AGENT PAT INSTRUMENT

PERSON : CHILD OBJECT HAND To Question answerer data base query handler, translator etc.

LOCATION

HOME

LOCATION

12

Stages of NLU
1. Parsing : - Verifies syntactic correctness - Creates parse tree - Employs knowledge of language syntax, morphology Semantic Interpretation : - Represents meaning of text - Frames, or other logic-based representations are used - Performing semantic consistency checks. World Knowledge Interpretation : - Produces expanded representation - uses necessary world knowledge for complete understanding.
13

2.

3.

Parsing Techniques
Grammars and Languages V = { A,B,..,Z, a,b,..z } String is constructed from concatenating elements of V L = { S/ S is a string } A language is a set of strings of finite length Well formed sentences are constructed using set of rules called grammar L(G) Denotes the Language Generated by grammar
14

Parsing Techniques (Contd..)

G = (Vn, Vt, s, P) Vn - Set of non terminal symbols Vt - Set of terminal symbols s - Starting symbol P Finite set of production rules or rewrite rules V = Vt U Vn U e, e the empty string Vt Vn = J
15

Parsing Techniques (Contd..)

Vt the terminals are symbols which cannot be decomposed further - Adjectives, nouns, verbs etc. Vn - the non terminals that can be decomposed further - noun phrase, verb phrase A general production rule P has the form xyz xwz x,y, and z belongs to V i.e., y should be rewritten as w in the context of x to z ; x and z can be even empty

16

Example of a Simple grammar

QN = { S, NP, N, VP, V, ART } QT = { boy, ate, tpffey, frog, flew, the, a } Rewrite rules P: S NP VP NP ART N VP V NP N boy  frog  toffey  for alternative choices V ate  Flew ART the  a

17

Example of a Simple grammar (Contd..)

S initial symbol (for sentence) NP noun phrase, VP verb phase N noun, V verb, ART article Example sentences from above grammar G - The boy ate a toffey - The frog flew a boy - A boy ate the frog To generate a sentence,
Start with S Apply rules from P sequentially till no non-terminal appears
18

Example of a Simple Grammar (Contd..)

The boy ate a toffey S NP VP ART N VP the N VP the boy VP the boy V NP the boy ate NP the boy ate ART N the boy ate a N the boy ate a toffey
19

Example of a Simple grammar (Contd..)

A grammar generates grammatically correct sentences. No guarantee for meaningful sentences Natural language can not be formally characterized by simple grammar (As above) Constrained / formal programming languages have been classified by grammar
20

Chomsky Hierarchy of Grammars

Type 0 grammar Most general xyz xwz y can not be e High power machine to recognize sentences is required

21

Context-Sensitive Grammar
Type - 1 grammar Also called context-sensitive grammar Restrictions Length of string on R.H.S. in a rule >= length of string on L.H.S. in rewrite rule xyz x w z, y must be a non-terminal w { e
22

Context Sensitive Grammar(Contd..) Typical grammar rules Capitals - non-terminals Small letters - terminals S S AB aA aA aS aAB BA ab aa

23

Context-Free Grammar
Type 2 grammar Also called context-free grammar Typical form A xyz A single non terminal Production Rules S aS S a Sb S aB S a AB A a B a
24

Regular grammar
Type 3 grammar Most restrictive Also called finite state or regular grammar Production Rules A aB A a

25

Types of Grammars (Contd..)

Regular and context-free languages are most widely studied and understood. Context-free languages are basis for formal programming languages. Type 0 and type 1 are not established More extensive grammar includes Prepositional Phrases PP Adjectives ADJ Determiners DET Adverbs ADV Auxiliary verbs AUX

26

Additional rewrite rules

PP PREP NP (in the house) VP V ADV (work hard) VP V PP (locked in the house) VP V NP PP (locked the dog in the house) VP AUX V NP (must do the job) DET ART ADJ (Determiners-either,next, DET ART other, both etc.) NP DET N The mean boy locked the dog in the house The cute girl worked to make some extra money
27

Basic Parsing Techniques

Parsing - Determining the syntactical structure of a sentence. - Inverse of sentence generation process. Parser : - Uses lexicon to determine the meaning of a word. Input string Parser Output representation structure

Lexicon
28

Parsing an input to create an output

The Lexicon
Lexicon : A dictionary of words containing syntactic, semantic and pragmatic information. The entries of a lexicon may not be the same.

29

The Lexicon (Contd..)

Example Lexicon : Word Type Features ------------------------------------------------------------------------------a Determiner { 3 s } 3 s means third person singular be Verb Trans : Intransitive boy Noun {3s} can Noun { 1s, 2s, 3s, 1p, 2p, 3p } Verb Trans : Intransitive orange Adjective {3s} Noun
30

Top-Down Versus Bottom-Up Parsing

A Top-down parser begins with a sentence Terminal symbols are replaced by input sentence words. Example Kathy jumped the horse S NP VP Noun VP Kathy VP Kathy V VP Kathy jumped NP Kathy jumped article N Kathy jumped the N Kathy jumped the horse
31

Top-Down Versus Bottom-Up Parsing (Contd..)

A Bottom-up parser is data driven because it begins with the actual words in sentence Kathy jumped the horse name jumped the horse name V the horse name V art horse name V art N NP V art N NP V NP NP VP S
32

Transition Networks
Used to represent natural language structures Consists of a number of nodes and labeled arcs. Nodes represent different states in a sentence Arcs represent rules or test conditions to make the transition from state to state
33

Transition Networks (Contd..)

Determiner N1 Adjective Determiner N1 Pronoun Proper Noun Jump
34

noun N2 N3

verb N4

The Child Runs

Adjective N2 Noun N3

Transition Networks (Contd..)

To move from N1 to N2 it is necessary to find an adjective, a pronoun, a determiner, a proper noun or none of these by jumping directly to N2. Examples Big white fluffy clouds Our bright children A large beautiful white flower Large green leaves
35

Augmented Transition Networks

Uses Recursive Transition Networks (RTN) RTN are more powerful than simple networks An RTN is a transition network which permits arc labels to refer to other networks and they in turn may refer back to the referring network.

36

Recursive Transition Network

Example : The big tree shades the old house by the stream (the big tree) NP S: S1 S2 Aux S3 NP S4 (shades) V S5 V (the old house) NP PP S6 POP POP

(a) Top Level RTN

37

Augmented Transition Networks (Contd..)

The DET NP : N1 NPR N3 N2 big ADJ
N

tree PP N4 POP POP

(b) Noun Phrase sub-network

38

Augmented Transition Networks (Contd..)

PREP PP P1 P2 NP P3 POP by the stream (c) Prepositional Phrase Network

Note : POP is used to signal the successful completion of the sub network.
39

Augmented Transition Networks (Contd..)

Include more semantic information into structure RTN with additional features is ATN. Additional sentence features include Number S/P Mood Declarative or Interrogative Tense Present, Past Additional tests performed for semantic features. Temporary storage registers are used in ATN. A set of registers for NP network A set of registrar for PP network Register contents are cleared when failure occurs After successful parsing, contents of registers are combined for
final sentence structure
40

Augmented Transition Networks (Contd..)

Word Definition Word Definition a Part-of-speech: article Part-of-speech: verb Root: a Root: like Number: Singular Like Number: Plural men Part-of-speech: noun Root: man Number: Plural Likes Part-of-speech: verb Root: like Number: Singular Part-of-speech: noun Root : dog Number: Singular

Dog

Dictionary of Entries for a Simple ATN

41