ST.

FRANCIS INSTITUTE OF TECHNOLOGY

MT. POINSUR, BORIVALI (W), MUMBAI
Name: TalishaTanna Roll No: 71
Class: BE CMPN B Batch: 04
Experiment No:03
Aim: Case Study on Natural Language Processing System

Theory:

Expert systems :

The expert systems are the computer applications developed to solve complex problems in a particular
domain, at the level of extra-ordinary human intelligence and expertise.

Characteristics of Expert Systems

 High performance
 Understandable
 Reliable
 Highly responsive

Capabilities of Expert Systems
The expert systems are capable of −
 Advising
 Instructing and assisting human in decision making
 Demonstrating
 Deriving a solution
 Diagnosing
 Explaining
 Interpreting input
 Predicting results
 Justifying the conclusion
 Suggesting alternative options to a problem

They are incapable of −

 Substituting human decision makers
 Possessing human capabilities
 Producing accurate output for inadequate knowledge base
 Refining their own knowledge

Components of Expert Systems

The components of ES include −
 Knowledge Base
 Inference Engine
 User Interface
Natural Language Processing
Natural language processing (NLP) is a branch of artificial intelligence that helps computers understand,
interpret and manipulate human language. NLP draws from many disciplines, including computer science
and computational linguistics, in its pursuit to fill the gap between human communication and computer
understanding.
Natural language processing (NLP) is an area of computer science and artificial intelligence concerned
with the interactions between computers and human (natural) languages, in particular how to program
computers to process and analyze large amounts of natural language data.

Applications of NLP are as follows:-

 Lexical semantics

o What is the computational meaning of individual words in context?

 Machine translation

o Automatically translate text from one human language to another. This is one of the most
difficult problems, and is a member of a class of problems colloquially termed "AI-
complete", i.e. requiring all of the different types of knowledge that humans possess
(grammar, semantics, facts about the real world, etc.) in order to solve properly.

 Named entity recognition (NER)

o Given a stream of text, determine which items in the text map to proper names, such as
people or places, and what the type of each such name is (e.g. person, location,
organization). Note that, although capitalization can aid in recognizing named entities in
languages such as English, this information cannot aid in determining the type of named
entity, and in any case is often inaccurate or insufficient. For example, the first word of a
sentence is also capitalized, and named entities often span several words, only some of
which are capitalized. Furthermore, many other languages in non-Western
script(e.g. Chinese or Arabic) do not have any capitalization at all, and even languages
with capitalization may not consistently use it to distinguish names. For
example, German capitalizes all nouns, regardless of whether they are names,
and French and Spanish do not capitalize names that serve as adjectives.
  Natural language generation

o Convert information from computer databases or semantic intents into readable human
language.

 Natural language understanding

o Convert chunks of text into more formal representations such as first-order

logic structures that are easier for computer programs to manipulate. Natural language
understanding involves the identification of the intended semantic from the multiple
possible semantics which can be derived from a natural language expression which
usually takes the form of organized notations of natural language concepts. Introduction
and creation of language metamodel and ontology are efficient however empirical
solutions. An explicit formalization of natural language semantics without confusions
with implicit assumptions such as closed-world assumption (CWA) vs. open-world
assumption, or subjective Yes/No vs. objective True/False is expected for the
construction of a basis of semantics formalization.

 Optical character recognition (OCR)

o Given an image representing printed text, determine the corresponding text.

 Recognizing Textual entailment
o Given two text fragments, determine if one being true entails the other, entails the other's
negation, or allows the other to be either true or false.

Image Captioning

Problem Definition:
The task of image captioning aims at automatic generation of a natural language description of an image
.It connects two major fields of artificial intelligence: computer vision and natural language processing.
Given an image, break it down to extract the different objects, actions, and attributes, and finally generate
a meaningful sentence (caption/description) for the image. A description must capture not only the objects
contained in an image, but it also must express how these objects relate to each other as well as their
attributes and the activities they are involved in.
Thus the problem boils down to two things - image analysis to get features, and then a language
model to generate meaningful captions.

Model:
The model is divided into two parts:
1. CNN-based Image Feature Extractor
2. LSTM(Long Short Term Memory)-based Sentence Generator

CNN-based Image Feature Extractor:

For feature extraction convolution neural network is used.
Step 1
Train (or download) a classification model (like AlexNet).

Step 2
Fine-tune the model
 1. Change the number of categories from 1000 to 20.
2. Remove the last full-connection layer.

Step 3
Feature extraction:
1. Extract all candidate boxes from the image (selective search).
2. For each region: Correct the size of the region to fit the CNN input, do a forward operation, and
output the fifth pooled layer (that is, the candidate box extracted features) to the hard disk.

Step 4
1. Train an SVM classifier (2 categories) to determine the category of objects in this candidate box.
2. Determine whether the SVM belongs to each of the categories. If so the result is positive, and
negative if not. For example, the following figure is the SVM for a dog classification.

Step 5
Use regression to fine-tune the position of the candidate boxes. For each class, train a linear regression
model to determine if the box is optimized.
LSTM-based Sentence Generator
LSTM(Long Short Term Model) is a type of RNN’ s used for remembering information over long period
of time. The central role in LSTM is played by cell state, which contains gates to optimally send
information through. The sigmoid layer outputs numbers between zero and one, describing how
much of each component should be let through by each gate.

The most common LSTM’ s contain three gates:

• Forget Gate: It decides how much of the previous information we are going to forget, given as:
ft = σ(Wfx.xt + Wfm.mt−1)
• Input Gate: The input gate decides how much of the new information we are going to store and use,
given as:
it = σ(Wix.xt + Wim.mt−1)
• Output Gate: The output gate decides how much of the information we are going to output, given as:
ot = σ(Wox.xt + Wom.mt−1)

Cell State is updated as follows:

ct = ft ◦ ct−1 + it ◦ tanh(Wcx.xt + Wcm.mt−1)
mt = ot ◦ ct
pt = Softmax(mt)
where ◦ represents the product with a gate value, and the various W matrices are trained to the parameters.
The last equation mt is what is used to feed to a Softmax, which will produce a probability distribution pt
over all words.
Training

LSTM decoder combined with a CNN image embedder to generate image captioning

The LSTM model is trained to predict each word in the sentence. Each word is predicted based
on the preceding words and the image features. The image feature vector is given as input only
for the first time. Each output by the memory cell gives out a vector, which is used to predict a
word based on the highest probability and also "forwarded" again as input to the memory cell.

Results:

Caption: two zebras standing next to each

other in a zoo.
Conclusion:
Natural language processing (NLP) is an area of computer science and artificial intelligence concerned
with the interactions between computers and human (natural) languages.