8/5/2014

INTRODUCTION TO SOFT
COMPUTING

Harshali Patil

Introduction

Evolution of Computing
Soft computing constituents
From conventional AI to Computational Intelligence
Machine Learning basics

8/5/2014

Problem Solving

Reasoning

New
Problem
1) What kind
of similar
problems you
have solved?

4) Is it
necessary to
retain the
knowledge?

Solution
2) How to reuse
the previous
knowledge?

3) Proposed
solution is
valid or not?

Case Based Reasoning

8/5/2014

Artificial Intelligence

If intelligence can be induced in machines it is

called as artificial intelligence.
Soft computing is a part of artificial intelligent
techniques
Closed related to machine
intelligence/computational intelligence

Soft Computing

Neural Networks

Neuro+ Derivative- =
Fuzzy
Free
Computing
Optimization

Soft Computing

Fuzzy Inference
systems

8/5/2014

What is Soft Computing?

The idea behind soft computing is to model cognitive

behaviour of human mind.

Soft computing is foundation of conceptual

intelligence in machines.

Unlike hard computing , Soft computing is tolerant of

imprecision, uncertainty, partial truth, and
approximation.

Soft Computing
According to Prof.
Prof. Zadeh:
Zadeh:
"...in contrast to traditional hard
computing, soft computing exploits the
tolerance for imprecision, uncertainty, and
partial truth to achieve tractability,
robustness, low solution-cost, and better
rapport with reality

8/5/2014

Soft Vs Hard Computing

HARD COMPUTING

Precise Models

Symbolic
Logic
Reasoning

Traditional
Numerical
Modeling and
Search

SOFT COMPUTING

Approximate Models

Approximate
Reasoning

Soft Computing

Hard Computing
Conventional computing requires a Soft
precisely stated analytical model.

Functional
Approximation
and Randomized
Search

computing

is

tolerant

of

imprecision.

Often requires a lot of computation Can solve some real world problems in
time.

reasonably less time.

Not suited for real world problems for Suitable for real world problems.
which ideal model is not present.
It requires full truth

Can work with partial truth

It is precise and accurate

Imprecise.

High cost for solution

Low cost for solution

8/5/2014

Overview techniques of SC
Neural Networks
Fuzzy Logic
Genetic Algorithm
Hybrid Systems

Evolution of Computing
1940s

1947
Cybemetics

1943 McCulloach
Pitts neuron model

1950s

1956 AI

1957 Perceptron

1960s

1960 LISP
language

1960 Adaline
Masaline

1965 Fuzzy
sets

1970s

Mid 1970s
knowledge
engineering
(expert system)

1974 Birth of back

propogation
algorithm
1975 Cognitron
Neocognitron

1974 Fuzzy
controller

1970s
Genetic
algorithm

1980 self organizing

map
1982 hopfield net
1983 Boltzmann
machine
1986
Backpropogation
algorithm boom

1985 Fuzzy
modelling

Mid 1980s
Artificial life
Immune
modelling

1980s

1990s

1990s Neuro
1990 Genetic
fuzzy modelling programming
1991 ANFIS

8/5/2014

SC constituents (the first three

items) and conventional AI
Methodology

Strength

Neural network

Learning and adaptation

Fuzzy set theory

Knowledge representation via fuzzy

if-then rules

Genetic algorithm and

simulated annealing

Systematic random search

Conventional AI

Symbolic manipulation

Character recognizer

8/5/2014

Features of Conventional AI

Conventional

AI manipulates symbols on the

assumption that human intelligence behavior can
be stored in symbolically structured knowledge
bases: this is known as: The physical symbol
system hypothesis

The

knowledge-based system (or expert system) is

an example of the most successful conventional AI
product

What is expert system?

An expert system is software that uses a knowledge

base of human expertise for problem solving, or to
clarify uncertainties where normally one or more
human experts would need to be consulted

8/5/2014

Expert system

Building blocks of expert system

Knowledge base: factual knowledge and heuristic

knowledge
Knowledge representation: in the form of rules
Problem solving model: forward chaining or
backward chaining
Note:
Knowledge

engineering:- building an expert system

Knowledge engineers:- practitioners.

8/5/2014

Definitions of AI

AI is the study of agents that exists in an

environment and perceive and act [S. Russel & P.
Norvig]
AI is the act of making computers do smart
things [Waldrop]
AI is a programming style, where programs
operate on data according to rules in order to
accomplish goals [W.A. Taylor]

Definitions of AI

AI is the activity of providing such machines as

computers with the ability to display behavior that
would be regarded as intelligent if it were
observed in humans [R. Mc Leod]
Expert system is a computer program using
expert knowledge to attain high levels of
performance in a narrow problem area [D.A.
Waterman]

10

8/5/2014

Definitions of AI

Expert system is a caricature of the human

expert, in the sense that it knows almost
everything about almost nothing [A.R. Mirzai]
AI is changing rapidly, these definitions are
already obsolete!

Applications of expert system

Diagnosis and Troubleshooting of Devices and

Systems of All Kinds
Planning and Scheduling
Configuration of Manufactured Objects from
Subassemblies
Financial Decision Making
Knowledge Publishing
Design and Manufacturing

11

8/5/2014

If the facts don't fit the

theory, change the facts.
- Albert Einstein

GEMS CBR for Remote Diagnostics

On-site Monitoring

Remote Data
Access

Phone, E-mail
FAX, Web
GE Regional
Service Team
Parts
Specialist

Diagnostics
Specialist

Technical
Answer Center

Information
System
Servers

Call Management/
Commitment Tracking
System
Monitoring &
Diagnostics
Problem/Solution DB
CBR using DB

Service
Engineer

12

8/5/2014

Machine Learning basics

Machine learning, a branch of artificial intelligence,

concerns the construction and study of systems that
can learn from data.

E.g. a machine learning system could be trained on email

messages to learn to distinguish between spam and nonspam messages. After learning, it can then be used to
classify new email messages into spam and non-spam
folders.

Research Areas
Intelligent Robots
 Bayesian inference and design
technique for uncertain scene
recognition
Combination Image filtering and
Bayesian inference
Navigation technique research for
autonomous mobile robot
Evolving a mobile robot controller

13

8/5/2014

Research Areas

Intelligent Agent
 Intelligent virtual secretary agent
Conversational agent
Intelligent assistants for smart
phone service

Research Areas

Bioinformatics
Bioinformatics: the collection, classification,
storage, and analysis of biochemical and biological
information using computers especially as applied
in molecular genetics and genomics
Classification techniques in Bioinformatics
Agent driven virtual cell modelling

14

8/5/2014

Research Areas

Ubiquitous
Developing an adaptation scheme in the context
of middleware and applications
Developing context-aware system for ubiquitous
systems
Developing basic theories and algorithms of the
advanced intelligent models for ubiquitous
environment

Research Areas

Intrusion Detection System (IDS)

HMM-based intrusion detection system
Generation of various intrusion patterns using
interactive genetic algorithm
Viterbi algorithm for intrusion type identification
Rule-based integration of multiple measure-models

15

8/5/2014

Research Areas

Biometrics
Analysis and evaluation techniques of fingerprint
recognition system
Development of classification and matching algorithm
for fingerprint recognition

Neuro Fuzzy and Soft

Computing Characteristics

With NF modeling as a backbone, SC can be

characterized as:
Human expertise (fuzzy if-then rules)

Biologically inspired computing models (NN)

New optimization techniques (GA, SA, RA)

Numerical computation (no symbolic AI so far, only
numerical)

New application domains: mostly computation
intensive like adaptive signal processing, adaptive
control, nonlinear system identification etc

16

8/5/2014

Neuro Fuzzy and Soft

Computing Characteristics

Model

free learning:-models are constructed based on

the target system only

Intensive computation: based more on computation

Fault tolerance: deletion of a neuron or a rule does not
destroy the system. The system performs with lesser
quality

Goal driven characteristics:- only the goal is
important and not the path.

Real world application:- large scale, uncertainties

Neural Network

DARPA Neural Network Study (1988, AFCEA

International Press, p. 60):

... a neural network is a system composed of many

simple processing elements operating in parallel
whose function is determined by network
structure, connection strengths, and the
processing performed at computing elements or
nodes.

17

8/5/2014

Definition of Neural Network

According to Haykin (1994), p. 2:

A neural network is a massively parallel distributed
processor that has a natural propensity for storing
experiential knowledge and making it available for
use. It resembles the brain in two respects:

Knowledge is acquired by the network through a learning

process.

Interneuron connection strengths known as synaptic

weights are used to store the knowledge

According to Nigrin (1993), p. 11:

A neural network is a circuit composed of a very
large number of simple processing elements that
are neurally based. Each element operates only on
local information.
Furthermore each element operates
asynchronously; thus there is no overall system
clock.

18

8/5/2014

According to Zurada (1992):

Artificial neural systems, or neural networks, are
physical cellular systems which can acquire, store
and utilize experiential knowledge.

Multi disciplinary view of

Neural Networks

19

8/5/2014

Fuzzy Logic

Origins: Multivalued Logic for treatment of imprecision

and vagueness

1930s: Post, Kleene, and Lukasiewicz attempted to represent

undetermined, unknown, and other possible intermediate
truth-values.

1937: Max Black suggested the use of a consistency profile to

represent vague (ambiguous) concepts.

1965: Zadeh proposed a complete theory of fuzzy sets (and its

isomorphic fuzzy logic), to represent and manipulate ill-defined
concepts.

FUZZY LOGIC LINGUISTIC

VARIABLES

Fuzzy logic gives us a language (with syntax and local

semantics) in which we can translate our qualitative
domain knowledge.

Linguistic variables to model dynamic systems

These variables take linguistic values that are

characterized by:

a label - a sentence generated from the syntax

a meaning - a membership function determined by a local
semantic procedure

20

8/5/2014

Linguistic variables

Linguistic variables associate a linguistic condition

with a crisp variable.
A crisp variable is the kind of variable that is used
in most computer programs: an absolute value.
A linguistic variable,
variable on the other hand, has a
proportional nature: in all of the software
implementations of linguistic variables, they are
represented by fractional values in the range of 0
to 1.

Linguistic variables

Linguistic variables in soup instructions

21

8/5/2014

An old friend comes into your shop asking to buy a few

widgets, and wants your best price. The onus is on you to
come up with a price given many parameters. Taking this
hypothetical case we need to account for:
Cost of the widgets
Normal markup
Shelf time of the product
Shelf life of the product
Length of the relationship
Customer payment history
Quantity of the sale
Repeat business potential

Parameters to consider pricing a widget

22

8/5/2014

Figure: Linguistic variable HOT

23

8/5/2014

Figure: F_OR operator (Fuzzy OR)

Figure: F_EQ (Fuzzy equal)

24

8/5/2014

FUZZY LOGIC REASONING

METHODS

The meaning of a linguistic variable may be interpreted as an

elastic constraint on its value.
These constraints are propagated by fuzzy inference operations,
based on the generalized modus-ponens.
An FL Controller (FLC) applies this reasoning system to a
Knowledge Base (KB) containing the problem domain heuristics.
The inference is the result of interpolating among the outputs of
all relevant rules.
The outcome is a membership distribution on the output space,
which is defuzzified to produce a crisp output.

There are two consistent logical argument constructions: modus

ponens ("the way that affirms by affirming") and modus tollens ("the
way that denies by denying"). Here are how they are constructed:

Modus Ponens: "If A is true, then B is true. A is true. Therefore, B is true."

Modus Tollens: "If A is true, then B is true. B is not true. Therefore, A is
not true.

There are two related incorrect and inconsist

constructions: affirming the consequent and denying the
antecedent.

Affirming the Consequent: "If A is true, then B is true. B is true.

Therefore, A is true."
Denying the Antecedent: "If A is true, then B is true. A is not true.
Therefore, B is not true."

25

8/5/2014

Example

Here is a sensible example, illustrating each of the

above:
"If it is a car, then it has wheels. It is a car.
Therefore, it has wheels." (Modus Ponens CORRECT)
"If it is a car, then it has wheels. It does not have
wheels. Therefore, it is not a car." (Modus Tollens CORRECT)

"If it is a car, then it has wheels. It has wheels.

Therefore, it is a car." (Affirming the Consequent INCORRECT.)

Comment: why is this incorrect? Well, the thing might

have wheels but that doesn't mean it has to be a car. It
might be a cart, or rollerblades, or a moped. It doesn't have
to be a car.

"If it is a car, then it has wheels. It is not a car.

Therefore, it does not have wheels." (Denying the
Antecedent - INCORRECT)

Comment: why is this incorrect? Consider

the argument for the "affirming the consequent" example.
Rollerblades are not cars, but they DO have wheels.

26

8/5/2014

Genetic algorithm
EVOLUTIONARY PROCESS

Steps involved in Genetic

Algorithm

The genetic algorithms follow the evolution process in the

nature to find the better solutions of some complicated
problems. Foundations of genetic algorithms are given in
Holland (1975) and Goldberg (1989) books.
Genetic algorithms consist the following steps:

Initialization
Selection
Reproduction with crossover and mutation

Selection and reproduction are repeated for each

generation until a solution is reached.
During this procedure a certain strings of symbols, known
as chromosomes, evaluate toward better solution.

27

8/5/2014

Hybrid Systems

Hybrid systems enables one to combine various

soft computing paradigms and result in a best
solution. The major three hybrid systems are as
follows:

Hybrid

Fuzzy Logic (FL) Systems

Hybrid

Neural Network (NN) Systems

Hybrid

Evolutionary Algorithm (EA) Systems

Applications of Soft Computing

Handwriting Recognition
Image Processing and Data Compression
Automotive Systems and Manufacturing
Soft Computing to Architecture
Decision-support Systems
Soft Computing to Power Systems
Neuro Fuzzy systems
Fuzzy Logic Control
Machine Learning Applications
Speech and Vision Recognition Systems
Process Control and So on

28

8/5/2014

29