1

SOFT COMPUTING FUNDAMENTS

OUTLINE
Introduction to Soft Computing Probabilistic Computing (PC) Fuzzy computing (FC) Fuzzy Theory Fuzzy Systems Neural computing (NC)
Artificial Neural Networks

Evolutionary Computing (EC)

Genetic Algorithms

Summary

INTRUDUCTION
The idea of soft computing started with Lofti A. Zadeh in 1981. Soft Computing is the fusion of methodologies designed to model and enable solutions to real world problems.

* Basically, soft computing is not a homogeneous body of concepts and techniques. *The role model for soft computing is the human mind

The dominant aim of soft computing is to exploit the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness, low solution-cost, and better raptor with reality . (Zadeh,1981)

Rather than a precise definition for soft computing, it is instead defined by extension.

Symbolic manipulation

AI

Machine Learning

Automatic improvement with experience

Cognitive Psychology

SOFT COMPUTING Statistics

Study of the mind

Uncertainty and Imprecision

Probability (not possibility)

Probabilistic Computing (PC)

Supervised learning
Classification problems Regression problems

Unsupervised learning
Clustering Dimensionality reduction.

*Bayes Theory *Dampster-Shafer Theory

Supervised Learning
Input

Inputs Outputs

Learning algorithm
Linear regression, polynomial regression, logistic regression, etc. Optimization functions

Obtain probable solution.

Training Set

Hypothesis

Output

Generate a hypothesis that classifies the data. A decision boundary can be established using linear regression.
X2

Optimization Algorithms: Gradient descendent.

X1

Unsupervised Learning
Input

Input Not Feedback

Learning algorithm
Generate a probabilistic model. Bayes Networks

Obtain probable solution.

DATA

Model

Output

10

There is not correct answer. There are not parameters to feet. The only data available is the input
X1

X2

In general, the true distribution of the data is unknown

11

Bayes Decision Theory

Belief Theory:
=
() ()

Classification

From the Bayes rule is assumed to have a models m where

Ethem Alpaydin, 2004

12

Bayesian Network
Belief networks or probabilistic networks Used as graphical models to represent the interaction between variables.

Taken from Wikipedia

13

Fuzzy computing (FC)

Introduced by Lofti Zadeh in 1965, as the fuzzy set theory.
REAL WORLD

Human Think
Vague Inexact

Ambiguous

Probable truth

Probabilistic

Linguistic Knowledge
Not quantifiable

Nature
Uncertain Imprecise

Imprecisely defined

14

Fuzzy theory
Imprecise definition

Membership Created by experts functions

Rules

Qualificative

Try to reproduce the reasoning to make a decision.

Probable truth

15

Fuzzy set is an extension of classical set theory where elements have degrees of membership.

Classical theory = 1 0

Fuzzy theory = 0

16

Fuzzy Systems

17

Fuzzification: With an input variable find the numeric values of the membership function.

18

Fuzzy inference: 1. Using linguistic rules, determine with the inputs which rules apply. 2. Determining conclusions.

19

Desfuzzification: Operation to obtain the most certain output.

20

NEURAL COMPUTING
Artificial Neural Networks (ANN): Non-linear approximation Emulate a simplified biological neuron. Mathematical or computational model.

Advantage: Size is not directly related with the number of features.

21

Biological Neuron
INPUTS

OUTPUT

22

McCulloch-Pitts Neuron
Bias INPUTS

= ( ) Activation function OUTPUT

Synaptic Weights =

Threshold

yi (t ) f i ( wij xi i )
j

23

Weight factors, Wj Determine the strain of the input vector. Activation Function: Describes the relationship of input-output. Could be a threshold function, sigmoid function(smooth), piecewise linear function, tangent hyperbolic function

24

Perceptron: example AND I1

W1 W2
x Wi I i +
i 1 2

F(x) = x O=x

I2

3 2.5

I1 0 0 1 1

I2 0 1 0 1

I1 AND I2

2 1.5 1 0.5 0

0 0 0 1

-0.5 -1 -1.5 -2 -1

-0.5

0.5

1.5

25

Single layer feed-forward network

26

Multi Layer Feed-forward netowrk

27

LEARNING METHODS IN ANN

ANN Learning algorithms

Supervised Learning

Reinforced Learning

Unsupervised Learning

Stochastic

Gradient descent

Hebbian

Competitive

EMS

Back propagation

28

Back propagation algorithm

The artificial neurons are organized in layers, and send their signals forward, and then the errors are propagated backwards.` The idea of the backpropagation algorithm is to reduce the error in the output. The backpropagation algorithm calculates how the error depends on the output, inputs, and weights

29

Classification of ANN

30

Optimization
Optimization Methods Linear Programming Classical Methods Newton, Fibonacci Non-Linear Programming Enumerative Methods Min-max algorithms Stochastic Methods

EA, GA,SA

31

Evolutionary Computing
Evolutionary algorithms (EA) are populationbased metaheuristics optimization algorithms. Use biology-inspired mechanisms like mutation, crossover, natural selection, and survival of the fittest.

Is more robust than other techniques in cases of noise, or input changes.

32

Basic cycle of evolutionary computing

Weise Thomas (2009)

33

A basic evolutionary algorithm

34

GENETIC ALGORITHMS
Is a subclass of Evolutionary algorithms. Mimic natural process as: selection, crosses over, mutation and accepting Random search Adaptive heuristic search. Survival of the fittest

35

Basic process

GENETIC OPERATORS

ADJUSTMENT AND EVALUATION

SELECTION

36

Basic genetic algorithm

1. Generate random population 2.Evaluate fitness f(x) for each chromosome x 3. New Population.
A. Selection select parents best fitness B. Crossover crossover the parents new offspring child C. Mutation using mutation probability mutate new offspring

4. Raplace 5. Test

37

38

Summary
Soft computing techniques deal with the uncertainty and imprecision of the real world, using algorithms that in contrast to hard computing can treat this problems with robustness and generate low cost solutions.
Systems based in probabilistic computing and fuzzy logic are approximation to the reasoning, so that it can be obtained a similar model to the real one but not exactly the same. Neural networks and evolutionary algorithms create functional approximations, for unknown systems, or very complex systems that cant be mathematically described.

39

References
J.-S.R. Jang, K. Goebel, Neuro-Fuzzy and Soft Computing. Ethem Alpaydin (2004), Introduction to Machine Learning, MIT Press, Cambridge, Massachustts. http://www.uncertainty-inengineering.net/uncertainty_models/fuzziness Weise Thomas (2009), Global Optimization Algorithms, Theory and Application, http://www.it-weise.de/