Introduction to Fuzzy

Logic Control

1
Outline
General Definition
Applications
Operations
Rules
Fuzzy Logic Toolbox
FIS Editor
Tipping Problem: Fuzzy Approach
Defining Inputs & Outputs
Defining MFs
Defining Fuzzy Rules
2
 General Definition
Fuzzy Logic - 1965 Lotfi Zadeh, Berkely
superset of conventional (Boolean) logic that has
been extended to handle the concept of partial truth
central notion of fuzzy systems is that truth values
(in fuzzy logic) or membership values (in fuzzy
sets) are indicated by a value on the range [0.0,
1.0], with 0.0 representing absolute Falseness and
1.0 representing absolute Truth.
deals with real world vagueness
Applications
Expert Systems
Control Units
Bullet train between Tokyo and
Osaka
Video Cameras
Automatic Transmissions
 Operations

A B

AB AB A
 Controller Structure
Fuzzification
Scales and maps input variables to fuzzy sets

Inference Mechanism
Approximate reasoning

Deduces the control action

Defuzzification
Convert fuzzy output values to control signals
 MATLAB fuzzy logic toolbox
MATLAB fuzzy logic toolbox facilitates the
development of fuzzy-logic systems using:
graphical user interface (GUI) tools
command line functionality
The tool can be used for building
Fuzzy Expert Systems
Adaptive Neuro-Fuzzy Inference Systems (ANFIS)

7
 Graphical User Interface (GUI) Tools
There are five primary GUI tools for building,
editing, and observing fuzzy inference systems in the
Fuzzy Logic Toolbox:
Fuzzy Inference System (FIS) Editor
Membership Function Editor
Rule Editor
Rule Viewer
Surface Viewer

8
MATLAB: Fuzzy Logic Toolbox

9
MATLAB: Fuzzy Logic Toolbox

10
Fuzzy Inference system

Two type of inference system

Mamdni inference method

Sugeno inference method

*Mamdani's fuzzy inference method, the most

common methodology

11
FIS Editor: Mamdani s inference
system

12
 Fuzzy Logic Examples using Matlab

To control the speed of a motor by changing the input

voltage When a set point is defined, if for some
reason, the motor runs faster, we need to slow it down
by reducing the input voltage. If the motor slows
below the set point, the input voltage must be
increased so that the motor speed reaches the set
point.

13
 Input/Output
Input status words be:
Too slow

Just right

Too fast

output action words be:

Less voltage (Slow down)
No change
More voltage (Speed up)

14
FIS Editor: Adding Input / Output

15
FIS Editor: Adding Input / Output

16
Membership Function Editor

17
Input Membership Function

18
Output Membership Function

19
Membership Functions

20
 Rules

Define the rule-base:

1) If the motor is running too slow, then more voltage.

2) If motor speed is about right, then no change.
3) If motor speed is to fast, then less voltage.

21
Member function Editor:
Adding Rules

22
Rule Base

23
Rule Viewer

24
Surface Viewer

25
 Save the file as one.fis.
Now type in the commend window to get
the result:
>>fis = readfis('one');
out=evalfis(2437.4,fis)
>>out =2.376

26
Sugeno-Type Fuzzy Inference

Takagi-Sugeno-Kang, method of fuzzy inference

similar to the Mamdani method in many respects
Fuzzifying the inputs and applying the fuzzy
operator, are exactly the same.
The main difference between Mamdani and
Sugeno is that the Sugeno output membership
functions are either linear or constant.

27
FIS Editor: Sugeno inference system

28
Add Input/output variables

29
Define Input/output variables

30
Add Input MF

31
Define Input MF

32
Add output MF

33
Define output MF

34
Add rules

35
Define Rule Base

36
View rules

37
Rules viewer

38
Surface viewer

39
 Advantages of the Sugeno Method
Sugeno is a more compact and computationally
efficient representation than a Mamdani system.
It is computationally efficient.
It works well with linear techniques (e.g., PID
control).
It works well with optimization and adaptive
techniques.
It has guaranteed continuity of the output surface.
It is well suited to mathematical analysis.

40
Advantages of the Mamdani Method

It is intuitive.
It has widespread acceptance.
It is well suited to human input.

41
Support Vector Machine & Its
Applications
 Overview
Introduction to Support Vector Machines (SVM)
Properties of SVM
Applications
Gene Expression Data Classification
Text Categorization if time permits
Discussion
 Support Vector Machine(SVM)
The fundamental principle of classification using the
SVM is to separate the two categories of patterns
Map data x into a higherdimensional feature space via
a nonlinear mapping.
The linear classification (regression) in the high
dimensional space is equivalent to the nonlinear
classification (regression) in the lowdimensional space
 Linear Classifiers

x yest
w x + b>0
f
denotes +1
denotes -1

f(x,w,b) = sign(w x + b)

w x + b<0

How would you

classify this data?
 Linear Classifiers
x yest
denotes +1 f
denotes -1 f(x,w,b) = sign(w x + b)

How would you

classify this data?
 Linear Classifiers
x
f yest
f(x,w,b) = sign(w x + b)
denotes +1
denotes -1

How would you

classify this data?
 Linear Classifiers

x f yest
f(x,w,b) = sign(w x + b)
denotes +1
denotes -1

Any of these
would be fine..

..but which is
best?
 Linear Classifiers
x f yest
denotes +1 f(x,w,b) = sign(w x + b)

denotes -1

How would you

classify this data?

Misclassified
to +1 class
 Classifier Margin

x f yest
denotes +1
denotes -1 f(x,w,b) = sign(w x + b)
Define the margin
of a linear
classifier as the
width that the
boundary could be
increased by
before hitting a
datapoint.
 Maximum Margin
1. Maximizing the margin is good
accordingf(x,w,b)
to intuition and PAC
= sign(w theory
x+ b)
denotes +1 2. Implies that only support vectors are
denotes -1 important; other The
training examples
maximum
are ignorable.
margin linear
3. Empirically it works very very
classifier iswell.
the
linear classifier
Support Vectors with the, um,
are those
datapoints that maximum margin.
the margin This is the
pushes up
against simplest kind of
SVM (Called an
LSVM)
Linear SVM
 Linear SVM Mathematically
x+ M=Margin Width

X-

What we know: +
(x x ) w 2
w. x+ + b = +1 M = =
w . x- + b = -1 w w
w . (x+-x-) = 2
 Linear SVM Mathematically
Goal: 1) Correctly classify all training data
wx i + b 1 if yi = +1
wx i + b 1 if yi = -1
yi ( wxi + b) 1for all i 2
2) Maximize the Margin M =
1 t w
same as minimize ww
2
We can formulate a Quadratic Optimization Problem and solve for w and b

1 t
Minimize ( w) = w w
2
subject to yi ( wxi + b) 1 i
 Solving the Optimization Problem
Find w and b such that
(w) = wTw is minimized;
and for all {(xi ,yi)}: yi (wTxi + b) 1
Needtooptimizeaquadraticfunctionsubjecttolinearconstraints.
Quadraticoptimizationproblemsareawellknownclassofmathematical
programmingproblems,andmany(ratherintricate)algorithmsexistfor
solvingthem.
ThesolutioninvolvesconstructingadualproblemwhereaLagrange
multiplier iisassociatedwitheveryconstraintintheprimaryproblem:
Find 1N such that
Q() =i - ijyiyjxiTxj is maximized and
(1) i y i = 0
(2) i 0 for all i
 The Optimization Problem Solution
The solution has the form:
w = i y i x i b= yk- wTxk for any xk such that k 0

Each non-zero i indicates that corresponding xi is a

support vector.
Then the classifying function will have the form:
f(x) = iyixiTx + b
Notice that it relies on an inner product between the test
point x and the support vectors xi we will return to this
later.
Also keep in mind that solving the optimization problem
involved computing the inner products xiTxj between all
pairs of training points.
Dataset with noise
denotes +1 Hard Margin: So far we require
all data points be classified correctly
denotes -1
- No training error
What if the training set is
noisy?
- Solution 1: use very powerful
kernels

OVERFITTING!
Soft Margin Classification
Slack variables i can be added to allow
misclassification of difficult or noisy examples.

What should our quadratic

11 optimization criterion be?
2 Minimize
R
1
w.w + Ck
2 k =1
7
 Hard Margin v.s. Soft Margin
The old formulation:
Find w and b such that
(w) = wTw is minimized and for all {(xi ,yi)}
yi (wTxi + b) 1

The new formulation incorporating slack variables:

Find w and b such that

(w) = wTw + Ci is minimized and for all {(xi ,yi)}
yi (wTxi + b) 1- i and i 0 for all i

Parameter C can be viewed as a way to control

overfitting.
 Linear SVMs:Overview
The classifier is a separating hyperplane.
Most important training points are support vectors; they
define the hyperplane.
Quadratic optimization algorithms can identify which training
points xi are support vectors with non-zero Lagrangian
multipliers i.
Both in the dual formulation of the problem and in the solution
training points appear only inside dot products:

Find 1N such that

Q() =i - ijyiyjxiTxj is maximized and
(1) iyi = 0
(2) 0 i C for all i

f(x) = iyixiTx + b
 Non-linear SVMs
Datasets that are linearly separable with some noise
work out great:
0 x

But what are we going to do if the dataset is just too hard?

0 x
How about mapping data to a higher-dimensional
space:
x2

0 x
 Non-linear SVMs: Feature spaces
General idea: the original input space can always be
mapped to some higher-dimensional feature space
where the training set is separable:

: x (x)
 The Kernel Trick
The linear classifier relies on dot product between vectors K(xi,xj)=xiTxj
If every data point is mapped into high-dimensional space via some
transformation : x (x), the dot product becomes:
K(xi,xj)= (xi) T(xj)
A kernel function is some function that corresponds to an inner product in
some expanded feature space.
Example:
2-dimensional vectors x=[x1 x2]; let K(xi,xj)=(1 + xiTxj)2,
Need to show that K(xi,xj)= (xi) T(xj):
K(xi,xj)=(1 + xiTxj)2,
= 1+ xi12xj12 + 2 xi1xj1 xi2xj2+ xi22xj22 + 2xi1xj1 + 2xi2xj2
= [1 xi12 2 xi1xi2 xi22 2xi1 2xi2]T [1 xj12 2 xj1xj2 xj22 2xj1 2xj2]
= (xi) T(xj), where (x) = [1 x12 2 x1x2 x22 2x1 2x2]
 What Functions are Kernels?
For some functions K(xi,xj) checking that
K(xi,xj)= (xi) T(xj) can be cumbersome.
Mercers theorem:
Every semi-positive definite symmetric function is a kernel
Semi-positive definite symmetric functions correspond to a
semi-positive definite symmetric Gram matrix:

K(x1,x1) K(x1,x2) K(x1,x3) K(x1,xN)

K(x2,x1) K(x2,x2) K(x2,x3) K(x2,xN)
K=

K(xN,x1) K(xN,x2) K(xN,x3) K(xN,xN)
Examples of Kernel Functions
Linear: K(xi,xj)= xi Txj

Polynomial of power p: K(xi,xj)= (1+ xi Txj)p

Gaussian (radial-basis function network):

2
xi x j
K (x i , x j ) = exp( )
2 2

Sigmoid: K(xi,xj)= tanh(0xi Txj + 1)

 Non-linear SVMs Mathematically
Dual problem formulation:
Find 1N such that
Q() =i - ijyiyjK(xi, xj) is maximized and
(1) iyi = 0
(2) i 0 for all i

The solution is:

f(x) = iyiK(xi, xj)+ b

Optimization techniques for finding is remain the same!

 Nonlinear SVM - Overview
SVM locates a separating hyper plane in the
feature space and classify points in that
space
It does not need to represent the space
explicitly, simply by defining a kernel
function
The kernel function plays the role of the dot
product in the feature space.
 Properties of SVM
Flexibility in choosing a similarity function
Sparseness of solution when dealing with large data sets

- only support vectors are used to specify the separating

hyper plane
Ability to handle large feature spaces

- complexity does not depend on the dimensionality of the

feature space
Over fitting can be controlled by soft margin approach

Nice math property: a simple convex optimization problem

which is guaranteed to converge to a single global solution
Feature Selection
 SVM Applications

SVM has been used successfully in

many real-world problems
- text (and hypertext) categorization
- image classification
- bioinformatics (Protein classification,
Cancer classification)
- hand-written character recognition
 Weakness of SVM
It is sensitive to noise
- A relatively small number of mislabeled examples can
dramatically decrease the performance

It only considers two classes

- how to do multi-class classification with SVM?
- Answer:
1) with output arity m, learn m SVMs
SVM 1 learns Output==1 vs Output != 1

SVM 2 learns Output==2 vs Output != 2

SVM m learns Output==m vs Output != m

2)To predict the output for a new input, just predict with
each SVM and find out which one puts the prediction the
furthest into the positive region.
 Some Issues
Choiceofkernel
Gaussianorpolynomialkernelisdefault
ifineffective,moreelaboratekernelsareneeded
domainexpertscangiveassistanceinformulating
appropriatesimilaritymeasures
Choiceofkernelparameters
inGaussiankernel
isthedistancebetweenclosestpointswithdifferent
classifications
Optimizationcriterion Hardmarginvs.Softmargin
Alengthyseriesofexperimentsinwhichvarious
parametersaretested
 WindPowerForecasting(WPF)
WPFisatechniquewhichprovidestheinformationof
howmuchwindpowercanbeexpectedatagiven
pointoftime.
Duetotheincreasingpenetrationofwindpowerinto
theelectricpowergrid.
Agoodshorttermforecastingwillensuregrid
stabilityandafavorabletradingperformanceonthe
electricitymarkets.
 -SVM
The objective function of the SVM is based on a
insensitive loss function. The formula for the SVM is
given as follows:
StructureofSVM
 Data Resolution
Theresolutionofthedatasetis10minutes.
Eachdatarepresentstheaveragewindspeedand
powerwithinonehour.
Thedatavaluesbetweentwoadjacentsamplesare
xj

linearlychanged,thatis:
) xi+1 + xi
x j (t ) = xi + .t 0 t dti
dti

xi xi +1
Whereisthetimeintervalbetweenand.
dti
 DataValue
Ts
Theaveragevalueofthedatawithincanbe
calculatedas
ti +Ts
) 1 )
x j (t ) =
Ts x j (t )dt
ti

where Ts =60minutesisusedintheveryshortterm
Ts
forecasting(lessthan6hours)and=2hoursisused
forshorttermforecasting.
 FixedStepPredictionScheme
Predictionhorizonofhsteps
fixedstepforecastingmeansonlythevalueofthe
nextsampleispredictedbyusingthehistorical
hth
data.
(t + h) = f (yt, yt-1,,yt-d)
Wheref isanonlinearfunctiongeneratedbySVM

yt
yt +h ispredictedwiththedatabefore(theredblocks),is
yt +h1 predictedwiththedatabefore(thegreenblocks)
yt 1
Windspeednormalization
Autocorrelationsofthewindspeedsamples
SVMmodelandtheRBFmodel
1h-ahead wind power prediction using
the SVM model.
 CONCLUSIONS
TheSVMhasbeensuccessfullyappliedtothe
problemsofpatternclassification,particularlythe
classificationoftwodifferentcategoriesofpatterns.
SVMmodelismoresuitableforveryshorttermand
shorttermWPF
ProvidesapowerfultoolforenhancingtheWPF
accuracy.