Neural Networks Lesson 9 - Fuzzy Logic: Prof. Michele Scarpiniti

Introduction Neurofuzzy Network
Neural Networks Lesson 9 Fuzzy Logic

Prof. Michele Scarpiniti
INFOCOM Dpt. - Sapienza University of Rome
http://ispac.ing.uniroma1.it/scarpiniti/index.htm michele.scarpiniti@uniroma1.it
Rome, 26 November 2009
M. Scarpiniti
Neural Networks Lesson 9 - Fuzzy Logic
1 / 37
Introduction Basic Concepts Fuzzy Logic
Neurofuzzy Network Neurofuzzy Network Fuzzy Clustering
M. Scarpiniti
2 / 37
Basic Concepts Fuzzy Logic
Introduction
Introduction
M. Scarpiniti
3 / 37
Introduction to Fuzzy Logic
Fuzzy logic is a form of multi-valued logic to deal with reasoning that is approximate rather than precise. Fuzzy logic emerged as a consequence of the 1965 proposal of fuzzy set theory by Lot Zadeh. Though fuzzy logic has been applied to many elds, from control theory to articial intelligence, it still remains controversial among most statisticians who prefer Bayesian logic, and some engineers who prefer traditional two-valued logic. A logic based on the two truth values True and False is sometimes inadequate when describing human reasoning. Fuzzy logic uses the whole interval between 0 (False) and 1 (True) to describe human reasoning.
M. Scarpiniti
4 / 37
Fuzzy sets
Fuzzy sets are a further development of the mathematical concept of a set. A set is any collection of objects which can be treated as a whole. Cantor described a set by its members, such that an item from a given universe is either a member or not. A set can be specied by its members, they characterize a set completely. The list of members A = {0, 1, 2, 3}. Nobody can list all elements of an innite set, we must instead state some property which characterizes the elements in the set, for instance the predicate x > 10. Following Zadeh many sets have more than an either-or criterion for membership. Take for example the set of young people. A one year old baby will clearly be a member of the set, and a 100 years old person will not be a member of this set, but what about people at the age of 20, 30, or 40 years? Another example is a weather report regarding high temperatures, strong winds, or nice days. Zadeh proposed a grade of membership, such that the transition from membership to non-membership is gradual rather than abrupt.
M. Scarpiniti Neural Networks Lesson 9 - Fuzzy Logic 5 / 37
Fuzzy sets
The grade of membership for all its members thus describes a fuzzy set. An items grade of membership is normally a real number between 0 and 1, often denoted . The higher the number, the higher the membership. Zadeh regards Cantors set as a special case where elements have full membership = 1. He nevertheless called Cantors sets nonfuzzy , but today the term crisp set is used.
Note that a fuzzy membership function is dierent from a statistical probability distribution. A possible event does not imply that it is probable. However, if it is probable it must also be possible. You might view a fuzzy membership function as your personal distribution, in contrast with a statistical distribution based on observations.
Fuzzy sets
Notice that Zadeh does not give a formal basis for how to determine the grade of membership. The membership for a 50 year old in the set young depends on ones own view. The grade of membership is a precise, but subjective measure that depends on the context.
Elements of a fuzzy set are taken from a universe. The universe contains all elements that can come into consideration and it depends on the context. For example the number between 0 and 100, or, for non-numerical quantity, bitter, sweet, sour, salt, hot,....
Membership function
Every element in the universe of discourse is a member of the fuzzy set to some grade, maybe even zero. The set of elements that have a non-zero membership is called the support of the fuzzy set. The function that ties a number to each element x of the universe is called the membership function (x). There are two alternative ways to represent a membership function: continuous or discrete. In the continuous form the membership function is a mathematical function. A membership function is for example bell-shaped (also called a -curve), s-shaped (called an s-curve), a reverse s-curve (called z-curve), gaussian, triangular, or trapezoidal. In the discrete form the membership function and the universe are discrete points in a list (vector). Sometimes it can be more convenient with a sampled (discrete) representation. A fuzzy set is normalized if its largest membership value equals 1.
Membership function: example

Following some well-known membership functions are reported. Triangular: (x) = max min x a c x , ba c b ,0
Trapezoidal: (x) = max min x a d x , ba d b ,0
M. Scarpiniti
9 / 37
Membership function
Other well-known membership functions are reported. Gaussian: (x) = exp (x a)2 2b 2
Bell-shaped: (x) = 1+ 1
xa 2b c
M. Scarpiniti
10 / 37
Singletons
A fuzzy set A is a collection of ordered pairs A = {(x, (x))} Item x belongs to the universe and (x) is its grade of membership in A. A single pair (x, (x)) is called a fuzzy singleton. Thus the whole set can be viewed as the union of its constituent singletons. It is often convenient to think of a set A just as a vector a = [(x1 ), (x2 ), . . . , (xn )]
M. Scarpiniti
11 / 37
Linguistic variables
Just like an algebraic variable takes numbers as values, a linguistic variable takes words or sentences as values. The set of values that it can take is called its term set. Each value in the term set is a fuzzy variable dened over a base variable. The base variable denes the universe of discourse for all the fuzzy variables in the term set. In short, the hierarchy is as follows: linguistic variable fuzzy variable base variable.
Example Let x be a linguistic variable with the label Age. Terms of this linguistic variable are from the term set T = {Old, VeryOld, NotSoOld, MoreOrLessYoung , QuiteYoung , Young , VeryYoung } Each term is a fuzzy variable dened on the base variable, which might be the scale from 0 to 100 years.
A primary term is a term or a set that must be dened a priori, for example Young and Old, whereas the sets VeryYoung and NotYoung are modied.
Linguistic variables: an example

Given the set X = {1.5, 1.6, 1.75, 1.8, 2}, we can dene the following characterization: 1 short = {1.5/1, 1.6/0.8, 1.75/0.5, 1.8/1, 2/0}; 2 normal = {1.5/0, 1.6/0.5, 1.75/1, 1.8/1, 2/0}; 3 tall = {1.5/0, 1.6/0.2, 1.75/0.5, 1.8/0.8, 2/1};
M. Scarpiniti
13 / 37
Operations on Fuzzy sets

Like for the traditional sets, some elementary operations can be developed for fuzzy sets too.
Complement: A (x) = 1 A (x)
Intersection: AB (x) = min (A (x), B (x))
Union: AB (x) = max (A (x), B (x))
M. Scarpiniti
14 / 37
Fuzzy logic
The fuzzy logic invalidate two of the strongholds of the classical logic:
1
Law (or principle) of the excluded third (or Tertium non datur in Latin): A A = True Principle of contradiction (or principium contradictionis in Latin): A A = False
In this way the reasoning for fuzzy logic is dierent from the classical one: Usually natural and articial languages have some rules, of the type: if then
For example: if it is raining then we get wet; if the pressure is high then the volume is small.
Fuzzy reasoning
In the fuzzy reasoning the linguistic variable contained in a rule, are dened by fuzzy sets: if x is A then y is B A and B are fuzzy sets dened on the universe X and Y ; the values x X and y Y ; the fuzzy rule denes a relation R on the space X Y ; a fuzzy relation is a fuzzy set dened on several domains: R = X Y Z ... For example: binary relation R =AB
M. Scarpiniti
R (x, y , z, . . .)
R (x, y ) = min (A (x), B (y ))

16 / 37
Fuzzy rules
A fuzzy rule is dened as a conditional statement in two forms:
1
Mamdami rules:
IF x1 is A1 and x2 is A2 and . . . and xN is AN THEN y1 is B1 and y2 is B2 and . . . and yM is BM

2
Takagi-Sugeno rules:
IF x1 is A1 and x2 is A2 and . . . and xN is AN THEN y = fk (x1 , x2 , . . . , xN )

The part included between the words IF and THEN is called antecedent, the last part, after the word THEN, is called consequent. The single assignment xi is Ai or yi is Bi is called atom.
Example IF temp IS HIGH AND press IS LOW THEN lever IS UP
Inference principle: modus ponens and modus tollens

Given two domains X and Y and a fuzzy relation R, and let us pose the fuzzy sets A dened on X and B dened on Y , then how can I produce a fuzzy reasoning? Classical logic uses two fundamental inference principles: Modus Ponens Premise: if (x is A) then (y is B) Antecedent: x is A Consequent: y is B Modus Tollens Premise: if (x is A) then (y is B) Antecedent: y is not B Consequent: x is not A
M. Scarpiniti
18 / 37
Fuzzy inference principle: generalized modus ponens

The modus ponens from standard logical propositional calculus cannot be used in the fuzzy logic environment causing such an inference can take place if, and only if, the fact or premise is exactly the same as the antecedent of the IF-THEN rule. In fuzzy logic the generalized modus ponens is used. It allows an inference when the fact is only partly known or when the fact is only similar but not equal to it.
Generalised Modus Ponens (for fuzzy logic) Premise: if (x is A) then (y is B) Antecedent: x is A Consequent: y is B B (y ) = max (min (A (x), R (x, y )))
X
Example Premise: if the tomato is red then it is sweet, possibly sweet-sour, and likely to be sour. Antecedent: The tomato is more or less red (Red = 0.8). Consequent: Taste = ?
Fuzzy inference principle: a rst example

As rst example we show an inference with one rule and one atom: C (y ) = max (min (A (x), R (x, y )))
X
where R (x, y ) = min (A (x), C (y )). Hence, we have C (y ) = maxX (min (A (x), A (x), C (y ))) = = min {maxX (min (A (x), A (x))) , C (y )} = min (q0 , C (y ))
M. Scarpiniti
20 / 37
Fuzzy inference principle: a second example

As a second example we show an inference with one rule and two atoms:
C (y ) = maxX (min (A (x1 ), B (x2 ), R (x1 , x2 , y ))) = = maxX (min (A (x1 ), B (x2 ), A (x1 ), B (x2 ), C (y ))) = = min {maxX1 (min (A (x1 ), A (x1 ))) , maxX2 (min (B (x2 ), B (x2 ))) , C (y ) = min (qx10 , qx20 , C (y ))
M. Scarpiniti
21 / 37
Fuzzy inference principle: a third example

As a third example we show an inference with two rules and two atoms each:
M. Scarpiniti
22 / 37
Defuzzication
Defuzzication is the process of producing a quantiable result in fuzzy logic. A fuzzy quantity is converted into a crisp one (a single number). Defuzzier is the system implementing the defuzzication. Several methods are implemented for defuzzication. 1 Centroid point: xC =
2
x(x)dx (x)dx x
Mean of the maximum: xmed = 1 N

N
MAX (xi )
i=1
Minimum Max: xM = min {MAX (xi )}

X
M. Scarpiniti
23 / 37
Neurofuzzy Network Fuzzy Clustering
Neurofuzzy Network
Neurofuzzy Network
M. Scarpiniti
24 / 37
Analogies and dierences between circuits and neurofuzzy networks

The fuzzy model can be seen as a digital circuit, designed for signal processing. Hence there exist an analogy between the fuzzy model and a digital circuit, shown in the following table: Circuits components topology synthesis Fuzzy Model rules reasoning clustering and rules inserting
The neurofuzzy network can be thought as a generalization of the classical digital circuits, but there exist a substantial dierence, show in the following table: Circuits indirect information Fuzzy Model direct information
The linguistic information can be used directly into the neurofuzzy network: this possibility is not done for the digital circuits, where these properties must be kept into account with particular data transformations.
Neurofuzzy networks
The term neurofuzzy refers to combinations of articial neural networks and fuzzy logic. Neurofuzzy hybridization results in a hybrid intelligent system that synergies these two techniques by combining the human-like reasoning style of fuzzy systems with the learning and connectionist structure of neural networks. Neurofuzzy hybridization is widely termed as Fuzzy Neural Network (FNN) or Neuro-Fuzzy System (NFS) in the literature. Neurofuzzy system incorporates the human-like reasoning style of fuzzy systems through the use of fuzzy sets and a linguistic model consisting of a set of IF-THEN fuzzy rules. The main strength of neurofuzzy systems is that they are universal approximators with the ability to solicit interpretable IF-THEN rules. The strength of neurofuzzy systems involves two contradictory requirements in fuzzy modeling: interpretability versus accuracy. In practice, one of the two properties prevails. We can show two type of neurofuzzy systems: the rst type following the Mamdamis rules and the second one following the Takagi-Sugenos rules.
M. Scarpiniti
26 / 37
Neurofuzzy networks
A neurofuzzy network of the rst type is depicted in the following gure: the output of this network is an expansion of the input following some basis given by the rules.
M. Scarpiniti
27 / 37
Neurofuzzy networks
A neurofuzzy network of the second type is depicted in the following gure: it is an example of a rst order function f (x). This kind of neurofuzzy network realize an input-output relation as a linear piecewise approximation with smoothing at the border. In fact each rule is locally dened on a small region of the input data.
M. Scarpiniti
28 / 37
Synthesis of neurofuzzy networks

The main problem in designing neurofuzzy systems is the determination of the fuzzy rules. The specication of such rules are established by determining high density regions in data space using clustering algorithms. Each rule corresponds to each cluster. Synthesis of a neurofuzzy networks is based on a clustering algorithm. In particular the membership function is evaluated on the basis of the corresponding cluster density: after choosing a shape for the membership function, its parameters are determined by its density. A possible form is the following: 1 (x) = a > 0, b > 0 2b , 1 + xv a where the distance between input data x and v can be measured in several ways.
Synthesis of neurofuzzy networks

A generic algorithm for the determination of he neurofuzzy network, as describe in the previous slide, is proposed below: Synthesis algorithm
1 2 3 4
choosing a membership function shape; applying a clustering algorithm to the training set; associating to each cluster a membership function; learning the membership function parameters proportionally to the corresponding cluster density; projecting the clusters to the axis of the input space, determining the fuzzy values Aik for the rules antecedent; using the proposed network obtaining the crisp result y after defuzzication.
Neural Networks Lesson 9 - Fuzzy Logic 30 / 37
M. Scarpiniti
Fuzzy Clustering
In fuzzy clustering, data elements can belong to more than one cluster, and associated with each element is a set of membership levels. These indicate the strength of the association between that data element and a particular cluster. We can modify well-known clustering algorithm in order to obtain new fuzzy version of this algorithm. Another procedure is to consider several procedures valid for fuzzy sets and develop new clustering algorithm. We will show two clustering algorithms, one for each category:
1 2
Fuzzy C-means algorithm; Min-Max algorithm.
M. Scarpiniti
31 / 37
Fuzzy C-means
Given as examples the N vectors Xk , k = 1, 2, . . . , N, the C-means algorithm is summarized below: Fuzzy C-means algorithm
1 2 3
choosing C vectors Vi (c = 1, 2, . . . , C ) as centroids; for each input vector Xk choosing a membership function ik (x); evaluate the new centroids Vi =
N m k=1 ik Xk N m k=1 ik
i = 1, 2, . . . , C
with m an integer greater than 1.

4
evaluating the sum of the distance of the new centroid from the old ones:
C
E =
i=1 5
Vi Vi
if E < with a small positive and arbitrary threshold stop the algorithm, else repeat from step 2.
Neural Networks Lesson 9 - Fuzzy Logic 32 / 37
M. Scarpiniti
Fuzzy Min-Max
The Min-Max clustering algorithm uses a neurofuzzy network called MinMax and is based on the subdivision of the data space into hypercubes (HC) dened by a couple of points.
1
When a new data point is presented the network evaluates, looking to its neighborhood, if this point can be included in a present hypercube with an expansion. This neighborhood is measured by a membership function. If there no exist such an hypercube, a new one is created and its vertexes are dened by the data point Xk . If two or more hypercubes are partially overlapped, we must contract these HCs, deciding to which HC a data point belongs.
2 3
M. Scarpiniti
33 / 37
Fuzzy Min-Max
An example of Min-Max clustering algorithm is reported in the following gure.
M. Scarpiniti
34 / 37
Example: Fuzzy controlling of distance

Let us consider a fuzzy controller for controlling a distance between two cars traveling on a road. The following gure demonstrates how the braking force B depends upon the distance D and the velocity v . It is assumed that the distance D is constant, and only the mapping, B = f (v ) , is modeled.
The membership functions are reported below
M. Scarpiniti
35 / 37
Example: Fuzzy controlling of distance

The considered rules are the following ones IF velocity is low, THEN braking force is small. IF velocity is medium THEN braking force is medium. IF velocity is high, THEN braking force is large while the relation B = f (v ) is the following
M. Scarpiniti
36 / 37
References
L.A. Zadeh. Fuzzy sets. Information and Control, Vol. 8, pp. 338-353, 1965. V. Kecman. Learning and Soft Computing. Support Vector Machines, Neural Networks and Fuzzy Logic Models The MIT Press, 2001. O. Maimon and L. Rokach. Soft Computing for Knowledge Discovery and Data Mining. Springer, 2008.
M. Scarpiniti
37 / 37

Neural Networks Lesson 9 - Fuzzy Logic: Prof. Michele Scarpiniti

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Neural Networks Lesson 9 - Fuzzy Logic: Prof. Michele Scarpiniti

Uploaded by

Copyright:

Available Formats

Introduction Neurofuzzy Network