Introduction & Basics Perceptrons Perceptron Learning and PLR Beyond Perceptrons Two-Layered Feed-Forward Neural Networks

5/13/2012
NEURAL NETWORKS
Chapter 20.5
Introduction & Basics Perceptrons Perceptron Learning and PLR Beyond Perceptrons Two-Layered Feed-Forward Neural Networks
1
2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, ET. AL.
5/13/2012
WHAT IS NEURAL NETWORK?
Inspired by biological nervous system. composed of a large number of highly interconnected processing elements. It resembles the brain in two respects: Knowledge is acquired by the network through a learning process. Interneuron connection strengths known as synaptic weights are used to store the knowledge.
2
5/13/2012
3
5/13/2012
INTRODUCTION
4
5/13/2012
INTRODUCTION
Known as:
Neural Networks (NNs) Artificial Neural Networks (ANNs) Connectionist Models Parallel Distributed Processing (PDP) Models
Neural Networks are a fine-grained, parallel, distributed computing model.
5
5/13/2012
INTRODUCTION
NN similar to brain:
knowledge is acquire experientially (learning) knowledge stored in connections (weights)
Brain composed of neurons cells:

dendrites collect input from other neurons single axon sends output to other neurons connected at synapses that have varying strength this model is greatly simplified
6
5/13/2012
INTRODUCTION
Constraints on human info. processing:

number of neurons: number of connections: neuron death rate: neuron birth rate: connection birth rate: 1011 104 per neuron 105 per day ~0 very slow
performance: about 102 msec, about 100 sequential neuron firings for "many" tasks
7
5/13/2012
INTRODUCTION
Attractions of NN approach:
can be massively parallel
MIMD, optical computing, analog systems from a large collection of simple processing elements emerges interesting complex global behavior
can do complex tasks

pattern recognition (handwriting, facial expressions, etc.) forecasting (stock prices, power grid demand) adaptive control (autonomous vehicle control, robot control)
is a robust computation
can handle noisy and incomplete data due to fine-grained distributed and continuous knowledge representation
8
5/13/2012
INTRODUCTION
Attractions of NN approach:
fault tolerant
ok to have faulty elements and bad connections
isn't dependent on a fixed set of elements and connections
degrades gracefully
continues to function, at a lower level of performance, when portions of the network are faulty
uses inductive learning

useful as a psychological model useful for a wide variety of high-performance applcations
9
5/13/2012
BASICS OF NEURAL NETWORKS
Neural network composition:

large number of units
simple neuron-like processing elements (PEs)
connected by a large number of links

directed from one unit to another
with a weight associate with each link

positive or negative real values means of long term storage adjusted by learning
and an activation level associated with each unit

result of the unit's processing unit's output
10
5/13/2012
Neural network configurations:

nodes: units
represent as a graph
edges: links
single-layered multi-layered feedback layer skipping fully connected (N2 links)

11
Layer 3 Layer 2
Layer 1
5/13/2012
Unit composition:
set of input links
from other units or sensors of the environment
set of output links

to other units or effectors of the environment
and an activation function

computes the activation level based on local info: wthe inputs from neighbors and the weights
1
Inputs
is a simple function of Activation the linear combination of its inputs
Output
wn
12
5/13/2012
Given n inputs, the unit's activation is defined by: a = g( (w1 * x1) + (w2 * x2) + ... + (wn * xn) )
where: wi are the weights xi are the input values g() is a simple non-linear function, commonly:
let ini be sum of wi xi for all i
step:
sign:
1
activation flips from 0 to 1 when ini >= threshold

activation flips from 1 to +1 when ini >= 0
1
1 sigmoid: activation transistions from 0 to 1 when ini 0 1/(1+e-x) where x is ini T

13
-1
5/13/2012
PERCEPTRONS: LINEAR THRESHOLD UNITS (LTU) LTUs where studied in the 1950s!
mainly as single-layered nets
since an effective learning algorithm was known
Perceptrons:
simple 1-layer network, units act independently composed of linear threshold units (LTU)
a unit's inputs, xi, are weighted, wi, and combined
step function computes activation level, a
x1 xi xn
w1
S
wn
14
5/13/2012
Threshold is just another weight (called the bias):

(w1 * x1) + (w2 * x2) + ... + (wn * xn) >= t is equivalent to (w1 * x1) + (w2 * x2) + ... + (wn * xn) + (t * -1) >= 0
PERCEPTRONS: LINEAR THRESHOLD UNITS (LTU)
-1 t x1 xn w1 a wn
15
5/13/2012
AND Perceptron:
inputs are 0 or 1 output is 1 when both x1 and x2 are 1
PERCEPTRONS: AND EXAMPLE
-1
.75
x1 x2 .5 a .5
.5*1+.5*1+.75*-1 = .25 output = 1
2-D input space
.5*1+.5*0+.75*-1 = -.25 output = 0
4 possible .5*0+.5*0+.75*-1 data points = -.75 output = 0 threshold is like a separating line
16
x1
1
0 0 1 x2
5/13/2012
OR Perceptron:
inputs are 0 or 1
PERCEPTRONS: OR EXAMPLE
-1
output is 1 when either x1 and/or x2 are 1
.25
x1 x2 .5 a .5
.5*1+.5*1+.25*-1 = .75 output = 1
2-D input space
.5*1+.5*0+.25*-1 = .25 output = 1
4 possible .5*0+.5*0+.25*-1 data points = -.25 output = 0 threshold is like a separating line
17
x1
1
0 0 1 x2
5/13/2012
PERCEPTRON LEARNING
How might perceptrons learn?

Programmer specifies:
numbers of units in each layer
connectivity between units
So the only unknown is the weights Perceptrons learn by changing their weights
supervised learning is used
the correct output is given for each training example
an example is a list of values for the input units correct output is a list desired values for the output units
18
5/13/2012
PERCEPTRON LEARNING: ALGORITHM
1. 2.
Initialize the weights in the network
Repeat until all examples correctly classified or some other stopping criterion is met
(usually with random values)
for each example e in training set do
b.
a. O = neural_net_output(network, e) T = desired output, i.e Target or Teacher's output c. update_weights(e, O, T)

Unlike other learning techniques, Perceptrons need to see all of the training examples multiple times. Each pass through all of the training examples is called an epoch.
19
5/13/2012
PERCEPTRON LEARNING: THE RULE
How should the weights be updated?

Determining how to update the weights is a case of the credit assignment problem. Perceptron Learning Rule:
wi = wi + Dwi where Dwi = a * xi * (T - O)
where xi is the value associated with ith input unit
a is a constant between 0.0 and 1.0

called the learning rate
20
5/13/2012
PERCEPTRON LEARNING: THE RULE

Dwi = a * xi * (T - O)
note it doesn't depend on wi
When wont the weight be change (i.e. Dwi = 0)?

correct output, i.e. T = O gives a * xi * 0 = 0 zero input, i.e. xi = 0 gives a * 0 * (T - O) = 0
What should happen to the weight if T=1 & O=0?

Increase it so that maybe next time the weighted sum will exceed the threshold causing output to be 1
What should happen to the weight if T=0 & O=1?

Decrease it so that maybe next time the weighted sum wont exceed the threshold causing output to be 0
21
5/13/2012
PERCEPTRON LEARNING: EXAMPLE
In class example, Learning Or
22
5/13/2012
PERCEPTRON LEARNING RULE (PLR)
PLR is also called the Delta Rule or the Widrow-Hoff Rule PLR is a variant of rule proposed by Rosenblatt in 1960 PLR is based on an idea of Hebb:
the strength of a connection between two units should be adjusted in proportion to the product of their simultaneous activations the product is used as a means of measuring the correlation between the values that are output by the two units
23
5/13/2012
PLR is a "local" learning rule only local information in the network is needed to update a weight PLR performs gradient descent in "weight space" this rule iteratively adjusts all of the weights so that at for each training example the error is monotonically non-increasing, i.e. ~decreases
24
5/13/2012
Perceptron Convergence Theorem says if a set of examples are learnable, then PLR will find the necessary weights
in a finite number of steps independent of the initial weights
This theorem says that if a solution exists, PLR's gradient descent is guaranteed to find an optimal solution (i.e., 100% correct classification) for any 1-layer neural network
25
5/13/2012
LIMITS OF PERCEPTRON LEARNING
What are the limitations of perceptron learning?

A single perceptron's output is determined by the separating hyperplane defined by (w1 * x1) + (w2 * x2) + ... + (wn * xn) = t
So, Perceptrons can only learn functions that are linearly separable (in input space).
26
5/13/2012
XOR Perceptron:
inputs are 0 or 1
PERCEPTRONS: XOR EXAMPLE
-1
output is 1 when x1 is 1 and x2 is 0 or x1 is 0 and x2 is 1
???
x1 x2 .5 a .5
2-D input space with 4 possible data points positives from negatives using a straight line?
27
x1
How do you separate
1
0 0 1 x2
5/13/2012
PERCEPTRON LEARNING SUMMARY
In general, the goal of learning in a perceptron is to adjust the separating hyperplane which divides an n-dimensional input space where n is the number of input units by modifying the weights (and biases) until all of the examples with target value 1 are on one side of the hyperplane, and all of the examples with target value 0 are on the other side of the hyperplane.
28
5/13/2012
BEYOND PERCEPTRONS
Perceptrons as a computing model are too weak because they can only learn linearly-separable functions. To enhance the computational ability, general neural networks have multiple layers of units. The challenge is to find a learning rule that works for multi-layered networks.
29
5/13/2012
BEYOND PERCEPTRONS
A feed-forward multi-layered network computes a function of the inputs and the weights. Input units (on left or bottom):
activation is determined by the environment
Output units (on right or top):

activation is the result
Hidden units (between input and output units):

cannot observe directly
Perceptrons have input units followed 30 by one layer of output units, i.e. no hidden units
5/13/2012
BEYOND PERCEPTRONS
NN's with one hidden layer of a sufficient number of units, can compute functions associated with convex classification regions in input space. NN's with two hidden layers are universal computing devices, although the complexity of the function is limited by the number of units.
If too few, the network will be unable to represent the function. If too many, the network will memorize examples and is subject to overfitting.
31
5/13/2012
TWO-LAYERED FEED-FORWARD NEURAL NETWORKS
Inputs Hidden Units Output Units Weights on links from input to hidden Weights on links from hidden to output Network Activations
ak=Ik Wk,j I1 I2
Aj
Wj,i
Ai
I3
I4 I5 I6
a1 = O1
a2 = O2
32
5/13/2012
ak=Ik Wk,j
Aj
Wj,i
Ai
Two Layered: count layers with units computing an activation Feed-forward: each unit in a layer connects forward to all of the units in the next layer no cycles
- links within the same layer - links to prior layers
I1 I2
I3
I4 I5 I6 Layer 1
33
a1 = O1
a2 = O2
no skipping layers
Layer 2
5/13/2012
NEURAL NETWORKS
Chapter 20.5
Two-Layered Feed-Forward Neural Networks Solving XOR Learning in Multi-Layered FeedForward NN Back-Propagation Computing the Change for Weights Other Issues & Applications
34
5/13/2012
XOR Perceptron?:
inputs are 0 or 1 output is 1 when I1 is 1 and I2 is 0 or I1 is 0 and I2 is 1
CONQUERING XOR
I1
.25 .5 .5 .5
OR
.5 O -.5
I2
Each unit in hidden layer acts like a perceptron learning a separating line
.5 .75
AND I1 1 0 0 1
top hidden unit acts like an OR perceptron bottom hidden unit acts like an AND perceptron
35
I2
5/13/2012
XOR Perceptron?:
inputs are 0 or 1 output is 1 when I1 is 1 and I2 is 0 or I1 is 0 and I2 is 1
CONQUERING XOR
I1
.25 .5 .5 .5
OR
.5 O -.5
I2
The output unit combines I1 these separating lines by intersecting the "half-planes" 1 defined by the separating lines
when OR is 1 and AND is 0 0 0
36
.5 .75
AND
I2 1
then output O, is 1
5/13/2012
LEARNING IN MULTI-LAYERED FEED-FORWARD NN
PLR doesn't work in multi-layered feed-forward nets since desired values for hidden units aren't known. Must again solve the Credit Assignment Problem
determine which weights to credit/blame for the output error in the network determine which weights in the network should be updated and how to update them
37
5/13/2012
LEARNING IN MULTI-LAYERED FEED-FORWARD NN
Back-Propogation:
method for learning weights in these networks generalizes PLR
Rumelhart, Hinton, Williams (re)discovered in 1986
Back-Propagation approach:
gradient-descent algorithm to minimize the error on the training data errors are propagated through the network starting at the output units and working backwards towards the input units
38
5/13/2012
BACK-PROPAGATION ALGORITHM
1. 2.
Initialize the weights in the network (usually random values like PLA) Repeat until all examples correctly classified or other stopping criterion is met for each example e in training set do
forward pass: Oi = neural_net_output(network, e) Ti = desired output, i.e Target or Teacher's output
a. b. c.
i.
calculate error (Ti - Oi) at the output units d. backward pass:

39
compute Dwj,i for all weights from hidden to output layer

ET. AL.
ii. compute Dwk,j for all weights from inputs to hidden layer 2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER,
5/13/2012
COMPUTING THE CHANGE FOR WEIGHTS
Back-propagation performs a gradient descent search in weight space to learn network weights. Given a network with n weights:
each configuration of weights is a vector, W, of length n that defines an instance of the network W can be considered a point in an n-dimensional weight space, where each dimension is associated with one of the connections in the network
40
5/13/2012
Given a training set of m examples:

each network defined by the vector W has an associated total error, E, on all of training data E the sum of the squared error (SSE) is defined as: E = E1 + E2 + ... + Em where each Ei is the squared error of the network on the ith training example
Given n output units in the network: Ei = ((T1 - O1)2 + (T2 - O2) 2 + ... + (T n - On) 2) / 2
Ti is the target value for the ith output unit Oi is the network output value for the ith output unit
41
5/13/2012
Visualized as a 2D error surface in weight space Each point in w1 w2 plane E is a weight configuration Each point has a total error E

2D surface represents errors for all weight configurations Goal is to find a lower point on the error surface (local minima) Gradient descent follows the direction of the steepest descent i.e. where E decreases the most
w2
.3
.8
w1
42
5/13/2012
The gradient is defined as: Gradient_E = [E/w1, E/w2, ..., E/wn] Then change the ith weight by: Dwi = - a * E/wi To compute the derivatives for calculating the gradient direction requires an activation function that is continuous, differentiable, non-decreasing and easily computed.
can't use the step function as in LTU's instead use the sigmoid function 1/(1+e-x) where x is ini the weighted sum of inputs
43
COMPUTING THE CHANGE FOR WEIGHTS: TWO-LAYER For weights between hidden and output units generalize PLR for sigmoid NEURAL NETWORK activation is:
5/13/2012
Dwj,i = -a * E/wj,i = -a * -aj * (Ti - Oi) * g'(ini) = a * aj * (Ti - Oi) * Oi * (1 - Oi) a learning rate parameter
wj,i weight on link from hidden unit j to output unit i
aj activation (i.e. output) of hidden unit j

Ti teacher output for output unit i Oi actual output of output unit i
44 g' 2derivative of M E S D . Sactivation function, whichYisCg' D Y g(1-g) sigmoid K R E N T N Y F R O M N O T E S B = ER, 001-2004 JA .
ET. AL.
5/13/2012
a1
w1,2
Dw1,2 = a
product of
I1 I2
learning rate
a1 activation along link (T2 O2) error O2 * (1 O2) g(in2)
I3
I4 I5 I6
O1
O2
45
COMPUTING THE CHANGE FOR WEIGHTS: TWO-LAYER For weights between inputs and hidden units: NEURAL NETWORK
5/13/2012
don't have teacher-supplied correct output values
infer the error at these units by "back-propagating"
error at an output units is "distributed" back to each of the hidden units in proportion to the weight of the connection between them
total error is distributed to all of the hidden units that contributed to that error
Each hidden unit accumulates some error from each of the output units to which it is connected
46
COMPUTING THE CHANGE FOR WEIGHTS: TWO-LAYER For weights between inputs and hidden units: NEURAL NETWORK Dwk,j = -a * E/wk,j
5/13/2012
= -a * -Ik * g'(inj) * S( wj,i * (Ti - Oi) * g'(ini) ) = a * Ik * aj * (1 - aj) * S( wj,i*(Ti-Oi)*Oi*(1-Oi) ) wk,j weight on link from input k to hidden unit j wj,i weight on link from hidden unit j to output unit i
a learning rate parameter

aj activation (i.e. output) of hidden unit j Ti teacher output for output unit i Oi actual output of output unit i 47
input value k
5/13/2012
Dw1,2 =
a*
I1 *
w1,2
product of
learning rate activation along link
a2
W2,i
I1 I2
a2 * (1 a2)
I3
I4 I5 I6
O1
g(in2)
error from outputs distributed back weight error g(ini)
S(w2,i* (Ti-Oi)* Oi*(1-Oi))

= w2,1* (T1-O1)* O1*(1-O1) + w2,2* (T2-O2)* O2*(1-O2)
O2
48
5/13/2012
EXAMPLE: COMMENTS ABOUT A4
1. 2.
Initialize the weights in the network (usually random values) Repeat until all examples correctly classified or other stopping criterion is met for each example e in training set do
a. b. c.
Forward Pass: Oi = neural_net_output(network, e) compute weighted sum, then sigmoid activation Ti = desired output, i.e Target or Teacher's output calculate error (Ti - Oi) at the output units d.
i.
Backward Pass:
49
compute Dwj,i = a * aj * (Ti - Oi) * Oi * (1 - Oi)
2 0 0ii. - 2 0 0 4 J A M E SDwk,j S Ka E NkT* aj * (1 O Mj) N O T(wS *(Ti-Oi.)*Oi*(1-Oi)) 1 compute D . = R * I N Y F R - a * S E j,i B Y C D Y E R , ET. AL.
5/13/2012
OTHER ISSUES
How should a network's error rate be estimated?

Report the average error rate by using an evaluation method such as cross-validation multiple times with different random initial weights.
How should the learning rate parameter be set?

Use a tuning set or cross-validation to train using several candidate values for alpha, and then select the value that gives the lowest error.
50
5/13/2012
OTHER ISSUES
How many hidden layers should be used?

usually just one hidden layer is used
How many hidden units should be in a layer?

too few and the concept can't be learn too many:
examples just memorized "overfitting", poor generalization
Use a tuning set or cross-validation to determine experimentally the number 51 of units that minimizes error.
5/13/2012
OTHER ISSUES
How many examples should be in training set?

The larger the better, but training takes longer.
To obtain 1 e correct classification on test set:

training set should be of size approximately n/e:
n is the number of weights in the network
e is test set error fraction between 0 and 1
train to classify 1 - e/2 of the training set correctly e.g. if n=80 and e=0.1 (i.e. 10% error on test set) 52
2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, training E T . of L . is 800 set A size
5/13/2012
OTHER ISSUES
When should training stop?

too soon and the concept isn't learned too late:
"overfitting", poor generalization
error rate will go up on the testing set
Train the network until the error rate on a tuning set begins increasing rather than training until the error (i.e. SSE) is minimized.
53
5/13/2012
APPLICATIONS
NETtalk (Sejnowski & Rosenberg, 1987):

learns to say text by mapping character strings to phonemes
Neurogammon (Tesauro & Sejnowski, 1989):

learns to play backgammon
Speech recognition (Waibel, 1989):

learns to convert spoken words to text
Character recognition (Le Cun et al., 1989):

54
learns to convert page image to text
5/13/2012
APPLICATIONS: ALVINN
ALVINN (Pomerleau, 1988): learns to control vehicle steering to stay in the middle of its lane topology: two-layered feed-forward network using back-propagation learning
topology: input
input is 480*512 pixel image 15 times per second
color image is preprocessed to obtain a 30*32 pixel image each pixel is one byte, an integer from 0 to 255 55 corresponding to the brightness of the image
5/13/2012
topology: output
output is one of 30 discrete steering positions
output unit 1 means sharp left output unit 30 means sharp right
target output is a set of 30 values

Gaussian distribution with a variance of 10 centered 2 on the desired steering directions: Oi = e[-(i-d) /10]
actual output for steering determined by

computing a least-squares best fit of output units' values to a Gaussian distribution with a variance of 10 peak of this distribution is taken as the steering direction
error for learning is: target output - actual output

topology: hidden
56
5/13/2012
Learning:
continuously learns on the fly by observing
human driver (takes ~5 minutes from random initial weights)
itself (do an epoch of training every 2 seconds there after)
problem with using real continuous data:

there aren't negative examples network may overfit data in recent images (e.g. straight road) at the expense of past images (e.g. road with curves)
solutions
generate negative examples by synthesizing views of the road that are incorrect for current steering
57 maintain a buffer of 200 real and synthesized images that keeps some images in many different 2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, steering. directions ET AL.
5/13/2012
Results:
has driven at speeds up to 70 mph has driven continuously for distances up to 90 miles
has driven across the continent during different times of the day and with different traffic conditions
can drive on:
single lane roads and highways multi-lane highways paved bike paths dirt roads
see for yourself (video) 58

5/13/2012
SUMMARY
Advantages
parallel processing architecture robust with respect to node failure
fine-grained, distributed representation of knowledge

robust with respect to noisy data incremental algorithm (i.e. learn as you go) simple computations empirically shown to work well for many problem domains
59
5/13/2012
SUMMARY
Disadvantages
slow training (i.e. takes many epochs) poor interpretability (i.e. difficult to get rules)
ad hoc network topologies (i.e. layouts)

hard to debug because distributed representations preclude content checking may converge to local, not global, minimum of error may be hard to describe a problem in terms of features with numerical values not known how to model higher-level cognitive mechanisms with NN model
60

Introduction & Basics Perceptrons Perceptron Learning and PLR Beyond Perceptrons Two-Layered Feed-Forward Neural Networks

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction & Basics Perceptrons Perceptron Learning and PLR Beyond Perceptrons Two-Layered Feed-Forward Neural Networks

Uploaded by

Copyright:

Available Formats

5/13/2012

WHAT IS NEURAL NETWORK?

Neural Networks are a fine-grained, parallel, distributed computing model.

Brain composed of neurons cells:

Constraints on human info. processing:

can do complex tasks

isn't dependent on a fixed set of elements and connections

uses inductive learning

BASICS OF NEURAL NETWORKS

Neural network composition:

connected by a large number of links

with a weight associate with each link

and an activation level associated with each unit

Neural network configurations:

BASICS OF NEURAL NETWORKS

single-layered multi-layered feedback layer skipping fully connected (N2 links)

BASICS OF NEURAL NETWORKS

set of output links

and an activation function

is a simple function of Activation the linear combination of its inputs

BASICS OF NEURAL NETWORKS

activation flips from 0 to 1 when ini >= threshold

1 sigmoid: activation transistions from 0 to 1 when ini 0 1/(1+e-x) where x is ini T

2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, ET. AL.

since an effective learning algorithm was known

2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, ET. AL.

Threshold is just another weight (called the bias):

PERCEPTRONS: LINEAR THRESHOLD UNITS (LTU)

PERCEPTRONS: AND EXAMPLE

2-D input space

.5*1+.5*0+.75*-1 = -.25 output = 0

2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, ET. AL.

output is 1 when either x1 and/or x2 are 1

2-D input space

.5*1+.5*0+.25*-1 = .25 output = 1

2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, ET. AL.

How might perceptrons learn?

connectivity between units

PERCEPTRON LEARNING: ALGORITHM

Initialize the weights in the network

(usually with random values)

for each example e in training set do

a. O = neural_net_output(network, e) T = desired output, i.e Target or Teacher's output c. update_weights(e, O, T)

2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, ET. AL.

PERCEPTRON LEARNING: THE RULE

How should the weights be updated?

a is a constant between 0.0 and 1.0

PERCEPTRON LEARNING: THE RULE

When wont the weight be change (i.e. Dwi = 0)?

What should happen to the weight if T=1 & O=0?

What should happen to the weight if T=0 & O=1?

PERCEPTRON LEARNING: EXAMPLE

In class example, Learning Or

PERCEPTRON LEARNING RULE (PLR)

PERCEPTRON LEARNING RULE (PLR)

PERCEPTRON LEARNING RULE (PLR)

LIMITS OF PERCEPTRON LEARNING

What are the limitations of perceptron learning?

PERCEPTRONS: XOR EXAMPLE

output is 1 when x1 is 1 and x2 is 0 or x1 is 0 and x2 is 1

How do you separate

2001-2004 JAMES D. SKRENTNY FROM NOTES BY C. DYER, ET. AL.

.51+.50+.75*-1 = -.25 output = 0

.51+.50+.25*-1 = .25 output = 1