Professional Documents
Culture Documents
=
=
n
i
i i
w x sign Y
1
u
y3 = sign[x1w13+x2w23-] = sign[1+(-1)-0.1] = sign[-0.1] = -1
y4 = sign[x1w14+x2w24-] = sign[1+(-1)-0.1] = sign[-0.1] = -1
y5 = sign[x1w15+x2w25-] = sign[1+(-1)-0.1] = sign[-0.1] = -1
y6 = sign[y3w36+ y4w46+y5w56-] = sign[-1+(-1)+(-1)-0.1] = sign[-
3.1] = -1
y7 = sign[y3w37+ y4w47+y5w57-] = sign[-1+(-1)+(-1)-0.1] = sign[-
3.1] = -1
[3 marks each]
[2 marks for label]
[3 marks for neurons and connection]
1
2
6
7
4
5
3
Input Hidden Output
6) Describe the adaptive learning rate used in training algorithm in a multilayer neural
network.
[5 marks]
The small learning rate parameter, , causes small changes to the weights in the
network from one iteration to the next, and thus leads to the smooth learning
curve.
On the other hand, if the learning rate parameter, , is made larger to speed up the
training process, the resulting larger changes in the weights may cause instability
and, as a result, the network may become oscillatory.
[1 mark]
To accelerate the convergence and yet avoid the danger of instability, the following
heuristics are applied:
- Heuristic 1. If the change of the sum of squared errors has the same algebraic
sign for several consequent epochs, then the learning rate parameter, should
be increased.
- Heuristic 2. If the algebraic sign of the change of the sum of squared errors
alternates for several consequent epochs, then the learning rate parameter,
should be decreased.
[4 marks]