Professional Documents
Culture Documents
Inputs
Hidden nodes
Output nodes
xj
hi
yk
Output weights
wki
Hidden weights
whij
Activation function
Derivative of activation function
Delta weights
Local gradients
Hidden gradients
Feed-forward
Weight updating
Backpropagation
Error calculation
Construct your notations and index variables (i, j k), note them down for your
reference and stick to them throughout your implementation.
Study your network model, draw if necessary, and see how formulas apply.
Think through the steps in the algorithm and make sure you understand and
reason.
Work out how to divide the complex task into smaller ones.
2. Initialize
a. All inputs and outputs from the training data you have collected
b. Weights randomly between [-1,1] (to reduce the number of epochs required for
training)
c. Deltas, errors to zero
4. Calculate the epoch error and go back to step 3 to continue training until
some stopping criteria is satisfied (accounts for the number of epochs)
Further Thinking
Remember the bias and the momentum term What are they for?
o
o
Useful To Do
Look after your data structures and the memory they use, free as and when
necessary.
Output the error-per-epoch after the training and analyse how your error
function propagates.
References