MultiLayer Perceptron Learning Algorithm

The algorithm for Perceptron Learning is based on the back-propagation rule discussed previously. This algorithm can be coded in any programming language, and in the case of this tutorial, Java for the applets. In this case we are assuming the use of the sigmoid function f(net) described earlier in the tutorial. This is because it has a simple derivative.

Algorithm:

1. Initialise weights and threshold.

   Set all weights and thresholds to small random values.

2. Present input and desired output

   Present input X_p = x₀ ,x₁ ,x₂ ,...,x_n-1 and target output T_p = t₀ ,t₁ ,...,t_m-1 where n is the number of input nodes and m is the number of output nodes. Set w₀ to be -ø, the bias, and x₀ to be always 1. For pattern association, X_p and T_p represent the patterns to be associated. For classification, T_p is set to zero except for one element set to 1 that corresponds to the class that X_p is in.

3. Calculate the actual output

   Each layer calculates the following:

   fy_pj = f [w₀x₀ + w₁x₁ + .... + w_nx_n]

   This is then passes this to the next layer as an input. The final layer outputs values o_pj.

4. Adapts weights

   Starting from the output we now work backwards.

   w_ij(t+1) = w_ij(t) + ñþ_pjo_pj , where ñ is a gain term and þ_pj is an error term for pattern p on node j.

   For output units

   þ_pj = ko_pj(1 - o_pj)(t - o_pj)

   For hidden units

   þ_pj = ko_pj(1 - o_pj)[(þ_p0w_j0 + þ_p1w_j1 + _....+ þ_pkw_jk)]

   where the sum(in the [brackets]) is over the k nodes in the layer above node j.

Previous:- MultiLayer Perceptron Learning Theory

Next:- Example 4: Two Dimensional Pattern Space revisted