Backpropagation Algorithm

Initialize all weights to small random numbers.

Until satisfied, Do For each training example, Do Input the training example to the network and compute the network outputs

For each output unit $k$

$$\delta_{k} \leftarrow o_{k}(1-o_{k})(t_{k}-o_{k})$$

For each hidden unit $h$

$$\delta_{h} \leftarrow o_{h}(1-o_{h})\sum_{k \in outputs}w_{h,k}\delta_{k}$$

Update each network weight $w_{i,j}$

$$w_{i,j} \leftarrow w_{i,j} + \Delta w_{i,j}$$

where

$$\Delta w_{i,j} = \eta \delta_{j} x_{i,j}$$