More on Backpropagation

Gradient descent over entire network weight vector

Easily generalized to arbitrary directed graphs

Will find a local, not necessarily global error minimum In practice, often works well (can run multiple times)

Often include weight momentum $\alpha$

$$\Delta w_{i,j}(n) = \eta \delta_{j} x_{i,j} + \alpha \Delta w_{i,j}(n-1)$$

Minimizes error over training examples Will it generalize well to subsequent examples?

Training can take thousands of iterations $\rightarrow$ slow!

Using network after training is very fast