ch4

Next: ch4 Up: ch4 Previous: ch4

Gradient Descent

$\begin{picture}(400,250) %% a pt = 1/72 in \put(0,0){\psfig{file=./bookps/parabo... ...t(60,80){\psfig{file=./bookps/parabola-slope-fat.ps,height=1.1in}} \end{picture}$

Gradient

$\begin{displaymath}\nabla E[\vec{w}] \equiv \left[\frac{\partial E}{\partial w_{... ...artial w_{1}}, \cdots \frac{\partial E}{\partial w_{n}}\right] \end{displaymath}$

Training rule:

$\begin{displaymath}\Delta \vec{w} = -\eta \nabla E[\vec{w}] \end{displaymath}$

i.e.,

$\begin{displaymath}\Delta w_{i} = -\eta \frac{\partial E}{\partial w_{i}}\end{displaymath}$

Don Patterson 2001-12-13