Gradient Descent Gradient-Descent( $training\_examples, \eta$)

Each training example is a pair of the form $\langle \vec{x}, t \rangle$, where $\vec{x}$ is the vector of input values, and $t$ is the target output value. $\eta$ is the learning rate (e.g., .05).

Initialize each $w_{i}$ to some small random value Until the termination condition is met, Do Initialize each $\Delta w_{i}$ to zero. For each $\langle \vec{x}, t \rangle$ in $training\_examples$, Do Input the instance $\vec{x}$ to the unit and compute the output $o$ For each linear unit weight $w_{i}$, Do

$$\Delta w_{i} \la \Delta w_{i} + \eta (t - o) x_{i}$$

For each linear unit weight $w_{i}$, Do

$$w_{i} \la w_{i} + \Delta w_{i}$$