Alternative Error Functions

Penalize large weights:

$$E(\vec{w}) \equiv \frac{1}{2}\sum_{d \in D} \sum_{k \in outputs} (t_{kd} - o_{kd})^2 + \gamma \sum_{i,j}w_{ji}^{2}$$

Train on target slopes as well as values:

$$E(\vec{w}) \equiv \frac{1}{2} \sum_{d \in D} \sum_{k \in outp... ...j_d} - \frac{\partial o_{kd}}{\partial x^j_d}\right)^2 \right]$$

Tie together weights: e.g., in phoneme recognition network