Alternative Error Functions

Penalize large weights: $E (\overset{⇀}{w}) \equiv \frac{1}{2} \sum_{d \in D} \sum_{k \in outputs} (t_{kd} - o_{kd})^{2} + γ \sum_{i, j} w_{ji}^{2}$
Add a term for errors in the slope of the target function $E (\overset{⇀}{w}) \equiv \frac{1}{2} \sum_{d \in D} \sum_{k \in outputs} [(t_{kd} - o_{kd})^{2} + μ \sum_{j \in inputs} {(\frac{\partial t_{kd}}{\partial x_{d}^{j}} - \frac{\partial o_{kd}}{\partial x_{d}^{j}})}^{2}]$
Tie together weights: e.g., in phoneme recognition network