Gradient Descent Algorithm

Gradient-Descent(training-examples, η )
- Each training example is a pair of the form $⟨ \overset{⇀}{x}, t ⟩$ , where $\overset{⇀}{x}$ is the vector of input values, and $t$ is the target output value. $η$ is the learning rate (e.g., .05).
Initialize each $w_{i}$ to some small random value
Until the termination condition is met, Do
1. Initialize each $Δ w_{i}$ to zero.
2. For each ⟨ x ⇀,t⟩ in training-examples, Do
  1. Input the instance $\overset{⇀}{x}$ to the unit and compute the output $o$
  2. For each linear unit weight $w_{i}$ , Do $Δ w_{i} \leftarrow Δ w_{i} + η (t - o) x_{i}$
3. For each linear unit weight $w_{i}$ , Do $w_{i} \leftarrow w_{i} + Δ w_{i}$