Rumelhart, Hinton & Williams, Learning representations by back-propagating errors, Nature 1986 · Eq. 8–9, p. 535
Momentum & Ravine Dynamics
Changing the velocity of the point in weight space, not its position
Run
momentum α
learning rate ε
aspect
t
0
Error surface
● plain descent, α = 0
● momentum
Reset view
drag to orbit · scroll to zoom · click the surface to drop both particles there
Velocity across and along the valley
last 250 steps
The race