Ackley, Hinton & Sejnowski, A Learning Algorithm for Boltzmann Machines, 1985 · §3.1–3.2, Eq. 8–10, appendix
The G-Divergence & Information Landscape
Kullback–Leibler optimization in a Boltzmann machine
Run
Gradient descent ε
Sign rule δ
Steepness
step
0
/
G over two weights
● gradient descent
● sign rule
Reset view
drag to orbit · scroll to zoom · click the surface to start both probes there
G over the steps
log scale
Gradient
true direction against the step taken