Ackley, Hinton & Sejnowski, A Learning Algorithm for Boltzmann Machines, 1985 · §3, Eq. 9–10, appendix

The Two-Phase Hebbian MachineLearning from the difference between p_ij and p′_ij

Clamped phasemeasure p_ij
Free-running phasemeasure p′_ij
Weight updateΔw ∝ p − p′

cycle 0 · G –

Clamped phase visible units held by the environment

waiting

Free-running phase nothing clamped, the network dreams

waiting

Settings

Temperature T Sweeps of statistics per phase

Update rule

Learning rate ε

Statistics

Noisy clamping Hidden units connected

G-measure Eq. 8, Σ P ln P/P′

P(V) against P′(V) environment, model

Gold: how often the environment shows each visible pattern. Violet: how often the free-running network produces it at equilibrium (exact, from every state). G is zero only when they match.

Weights Hinton diagram

Square area is |w|, cyan excitatory and crimson inhibitory. The diagonal holds each unit’s bias, its link to the always-on true unit (§2.1). Dots are pairs with no link.