Dayan, Hinton, Neal & Zemel, The Helmholtz Machine, Neural Computation 1995 · recognition φ and generation θ

Two Pathways: Recognition and GenerationBottom-up inference and a top-down world model, with different weights

recognition weights
units
input top unit

The two pathways click pixels to change the input · click a unit to inspect it

Every pair unit generative template θ beside recognition filter φ

Cyan positive, crimson negative, square area ∝ |w|. Top row of each grid: the top pixels; bottom row: the shifted pixels.

Learned φ against tied φ = θᵀ averaged over every shifter pattern