Rumelhart, Hinton & Williams, Learning representations by back-propagating errors, Nature 1986 · fig. 1

The Mirror Symmetry ResolverTwo hidden units, opposite weights, and the 1 : 2 : 4 cancellation

The two hidden units as balances tilt ∝ Σ skwk, positive weights on the right pan

Each bit that is on puts a block of size |w| on the pan of its weight's sign. The dashed mark is the threshold: past it, the total input x = Σ + θ is positive and the unit turns on.

Cancellation engine H1: left half against its mirror

Experiments sizes, ratios and symmetry of the weights

|w| bits 1 & 6
|w| bits 2 & 5
|w| bits 3 & 4
by Δw

All 64 patterns

Re-learn it with back-propagation the paper's run: ε = 0.1, α = 0.9, start uniform in ±0.3, gradient summed over all 64 cases