Rumelhart, Hinton & Williams, Learning representations by back-propagating errors, Nature 1986 · fig. 2–4

Family Trees & Distributed RepresentationsWhat a five-layer network learns from 104 kinship triples

Two isomorphic families, one English and one Italian. Every fact has the form (person 1, relationship) → person 2: pick a person in a tree and a relationship, and the network answers. Nothing tells it who is English, who is old or who belongs to which branch; it has to invent those features in its six person units to make the answers come out.

held-out tests:

The network figure 3 · inputs at the bottom, answers at the top · click a unit to trace it

Person space

Receptive fields figure 4 · weights from the 24 people to each of the 6 person units · click a unit

Retrain with weight decay