Rumelhart, Hinton & Williams, Learning representations by back-propagating errors, Nature 1986 · fig. 5, p. 535
The Unrolled Recurrent Net
An iterative net, the layered net it equals, and back-propagation through time
Play
steps T
tied (recurrent)
untied (deep net)
phase
The iterative net
one copy of each unit, its states kept as a history
The equivalent layered net
one layer per step, each weight once between every two layers
Each set of corresponding weights
The error going back
size of δ at each step, log scale
Training
every case, all steps of each
Reset weights
ε
α
Run it longer