# Geoffrey Hinton's AI Legacy > Exploring the Research, Mathematics, and Ideas That Shaped Modern Artificial Intelligence. An independent, open-access educational archive exploring Geoffrey Hinton's scientific contributions through original research papers, mathematical explanations, PyTorch implementations, experiments, and video lectures. This is an independent educational project by Mohammed Fahd Abrah. It is not an official Geoffrey Hinton resource and is not affiliated with or endorsed by Geoffrey Hinton or the University of Toronto. The original publications remain the authoritative sources; all research is credited to Geoffrey Hinton and his collaborators. Each paper page covers the problem the paper addressed, its core ideas, the mathematics step by step, a PyTorch implementation, experiments, limitations, its influence on later work, and, where available, video lectures. The site is a single page (https://geoffrey-hinton-ai-legacy.org/) whose papers are selected from the sidebar. ## Papers covered - Relaxation and Its Role in Vision (PhD Thesis) (1977, Hinton) - A Learning Algorithm for Boltzmann Machines (1985, Ackley, Hinton, Sejnowski) - Learning Representations by Back-Propagating Errors (1986, Rumelhart, Hinton, Williams) - Distributed Representations (1986, Hinton, McClelland, Rumelhart) - Learning Internal Representations by Error Propagation (1986, Rumelhart, Hinton, Williams) - Adaptive Mixtures of Local Experts (1991, Jacobs, Jordan, Nowlan, Hinton) - How Neural Networks Learn from Experience (1992, Hinton) - Autoencoders, Minimum Description Length and Helmholtz Free Energy (1994, Hinton, Zemel) - The Helmholtz Machine (1995, Dayan, Hinton, Neal, Zemel) - The Wake-Sleep Algorithm for Unsupervised Neural Networks (1995, Hinton, Dayan, Frey, Neal) - Products of Experts (1999, Hinton) - Training Products of Experts by Minimizing Contrastive Divergence (2002, Hinton) - Stochastic Neighbor Embedding (2002, Hinton, Roweis) - A Fast Learning Algorithm for Deep Belief Nets (2006, Hinton, Osindero, Teh) - Reducing the Dimensionality of Data with Neural Networks (2006, Hinton, Salakhutdinov) - Restricted Boltzmann Machines for Collaborative Filtering (2007, Salakhutdinov, Mnih, Hinton) - Visualizing Data using t-SNE (2008, van der Maaten, Hinton) - Deep Boltzmann Machines (2009, Salakhutdinov, Hinton) - Rectified Linear Units Improve Restricted Boltzmann Machines (2010, Nair, Hinton) - Deep Neural Networks for Acoustic Modeling in Speech Recognition (2012, Hinton, Deng, Yu) - ImageNet Classification with Deep Convolutional Neural Networks (2012, Krizhevsky, Sutskever, Hinton) - On the Importance of Initialization and Momentum in Deep Learning (2013, Sutskever, Martens, Dahl, Hinton) (coming soon) - Dropout: A Simple Way to Prevent Neural Networks from Overfitting (2014, Srivastava, Hinton, Krizhevsky, Sutskever, Salakhutdinov) (coming soon) - Distilling the Knowledge in a Neural Network (2015, Hinton, Vinyals, Dean) (coming soon) - Layer Normalization (2016, Ba, Kiros, Hinton) (coming soon) - Using Fast Weights to Attend to the Recent Past (2016, Ba, Hinton, Mnih, Leibo, Ionescu) (coming soon) - Dynamic Routing Between Capsules (2017, Sabour, Frosst, Hinton) (coming soon) - Matrix Capsules with EM Routing (2018, Hinton, Sabour, Frosst) (coming soon) - How to Represent Part-Whole Hierarchies in a Neural Network (2021, Hinton) - The Forward-Forward Algorithm: Some Preliminary Investigations (2022, Hinton) ## Interactive playgrounds - [The Puppet-Finding Playground](https://geoffrey-hinton-ai-legacy.org/playgrounds/puppet-relaxation/): Parallel LP relaxation in visual perception - [Joint Zone Tester](https://geoffrey-hinton-ai-legacy.org/playgrounds/joint-zones/): Space occupancy and separating borders - [Hyperspace Vector & Polytope Explorer](https://geoffrey-hinton-ai-legacy.org/playgrounds/hyperspace-polytope/): Continuous relaxation in linear programming - [Gomory Cutting Plane & Fractional Vertex Slicer](https://geoffrey-hinton-ai-legacy.org/playgrounds/gomory-slicer/): Integer linear programming in perception - [Master Relaxation Equation Dynamics Lab](https://geoffrey-hinton-ai-legacy.org/playgrounds/relaxation-dynamics/): Numerical stability and phase transitions - [Generators & Lazy Search Expansion](https://geoffrey-hinton-ai-legacy.org/playgrounds/lazy-generators/): Pruning combinatorial growth in relaxation - [The Search Arena](https://geoffrey-hinton-ai-legacy.org/playgrounds/search-arena/): Waltz filtering vs. the simplex method vs. parallel relaxation - [SETTLE: Schema Demons & Gated Forward Chaining](https://geoffrey-hinton-ai-legacy.org/playgrounds/settle-system/): Rules that only fire when relaxation trusts their premises - [The Stochastic Neuron & Simulated Annealing Chamber](https://geoffrey-hinton-ai-legacy.org/playgrounds/annealing-chamber/): Escaping local minima - [The Two-Phase Hebbian Machine](https://geoffrey-hinton-ai-legacy.org/playgrounds/two-phase-hebbian/): Learning from clamped and free-running statistics - [The Communication Bottleneck](https://geoffrey-hinton-ai-legacy.org/playgrounds/encoder-bottleneck/): Solving the encoder problem with a Boltzmann machine - [Code Repulsion & Territory Competition](https://geoffrey-hinton-ai-legacy.org/playgrounds/code-repulsion/): How the learning rule pushes clashing codes apart in hidden space - [Hinton Diagrams](https://geoffrey-hinton-ai-legacy.org/playgrounds/hinton-diagram/): The recursive weight matrix of a Boltzmann machine - [The G-Divergence & Information Landscape](https://geoffrey-hinton-ai-legacy.org/playgrounds/g-divergence/): Kullback–Leibler optimization in a Boltzmann machine - [Weight Divergence & the Noisy Clamping Lab](https://geoffrey-hinton-ai-legacy.org/playgrounds/noisy-clamping/): Why rare patterns demand infinite energies, and how noise prevents it - [The Speed–Accuracy Trade-Off](https://geoffrey-hinton-ai-legacy.org/playgrounds/speed-accuracy/): Annealing speed, completion errors and reaction times in a Boltzmann machine - [The Forward–Backward Chain Rule Lab](https://geoffrey-hinton-ai-legacy.org/playgrounds/backprop-stepper/): Back-propagation in a 2-2-1 network, one micro-step at a time - [The Mirror Symmetry Resolver](https://geoffrey-hinton-ai-legacy.org/playgrounds/mirror-symmetry/): Two hidden units, opposite weights, and the 1 : 2 : 4 cancellation - [Family Trees & Distributed Representations](https://geoffrey-hinton-ai-legacy.org/playgrounds/family-trees/): What a five-layer network learns from 104 kinship triples - [Receptive Fields & Weight Decay](https://geoffrey-hinton-ai-legacy.org/playgrounds/receptive-fields/): Reading the six person units of the family-tree network - [Momentum & Ravine Dynamics](https://geoffrey-hinton-ai-legacy.org/playgrounds/momentum-ravine/): Changing the velocity of the point in weight space, not its position - [The Unrolled Recurrent Net](https://geoffrey-hinton-ai-legacy.org/playgrounds/bptt-unrolled/): An iterative net, the layered net it equals, and back-propagation through time - [Local Minima & Extra Dimensions](https://geoffrey-hinton-ai-legacy.org/playgrounds/local-minima/): Why a few more connections let gradient descent get around the barriers - [Saturation & the Tolerance Window](https://geoffrey-hinton-ai-legacy.org/playgrounds/tolerance-saturation/): Why the paper counted any output above 0.8 or below 0.2 as correct - [Coarse Coding & the Hypercylinder](https://geoffrey-hinton-ai-legacy.org/playgrounds/coarse-coding/): Why broad, overlapping receptive fields pinpoint a feature more accurately - [The Binding Problem](https://geoffrey-hinton-ai-legacy.org/playgrounds/binding-problem/): Which x goes with which y, and what it costs to know - [Two Pathways: Recognition and Generation](https://geoffrey-hinton-ai-legacy.org/playgrounds/helmholtz-pathways/): Bottom-up inference and a top-down world model, with different weights ## Optional - [Sitemap](https://geoffrey-hinton-ai-legacy.org/sitemap.xml) - [About the author](https://www.programming-ocean.com/Mohammed-Al-Abrah-Profile.php)