Hopfield Network & Associative Memory

Store patterns as energy minima in a Hopfield network. Corrupt a pattern and watch the network recover it via gradient descent on the energy landscape. Explore capacity limits and spurious attractors.

Hopfield (1982): W_ij = (1/N)Σ_μ ξ_i^μ ξ_j^μ (Hebb's rule). Energy E = -½ Σ W_ij s_i s_j. Capacity: M_max ≈ 0.138·N patterns (Amit-Gutfreund-Sompolinsky 1985). Above capacity: spurious attractors, retrieval fails. Modern Hopfield (Ramsauer 2020): exponential storage capacity M ~ exp(N/2) with softmax update rules — equivalent to attention mechanism in Transformers.