Hopfield nets

Architecture:

• Recurrent connections
• Binary theshold units

Idea: If connections are symmetric, there is a global energy function.

Each binary configuration has an energy. Binary threshold decision rule causes network to settle to a minimum of this energy function.

Global energy:

$E = - \sum_i s_i b_i - \sum_{i<j} s_i s_j w_ij$

• $w_ij$ is connection weight.
• Binary states of two neurons

Local computation for each unit:

$\delta E_i = E(s_i=0) - E(s_i=1) = b_i \sum_j s_j w_{ij}$