Restricted Boltzmann Machine

Energy E = −b·v − c·h − v·W·h  ·  Gibbs sampling between visible & hidden layers

Parameters

1.0
0.10
1

Energy Function

E(v,h) = −Σ b_i v_i − Σ c_j h_j
− Σ_ij v_i W_ij h_j
P(v,h) ∝ e^{−E(v,h)/T}
P(h_j=1|v) = σ(c_j + Σ_i v_i W_ij)
P(v_i=1|h) = σ(b_i + Σ_j W_ij h_j)
4×4 visible (16 units) + 4 hidden. Gibbs sampling alternates visible/hidden. Contrastive divergence CD-k approximates gradient.