Mutual Information — Blahut-Arimoto Algorithm

The Blahut-Arimoto algorithm computes channel capacity C = max_{p(x)} I(X;Y). Watch alternating optimization converge to the capacity-achieving input distribution.

Channel Matrix

Click cells to edit P(Y|X). Row = input, Col = output.

Convergence

Algorithm

Step 1: c(x,y) = P(y|x) exp(Σ_x q(x)·ln P(y|x))
Step 2: q(x) ← Σ_y c(x,y) / Σ_{x'y'} c(x',y')
Convergence: I(q) → C
Capacity C = ?
Run algorithm...

Optimal Input Distribution