Entropy Visualization

Binary source p(1) = 0.50

Joint: p(X=1) = 0.50

Joint: p(Y=1) = 0.50

Correlation ρ = 0.00

KL: q(1) = 0.50

Markov transition p(1→0) = 0.20

H(p) = -p·log₂(p) - (1-p)·log₂(1-p) — Binary entropy curve. Orange dot = current p.

Venn: H(X), H(Y), I(X;Y), H(X|Y), H(Y|X), H(X,Y) decomposition.

KL divergence D(P‖Q): asymmetric distance from P to Q over binary alphabet.

Markov chain entropy rate H(X) = stationary distribution × conditional entropy.