Entropy Rate — Hidden Markov Models

Visualize how hidden state uncertainty drives the entropy rate of an HMM's observed emission sequence.

Transition p(stay in state) 0.80 Emission confusion ε 0.20 History length 200

Computing…

h(X) = lim H(Xₙ|X₁…Xₙ₋₁)

HMM: 2 hidden states S∈{0,1} with Markov transitions, each emitting symbol ∈{A,B} with confusion ε. Entropy rate = "average surprise per symbol" — plotted vs block length n. True entropy rate from Blackwell's measure.