Rényi Entropy
H_q — parameterized entropy family from min-entropy to max-entropy
Distribution
Shape
Uniform
Peaked (Gaussian)
Bimodal
Power law
Exponential
Near-Dirac
Bins n
32
Shape parameter
0.5
Entropy sweep
q min
0.1
q max
5
Rényi entropy of order q:
H_q = log(Σ pᵢ^q) / (1−q)
Limits:
q→0: H_0 = log(support size)
q→1: H_1 = Shannon = −Σpᵢ log pᵢ
q→∞: H_∞ = −log(max pᵢ) (min-entropy)
Monotonically decreasing in q. Converges for all q>0. Generalizes Tsallis entropy.
H_0 (Hartley):
—
H_1 (Shannon):
—
H_2 (collision):
—
H_∞ (min):
—