Rényi Entropy

Generalizing Shannon entropy with order α

Parameters

Preset distributions:

Drag bars to edit distribution

H_α (Rényi)
H_1 (Shannon)
H_0 (Hartley)
H_2 (Collision)
H_∞ (Min-ent)
Rényi entropy of order α:
H_α = 1/(1-α) · log₂(Σ pᵢᵅ)

α→1: Shannon entropy H = -Σpᵢlog₂pᵢ
α=0: log₂(support size)
α=2: collision entropy
α→∞: min-entropy -log₂(max pᵢ)

All orders satisfy H_α ≥ H_β for α≤β. Maximum is log₂(n) for uniform distribution.