Rate-Distortion Theory

Shannon's rate-distortion theorem (1959): the minimum bits/sample needed to reconstruct a source within distortion D. For a Gaussian source N(0,σ²) with MSE distortion: R(D) = ½log₂(σ²/D) for D ≤ σ². Below the curve is impossible; above it is achievable.

Source variance σ²2.0
Rate R (bits)1.5
Distortion D1.0
R(D) [Gaussian]— bits
D(R) = σ²·2^{−2R}—
Shannon entropy bound— bits
Current D/σ²—
R(D) = ½ log₂(σ²/D) bits/sample | D(R) = σ² · 2^{−2R}