Information Theory: Channel Coding & Shannon Capacity
BSC · BEC · AWGN · Noisy Channel Theorem · Shannon Limit · LDPC
BSC capacity
BEC capacity
AWGN capacity
Current rate R
Channel Parameters
Channel type
BSC (Binary Symmetric Channel)
BEC (Binary Erasure Channel)
AWGN (Gaussian)
Error probability p (BSC/BEC)
0.1
SNR (dB) for AWGN
5
Code rate R
0.5
Channel capacity C:
—
bits/use
Rate R:
—
bits/use
Achievable?
—
Shannon's Noisy Channel Theorem:
For R < C: ∃ codes with P_e→0 as n→∞
For R > C: P_e bounded away from 0
BSC capacity:
C = 1 − H_b(p) = 1 − h(p,1-p)
h(p,q) = −p log p − q log q
BEC capacity:
C = 1 − ε
(ε = erasure probability)
AWGN capacity:
C = ½ log₂(1 + SNR)
Shannon limit (BPSK):
Eb/N₀ ≥ ln 2 ≈ −1.59 dB
LDPC codes:
approach Shannon limit
within 0.0045 dB (turbo codes similar)