Shannon Channel Capacity
Mutual Information · Noisy Channel Theorem
Channel Parameters
Signal-to-Noise Ratio (dB):
10 dB
Bandwidth (Hz):
1000
Input Distribution α:
0.50
BSC Error Probability ε:
0.10
—
AWGN Cap (bps)
—
BSC Cap (bits)
—
I(X;Y) bits
—
H(X) bits
Shannon-Hartley:
C = B·log₂(1+S/N)
BSC capacity:
C = 1 − H_b(ε)
where H_b(ε) = −ε log₂ε − (1−ε)log₂(1−ε)
Mutual information:
I(X;Y) = H(X) − H(X|Y)
= Σ p(x,y) log₂[p(x,y)/p(x)p(y)]
Capacity = max over input distributions p(X).