Optimal compression tradeoff on the information plane
Controls
I(X;Y) = — I(T;X) = — I(T;Y) = —
The Information Bottleneck tradeoff: min I(T;X) − β·I(T;Y)
Each point on the curve corresponds to one β. The curve traces from (0,0) (no info) to (H(X), I(X;Y)) (full info). The bottleneck is how much of X can be discarded while retaining I(T;Y).