Information Bottleneck

The Information Bottleneck (Tishby et al. 1999) formalizes the tradeoff between compressing X into T while preserving information about Y. The IB curve traces achievable (I(X;T), I(T;Y)) pairs — the upper-left boundary. The parameter β controls the tradeoff: small β favors compression (fewer bits about X), large β allows more complexity to capture more of Y.