The information bottleneck is an information-theoretic framework for learning compressed data representations that preserve maximal information about a target variable. It formalizes the trade-off between compression (minimizing the mutual information I(X;T) between input X and representation T) and prediction (maximizing I(T;Y) between T and target Y) through the Lagrangian L = I(X;T) - beta*I(T;Y), where beta controls the compression-prediction trade-off. The framework persists through algorithmic optimization: the Information Bottleneck method iteratively updates P(T|X) to maximize the bottleneck functional, converging to a representation that discards irrelevant information while retaining predictive structure. [formal: collisio | substrate: mind | horizon: hours | explicit: yes | epoch: 0.03]
Accepted ontology entry
information bottleneck
The information bottleneck is an information-theoretic framework for learning compressed data representations that preserve maximal information about a target variable. It formalizes the trade-off between compression (minimizing the mutual…
Definition
Why it is in scope
A human-made information-theoretic framework for learning compressed representations that retain only the information relevant to a target variable, formalized as an optimization over the trade-off between compression rate and predictive accuracy.
Names and aliases
- information bottlenecken · CANONICAL
Relations from this entry
- cmsftujnv00f0qszgpolboiajDEPENDS_ON →
The information bottleneck method is defined through mutual information and rate-distortion trade-offs — both core information-theoretic quantities. Remove information theory and the information bottleneck ceases to operate as a measurable, operational framework.
- cmsftujnv00f0qszgpolboiajDERIVED_FROM →
The information bottleneck principle was derived from information theory — specifically from Shannon's framework for quantifying information transfer. Claude Shannon's information theory (1948) predates and feeds into the information bottleneck method (Tishby et al., 1999). Which existed first? Information theory clearly came first.
Relations to this entry
No accepted relations in this direction.
Record identity
- Created
- Aug 5, 2026, 9:10 AM UTC
- Content hash
- 4db7cdb50a4775d68824fa477e3486e29ff8d959649aa99da8917e73b5fc052e