Conditional entropy is a human-made measure of remaining uncertainty in a random variable Y given knowledge of another random variable X. For discrete variables, H(Y|X) = -∑_x∑_y p(x,y) log p(y|x) = ∑_x p(x) H(Y|X=x). It extends Shannon entropy to the setting of dependent variables and satisfies the chain rule H(X,Y) = H(X) + H(Y|X). The measure is always non-negative and equals H(Y) when X and Y are independent. Conditional entropy persists through its role in defining mutual information (I(X;Y) = H(Y) - H(Y|X)), channel capacity, and conditional independence structures in graphical models. [formal: conditional-entropy | substrate: mind | horizon: a life | explicit: yes | epoch: 0.01]
Accepted ontology entry
conditional-entropy
Conditional entropy is a human-made measure of remaining uncertainty in a random variable Y given knowledge of another random variable X. For discrete variables, H(Y|X) = -∑_x∑_y p(x,y) log p(y|x) = ∑_x p(x) H(Y|X=x). It extends Shannon en…
Definition
Why it is in scope
Conditional entropy is a human-made measure of remaining uncertainty in one random variable given knowledge of another. It is the constructed map from information theory that quantifies H(Y|X) = -∑ p(x,y) log p(y|x), extending Shannon entropy to the setting of dependent variables. It persists through the mathematical framework of probability theory and its applications in communications, statistics, and machine learning.
Names and aliases
- conditional-entropyen · CANONICAL
Relations from this entry
- cmsfrqasq00aqqszg1stkbizqDERIVED_FROM →
Conditional entropy H(Y|X) extends Shannon entropy H(X) to the setting of dependent variables. Entropy existed first (Shannon 1948) and conditional entropy was derived from it by introducing the conditioning variable. The chain rule H(X,Y)=H(X)+H(Y|X) shows conditional entropy as the extension of entropy to joint distributions.
- cmsfrqasq00aqqszg1stkbizqINSTANCE_OF →
Conditional entropy is a specific kind of entropy — the entropy of one random variable given knowledge of another. A competent speaker would call it 'an entropy measure' — it is a specific case within the general concept of entropy.
Relations to this entry
- cmsfn3z6r001oqszgsol6nolu← DERIVED_FROM
Mutual information I(X;Y) = H(Y) - H(Y|X) is derived from the conditional entropy formula. Conditional entropy extends Shannon entropy to dependent variables; mutual information measures the reduction in conditional entropy compared to marginal entropy. Conditional entropy is the more fundamental construct that mutual information derives from.
Record identity
- Created
- Sep 3, 2026, 11:26 PM UTC
- Content hash
- eaa28b35d7c16bde9878bcb34d5e307a08a902e6624bab4096d920e499da76d3