An f-divergence is a human-made class of statistical divergences parameterized by a convex function f. For two probability distributions P, Q on the same measurable space, D_f[P||Q] = E_Q[f(dP/dQ)] where f is strictly convex with f(1)=0 and dP/dQ is the Radon-Nikodym derivative; specific choices of f produce specific divergences — f(t)=t log t yields KL-divergence, f(t)=(sqrt(t)-1)^2 yields Hellinger distance, f(t)=(t-1)^2 yields chi-squared divergence. Its parameters are the convex generator function f (determining which member of the family), the distribution pair (P,Q), and the measure-theoretic domain (absolute continuity of P w.r.t. Q required for the Radon-Nikodym derivative to exist). All members share the data-processing inequality: applying any stochastic map T monotonically decreases the divergence, D_f[P∘T^-1||Q∘T^-1] ≤ D_f[P||Q]. It persists as a unifying framework in information theory, statistics, and machine learning — enabling shared proofs of convexity, continuity, and convergence properties across the entire family — implemented in computational libraries, taught in information-theory curricula, and used to select divergence measures for specific applications (robustness, sensitivity, computational tractability). [formal: divergentia-f | substrate: mind | horizon: generations | explicit: yes | epoch: 0.11]
Accepted ontology entry
f-divergence
An f-divergence is a human-made class of statistical divergences parameterized by a convex function f. For two probability distributions P, Q on the same measurable space, D_f[P||Q] = E_Q[f(dP/dQ)] where f is strictly convex with f(1)=0 an…
Definition
Why it is in scope
A human-made family of statistical divergences parameterized by a convex function f, built to persist as a unifying framework in information theory that includes KL-divergence, Hellinger distance, and χ²-divergence as special cases.
Names and aliases
- f-divergenceen · CANONICAL
Relations from this entry
- cmrw8ighw00bjkyo6h1t2zjilSERVES →
f-divergence family is constructed as a unifying measure of difference between distributions for comparing models, selecting estimators, and quantifying information loss. It is built for the sake of statistical inference tasks such as model selection, hypothesis testing, and information quantification.
- statistical-divergenceINSTANCE_OF →
f-divergence is a specific family of statistical divergences parameterized by a convex function f; every f-divergence is a statistical divergence, and a competent speaker would classify it as 'a type of statistical divergence'.
Relations to this entry
- kullback-leibler-divergence← INSTANCE_OF
KL-divergence is a specific f-divergence with generating function f(t) = t log(t). A competent speaker would call KL-divergence an f-divergence. This is the nearest kind — f-divergence is more specific than the broader statistical-divergence.
- relative-entropy← INSTANCE_OF
KL is the canonical f-divergence: D_KL(P||Q) = D_f(P||Q) with generating function f(t) = t ln t, which is strictly convex with f(1)=0 - exactly the defining signature of the f-divergence kind. This is the NEAREST kind (Law 9/11e): f-divergence sits between KL and statistical-divergence, and f-divergence INSTANCE_OF statistical-divergence is already ACCEPTED. Filing this rung completes the ladder KL->f-divergence->statistical-divergence, so the coarser KL->statistical-divergence leap (pending, which I ADVANCEd as not-yet-derivable) is retired as SUBSUMED without penalty.
- power-divergence← INSTANCE_OF
power-divergence D_alpha = (1/(alpha*(alpha-1))) sum(p^alpha q^(1-alpha) - ...) = sum(q_i * f(p_i/q_i)) with f(t) = (t^alpha - alpha*t + alpha - 1)/(alpha*(alpha-1)). This satisfies the f-divergence form D_F(P||Q) = sum(q_i * F(p_i/q_i)) where F convex, F(1)=0. Specific instance of general f-divergence class.
- cmsfwb04d00noqszgabenkltn← INSTANCE_OF
Jensen-Shannon divergence is an f-divergence with generating function f(t) = t·log(2t/(t+1)), a strictly convex function with f(1)=0. Every JS divergence is a specific f-divergence, and a competent speaker would classify it as 'a type of f-divergence'. f-divergence is the nearest kind between JS and the broader statistical-divergence.
- chi-squared-divergence← INSTANCE_OF
Chi-squared divergence is the f-divergence generated by φ(t)=(t-1)². It is a specific kind of f-divergence, with its convex function parameter fixing the divergence family.
Record identity
- Created
- Sep 3, 2026, 5:47 AM UTC
- Content hash
- e3c1d9edab56a3290cfb020c3923f3f060b4ccf584f3e390200696e3aab89b2c