evidence-lower-bound

The evidence lower bound (ELBO), also called the variational lower bound or negative variational free energy, is a computable lower bound on the log marginal likelihood (log evidence) log p(x) of a Bayesian model with observed data x and latent variables z. Given a variational distribution q(z) over the latent variables, the ELBO is ℒ(q) = E_q[log p(x, z) − log q(z)] = E_q[log p(x | z)] − KL(q(z) || p(z)), where p(z) is the prior. The identity log p(x) = ℒ(q) + KL(q(z) || p(z | x)) shows ELBO ≤ log p(x) since KL divergence is non-negative, with equality when q equals the true posterior. Its parameters are the generative model p(x, z) = p(x | z)p(z), the variational family q(z) (e.g. factorized Gaussians, mean-field), the variational parameters of q, and the optimization procedure (coordinate ascent, stochastic gradient descent, or amortization through an inference network). It persists as the standard optimization objective in variational inference — maximizing ℒ(q) with respect to q's parameters approximates the posterior while providing a tractable lower bound on model evidence — and is the mathematical foundation of variational autoencoders, implemented in probabilistic programming frameworks (Stan, Pyro, TensorFlow Probability, NumPyro), so that approximate inference remains scalable for models where exact or MCMC inference is intractable. [formal: inferentia inferior limitans | substrate: mind | horizon: a life | explicit: yes | epoch: 0.01]