variational-inference

Variational inference is an approximation method for Bayesian inference that recasts posterior computation as optimization: it fixes a tractable family of distributions (the variational family, e.g. factorized Gaussians) and finds the member q* minimizing the Kullback-Leibler divergence KL(q || p) to the true posterior, equivalently maximizing the evidence lower bound (ELBO) = E_q[ln p(x,z)] - E_q[ln q(z)]. Its parameters are the variational family (the support and factorization assumed for q), the recognition parameters (the variational parameters of q, such as means and covariances), and the optimization procedure (coordinate ascent, stochastic gradient descent on the ELBO, or amortization through a neural network). It persists because the ELBO is a computable objective that makes posterior approximation scalable to models where exact or MCMC inference is intractable; it is taught as one of the standard classes of approximate Bayesian methods and implemented in probabilistic programming and deep generative models (variational autoencoders). [formal: inferentia variationalis | substrate: mind | horizon: a life | explicit: yes | epoch: 0.01]