A real-valued function on R^n defined as log-sum-exp(x) = log(∑_{i=1}^n exp(x_i)), where log is the natural logarithm. It is a smooth, differentiable approximation to max(x), with the bound max(x) ≤ log-sum-exp(x) ≤ max(x) + log(n). The function persists through numerical computation practice as the log-sum-exp trick: log-sum-exp(x − c) = log(∑_i exp(x_i − c)) for any constant c, commonly used with c = max(x) to prevent overflow in probability computations. It appears in the derivation of the cross-entropy loss, in computing the gradient of the softmax function, in variational lower bounds via Jensen's inequality (log-sum-exp(x) ≥ x for all x when normalized), and in convex optimization as a convex function with gradient given by the softmax operator. [formal: logarithmus summa exponentialis | substrate: mind | horizon: hours | explicit: yes | epoch: 0.01]
Accepted ontology entry
log-sum-exp
A real-valued function on R^n defined as log-sum-exp(x) = log(∑_{i=1}^n exp(x_i)), where log is the natural logarithm. It is a smooth, differentiable approximation to max(x), with the bound max(x) ≤ log-sum-exp(x) ≤ max(x) + log(n). The fu…
Definition
Why it is in scope
A smooth approximation to the maximum function: log-sum-exp(x) = log(∑_i exp(x_i)). Human-made computational construct used in optimization, machine learning, and probability to replace the non-differentiable max with a differentiable surrogate, and in numerically stable computation of log-probabilities via the log-sum-exp trick.
Names and aliases
- log-sum-expen · CANONICAL
Relations from this entry
- cmrwjrnhj00e5soaclc3n5sr2INSTANCE_OF →
log-sum-exp IS a specific kind of numerical method — it is a technique for computing log(sum(exp(x))) in a numerically stable way. The nearest kind is numerical-method: a procedure for approximating mathematical computations. Direction is specific→general.
- exponential-functionDEPENDS_ON →
Removal test (Laws 2b/8b): log-sum-exp is defined as log(∑_i exp(x_i)) — the exponential function is inside its own formula. Remove exponential-function now and log-sum-exp stops operating: it cannot be evaluated, its log-sum-exp trick (shifting by c = max(x) to prevent overflow) is exactly a property of exp, and its gradient (the softmax operator) is exp(x_i − log-sum-exp(x)). The dependency is not chronology or mere sayability — exp is a load-bearing component of the definition, the same relation as the accepted log-sum-exp DEPENDS_ON logarithm edge for the log half of the formula.
- cmrwraj00012csoaclnxvq1ekDEPENDS_ON →
log-sum-exp(x) = log(∑_i exp(x_i)) is defined using the logarithm. Remove the logarithm function and log-sum-exp has no definition — its very identity depends on the logarithm operation. The removal test passes: without logarithm, log-sum-exp ceases to operate as a function.
Relations to this entry
No accepted relations in this direction.
Record identity
- Created
- Sep 5, 2026, 5:51 AM UTC
- Content hash
- 2ca9710d9f8eb0c9221ba08b5cab83267d80f1263f81d7036d233a5390b10fe5