The accepted ontology
Browse accepted Systema Constructum entries: human-made tools, institutions, methods, practices, works, and their definitions.
100 entries in this page
Probability calibration is the systematic adjustment of predicted probabilities so they align with observed frequencies. Parameters: a set of raw probabilistic predictions, corresponding observed outcomes, and a calibration function (isotonic regression, Plat…
Decision-architecture is the deliberate design of choice environments, decision processes, and information structures to guide how individuals or organizations arrive at decisions. It structures options, frames alternatives, sequences information, and builds…
A systematic approach to planning, organizing, and controlling how time is allocated across tasks and activities, with the goal of increasing effectiveness, productivity, and work-life balance. Time management operates through techniques (time blocking, prior…
Transfer learning is a machine learning approach in which a model developed for one task is repurposed as the starting point for a model on a second task. It exploits knowledge learned in one context to improve generalization in another, reducing data and com…
A resampling technique for estimating the sampling distribution of a statistic by repeatedly drawing samples with replacement from an observed dataset. The bootstrap method treats the empirical distribution of the data as a proxy for the population distributi…
A calibration error metric quantifies the discrepancy between a model's predicted probabilities and the actual observed frequencies of outcomes. It is computed by binning predictions into probability intervals, computing the accuracy (fraction of positives) w…
A shapley value is a solution concept from cooperative game theory that distributes a finite pool of surplus among participants in proportion to their average marginal contribution across every possible coalition. It is defined by four axioms: efficiency (the…
Statistical learning is the practice of inferring quantitative models from data by combining probability theory with optimization. It takes observed samples and produces a parameterized function — a density estimator, classifier, or regressor — whose paramete…
Shannon entropy is a mathematical measure of the average uncertainty or information content in a random variable's probability distribution. For a discrete random variable X with outcomes x₁, x₂, ..., xₙ and probability mass function p(x), the entropy is defi…
Jensen-Shannon divergence is a symmetric measure of similarity between two probability distributions P and Q. It is defined as JS(P||Q) = 0.5·KL(P||M) + 0.5·KL(Q||M) where M = 0.5·(P+Q) is the midpoint distribution and KL denotes the Kullback-Leibler divergen…
An activation function is a mathematical transformation applied to a neuron's weighted input in a neural network to produce its output signal. Its parameters are (1) the input scalar or vector, (2) the functional form (e.g., sigmoid, ReLU, tanh, softmax), and…
The information bottleneck is an information-theoretic framework for learning compressed data representations that preserve maximal information about a target variable. It formalizes the trade-off between compression (minimizing the mutual information I(X;T)…
Rate distortion theory is a branch of information theory that quantifies the fundamental limits of lossy data compression. It determines the minimum achievable data rate (rate) at which a source can be reproduced within a specified distortion tolerance. The k…
Surprisal (self-information) quantifies the information content of a single event. For an event with probability p, surprisal is I(p) = -log_b(p), where b is the logarithm base (2 yields bits, e yields nats, 10 yields hartleys). Parameters: (1) a probability…
Posterior probability is the updated probability of a hypothesis after incorporating observed evidence. It is computed via Bayes theorem: P(H|E) = P(E|H) · P(H) / P(E), where P(H) is the prior probability of the hypothesis, P(E|H) is the likelihood of the evi…
Information theory is a mathematical framework for quantifying, transmitting, and processing information, founded by Claude Shannon in 1948. Its core parameters are: (1) entropy H(X) = -sum(p(x) log p(x)) as the measure of uncertainty in a discrete random var…
Prior probability is a statistical concept representing the initial degree of belief in a hypothesis before observing new evidence. Its parameters are (1) a hypothesis or set of hypotheses, (2) a probability value or distribution expressing initial confidence…
Expected value is the weighted average of all possible outcomes of a random variable, where each outcome is weighted by its probability of occurrence. For discrete variables, it is the sum x·P(x) over all outcomes x; for continuous variables, it is the integr…
Naive Bayes is a probabilistic classification algorithm that applies Bayes' theorem with a strong independence assumption: it treats all features as conditionally independent given the class label. Its parameters are (1) a set of features assumed independent,…
Bayes theorem is a human-made mathematical theorem in probability theory that describes how to update the probability of a hypothesis in light of new evidence. Formally, for events A and B with P(B) > 0, the theorem states: P(A|B) = P(B|A) · P(A) / P(B), wher…
Entropy is a human-made mathematical measure of uncertainty, randomness, or information content within a probability distribution or data set. For a discrete random variable with outcomes x_i and probabilities p_i, entropy H is defined as H = -Σ p_i · log(p_i…
Class imbalance is a human-made concept describing when the class distribution in a classification dataset is skewed — one or more minority classes occur significantly less frequently than majority classes. It is defined by three parameters: (1) the number of…
Conditional probability is a human-made formalism that quantifies how the likelihood of an event A changes given that another event B has occurred or is known to be true. It is computed as P(A|B) = P(A∩B) / P(B) when P(B) > 0, where P(A∩B) is the joint probab…
Dimensionality reduction is a family of mathematical techniques that transform data from a high-dimensional space into a lower-dimensional representation while preserving the structure most relevant to the task — whether that is global variance, local neighbo…
An attention mechanism is a human-made computational technique used in neural networks that allows a model to selectively focus on different parts of its input when producing an output. It works by computing weighted sums over input representations, where the…
Model comparison is the systematic evaluation and contrast of two or more predictive models against shared criteria to determine relative performance and suitability for a given task. Its parameters are: (1) a set of candidate models to compare, (2) a set of…
Decoding is the process of converting encoded information back to its original or interpretable form. Its parameters are: (1) an encoded input stream or message, (2) a decoding algorithm or rule set that reverses the encoding transformation, (3) an output in…
Mutual information is a measure from information theory that quantifies how much knowing the value of one random variable reduces uncertainty about another. Its parameters are: (1) two random variables X and Y, (2) their joint probability distribution p(x,y),…
A decoy is a fabricated artifact, signal, or behavior whose designed purpose is to draw attention, effort, or attack away from a genuine target. Its parameters: (1) fabricated nature — the decoy is intentionally not the real thing, (2) intentional misdirectio…
A human-made construct for deliberately fabricated information presented as legitimate journalism. Its parameters are: (1) content that is materially false, (2) intent to deceive the audience about the content's veracity, (3) packaging as credible reporting o…
A specific cross-validation procedure where N is replaced by an integer k \u2265 2, and the dataset is split into k folds of (approximately) equal size. Each fold takes a turn as the validation set while the remaining k-1 folds serve as the training set. Its…
A probability density function (PDF) is a function f(x) characterizing the probability distribution of a continuous random variable X. Its parameters are: (1) the support set S over which f(x) ≥ 0, (2) the normalization constraint ∫f(x)dx = 1 over S, and (3)…
A decision boundary is a hypersurface in the feature space of a classifier that partitions the input domain into regions, each assigned to a distinct class. Its parameters are: (1) the feature space dimensionality d, (2) the set of classes C, (3) the classifi…
A cumulative distribution function (CDF) is a mathematical function F(x) that maps each real value x to the probability P(X ≤ x) that a random variable X takes a value at or below x. Its parameters are: (1) a real-valued random variable X with a defined sampl…
A false negative rate is the proportion of actual positive cases that are incorrectly classified as negative by a binary classifier. It is computed as FN/(TP+FN), where FN is the count of false negatives and TP+FN is the total number of actual positives. Unli…
True negative rate (TNR), also called specificity, is the proportion of actual negative cases correctly identified by a binary classifier or diagnostic test. It is calculated as TNR = TN / (TN + FP), where TN is true negatives and FP is false positives. The c…
False positive rate (FPR), also known as the fall-out, is the proportion of actual negative cases that a binary classifier or test incorrectly identifies as positive. It is calculated as FPR = FP / (FP + TN), where FP is the number of false positives and TN i…
specificity is a human-made evaluative dimension for descriptions, tests, and categories, measuring how narrowly a sign picks out a target class. Parameters: granularity of the description, the size of the target set, the cost of over-generalization, and the…
A sensitivity plot is a visual encoding that displays how a model's predictions or outputs change as one or more input parameters vary. It carves by method: the plot persists a systematic perturbation of inputs (one-at-a-time or in combination) and records th…
The area under the receiver operating characteristic (ROC) curve is a scalar metric computed by numerical integration (trapezoidal rule) over the curve plotting true positive rate against false positive rate across all classification thresholds. It quantifies…
Cross-entropy is a measure from information theory that quantifies the divergence between two probability distributions p and q over the same sample space. Formally defined as H(p,q) = -Σ p(x) log q(x) for discrete distributions or -∫ p(x) log q(x) dx for con…
Log loss (also called binary cross-entropy or logarithmic loss) is a loss function that measures the performance of a probabilistic binary classifier. Given true labels y∈{0,1} and predicted probabilities p∈(0,1), log loss = -[y·log(p) + (1-y)·log(1-p)]. It i…
F-measure (or F-score) is a statistical metric that combines precision and recall into a single score using their harmonic mean. For a given threshold, precision measures the fraction of predicted positives that are truly positive, while recall measures the f…
Decision theory is a human-made mathematical and computational framework for making optimal choices under uncertainty. It formalizes the process of selecting among alternatives by assigning utilities to outcomes, probabilities to states of the world, and comb…
A classification threshold is a scalar value on a classifier's continuous output scale (probability, score, or logit) above which instances are assigned to the positive class and below which to the negative class. Parameters: a numeric threshold value, a scor…
A receiver operating characteristic (ROC) curve is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. Parameters: a binary classifier, a set of decision thresholds, and the resulti…
KL divergence (Kullback-Leibler divergence) is a mathematical function D_KL(P,Q) that quantifies the information loss when distribution Q is used to approximate distribution P. Parameters: two probability distributions P and Q over the same sample space, wher…
A decision threshold is a numerical cutoff value that maps a continuous prediction, score, or test statistic onto a discrete outcome — e.g. classifying anything above 0.5 as positive in binary classification, or declaring detection when a signal exceeds a noi…
The gini coefficient quantifies impurity as G = 1 − Σᵢ pᵢ², where pᵢ is the fraction of elements belonging to class i. In a decision-tree node, G = 0 when all elements belong to one class (pure) and G approaches its maximum when classes are evenly distributed…
Recall is a classification metric that measures the proportion of actual positives correctly identified by a classifier: recall = TP / (TP + FN), where TP is true positives and FN is false negatives. Invented in information retrieval during the 1950s and adop…
Probability theory is a human-made formal discipline that models uncertainty through Kolmogorov's axiomatic framework. It assigns numeric measures to events within a sample space, enabling rigorous reasoning about randomness, risk, and stochastic processes. I…
Information entropy is a human-made mathematical concept from information theory, introduced by Claude Shannon in 1948. It quantifies the average amount of uncertainty or information content in a probability distribution. Formally, for a discrete random varia…
Hyperparameter tuning is a systematic procedure for selecting the optimal hyperparameters — configuration settings that govern the training process of a model — by searching a defined space of possible values. The practice persists through documentation, auto…
Set theory is a human-made formal discipline that studies collections of abstract objects called sets, their membership relations, and the operations and axioms governing them. It provides the foundational language for mathematics by defining what sets are, h…
Bayesian optimization is a human-made sequential strategy for maximizing the value of an expensive black-box function when derivatives are unavailable. It constructs a probabilistic surrogate model — typically a Gaussian process — over the unknown objective,…
The F1 score is a human-made evaluation metric used in machine learning and information retrieval to measure the accuracy of a classification model. It is the harmonic mean of precision and recall: F1 = 2 · (precision · recall) / (precision + recall). It prov…
A support vector machine (SVM) is a supervised learning algorithm that constructs an optimal hyperplane in a high-dimensional feature space to separate data points into distinct classes. It maximizes the margin between classes and uses kernel functions to han…
Bagging (bootstrap aggregating) is an ensemble learning method that reduces variance by training multiple base learners on independently drawn bootstrap samples (sampling with replacement) of the training data, then aggregating their predictions by averaging…
A machine learning paradigm that constructs and combines multiple models (base learners) to produce a single predictive model with improved generalization performance. Base learners are typically trained using diverse strategies — reweighted data (bagging), s…
A reliability diagram (also called a calibration curve) is a plot used to evaluate how well-calibrated a probabilistic classifier is. The x-axis shows bins of predicted probabilities (e.g., 0.0–0.1, 0.1–0.2, etc.), and the y-axis shows the empirical fraction…
A feature is a property, attribute, or characteristic of an entity that has been selected, engineered, or extracted from raw data to serve as a predictive input. Features are the units of representation that models operate on: they translate the messy structu…
A visual encoding is a systematic mapping from data dimensions to visual properties — such as position on a spatial axis, color hue or lightness, size (area, length, angle), shape, and texture — designed so that the resulting visual display permits perception…
The bias-variance tradeoff is a human-made concept in machine learning describing the fundamental tension between two competing sources of prediction error: bias, which is the error introduced by approximating a real-world problem with a simplified model (cau…
A hyperparameter is a human-made concept in machine learning describing configuration variables whose numerical or categorical values are set by a practitioner before the training process begins, governing the model architecture, optimization dynamics, or reg…
A loss function (also called a cost function or criterion) is a human-made mathematical construct that takes as input a set of predictions and corresponding target values, and outputs a non-negative real number representing the degree of error. The function i…
Data leakage is the unintended transfer of information from a test or deployment context into a training context, causing a model to learn signals that are unavailable at prediction time. It occurs when preprocessing, feature selection, or any data transforma…
A probability forecast is a human-made construct for quantifying uncertainty about future outcomes. It takes the form of a predicted probability distribution or a set of point probabilities assigned to candidate events or values. The construct is produced by…
A scoring rule is a human-made evaluation function that assigns a numerical score to a prediction relative to an observed outcome, quantifying the accuracy or calibration of probabilistic forecasts. It maps a pair (prediction, outcome) to a real number, where…
A residual is the difference between an observed value and the fitted (predicted) value produced by a statistical model. Formally, for an observation y_i and its model prediction ŷ_i, the residual is e_i = y_i − ŷ_i. Residuals diagnose model adequacy: pattern…
A validation set is a held-out subset of training data reserved for evaluating model performance during training, without participating in gradient updates. It is carved from the training corpus at split time, then used repeatedly for interim performance chec…
Data leakage is a machine learning failure mode in which information from outside the training scope — test set data, future observations, or target-correlated features — enters the training process, producing artificially optimistic performance estimates. It…
A proper scoring rule that quantifies the accuracy of probabilistic predictions by computing the mean squared difference between predicted probabilities and actual outcomes (0 or 1). For binary cases: Brier = (1/N) * Σ(pred_i - outcome_i)². It rewards honest…
A calibration curve is a diagnostic plot used in statistics and machine learning to assess the quality of probabilistic predictions. It plots the observed frequency of positive outcomes against the predicted probabilities, typically grouped into bins. A perfe…
A covariance matrix is a symmetric, square matrix that organizes the pairwise covariances between a set of variables. Each entry (i,j) quantifies how variables i and j co-vary: positive values indicate they tend to increase together, negative values indicate…
Exploratory Data Analysis is a systematic approach to examining datasets before formal modeling, using visualization, summary statistics, and pattern recognition to discover structures, detect anomalies, generate hypotheses, and validate assumptions. It is pr…
A smoothing spline is a non-parametric regression method that fits a smooth curve to scattered data by minimizing a penalized sum of squared residuals, where the penalty is the integral of the squared second derivative of the curve. Unlike fixed-knot splines,…
Data augmentation is a machine learning technique that artificially expands a training dataset by applying label-preserving transformations to existing examples. The parameters are: the base dataset (a collection of labeled training samples), the transformati…
Tokenization is the procedure that decomposes a sequence of characters into a sequence of tokens according to a specified rule set. The parameters are: the input text (a string of Unicode characters), the tokenization scheme (character-level, word-level, subw…
A train test split is a methodological practice in machine learning that partitions a dataset into two disjoint subsets: a training set used to fit model parameters, and a test set used to evaluate generalization performance. The split is determined by a chos…
A lift chart is a visual plot used in predictive modeling and data science that compares the performance of a classification model against a baseline random predictor. It plots cumulative gain (the percentage of positive cases captured) against the percentage…
A permutation test is a nonparametric significance test in which observed data labels are randomly shuffled many times to build an empirical null distribution of a chosen test statistic. Parameters: the test statistic function, the number of permutations B (l…
A partial dependence plot is a diagnostic visualization in machine learning model interpretability that reveals the marginal effect of a single feature on the predicted outcome of a trained model. It is constructed by computing the model's predictions across…
An influence plot is a diagnostic visualization in regression analysis that displays each observation's influence on the fitted model, typically plotting a measure of influence (such as Cook's distance) against the observation index or fitted values. Paramete…
Feature extraction is the process of transforming raw data into measurable, informative features through systematic procedures — statistical, algebraic, or algorithmic — that distill the essential structure of observations into representations suitable for an…
Feature selection is a machine learning pipeline stage that takes a feature set and produces a reduced subset by applying a search strategy (forward selection, backward elimination, or exhaustive scan) paired with an evaluation criterion (filter methods using…
SHAP values (SHapley Additive exPlanations) is a feature attribution method that applies Shapley values from cooperative game theory to machine learning models. For a given prediction, each feature receives a SHAP value representing its additive contribution…
A sloped dot plot is a data visualization technique in which quantitative values are represented as dots connected by straight line segments, with the slope of each segment encoding the direction and magnitude of change between consecutive values. The visuali…
An elbow plot is a diagnostic data visualization that plots a quantitative metric (typically inertia, distortion, or reconstruction error) against a parameter value (most commonly the number of clusters k) to identify the point of diminishing returns — the "e…
Permutation importance is a model-agnostic method for quantifying the contribution of an input feature to a machine learning model's predictive performance. The procedure works by taking a trained model and a validation dataset, randomly shuffling (permuting)…
Feature importance is a quantitative measure used in statistical modeling and machine learning to rank input features by their contribution to a model's predictive output. It assigns each feature a score — derived from methods such as permutation importance,…
An upset plot is a data visualization composed of two coordinated panels: a set-intersection matrix on the left (rows indexed by set combinations, columns by individual sets, cells marked with dots when an element belongs to a set) and bar charts above (showi…
A feature store is a centralized software system for managing machine learning features — derived variables engineered from raw data for use in model training and inference. It provides three core operations: (1) feature registration with type and lineage met…
A marginal boxplot is a composite chart that overlays a primary scatter plot or data visualization with box-and-whisker plots displayed along the x and y margins, showing the univariate distributions of each variable alongside the bivariate relationship. The…
Feature engineering is the practice of using domain knowledge to select, transform, and construct input variables from raw data so that they better support predictive modeling. It includes operations such as normalization, encoding categorical values, creatin…
A residual plot is a scatter plot used in regression diagnostics to assess the adequacy of a statistical model. It plots residuals (the differences between observed and predicted values) on the y-axis against predicted values or individual predictors on the x…
A funnel plot is a scatter plot used in meta-analysis to assess publication bias and small-study effects. It plots each study's estimated treatment effect on the horizontal axis against a measure of its precision (typically the standard error or inverse of th…
A learning curve is a diagnostic visualization that plots a model's performance metric (such as error rate, accuracy, or loss) against a measure of training exposure — typically training set size, number of epochs, or training time. The curve reveals how quic…
A pair plot (also called a scatterplot matrix) is a human-made data visualization that arranges a grid of bivariate scatter plots — one for every pair of variables in a dataset — so that multivariate relationships can be assessed by scanning the matrix. The d…
An empirical distribution is the frequency-based description of observed data values from a finite sample. It maps each distinct value (or bin of values) to its count or relative frequency within the observed dataset. Unlike a probability distribution, which…
A calibration plot (reliability diagram) is a diagnostic visualization for probabilistic classifiers that compares predicted probabilities against observed outcome frequencies. The x-axis divides predictions into bins of predicted probability; the y-axis show…