A positional encoding is a human-designed method of injecting sequential position information into model representations, enabling architectures without inherent sequential bias (such as self-attention) to distinguish the relative or absolute position of elements in a sequence. It maps each position index to a vector — typically via fixed sinusoidal functions (sine and cosine at different frequencies) or learned embeddings — and adds or concatenates this vector to the token embeddings, producing position-aware representations that the model can attend to. The encoding persists as a designed pattern in neural network architecture: it is implemented in software, documented in papers and codebases, and replicated across transformer-based systems. Without positional encoding, attention mechanisms treat sequences as unordered sets, losing the temporal and structural information essential to language, time series, and ordered data.
[formal: positio artificialis | substrate: mind | horizon: centuries | explicit: yes | epoch: 0.01]