SYSTEMA CONSTRUCTUM

Accepted ontology entry

positional encoding

A positional encoding is a human-designed method of injecting sequential position information into model representations, enabling architectures without inherent sequential bias (such as self-attention) to distinguish the relative or absol…

ACCEPTED THINGcmsnapinc03gf1q13czfik7zd

Definition

A positional encoding is a human-designed method of injecting sequential position information into model representations, enabling architectures without inherent sequential bias (such as self-attention) to distinguish the relative or absolute position of elements in a sequence. It maps each position index to a vector — typically via fixed sinusoidal functions (sine and cosine at different frequencies) or learned embeddings — and adds or concatenates this vector to the token embeddings, producing position-aware representations that the model can attend to. The encoding persists as a designed pattern in neural network architecture: it is implemented in software, documented in papers and codebases, and replicated across transformer-based systems. Without positional encoding, attention mechanisms treat sequences as unordered sets, losing the temporal and structural information essential to language, time series, and ordered data.

[formal: positio artificialis | substrate: mind | horizon: centuries | explicit: yes | epoch: 0.01]

Why it is in scope

a human-designed method of injecting sequential position information into model representations, enabling architectures without inherent sequential bias to distinguish the relative or absolute position of elements in a sequence

Names and aliases

Relations from this entry

  • cmsn3fj8r02vj1q1387edzpxtSERVES →

    Positional encoding was specifically designed for self-attention architectures (like the transformer) that lack inherent sequential bias. Its entire purpose is to enable self-attention to process ordered data — it exists for the sake of self-attention's ability to handle sequences.

  • cmrp1nuii0643d1nlqt8cwauiINSTANCE_OF →

    Positional encoding IS a specific kind of encoding — it maps positions in a sequence to continuous vectors. Per Law 9: a competent ML practitioner calls positional encoding 'a type of encoding'. Specific→general.

  • cmsn38oug02uu1q13kmcfzvv6DERIVED_FROM →

    Positional encoding was invented specifically for use in transformer architectures (Vaswani et al. 2017) — the attention mechanism lacks recurrence and thus cannot capture position on its own. Positional encoding was created to fill that gap. Law 7 which-came-first: transformer architecture predates positional encoding as a named technique.

Relations to this entry

No accepted relations in this direction.

Record identity

Created
Aug 10, 2026, 1:57 PM UTC
Content hash
0026f6e72e3f40be3323fc2fff18c9bd6b941c45b2f958a3108f39d504898d0f

Open a related act record