Quantization is the technique of mapping continuous or high-precision numerical values to a discrete, lower-precision representation. Its defining parameters are: (1) source precision — the original numeric format (e.g., FP32, FP16), (2) target precision — the reduced representation (e.g., INT8, INT4, binary), (3) mapping strategy — uniform, per-tensor, or per-channel scaling, and (4) calibration data — a representative dataset used to determine quantization ranges without retraining. It persists through compiler toolchains (e.g., TensorRT, TVM, ONNX Runtime) and model serialization formats that embed scale and zero-point metadata. [formal: quantization | substrate: behavior | horizon: hours | explicit: yes | epoch: 0.01]
Accepted ontology entry
quantization
Quantization is the technique of mapping continuous or high-precision numerical values to a discrete, lower-precision representation. Its defining parameters are: (1) source precision — the original numeric format (e.g., FP32, FP16), (2) t…
Definition
Why it is in scope
The human-made technique of reducing numerical precision in computational models (e.g., mapping float32 parameters to int8) to decrease model size and inference cost while preserving behavioral approximation.
Names and aliases
- quantizationen · CANONICAL
Relations from this entry
- cmrs990nt00goollh7iroikawINSTANCE_OF →
Test: is quantization a specific kind of compression? Yes — it reduces the information content and storage size of model parameters by mapping high-precision values to lower-precision discrete representations. A competent ML practitioner would call quantization 'a type of compression' — specifically, precision reduction as a compression technique. Per Law 9: specific→general.
- cmr9uz3vv00elhcxfruyltnd4DERIVED_FROM →
which-came-first-existed-and-fed-into: measurement (assigning numerical values to observations) predates quantization (converting continuous values to discrete levels) as a practice. One must first have measurement to quantify measured values. Quantization as a technique for discretizing measurement data was derived from the prior concept of measurement.
- cmr9lxelr009uhcxflfegr6mfINSTANCE_OF →
quantization IS a specific kind of algorithm: it converts continuous values to discrete levels using a defined procedure. A competent speaker would call quantization 'an algorithm' or 'an algorithmic technique.' Algorithm is the nearest accepted kind.
- cmsldf7d206tqnobp13puglliSERVES →
Quantization is built and maintained for the sake of model compression: in ML it reduces precision of model parameters to shrink model size and speed up inference. Its designed purpose in the ML context is to compress models while preserving acceptable accuracy.
Relations to this entry
- cmsrzc22l00r2h7yukrx2iaso← DEPENDS_ON
Dither needs quantization to operate now: the removal test — remove quantization and dither stops functioning because its sole purpose is to randomize quantization error. Without quantization, there is no quantization error for dither to address, and the technique has no operation. This is not merely sayable: dither as a concept would disappear without quantization, and practically, dither algorithms are meaningless without a quantization step to act upon.
- cmsrzc22l00r2h7yukrx2iaso← DERIVED_FROM
Which came first? Quantization as a technique predates dither. Dither emerged from quantization practice as a method to randomize quantization error. Quantization fed into the development of dither — you cannot have a dither technique without first having quantization.
- cmss1yi0800xsh7yuo9hcrixj← DEPENDS_ON
Noise-shaping needs quantization to operate now — it shapes quantization error by feeding it back through a filter. Remove quantization and noise-shaping has no error to shape. The removal test (Law 8) passes.
- cmsrzc22l00r2h7yukrx2iaso← SERVES
Dither is added and maintained for the sake of improving quantization quality — it decorrelates quantization error. For whose sake? quantization. The servant (dither) points at the master (quantization). Law 8d.
- cmss1yi0800xsh7yuo9hcrixj← SERVES
noise-shaping serves quantization — it is a technique designed to improve quantized audio by redistributing quantization noise to less perceptually salient frequency bands. Its purpose by design is to make quantization better.
- cmss1yi0800xsh7yuo9hcrixj← DERIVED_FROM
Noise shaping evolved from the study of quantization error. Quantization is the fundamental concept that predates noise shaping; noise shaping is a specialized technique that shapes quantization error spectra, building directly on quantization theory.
- cmsuhqssz000bs2m7p3thvvk3← DEPENDS_ON
Quantization is a core stage in every audio-codec pipeline; remove quantization and the codec cannot reduce precision to compress data.
- cmsvxfba9000uxm1h6qa55klp← DERIVED_FROM
Quantization as a concept predates the specific parameter bit-depth; bit-depth is the numerical parameterization of quantization resolution. Which came first? Quantization (the general concept of mapping continuous values to discrete levels) existed before the specific parameter bit-depth was introduced to specify how many levels.
- cmsvxfba9000uxm1h6qa55klp← DEPENDS_ON
bit-depth as an audio concept specifies the number of quantization levels (2^N amplitude steps). Remove the concept of quantization and bit-depth loses its entire meaning in audio — it is the parameter that defines quantization fineness. This is a genuine present-tense dependency: the concept of bit-depth operates only in the context of quantization.
- cmsvmvim6005w5xhk8rw0l904← DEPENDS_ON
Pre-echo IS the temporal spreading of quantization error across finite analysis windows. Remove quantization entirely — no discrete amplitude levels, no quantization error — and pre-echo has no mechanism and cannot exist. The accepted definition of pre-echo carves it by reference to 'quantization error' and 'coding window length.' Removal test passes: X (pre-echo) stops existing without Y (quantization).
- cmsvxf8u3000cxm1htic9sy4w← SERVES
Dithering is built for the sake of quantization: it adds controlled random noise before quantization to mask quantization artifacts and reduce perceptible distortion. Law 8d: X SERVES Y means X is designed to further Y's operation. Dithering's sole purpose is improving quantization quality.
- transform-coding← DEPENDS_ON
Transform coding requires quantization of transform coefficients to operate — remove quantization and transform coding collapses into a bare transform, losing its core compression mechanism. The removal test passes: without quantization, there is no transform coding.
- cmsvxf8u3000cxm1htic9sy4w← DEPENDS_ON
Dithering requires quantization to operate — it is the addition of noise prior to quantization to mask quantization artifacts. Remove quantization and dithering has no function or meaning. The removal test passes.
- bit-crusher← DEPENDS_ON
Bit-crusher operates by reducing bit depth — which IS quantization (mapping continuous amplitude values to a finite set of discrete levels). Remove quantization and bit-crushing ceases to operate: its core mechanism is quantization noise generation. Pinned sense: quantization as the amplitude discretization process applied to audio signals.
- audio-dithering← SERVES
Audio dithering is designed and maintained for the sake of improving quantization quality by decorrelating quantization error and preventing distortion artifacts. The servant points at the master per Law 8d.
- audio-dithering← DEPENDS_ON
Audio dithering is applied before quantization or bit-depth reduction to decorrelate quantization error. Remove quantization and dithering has no operation to improve.
Record identity
- Created
- Aug 10, 2026, 3:51 PM UTC
- Content hash
- 21ae72ca492f281f5f8bb3d008c02373a70c37022547f31f506c7bf0ff81569d