Fitting the Ocean into a Cup
Quantization is fundamentally a mapping problem. Your model weights might span a continuous range like -10.5 to +20.2. You need to represent that entire range using only the 256 discrete integers available in INT8 (-128 to 127).
The standard approach is affine quantization (also called asymmetric quantization), which uses a linear mapping:
$$\text{Real Value} = \text{Scale} \times (\text{Integer Value} - \text{Zero Point})$$
Two parameters define this mapping:
- Scale — the size of one integer "step" in real-value units. If Scale = 0.12, then moving from integer 5 to integer 6 represents a real-value change of 0.12.
- Zero Point — the integer that corresponds to the real number
0.0. This is necessary because the real-value range is often asymmetric (e.g.,0.0to10.0doesn't straddle zero), so the integer zero doesn't naturally align with the real zero.
Why Zero Point Matters
Consider a ReLU activation layer. Its outputs are always non-negative (range: 0.0 to some positive max). If you naively mapped this to -128 to 127, you'd waste half your integer range on negative numbers that never appear. The Zero Point shifts the mapping so that real 0.0 aligns with the integer that best uses the available range.
Symmetric quantization is a simpler variant that forces Zero Point = 0, centering the mapping around zero. It's less flexible but easier to implement and often used for weights (which tend to be roughly zero-centered after training).
Quantization Error
Every real value gets rounded to the nearest integer bin. The difference between the original value and the reconstructed value is called quantization error or quantization noise. This error is bounded by Scale / 2 — so a larger Scale (coarser mapping) means more error. The goal of calibration (covered next) is to choose Scale and Zero Point to minimize this error across the values that actually matter.
Interviewers often ask: "What's the difference between symmetric and asymmetric quantization, and when would you use each?" The clean answer: symmetric quantization (Zero Point = 0) is simpler and works well for weights, which are typically zero-centered. Asymmetric quantization is better for activations that have non-symmetric ranges (e.g., post-ReLU outputs that are always ≥ 0), because it uses the full integer range without wasting half of it on values that never appear.