https://k4i.top/tags/low-precision/Recent content in Low-Precision on k4i's blogHugo -- gohugo.ioensky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Fri, 05 Jun 2026 11:00:00 +0800https://k4i.top/posts/llm-quantization-low-precision-serving/Fri, 05 Jun 2026 11:00:00 +0800sky_io@outlook.com (K4i)Fri, 05 Jun 2026 00:26:17 +0800https://k4i.top/posts/llm-quantization-low-precision-serving/<p>This series is for quantization and low-precision serving. It deserves its own track because quantization touches representation, error control, calibration data, kernel support, KV cache, memory bandwidth, and quality regression.</p> <h2 id="existing-posts">Existing Posts</h2> <ol> <li><a href="https://k4i.top/posts/llm-quantization/">A Survey of LLM Quantization: From Linear Quantization to Codebooks</a></li> </ol> <h2 id="planned-posts">Planned Posts</h2> <ul> <li>KV cache quantization: beyond weight memory, the cache is often the real footprint</li> <li>FP8 serving: E4M3 / E5M2, activation scales, and Tensor Core paths</li> <li>INT4 weight-only serving: why saving memory does not always mean going faster</li> <li>GPTQ / AWQ / SmoothQuant engineering boundaries</li> <li>NF4 / AQLM: why lower bit widths need codebooks</li> <li>Quantization benchmarks: measuring the quality, speed, and memory triangle</li> </ul> <h2 id="questions">Questions Each Post Should Answer</h2> <ul> <li>Are we quantizing weights, activations, KV cache, or a communication/storage format?</li> <li>Does the benefit come from capacity, HBM bandwidth, Tensor Core throughput, or disk size?</li> <li>Does the error mainly come from outliers, scale granularity, rounding, or clipping?</li> <li>How do vLLM / SGLang load, observe, and roll back this choice?</li> </ul>K4ifeatured imagellmquantizationlow-precisionint4fp8servingAILLM Quantization and Low-Precision Serving