https://k4i.top/tags/decode/Recent content in Decode on k4i's blogHugo -- gohugo.ioensky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Fri, 05 Jun 2026 22:30:00 +0800https://k4i.top/posts/prefill-vs-decode/Fri, 05 Jun 2026 22:30:00 +0800sky_io@outlook.com (K4i)Fri, 05 Jun 2026 22:30:00 +0800https://k4i.top/posts/prefill-vs-decode/<p>LLM inference looks like one operation: send a prompt, get tokens back. under the hood it is two workloads sharing the same model weights.</p> <p><strong>prefill</strong> processes the input prompt and builds the initial KV cache. <strong>decode</strong> generates new tokens one step at a time while reading that cache. the weights are the same, but the hardware bottleneck is not. prefill behaves like a large batched matrix multiplication problem; decode behaves like a stream of small queries repeatedly reading a growing memory table.</p>K4ifeatured imagellminferenceprefilldecodekv-cacheservingsystemsAILLM Inference Internals