https://k4i.top/tags/ai-infra/Recent content in Ai-Infra on k4i's blogHugo -- gohugo.ioensky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Thu, 04 Jun 2026 22:10:00 +0800https://k4i.top/posts/vllm-sglang-source-reading/Thu, 04 Jun 2026 22:10:00 +0800sky_io@outlook.com (K4i)Fri, 05 Jun 2026 00:26:17 +0800https://k4i.top/posts/vllm-sglang-source-reading/<p>This series is for source reading and engineering follow-through. The goal is not to translate files line by line, but to locate core inference-engine mechanisms in real code paths and verify their behavior with benchmarks or small experiments.</p> <h2 id="reading-order">Reading Order</h2> <p>Planned posts will follow the request lifecycle:</p> <ol> <li>Request lifecycle: from OpenAI API to one forward pass</li> <li>Scheduler loop: waiting queue, running queue, token budget, and decode priority</li> <li>vLLM Block Manager: from logical blocks to physical KV blocks</li> <li>SGLang Radix Cache: why prefix reuse wants a tree</li> <li>What a prefix cache hit actually saves</li> <li>Chunked prefill parameters, scheduling branches, and benchmarks</li> <li>Why structured output / FSM decoding is a strong SGLang use case</li> </ol> <h2 id="format">Standard Format</h2> <p>Each source-reading post should answer four questions:</p>K4ifeatured imagellminferencevllmsglangsource-readingai-infraAIvLLM and SGLang Source Reading