https://k4i.top/tags/model-runner/Recent content in Model-Runner on k4i's blogHugo -- gohugo.ioensky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Tue, 23 Jun 2026 10:30:00 +0800https://k4i.top/posts/model-runner-scheduler-output-to-gpu-forward/Tue, 23 Jun 2026 10:30:00 +0800sky_io@outlook.com (K4i)Tue, 23 Jun 2026 10:30:00 +0800https://k4i.top/posts/model-runner-scheduler-output-to-gpu-forward/<p>Scheduler decides <strong>what to run this step</strong>. ModelRunner decides <strong>how to run it on GPU</strong>. If Scheduler compresses dynamic request queues into <code>SchedulerOutput</code>, ModelRunner translates that output into contiguous tensors, KV cache slots, attention metadata, forward context, logits, and sampled tokens.</p> <p>So yes, ModelRunner is the execution core of inference. It does not own HTTP serving or global queue policy, but once <code>SchedulerOutput</code> exists, the model starts running here.</p> <p>Read this after the <a href="https://k4i.top/posts/scheduler-request-queue-to-scheduler-output/">Scheduler</a> post. If the whole path is still fuzzy, start from the <a href="https://k4i.top/posts/request-lifecycle-openai-to-forward-pass/">request lifecycle</a> overview.</p>K4ifeatured imagellminferencevllmsource-readingmodel-runnerai-infraAIvLLM and SGLang Source Reading