https://k4i.top/tags/scheduler/Recent content in Scheduler on k4i's blogHugo -- gohugo.ioensky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Tue, 23 Jun 2026 11:20:00 +0800https://k4i.top/posts/scheduler-request-queue-to-scheduler-output/Tue, 23 Jun 2026 11:20:00 +0800sky_io@outlook.com (K4i)Tue, 23 Jun 2026 11:20:00 +0800https://k4i.top/posts/scheduler-request-queue-to-scheduler-output/<p>In the request lifecycle, Scheduler is the easiest piece to underestimate. The HTTP server admits requests, and ModelRunner executes batches on GPU. Scheduler answers the per-step question in between: <strong>who runs now, how many tokens do they get, and can the KV cache hold the result?</strong></p> <p>The three lifecycle posts fit together like this:</p> <table> <thead> <tr> <th>post</th> <th>question</th> </tr> </thead> <tbody> <tr> <td><a href="https://k4i.top/posts/request-lifecycle-openai-to-forward-pass/">request lifecycle</a></td> <td>how a request reaches EngineCore</td> </tr> <tr> <td>Scheduler</td> <td>EngineCore decides <strong>what to run</strong> in each step</td> </tr> <tr> <td><a href="https://k4i.top/posts/model-runner-scheduler-output-to-gpu-forward/">ModelRunner</a></td> <td><code>SchedulerOutput</code> becomes <strong>how to run on GPU</strong></td> </tr> </tbody> </table> <p>Scheduler output is not a vague &ldquo;batch.&rdquo; It is a concrete <code>SchedulerOutput</code>: which requests are new, which are already cached on workers, how many tokens each request gets, which KV blocks were allocated, which requests were preempted, and which finished requests must be cleaned up.</p>K4ifeatured imagellminferencevllmsource-readingschedulerai-infraAIvLLM and SGLang Source Reading