https://k4i.top/zh/tags/batching/Recent content in Batching on k4i's blogHugo -- gohugo.iozhsky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Wed, 22 Apr 2026 10:30:00 +0800https://k4i.top/zh/posts/continuous-batching/Wed, 22 Apr 2026 10:30:00 +0800sky_io@outlook.com (K4i)Sat, 30 May 2026 00:04:33 +0800https://k4i.top/zh/posts/continuous-batching/<h2 id="batching-problem">batching 问题</h2> <p>batching 是 LLM serving 系统让 GPU 忙起来的基本手段。单个请求通常无法充分利用 GPU，但多个请求放在一起，就能把很多小矩阵运算变成更大的矩阵运算。问题是：请求不会同时结束。</p>K4ifeatured imagellminferencesystemsbatchingschedulingAILLM Inference Internals