https://k4i.top/tags/kernel/Recent content in Kernel on k4i's blogHugo -- gohugo.ioensky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Fri, 05 Jun 2026 11:10:00 +0800https://k4i.top/posts/llm-attention-kernels-gpu-primitives/Fri, 05 Jun 2026 11:10:00 +0800sky_io@outlook.com (K4i)Fri, 05 Jun 2026 00:26:17 +0800https://k4i.top/posts/llm-attention-kernels-gpu-primitives/<p>This series is for kernels and GPU primitives. The mechanism series explains why a serving system needs an optimization; this series explains how the optimization works at the kernel and memory-access level.</p> <h2 id="existing-posts">Existing Posts</h2> <ol> <li><a href="https://k4i.top/posts/fused-softmax/">Fused Softmax in Triton</a></li> <li><a href="https://k4i.top/posts/online-softmax/">Online Softmax: Tiling for Arbitrarily Large Rows</a></li> </ol> <h2 id="planned-posts">Planned Posts</h2> <ul> <li>FlashAttention: how online softmax becomes IO-aware attention</li> <li>From FlashAttention to PagedAttention: how attention kernels and cache layout constrain each other</li> <li>PagedAttention kernels: how block tables enter the attention memory path</li> <li>Triton profiling: using roofline thinking for bandwidth-bound and compute-bound kernels</li> <li>Why decode kernels are often limited by HBM bandwidth</li> </ul> <h2 id="questions">Questions Each Post Should Answer</h2> <ul> <li>Which memory access does this kernel remove or reduce?</li> <li>How does data move through HBM, L2, shared memory, and registers?</li> <li>Does it improve prefill, decode, or both?</li> <li>How is it coupled to vLLM / SGLang serving parameters or cache layout?</li> </ul>K4ifeatured imagellmattentiontritoncudagpukernelAILLM Attention Kernels and GPU Primitives