https://k4i.top/zh/tags/rlhf/Recent content in Rlhf on k4i's blogHugo -- gohugo.iozhsky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Tue, 07 Jul 2026 10:00:00 +0800https://k4i.top/zh/posts/kl-divergence-not-a-distance/Tue, 07 Jul 2026 10:00:00 +0800sky_io@outlook.com (K4i)Tue, 07 Jul 2026 10:00:00 +0800https://k4i.top/zh/posts/kl-divergence-not-a-distance/<p>KL divergence 最容易被误读成“两个分布之间的距离”。这个说法有一半对：它确实在比较两个分布；但另一半很危险：<strong>KL 不是距离，因为方向有意义。</strong> 更准确地说，KL 虽然满足非负性，而且只有 \(P=Q\) 时才为 0，却不满足 metric 要求的对称性和三角不等式。</p>K4ifeatured imagedeep-learningkl-divergencecross-entropyrlhfinformation-theoryAI