https://k4i.top/zh/categories/ai/Recent content in AI on k4i's blogHugo -- gohugo.iozhsky_io@outlook.com (K4i)sky_io@outlook.com (K4i)All content is subject to the license of <a rel="license noopener" href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> .Mon, 16 Feb 2026 23:30:00 +0800https://k4i.top/zh/posts/batch-vs-stochastic-gradient-descent/Mon, 16 Feb 2026 23:30:00 +0800sky_io@outlook.com (K4i)Sun, 19 Apr 2026 23:18:51 +0800https://k4i.top/zh/posts/batch-vs-stochastic-gradient-descent/<h2 id="introduction">引言</h2> <p>从上一篇前向传播与反向传播的文章中，我们已知梯度下降的更新公式：</p> <p>\[<br /> \theta \leftarrow \theta - \eta \nabla C<br /> \]</p> <p>其中 \(\nabla C\) 是<strong>整个训练数据集</strong>上的损失梯度。但计算 \(\nabla C\) 意味着要对每一个训练样本运行前向传播和反向传播，然后取平均。当数据集有数百万样本时，这是极其昂贵的。</p>K4ifeatured imagegradient-descentAIhttps://k4i.top/zh/posts/forward-and-backward-propagation/Mon, 16 Feb 2026 23:26:00 +0800sky_io@outlook.com (K4i)Sun, 19 Apr 2026 23:18:51 +0800https://k4i.top/zh/posts/forward-and-backward-propagation/<h2 id="deep-neural-network">深度神经网络</h2> <figure><img src="https://k4i.top/images/posts/forward-backward-propagation/neural_network_example.svg" alt="图 1： 神经网络示例" width="70%"><figcaption> <p><span class="figure-number">图 1： </span>神经网络示例</p> </figcaption> </figure> <p>每个层可以用数学形式描述如下，从输入层逐层计算到最终输出层的过程称为<strong>前向传播（forward propagation）</strong>：</p>K4ifeatured imagegradient-descentAI