© 2026 LIN TIANYU. All rights reserved. Preview Version.

Powered bynextjs-blog

GitHub LinkedIn

ArticlePDF机器学习

梯度下降（Gradient Descent）

PublishedJuly 24, 2026Document9 pages

Contents

一维直觉：用导数下山 (1D Intuition: Descending with Derivatives)
抛物线例子与下山规则 (The Parabola & the Descent Rule)
局限：复杂函数的局部最小值 (Pitfall: Local Minima)
凸性：成功的关键 (Convexity: The Key Property)
为什么凸性保证下山成功 (Why Convexity Guarantees Success)
线性函数：最优下降方向 (Linear Functions: The Best Direction)
局部线性化：泰勒展开与梯度 (Local Linearization: Taylor & the Gradient)
为什么可以假设局部线性 (Why "Locally Linear")
高维的导数——梯度 (The Gradient)
较复杂例子：二次型的梯度 (Gradient of a Quadratic Form)
定义梯度下降算法 (Defining Gradient Descent)
应用：线性回归 (Application: Linear Regression)
推导 MSE 的梯度 (Deriving the Gradient of MSE)
在损失曲面上的完整下降轨迹 (Full Descent Trajectory on the Loss Surface)
两个实用优点 (Two Practical Advantages)
随机梯度下降 (Stochastic Gradient Descent, SGD)
步长选择与动量 (Choosing the Step Size & Momentum)
选的办法 (Picking eta)
动量 (Momentum)
加速梯度下降 (Accelerated Gradient Descent)

1

2

3

4

5

6

7

8

9

Thanks for reading.