矩阵梯度

错误的证明

\[
\begin{align}
\nabla_Z f(Z) =& \nabla_Z\begin{bmatrix}
z_{11} & \cdots & z_{1m}\\
\vdots & & \\
z_{n1} & \cdots &z_{nm}
\end{bmatrix}^\top
\begin{bmatrix}
z_{11} & \cdots & z_{1m}\\
\vdots & & \\
z_{n1} & \cdots &z_{nm}
\end{bmatrix} \label{错误的z}\\
=& \nabla_Z\begin{bmatrix}
z_{11} & \cdots & z_{n1}\\
\vdots & & \\
z_{1m} & \cdots &z_{nm}
\end{bmatrix}
\begin{bmatrix}
z_{11} & \cdots & z_{1m}\\
\vdots & & \\
z_{n1} & \cdots &z_{nm}
\end{bmatrix} \notag \\
=& \nabla_Z\begin{bmatrix}
z_{11}z_{11}+z_{21}z_{21}+\cdots +z_{n1}z_{n1} & \cdots & z_{11}z_{1m}+z_{21}z_{2m}+\cdots +z_{n1}z_{nm}\\
\vdots & & \\
z_{1m}z_{11}+z_{2m}z_{21}+\cdots +z_{nm}z_{n1} & \cdots & z_{1m}z_{1m}+z_{2m}z_{2m}+\cdots +z_{nm}z_{nm}
\end{bmatrix} \notag \\
=& \begin{bmatrix}
\frac{\partial}{\partial z_{11}} z_{11}z_{11}+z_{21}z_{21}+\cdots +z_{n1}z_{n1} & \cdots & \frac{\partial}{\partial z_{1m}}z_{11}z_{1m}+z_{21}z_{2m}+\cdots +z_{n1}z_{nm}\\
\vdots & & \\
\frac{\partial}{\partial z_{n1}}z_{1m}z_{11}+z_{2m}z_{21}+\cdots +z_{nm}z_{n1} & \cdots & \frac{\partial}{\partial z_{nm}}z_{1m}z_{1m}+z_{2m}z_{2m}+\cdots +z_{nm}z_{nm}
\end{bmatrix} \notag \\
= & \begin{bmatrix}
2z_{11} & z_{11} & \cdots & z_{11}\\
& 2z_{22} \\
\vdots & \\
z_{nm} & z_{nm} & \cdots & 2z_{nm}
\end{bmatrix} \notag
\end{align}
\]

发现证不出。

正确的证明

在公式[latex]\eqref{错误的z}[/latex]已经出错了！按照定义，梯度只能作用于返回实数的函数。

既然[latex]f(Z)=Z^\top Z=2Z[/latex]是对的，那么Z必须为向量，或n行1列矩阵。下面我们把Z当作n行1列矩阵，因为我们要用矩阵转置、矩阵乘法等操作。

严格按照定义，把[latex]f(Z)=Z^\top Z[/latex]代入。

\[\begin{align*}
\nabla_Z Z^\top Z =& \begin{bmatrix}
\frac{\partial Z^\top Z}{\partial z_{1}} \\
\vdots \\
\frac{\partial Z^\top Z}{\partial z_{n}}
\end{bmatrix} \\
=& \begin{bmatrix}
\frac{\partial z_1^2+z_2^2+\cdots+z_n^2}{\partial z_{1}} \\
\vdots \\
\frac{\partial z_1^2+z_2^2+\cdots+z_n^2}{\partial z_{n}}
\end{bmatrix}\\
=& \begin{bmatrix}
2z_1 \\
\vdots \\
2z_n
\end{bmatrix} \\
=& 2 \begin{bmatrix}
z_1 \\
\vdots \\
z_n
\end{bmatrix} \\
=& 2Z
\end{align*}\]

证毕。