We would like a matrix norm $\|\cdot\|$ to be invariant under orthogonal transformations in the sense that $\|A\|=\|UAV\|$ whenever $U,V$ are orthogonal mappings. But by the singular value decomposition, we know that every matrix norm invariant under orthogonal transformations is of the form $\|A\|=f(\sigma_1(A),\dots,\sigma_n(A))$ for some (symmetric) function $f:[0,\infty)^n\rightarrow[0,\infty)$ where $\sigma_1(A)\geq\dots\geq\sigma_n(A)$ are the singular values of $A$.

The power of the Schatten norm $\|A\|_p^p$ for $p$ even is notable since $\|A\|_p^p$ is both a homogeneous degree $p$ polynomial in the singular values of $A$ and also in the coefficients of $A$.

The Schatten norms $\|A\|_p^p$ for positive integer $p$ are natural in the sense that the ring of symmetric polynomials in $n$-variables over the field of real numbers is generated by the polynomials $x_1^p+\dots+x_n^p$. Furthermore, the ring of symmetric polynomials in $n$-variables over the field of real numbers is dense in the space of all symmetric continuous functions $f:\mathbb{R}^n\rightarrow\mathbb{R}$ in the topology of uniform convergence on compact sets by the Stone-Weierstrass theorem. In particular, the Schatten norms $\|A\|_p$ completely determine $A$ up to the equivalence relation $A\simeq UAV$ where $U,V$ are orthogonal.

**Schatten norms of Jacobians**

Suppose that $f:\mathbb{R}^m\rightarrow\mathbb{R}^n$ and $A=J(f)(\mathbf{x}_0)$. Then
$$
\lim_{\epsilon\rightarrow 0}\frac{f(\mathbf{x}_0+\epsilon\mathbf{x})-f(\mathbf{x}_0)}{\epsilon}=A\mathbf{x},
$$
so
$$\lim_{\epsilon\rightarrow 0}\left\|\frac{f(\mathbf{x}_0+\epsilon\mathbf{x})-f(\mathbf{x}_0)}{\epsilon}\right\|^2=\|A\mathbf{x}\|^2.
$$ If $\mathbf{x}$ is a random variable, then the distribution of $\|A\mathbf{x}\|^2$ tells us how much $f(\mathbf{z})$ varies when $\mathbf{z}$ is approximately $\mathbf{x}_0$, but when $\mathbf{x}$ is standard Gaussian, the moment generating function of the distribution of $\|A\mathbf{x}\|^2$ can be characterized in terms of a Schatten norm generating function.

Suppose that $\mathbf{x}$ is a Gaussian random variable with mean $0$ and identity covariance matrix. Then $\|A\mathbf{x}\|^2=\sigma_1^2 X_1^2+\dots+\sigma_n^2 X_n^2$ where $X_1,\dots,X_n$ are independent standard normally distributed random variables of one variable. Therefore,
$$
\begin{split}
E\big(\exp(t\|A\mathbf{x}\|^2)\big)
& =E\Big(\exp\big(t\sigma(A)_1^2X_1^2+\cdots+t\sigma_n(A)^2X_n^2\big)\Big)\\
& =E\Big(\exp\big(t\sigma_1(A)^2X_1^2)\cdots\exp(t\sigma_n(A)^2X_n^2\big)\Big)\\
& =E\Big(\exp\big(t\sigma_1(A)^2X_1^2\big)\Big)\cdots E\Big(\exp\big(t\sigma_n(A)^2X_n^2\big)\Big)\\
& =\frac{1}{\sqrt{1-2t\sigma_1(A)}}\cdots\frac{1}{\sqrt{1-2t\sigma_n(A)}}.
\end{split}$$

We then take logarithms to obtain
$$
\begin{split}
\ln\Big(E\big(\exp(t\|A\mathbf{x}\|^2\big)\Big) &=-\frac{1}{2}\big(\ln(1-2t\sigma_1(A)^2)+\dots+\ln(1-2t\sigma_n(A)^2)\big)\\
&=\frac{1}{2} \sum_{k=1}^n\sum_{j=1}^\infty\frac{(2t\sigma_k(A)^2)^j}{j}\\
&=\sum_{k=1}^n\sum_{j=1}^\infty\frac{1}{2}\frac{(2t)^j}{j}\sigma_k(A)^{2j}\\
&=\sum_{j=1}^\infty\frac{1}{2}\frac{(2t)^j}{j}\|A\|_{2j}^{2j}.
\end{split}$$

**Schatten norms of Hessians at local minima**

Suppose that $f:\mathbb{R}^m\rightarrow\mathbb{R}$ is a function that attains a local minimum on the input $\mathbf{x}_0$ and $H(f)(\mathbf{x}_0)=B$. Then
$\lim_{\epsilon\rightarrow 0}\frac{1}{\epsilon^2}\cdot(f(\mathbf{x}_0+\epsilon\mathbf{x})-f(\mathbf{x}_0))=\frac{1}{2}\cdot\langle B\mathbf{x},\mathbf{x}\rangle.$ Therefore, the distribution of $\langle B\mathbf{x},\mathbf{x}\rangle$ when $\mathbf{x}$ is a standard Gaussian random variable tells how much, but this distribution is also characterized by a Schatten norm generating function.

Suppose that $B$ is a positive semidefinite matrix and $A=\sqrt{B}$ and $\mathbf{x}$ is still a standard Gaussian random variable. Then since $\langle B\mathbf{x},\mathbf{x}\rangle=\langle A^2\mathbf{x},\mathbf{x}\rangle=\langle A\mathbf{x},A\mathbf{x}\rangle=\|A\mathbf{x}\|^2$, we have
$$
\ln\Big(E\big(\exp(t\cdot\langle B\mathbf{x},\mathbf{x}\rangle)\big)\Big)=\sum_{j=1}^\infty\frac{1}{2}\cdot\frac{(2t)^j}{j}\|A\|_{2j}^{2j}=\sum_{j=1}^\infty\frac{1}{2}\frac{(2t)^j}{j}\|B\|_j^j.$$