Skip to main content
7 of 8
Formula hyperlinking
Daniele Tampieri
  • 6.4k
  • 7
  • 30
  • 45

$\newcommand\R{\mathbb R}$Let $\R:=R$. Suppose that $|f''(x)(h,h)|\le C|h|^2$ for all $x$ and $h$ in $\R^m$ -- this is how we interpret the condition "Hessian matrix of $f$ is upper bounded by some constant $C$". Of course, here $f''(x)$ is the bilinear form that is the second derivative of $f$ at $x$, so that $f''(x)(h,k)=h^\top H(x)k$ for all $x,h,k$ in $\R^m$, where $H(x)$ is the Hessian matrix of $f$ at $x$.

Consider first the case $m=1$. Take any $x\in\R$ and any real $h>0$. Then $$0\le f(x+h)\le f(x)+f'(x)h+Ch^2/2,$$ whence $$f'(x)\ge-\frac{f(x)}h-\frac C2\,h\ge-K\sqrt{f(x)},$$ where $K:=\sqrt{2C}$. Similarly (or by the left-right symmetry), $f'(x)\le K\sqrt{f(x)}$, and hence $$|f'(x)|\le K\sqrt{f(x)} \label{1}\tag{1}$$ for all $x\in\R$.

Now take any natural $m$. Considering the restrictions of $f$ to all straight lines in $\R^m$, we see that \eqref{1} holds for all $x\in\R^m$, where now $|f'(x)|$ denotes the norm of the linear form $f'(x)$ that is the derivative of $f$ at $x$, so that $f'(x)(h)=h^\top\,\nabla f(x)$ for all $h\in\R^n$ and hence $|f'(x)|=|\nabla f(x)|$.

Thus, the desired conclusion holds, in general.

Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229