Let $f: \mathbb{R}^d\rightarrow\mathbb{R}_{\geq 0}$ be a function which is convex and smooth (i.e., in $C^{\infty}$). If $x^* \in \mathbb{R}^d$ is the (global) minimum of $f$, it is well known that gradient descent with a (sufficiently small) fixed step size $\eta$, $$ x_{n+1} = x_{n} - \eta \nabla f(x_n), $$ converges to the global minimum $x_n \rightarrow x^*$.
When a finite minimizer $x^*$ does not exist (i.e., when $\forall x \in \mathbb{R}^d : \nabla f(x)\neq 0$), one might expect that $x_n/||x_n||$ should converge to a finite solution. However, there are counterexamples to this claim.
Question: What additional assumptions (as "light" as possible) do we need to add to have on $f$ so that $x_n/||x_n||$ would converge (for a sufficiently small $\eta$)? The counterexample above suggests that the Hessian of $f$ perhaps should have a bounded eigenvalue ratio. Is this enough?
Thanks in advance!
