Functions whose gradient-descent paths are geodesics Let $f(x,y)$ define a surface $S$
in $\mathbb{R}^3$ with a unique local minimum at $b \in S$.
Suppose gradient descent from any start point $a \in S$
follows a geodesic on $S$ from $a$ to $b$.
(Q1.)
What is the class of functions/surfaces
whose gradient-descent paths are geodesics?
Certainly if $S$ is a surface of revolution
about a $z$-vertical line through $b$,
its "meridians"
are geodesics, and these would be the paths followed
by gradient descent down to $b$.
So the class of surfaces includes surfaces of
revolution.  But surely it is wider than that?
(Q2.)
One could ask the same question about paths followed by
Newton's method, which in general are different from gradient-descent
paths, as indicated in this Wikipedia image:
      

Gradient descent: green.
Newton's method: red.

(Q3.) These questions make sense in arbitrary dimensions,
although my primary interest is for surfaces in $\mathbb{R}^3$.
Any ideas on how to formulate my question as constraints on $f(\;)$,
or pointers to relevant literature,
would be appreciated.  Thanks!
 A: Here is a function $f(x,y)$ which is 0 inside the square $C=[\pm1,\pm1]$,
and outside that square
has value equal to the Euclidean distance $d( p, C )$ from $p=(x,y)$ to the boundary of $C$.
[I am trying to follow Pietro Majer's suggestion, as far as I understand it.]
It is not a surface of revolution
(but it is centrally symmetric).
Are its gradient descent paths geodesics?
I think so...



Left above: $f(x,y)$. Right above: Level sets of $f$.
Below: $\nabla f$.



And here (below) is a closeup of the function defined using squared distance
$[d( p, C )]^2$, as per Willie Wong's suggestion:

A: For (Q1). The tangent space of $S$ is generated by the gradient flow vector field $v = (|\nabla f|^2, \nabla f)$ and the tangents to the level sets $w= (0, \nabla^\perp f)$. The geodesic constraint can be imposed as the condition "no sideways acceleration", which means that $[(\nabla f \cdot \nabla )v] \cdot w = 0$. This implies that $\nabla^2_{ij} f \nabla^if \nabla^{(\perp)j}f = 0$. In other words, the eigendirections of the Hessian of $f$ must be $\nabla f$ and its orthogonal, or that $\nabla f$ is parallel to $\nabla |\nabla f|^2$. So this means that $f$ and $|\nabla f|^2$ share the same level sets. (This same characterization is valid for any dimension; so also answers (Q3). )
In particular, this answers Denis Serre's (Q4) in the positive. 
