3
$\begingroup$

For suitably smooth functions $f(x,y): \mathbb{R}^2 \rightarrow \mathbb{R}$, the concept of stationary points, in particular local minima, can as easily be mathematically defined as it is intuitively understood visually.

Now, visually, one can also often easily identify valleys within functions. Think for example of a part of a waterslide.

The functions $f(x,y) = x^2 + (y/c)^2$, $|c| > 1$ for instance have a stationary point at the origin, and a valley along the y-axis. So we might consider every point $(0,y)$ as part of a stationary curve. However, despite the ease of the intuition, I do not know a mathematical definition that would fit in a more general case.

Do you know of any such generalization and if so, how to determine such?

Naively, one might first think that only one component of the gradient needs to vanish, but subject to rotation, this is true for every point of a smooth function.


Edit (due to discussions below):

As demonstrated below, two approaches on my side to more narrowly define what I think I am searching for have somewhat failed. Ironically, I can not give an objective answer why I feel so, as such an objective answer remains my question. I am putting these here to separate them from the original question.

Faulty physical motivation:

Physically, while a locally minimal point can be thought of as a ball remaining still on a surface, a valley might be where every part of a chain remains still (or while sliding down, follows a one-dimensional track). One may translate the latter into a rigorous mathematical property, but it seems there should be something easier and local.

Faulty mathematical definition:

A stationary curve (in two dimensions) of $f(x,y)$ may be defined as any path $p: (0,1) \rightarrow \mathbb{R}^2$ that fulfills the following property: there exists $\delta > 0$ such that for every $0 < \varepsilon < \delta$ and every open interval $I \subset (0,1)$, there is no path $p_{\varepsilon}: I \rightarrow \mathbb{R}^2 \setminus p(I)$ for which $f(p_{\varepsilon}(s)) < f(p(s))$ and $\|p_{\varepsilon}(s) - p(s)\| < \varepsilon$ for all $s \in I$.

$\endgroup$
4
  • $\begingroup$ A question to clarify what you're looking for: Why do you not see/wish to include the x-axis as a stationary curve for the function you used as an example? $\endgroup$
    – Yankl
    Commented Jun 9, 2022 at 15:06
  • $\begingroup$ To add onto my previous comment, it seems that the condition which you're looking for is that the curve is tangent to the gradient of $f$. $\endgroup$
    – Yankl
    Commented Jun 9, 2022 at 15:18
  • $\begingroup$ What you describe are the parts of the function where $f$ has constant values. This is unfortunately not what I mean. $\endgroup$ Commented Jun 9, 2022 at 17:26
  • $\begingroup$ It sounds like you want negative-gradient integral curves that also extremize some condition involving second derivatives. Maybe integral curves whose initial condition is a direction of maximum curvature at a critical point, or integral curves that are always tangent to an eigenvector of the Hessian matrix? Then could check that solutions are accumulators for other integral curves, to verify the intuition of a waterslide (water always descends along gradient, and will accumulate to the "minimum curve" of the slide) $\endgroup$
    – Neal
    Commented Jun 14, 2022 at 16:12

2 Answers 2

4
$\begingroup$

$\newcommand{\ep}{\varepsilon}$As noted in a comment by Yankl, a stationary curve can be defined as one with tangent in the direction of the gradient of $f$. So, a stationary curve can be given as a solution to the system \begin{equation} \dot x=f_x,\quad \dot y=f_y \end{equation} of ODEs, where $f_x$ and $f_y$ are the partial derivatives of $f(x,y)$ with respect to $x$ and $y$.

We can further say that a stationary curve $C$ for $f$ is the bottom of a valley (BoV) if the second directional derivative of $f$ at any point on $C$ in the direction $(f_y,-f_x)$, orthogonal to the gradient, is $>0$. We can also say that a stationary curve $C$ for $f$ is the top of a ridge (ToR) if $C$ is the BoV for the function $-f$.

For instance, if $f(x,y)=x^2-y^2$, then the projections of the stationary curves onto the $xy$-plane are the hyperbolas $\{(x,y)\colon xy=c\}$ for real $c$. The parts $\{(x,y)\colon xy=c,y^2>x^2\}$ of these projections correspond to the BoV's, while the parts $\{(x,y)\colon xy=c,y^2<x^2\}$ of these rojections correspond to the ToR's.

These BoV's and ToR's for $c=-1/10$ are shown (in blue) in the picture below (two BoV's and two ToR's):

enter image description here


The OP has added the following:

Edit: I think I may give a more proper mathematical definition thanks to the discussion below. It probably still lacks some additional care, and in particular, it does not answer how to obtain such practically.

A stationary curve (in two dimensions) of $f(x,y)$ may be defined as any path $p: (0,1) \rightarrow \mathbb{R}^2$ that fulfills the following property: there exists $\delta > 0$ such that for every $0 < \varepsilon < \delta$ and every open interval $I \subset (0,1)$, there is no path $p_{\varepsilon}: I \rightarrow \mathbb{R}^2 \setminus p(I)$ for which $f(p_{\varepsilon}(s)) < f(p(s))$ and $\|p_{\varepsilon}(s) - p(s)\| < \varepsilon$ for all $s \in I$.

Well, if this is the definition of a stationary curve, then the OP's question has been answered. However, this definition is not good.

Indeed, let $f$ be the "slide" function given by the formula $f(x,y):=x^2+y$. Then apparently the OP (and everyone) would agree that the path $p$ given by the formula $p(s):=(0,s)$ corresponds to a stationary, bottom-of-a-valley curve for $f$; see a graph of $f$ together with the (red) stationary curve:

enter image description here

However, letting $p_\ep(s):=(\ep/2,s-\ep^2/2)$ for $\ep\in(0,1)$, we see that $p_\ep((0,1))\cap p((0,1))=\emptyset$, $f(p_\ep(s))<f(p(s))$ and $\|p_\ep(s)-p(s)\|<\ep$ for all $s\in(0,1)$, so that the path $p$ is not stationary in the sense of the highlighted definition.

Similarly to the first example in this answer, with $f(x,y)=x^2-y^2$, we can describe the stationary curves for $f(x,y):=x^2+y$: These are the curves whose projections onto the $xy$-plane are the "exponential" curves $\{(x,y)\colon x=ce^{2y}\}$ for real $c$. Each of these curves is the BoV in its entirety. The red curve in the latter picture corresponds to $c=0$, and the black curve there corresponds to $c=-1/20$. Shown in the latter picture also is the (dashed) curve on the surface whose projection onto the $xy$-plane is the straight line orthogonal to the projection onto the $xy$-plane of the black stationary curve at the point with $y=3/5$; this dashed curve on the surface is orthogonal to the black BoV itself at the point on the surface with $y=3/5$ and $x=ce^{2y}$ (still with $c=-1/20$). Walking any distance along the dashed curve on the surface left or right away from the black BoV will increase the vertical coordinate of the walker (because here $f$ is convex). So, the black curve is truly the bottom of a valley.


Here is yet another example, now with all the stationary curves congruent to one another: Let $f(x,y):= x + x^3/3 + y$. Then the projections of the stationary curves onto the $xy$-plane are given by equations $y=c + \arctan x$, where $c$ is any real number, so that the surface that is the graph of the function $f$ is the disjoint union of the family of stationary curves $L_c$ indexed by the real-valued parameter $c$. (Parts of these curves with $x>0$ are the BoV's and parts of these curves with $x<0$ are the ToR's.) It follows that, for any real values $c_1$ and $c_2$ of the parameter $c$, the stationary curve $L_{c_2}$ is obtained from the stationary curve $L_{c_1}$ by the parallel translation $(x,y,z)\mapsto(x,y+c_2-c_1,z+c_2-c_1)$. Therefore, here two different walkers walking with the same speed (say down) two different stationary lines (with different values of the parameter $c$) starting at the points of these two stationary lines with the same coordinate $x$ will have exactly the same experiences. So, there is no reason whatsoever to view any one of the stationary curves as special.

Here is a graph of this function $f$ together with the three (blue) stationary curves corresponding to $c\in\{-1,0,1\}$:

enter image description here

$\endgroup$
29
  • $\begingroup$ The second directional derivative orthogonal to the gradient was indeed something I considered. It may be that the curve I am looking for is contained in the set of curves you define, but my intuition tells me that the set is not yet what I am searching for. I think some physical chain would indeed slide along these curves (assuming zero inertia), but when fixing one point of such a chain, it would not stay on its curve, while only a chain of finite length would remain still when symmetrically placed within $x=0$. I would expect only a zero-set to be stationary curves in generic cases. $\endgroup$ Commented Jun 9, 2022 at 17:11
  • $\begingroup$ @SebastianK. : I don't understand your comment. In particular, what do you mean by "when fixing one point of such a chain, it would not stay on its curve"? What do you mean by "fixing"? What do you mean by $it"? What do you mean by "stay"? $\endgroup$ Commented Jun 9, 2022 at 17:21
  • $\begingroup$ What maybe comes closer is that not only the second derivatives along straight lines orthogonal to the gradient must be larger than zero, but that this also holds true along any circle which tangent is orthogonal to the gradient at the point where it intersects the supposed stationary curve. This seems to be more in line with the physical interpretation, though I have not rigorously checked this. $\endgroup$ Commented Jun 9, 2022 at 17:21
  • $\begingroup$ I meant this quite literally in a physical sense. Take a real saddle (with exactly the surface of $f$), place a metal chain on it, put your finger onto one of its segments and see what happens. I hope this is a bit clearer. Well, obviously, the problem is that I can not give you a clear mathematical definition. $\endgroup$ Commented Jun 9, 2022 at 17:25
  • 1
    $\begingroup$ @SebastianK. : The ideas that you presented contradict one another, as it has now been explained at the end of my answer. I think it is now time to finalize this matter. $\endgroup$ Commented Jun 10, 2022 at 0:16
0
$\begingroup$

I think you might be interested in the acceleration of integral curves of the gradient vector field (aka, "gradient flow curves").

Suppose $f:\mathbb{R}^n\to\mathbb{R}$ is at least $C^2$, with gradient $\nabla f$ and Hessian $\operatorname{H}f$.

If $\gamma$ is an integral curve for $\nabla f$, then for all $t$, $$ \ddot{\gamma}(t) = \operatorname{H}f_{\gamma(t)} \dot{\gamma}(t) $$ This follows from $\nabla f_q \approx_{q\to p} \nabla f_p + \operatorname{H}f_p (q-p)$ applied to the fact that $\dot{\gamma} = \nabla f$.

Since $f\in C^2$, its Hessian is symmetric, so admits an eigenvalue decomposition. Say a gradient flow curve of minimal acceleration is a gradient flow curve $\gamma$ with $\dot{\gamma}$ always lying in a minimum-eigenvalue eigenspace of $\operatorname{H}f$. (This need not always happen.)

Reversing time, so $t\to -\infty$, gradient flow curves will tend toward minimal acceleration curves. Here is a quick qualitative argument: If $e_j, \lambda_j$ is an orthonormal basis of eigenvectors of $\operatorname{H}f$ at point, write in coordinates $\dot{\gamma} = c_1v_1+\ldots +c_nv_n$. Now observe that $c_j(t - dt) \approx c_j(t) / (1 + dt \lambda_j)$, so the larger $\lambda_j$, the faster the component of $\dot{\gamma}$ in the direction of $v_j$ decays as $t\to -\infty$.

This tendency confirms the intuition of a waterslide: as water tumbles downhill (following exactly gradient descent), it accumulates toward a curve of minimal acceleration.

I would expect physicists have studied this concept as it is similar to the Hamiltonian formulation of classical mechanics. But I don't know the keywords to search.

Examples

Just to check whether this concept tracks with the desired intuition, this section works through the examples shared by Iosif Pinelis above, plus one more to illustrate.

(A) $f(x,y) = x^2 + y$

The Hessian of $f$ has two eigenvalues, $2$ and $0$, with eigenspaces parallel to the $x$- and $y$-axes respectively. For initial data $c_x, c_y$ the gradient flow curves have the form $ \gamma(t) = \big( c_x \exp(2t), t + c_y \big) $. Thus the minimum acceleration curve is the y-axis and as $t\to -\infty$ the gradient flow paths accumulate to the $y$-axis as expected.

(B) $f(x,y) = x^2 - y^2$

The Hessian of $f$ has two eigenvalues at each point, $2$ and $-2$, with eigenspaces parallel to the $x$- and $y$-axes respectively. For initial data $c_x, c_y$ the gradient flow curves have the form $ \gamma(t) = \big( c_x \exp(2t), c_y\exp(-2t) \big) $ and the two minimal acceleration gradient flow curves are the positive and negative y axes: exactly how we would expect water to flow off a saddle.

(C) $f(x,y) = x + \frac{1}{3}x^3 + y$

This example illustrates that curves of minimal acceleration need not exist. The Hessian of $f$ at $(x,y)$ has two eigenvalues, $2x$ and $0$, with eigenspaces parallel to the $x$-axis and the $y$-axis respectively.

The gradient flow curves have the form $\gamma(t) = \big( \tan(c_x + t), c_y + t \big)$. As $\dot{\gamma}(t) = \big(\sec^2(c_x + t), 1\big)$ we see that no gradient flow curves are ever tangent to an eigenspace of the Hessian. So there are no minimal acceleration gradient flow curves.

(D) $f(x,y) = \sin(x) + y$

Here's a picture of the graph of $f$:

graph of <span class=$\sin(x)+y$" />

Intuition is that the minimal acceleration gradient curves lie along the valleys of this slope. The gradient of $f$ is $\nabla f = \big( \cos(x), 1\big)$ and computing the Hessian of $f$ yields $$ \operatorname{H}f = \begin{bmatrix} -\sin(x) & 0 \\ 0 & 0 \end{bmatrix} $$

Gradient flow curves for initial data $c_x,c_y$ are $ \gamma(t) = \big( 2\arctan(\tanh((t + c_x)/2), t + c_y \big)$, except for a degenerate family of paths moving at constant speed parallel to the $y$ axis when $x = \pi/2 + k\pi$ corresponding to the ridges and valleys.

The eigenvalues of the Hessian matrix are $-\sin(x), 0$ with eigenspaces parallel to the $x$ and $y$ axes respectively (away from $x$ a multiple of $2\pi$, in which case there is a single $0$-eigenspace). The only paths that are always tangent to an eigenspace are the degenerate paths $(c_x, t+c_y)$ when $c_x = \pi/2 + k\pi$ ($k\in\mathbb{Z}$).

When $k$ is odd these degenerate paths are gradient flow curves of minimal acceleration (Hessian eigenvalue $0$) and as expected the gradient flow accumulates to these paths as $t\to -\infty$.

As an aside: at $x = \pi/2 + k\pi$ ($k$ odd) the function $f$ is equal to example (A), the "waterslide," to second order (and scaling and addition of a constant). So it is no surprise that the gradient curves behave the same way.

To illustrate, here is a plot of the gradient flow of $f$ superimposed on a contourplot of $f$: gradient streamplot of <span class=$\sin(x)+y$" />

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.