# Gradient descent under the presence of symmetries

Let $$M$$ be a Riemannian manifold (I'm happy to assume it is Euclidean space) with a function $$f: M \to \mathbb R$$ and a group of isometries $$G$$ acting on $$M$$ and preserving $$f$$, i.e., $$f(gm) = f(m)$$ for all $$g \in G, m \in M$$. We assume that $$G$$ is a lie group (and even a torus) for simplicity.

In this situation, let $$X_f = \nabla f$$ be the vector field given by the gradient of $$f$$ at every point with a resulting gradient flow (flowing towards minima). The flow might not be defined at every point but we will not worry about that and only consider the set of values at which it is well defined. Moreover, we assume that we start the gradient flow at a point $$m \in M$$ so that a local minima is reached after a finite amount of time, let $$\gamma: [0,T] \to M$$ be the corresponding path so that $$\gamma(0) = m$$ and $$\gamma(T)$$ is the corresponding local minima.

Is it true that if I instead consider gradient flow from initial point $$gm$$, then a minima is once again reached in finite time and moreover, this new minima is in the $$G$$-orbit of $$\gamma(T)$$?

The impetus behind this question is that I have an intuition that in the above set up, the correct space to consider gradient flow dynamics on is not $$M$$ but rather $$M/G$$. However, this idea wouldn't even get off the ground if the $$G$$-orbit of a minima one reaches by gradient flow depends on which point in a $$G$$-orbit we start at.

• Shouldn't the symmetries in $G$ also preserve the Riemannian structure on $M$? Commented Oct 2, 2023 at 14:59
• Your stipulation of finite time seems unusual and not necessary to the rest of the problem. Taking $M$ to be Euclidean space and $f(x) = |x|^2$, you only get convergence of the gradient flow $\dot x = -\nabla f$ to the minimizer $x = 0$ from an arbitrary starting point in the limit as time grows infinite. Commented Oct 2, 2023 at 16:05
• @DanielShapero That assumption was motivated by the application in mind although perhaps the answer will turn out to be more general. Commented Oct 2, 2023 at 19:55

Since $$G$$ preserves $$f$$, it preserves $$df$$ where defined, and preserves the set of points where $$df$$ is defined. Since $$G$$ preserves $$g$$ and $$df$$ it preserves $$X=\nabla f$$. So it takes flow lines of $$X$$ to flow lines of $$X$$. So the flow line of $$X$$ through $$m$$ maps by any element $$g \in G$$ to a flow line of $$X$$ through $$gm$$. We need to be a little careful: if $$f$$ is only $$C^1$$ along this flow line, we might have other flow lines as well through the same point, because we can't apply existence and uniqueness unless we have sufficient smoothness of $$X$$, hence of $$f$$. But on the open set where $$f$$ is $$C^2$$, we have existence and uniqueness as well.