Skip to main content
edited body
Source Link
Asvin
  • 7.7k
  • 2
  • 21
  • 52

Let $M$ be a Riemannian manifold (I'm happy to assume it is Euclidean space) with a function $f: M \to \mathbb R$ and a group of symmetriesisometries $G$ acting on $M$ and preserving $f$, i.e., $f(gm) = f(m)$ for all $g \in G, m \in M$. We assume that $G$ is a lie group (and even a torus) for simplicity.

In this situation, let $X_f = \nabla f$ be the vector field given by the gradient of $f$ at every point with a resulting gradient flow (flowing towards minima). The flow might not be defined at every point but we will not worry about that and only consider the set of values at which it is well defined. Moreover, we assume that we start the gradient flow at a point $m \in M$ so that a local minima is reached after a finite amount of time, let $\gamma: [0,T] \to M$ be the corresponding path so that $\gamma(0) = m$ and $\gamma(T)$ is the corresponding local minima.

Is it true that if I instead consider gradient flow from initial point $gm$, then a minima is once again reached in finite time and moreover, this new minima is in the $G$-orbit of $\gamma(T)$?

The impetus behind this question is that I have an intuition that in the above set up, the correct space to consider gradient flow dynamics on is not $M$ but rather $M/G$. However, this idea wouldn't even get off the ground if the $G$-orbit of a minima one reaches by gradient flow depends on which point in a $G$-orbit we start at.

Let $M$ be a Riemannian manifold (I'm happy to assume it is Euclidean space) with a function $f: M \to \mathbb R$ and a group of symmetries $G$ acting on $M$ and preserving $f$, i.e., $f(gm) = f(m)$ for all $g \in G, m \in M$. We assume that $G$ is a lie group (and even a torus) for simplicity.

In this situation, let $X_f = \nabla f$ be the vector field given by the gradient of $f$ at every point with a resulting gradient flow (flowing towards minima). The flow might not be defined at every point but we will not worry about that and only consider the set of values at which it is well defined. Moreover, we assume that we start the gradient flow at a point $m \in M$ so that a local minima is reached after a finite amount of time, let $\gamma: [0,T] \to M$ be the corresponding path so that $\gamma(0) = m$ and $\gamma(T)$ is the corresponding local minima.

Is it true that if I instead consider gradient flow from initial point $gm$, then a minima is once again reached in finite time and moreover, this new minima is in the $G$-orbit of $\gamma(T)$?

The impetus behind this question is that I have an intuition that in the above set up, the correct space to consider gradient flow dynamics on is not $M$ but rather $M/G$. However, this idea wouldn't even get off the ground if the $G$-orbit of a minima one reaches by gradient flow depends on which point in a $G$-orbit we start at.

Let $M$ be a Riemannian manifold (I'm happy to assume it is Euclidean space) with a function $f: M \to \mathbb R$ and a group of isometries $G$ acting on $M$ and preserving $f$, i.e., $f(gm) = f(m)$ for all $g \in G, m \in M$. We assume that $G$ is a lie group (and even a torus) for simplicity.

In this situation, let $X_f = \nabla f$ be the vector field given by the gradient of $f$ at every point with a resulting gradient flow (flowing towards minima). The flow might not be defined at every point but we will not worry about that and only consider the set of values at which it is well defined. Moreover, we assume that we start the gradient flow at a point $m \in M$ so that a local minima is reached after a finite amount of time, let $\gamma: [0,T] \to M$ be the corresponding path so that $\gamma(0) = m$ and $\gamma(T)$ is the corresponding local minima.

Is it true that if I instead consider gradient flow from initial point $gm$, then a minima is once again reached in finite time and moreover, this new minima is in the $G$-orbit of $\gamma(T)$?

The impetus behind this question is that I have an intuition that in the above set up, the correct space to consider gradient flow dynamics on is not $M$ but rather $M/G$. However, this idea wouldn't even get off the ground if the $G$-orbit of a minima one reaches by gradient flow depends on which point in a $G$-orbit we start at.

Source Link
Asvin
  • 7.7k
  • 2
  • 21
  • 52

Gradient descent under the presence of symmetries

Let $M$ be a Riemannian manifold (I'm happy to assume it is Euclidean space) with a function $f: M \to \mathbb R$ and a group of symmetries $G$ acting on $M$ and preserving $f$, i.e., $f(gm) = f(m)$ for all $g \in G, m \in M$. We assume that $G$ is a lie group (and even a torus) for simplicity.

In this situation, let $X_f = \nabla f$ be the vector field given by the gradient of $f$ at every point with a resulting gradient flow (flowing towards minima). The flow might not be defined at every point but we will not worry about that and only consider the set of values at which it is well defined. Moreover, we assume that we start the gradient flow at a point $m \in M$ so that a local minima is reached after a finite amount of time, let $\gamma: [0,T] \to M$ be the corresponding path so that $\gamma(0) = m$ and $\gamma(T)$ is the corresponding local minima.

Is it true that if I instead consider gradient flow from initial point $gm$, then a minima is once again reached in finite time and moreover, this new minima is in the $G$-orbit of $\gamma(T)$?

The impetus behind this question is that I have an intuition that in the above set up, the correct space to consider gradient flow dynamics on is not $M$ but rather $M/G$. However, this idea wouldn't even get off the ground if the $G$-orbit of a minima one reaches by gradient flow depends on which point in a $G$-orbit we start at.