2
$\begingroup$

Context:

I'm working on a convergence theorem for an accelerated version of an iterative optimisation algorithm. At regularly-spaced intervals during the algorithm, a number of previous (unaccelerated) iterates are linearly combined to form a new (accelerated) iterate.

I need to know if the map the combines the iterates is a ($C^1$-)diffeomorphism.

Details:

The unaccelerated algorithm produces a sequence $x_0,\dots, x_k, x_{k+1}$ of linearly independent vectors in $\mathbb{R}^n$.

The map $R$ takes the most recent iterate $x = x_{k+1}$ to the new iterate $R(x)$: $$R : \mathbb{R}^{n} \to \mathbb{R}^n, \quad x \mapsto R(x) = \sum_{j=0}^k c_j x_j.$$ The coefficients $c_j$ depend on $x$, and are the solution to an equality-constrained optimisation problem. I have a closed-form expression for $c$: $$c = \frac{(U^{\top}U)^{-1}\mathbf{1}}{\mathbf{1}^{\top}(U^{\top}U)^{-1}\mathbf{1}},$$ where $\mathbf{1} \in \mathbb{R}^{k+1}$ is a column vector of ones, and where the matrix $U \in \mathbb{R}^{n\times(k+1)}$ has rank $k+1$, and is defined by $$U = [x_1 - x_0, x_2 - x_1, \dots, x - x_k].$$ (The important term is the $x$ in the rightmost column.)

Is the map $R$ a diffeomorphism?

Or is there a manifold on which it is?

Clearly $R$ is injective and differentiable. Is it invertible at each $x\in \mathbb{R}^n$ such that the aforementioned linear-independence/full-rank conditions hold? Is the inverse (once-)differentiable at these points?

EDIT:

I'm assuming that $x_{k+1}$ is such that $U$ has rank $k+1$ for all $x$ in an open neighbourhood of $x_{k+1}$. If this were not the case, my algorithm would skip the application of $R$. So the revised question becomes, is $R$ a diffeomorphism when restricted to that neighbourhood?

$\endgroup$
4
  • $\begingroup$ I am sorry I do not quite understand that why $U$ has rank $k+1$ or you mean that you restrict your map to the $x$ such that $U$ has rank $k+1$? $\endgroup$
    – Siming Tu
    Commented Jul 25, 2017 at 15:56
  • $\begingroup$ @SimingTu: Thanks, good point. I mean that I restrict the map to the $x$ such that $U$ has rank $k+1$. If it so happens that the iterative algorithm produces $x = x_{k+1}$ that results in a $U$ with lower rank, then the mapping $R$ will not be applied. We can assume that $x$ is in an open neighbourhood such that $U$ has rank $k+1$ for all $x_{k+1}$ in that neighbourhood. $\endgroup$ Commented Jul 26, 2017 at 13:11
  • $\begingroup$ The image of $R$ is contained in the linear space spanned by $x_0, x_1,ldots,x_k$ (In fact just R(x) is a convex combination of the $k+1$ vectors) . Then the dimension of the domain $n$ is large than the dimension of the image $k+1$ so they can not be homeomorphic to each other so $R$ can not be a diffeomorphism I think. $\endgroup$
    – Siming Tu
    Commented Jul 27, 2017 at 12:29
  • $\begingroup$ @SimingTu. I think you're right, thanks. One concern is that I don't know the dimension of the domain of $R$, because of the restriction on the rank of $U$. Since $x_0,\dots, x_k, x_{k+1}$ are assumed to be linearly independent, we know that $x_{k+1}$ is in a space of dimension $n - (k+1)$. So $R$ could only possibly be a diffeomorphism if $n = 2(k+1)$. Thanks for the answer! $\endgroup$ Commented Jul 28, 2017 at 9:18

0

You must log in to answer this question.