Minimal norm problem with linear combination of translation operator to be estimated

Question

Follow up question from this one

Suppose $X = L^2(G)$, where $G$ is some locally compact group. Let $x, y \in G$ I for fixed $n$ I am seeking for an operator $H \in B(X)$ of the form

$$ H = H(\alpha_1,\ldots,\alpha_n,x_1\,\ldots, x_n)= \sum_{1 \leq i \leq n} \alpha_i T_{x_i} $$

Such that the norm

$$ f(H) = \frac{1}{2} \left\lVert Hx - y\right\rVert_2^2 $$

is minimized.

Questions:

1. How can I prove a minimizer exists?
2. Is there an explicit description of such minimizer (i.e. a formula maybe?)

As a more concrete example I have the following, maybe this can help to clarify where I am going:

Let $X = L^2(\mathbb{R},e^{-t^2}dt)$, $x(t) = t\times u(t)$, where $$ u(t) = \begin{cases} 1 & \text{t $\geq$ 0} \\ 0 & \text{otherwise} \end{cases}. $$ Let also $y(t) = e^{-t^2}$. Both $x, y \in X$. Fix $n = 3$ then I suspect that in this situation the minimizer is unique.

See the picture below:

What I am attempting to do with this question is to generalize the problem and phrase it in terms of finding a suitable linear operator in some Hilbert space, and maybe Banach Space.

Question 3. Can I solve the case I just mentioned uniquely? how much can I generalize?

Update For the case 3. I've started doing some calculations

First of all:

Proposition 1 The differential of $\alpha T_{x}$ w.r.t. $\alpha$ the map $D_{\alpha} [\alpha T_x] : h \to h T_x$. Proof: Straightforward using the definition of Frechet derivative.

Proposition 2 The differential of $\alpha T_x$ w.r.t. $x$ is the map $D_x [\alpha T_x] = \alpha T_x \circ \frac{d}{dx}$ Proof: This follows from $$ \frac{\alpha T_{x + h} - \alpha T_x}{h} = \alpha T_x \frac{T_h - I}{h} \to \alpha T_x \circ \frac{d}{dx} $$

Now where I am stuck is to calculate the derivative of

$$ f(\alpha_0,\alpha_1,\alpha_2,x_0,x_1,x_2) = \frac{1}{2}\left\lVert H(\alpha_0,\alpha_1,\alpha_2,x_0,x_1,x_2)x - y \right\rVert $$

I do know however that I have to use the chain rule. I am just a bit confused on how. Once I do this I would be posing the result to 0 to find the critical point and this would give me the result I think I am seeking.

I think I should be getting a result like

$$ D_{\alpha_0,...,x_2} f = \left\langle \cdot x, H(\alpha_0,\ldots,x_2)x - y \right\rangle \left[ \begin{array}{c} T_{x_1} \\ T_{x_2} \\ T_{x_3} \\ \alpha_1 T_{x_1} \circ \frac{d}{dx} \\ \alpha_2 T_{x_2} \circ \frac{d}{dx}\\ \alpha_3 T_{x_3} \circ \frac{d}{dx} \end{array} \right] $$

But I am not sure of this last calculation.

Answer 1

I think, fundamentally, you are trying to make the question a lot more complicated than it really is and your approach is doomed to fail. Here are some problems.

You cannot expect to use "linear algebraic" techniques in general for these problems

Here I include infinite dimensional (functional analytic) cases. The reason is that, to reduce the problem to a linear algebraic one, fundamentally you need to have some linear structure. In the case referenced in your linked question where you try to minimize $\|Hx - y\|$ by changing $x$ for a fixed $H$ and $y$, you are using that for a linear operator $H$, its range $\{Hx: x\in X\}$ is a linear space.

In your case here there is no linearity associated to the translation operators $T_x$: the set $\{H(\vec{\alpha}, \vec{\xi})x : \vec{\alpha}\in \mathbb{R}^n, \vec{\xi} \in G^n\}$ is not closed under addition. To make the set a linear subspace, you need to consider the set of ALL finite linear combinations of translations of $x$; but as David Gao observed such a space need not be a (topologically) closed subspace of $X$.

You cannot expect to use "geometric" techniques in general for these problems

The next best thing, when one doesn't have a linear problem, is when one has infinitesimally a linear problem. In this case we can rely on geometry to help. But as I observed in a comment, generically the set $\{H(\vec{\alpha}, \vec{\xi})x : \vec{\alpha}\in \mathbb{R}^n, \vec{\xi} \in G^n\}$ is not even a $C^1$ manifold. So in general, one cannot even use finite dimensional differential calculus to solve your problem.

---

Well, if we throw away linear algebra and calculus, we may still have left topology. But

Your problem may fail to be compact

In general a tried and true method for proving existence of a minimizer without using calculus or linear algebra is this:

1. First prove that the function you want to optimize is bounded below.
2. Next prove that there is a compact set $K$ in the domain such that outside of $K$ there cannot be an optimizer.
3. Finally prove that the function you wish to optimize is continuous (or at least semi-continuous of the right type).
4. Conclude that any minimizing sequence is eventually in $K$ and hence has an accumulation point. Continuity forces the accumulation point to be a minimizer.

In your case the first and third steps are easy (thinking of $f(H)$ as a function defined on $\mathbb{R}^n\times G^n$). But step 2 is hard, with the difficulty primarily coming from the group $G$. In fact, there are counterexamples showing that the minimization problem is not always solvable.

---

Failing the abstract arguments, perhaps it is much easier to just go back to basics. In your case: let $$ E_2(a) = \int_a^\infty x^2 e^{-x^2} dx $$ $$ E_1(a) = \int_a^\infty x e^{-x^2} dx $$ $$ E_0(a) = \int_a^\infty e^{-x^2} dx $$ we find $$ \langle T_{a}x, T_b x\rangle = E_2(\max(a,b)) - (a+b)E_1(\max(a,b)) + ab E_0(\max(a,b)) $$ and $$ \langle T_a x, y \rangle = \frac12 E_1(\sqrt{2}a) - \frac{a}{\sqrt{2}} E_0(\sqrt{2}a) $$ This allows you to expand, for $$ H = \alpha_1 T_{x_1} + \alpha_2 T_{x_2} + \alpha_3 T_{x_3} $$ with $x_1 \leq x_2 \leq x_3$, the function $f(H)$ as a continuously differentiable function of $(\vec{\alpha}, \vec{x})$. (Note that $E_2, E_1, E_0$ can all be expressed in terms of the exponential and the error function.) The outcome of this computation will be exactly the same as what you would obtain if you use the method you described whereby you take the formal differential of the operators involved, but can now be carried out by a reasonably well-practiced undergraduate.

Of course, solving the resulting equations for the critical point will involve evaluating the error function, so may not be too possible explicitly. But one can potentially appeal to topological methods such as Poincare-Hopf to establish existence.