Minimal norm problem whose unknown is an operator

Question

Generally given an Hilbert space $X$ with and a bounded linear operator $H : X \to X$ given a vector $y \in X$ we seek an $x \in X$ such that

$$ f(x) = \frac{1}{2} \left\lVert Hx - y \right\rVert_2^2 $$

Is minimal to calculate the Frechet differential we could calculate

$$ f(x + h) - f(x) = \left\langle Hh, Hx - y \right\rangle + \frac{1}{2} \left\lVert Hh \right\rVert_2^2 $$

Where the differential $D[f](h) := \left\langle Hh, Hx - y \right\rangle$, to find the minimum of such functional we seek an $x \in X$ such that for all $h \in X$ we have

$$ 0 = D[f](h) := \left\langle Hh, Hx - y \right\rangle = \left\langle h, H^*Hx - H^*y \right\rangle $$

and therefore we want to find $x$ such that

$$ H^* H x = H^* y \iff x = (H^*H)^{-1} H^* y $$

which is the well known pseudoinverse.

Now in a problem I have instead of attempting to find $x \in X$ I am attempting to find $H \in B(X)$ given both $x, y \in X$. Now defining

$$ F(H) = \frac{1}{2} \left\lVert Hx - y \right\rVert_2^2 $$

and calculating

$$ F(H + E) - F(H) = \left\langle Ex, Hx - y \right\rangle + \frac{1}{2} \left\lVert Ex \right\rVert_2^2 := D[f][E] + \frac{1}{2} \left\lVert Ex \right\rVert_2^2 $$

the differential yields

$$ 0 = D[f](E) = \left\langle Ex, Hx - y \right\rangle $$

Which is the best characterization I can give at the moment of $H$, I wonder however:

1. Can something precise can be said about $H$, maybe characterizing it by an equation not involing $E$?
2. In addition to 1. can we say something when we know $H$ is of the form: $$ H = \sum_{1 \leq k \leq n} \alpha_j T_{x_j} $$

where each $\alpha_j \in \mathbb{R}$ and $T_{x_j}f(x) = f(x - x_j)$ (i.e. the translation operator, assuming $X = L^2(G)$ where $G$ is some locally compact group).

Answer 1

Let me try to explain the series of comments by David Gao and transform it into a complete answer. It is worth keeping in mind the types of objects here:

The elements $x, y$ are considered to be given and fixed.
Corresponding to $x$ and $y$ we define the functional $F:B(X) \to \mathbb{R}$ with the expression $F(H) = \frac12 \| Hx - y\|^2$.
Our goal is to find a (all?) minimizer of $F$.

Case 1: $x = 0$

If the parameter $x$ is the zero element of $X$, then the expression for $F$ reduces to $$ F(H) = \frac12 \|y\|^2 $$ which is constant. And hence every $H\in B(X)$ is the global minimum of $F$.

Case 2: $x = 0$

Observe that by definition $F(H) \geq 0$ for any $H$. Hence if we find an $H_0$ such that $F(H_0) = 0$, then $H_0$ would be a global minimum.

In order for $F(H_0) = 0$, we need to $H_0x - y = 0$. This we see that

a sufficient condition for $H_0\in B(X)$ to be a minimizer of $F$ is that $H_0x = y$.

A particular example of one such operator is given by $$ H_*: X \ni \xi \mapsto \frac{\langle x,\xi\rangle}{\langle x,x\rangle} y $$ (In the case where $X$ is a complex inner product space, I prefer by inner products to be linear [and not conjugate linear] in the second slot.) In fact, if $A$ is the subset $A:= \{H\in B(X) | Hx = 0 \}$, then given any $L \in A$ we see that $H_0 = H_* + L$ is a minimizer. Similarly, given any other $H_0$ satisfying $H_0 x = y$, we find $H_0 - H_* \in A$.

Hence

a sufficient condition for $H_0$ to be a minimizer of $F$ is that $H_0$ belongs to the affine subspace $H_* + A$ in $B(X)$.

Finally, given $H$ such that $Hx = z \neq y$, so that $F(H) > 0$. Consider now

$$ F((1-\lambda)H + \lambda H_*) = \frac12 \| (1-\lambda)z - (1-\lambda) y)\|^2 = (1-\lambda)^2 F(H) $$

This shows that $H$ cannot be even a local minimizer for $F$ (just look at all $\lambda$ small and positive).

this shows that the requirement $H_0 \in H_* + A$ is also necessary for $H_0$ to be a minimizer of $F$.

---

The variational formula

Maybe it is worthwhile to see how this is implied by the variational formula you derived.

What you've found is that the first variation of $F$ is the assignment $$ B(X) \ni E \mapsto \Re \langle E x, Hx - y\rangle $$ When $x = 0$, then this assignment is always zero; this agrees with our previous assessment that when $x = 0$ any $H$ is a solution.

When $x \neq 0$, given any $z \in X$ we can find $E_z : =\xi \mapsto \frac{\langle x,\xi\rangle}{\langle x,x\rangle} z$ such that $E_z x = z$. Therefore, in order for the first variation to vanish for all $E$, it must also vanish for all $E_z$, and hence we must have that $\langle z, Hx - y\rangle = 0$ for all $z\in X$. But this implies $Hx - y = 0$, again in agreement with the analyses above.

---

A philosophical comment

The non uniqueness of the solution should not be surprising. Even in the case where you are solving for $x$ using the pseudo inverse, the operator $H^*H$ is in general not invertible (consider the case where $H$ has a kernel). Our situation here is similar (the act of evaluating $H\in B(X)$ at a point $x\in X$ is a linear functional of $B(X)$, with $A$ being its kernel).

The availability of $H_*$ can also be drawn in parallel to the case where one is solving for $x$: in the case where $y$ belongs to the range of $H$, then any pre-image of $y$ is obviously a minimizer of $f$; here again we are solving the equation $Hx = y$.