# Best approximation with tensors of rank $\ge2$

Let $$k\in\mathbb N$$, $$H_i$$ be a (finite-dimensional, if necessary) $$\mathbb R$$-Hilbert space for $$i\in I:=\{1,\ldots,k\}$$, $$H:=\bigotimes_{i\in I}H_i$$ denote the tensor product$$^1$$ of $$(H_i)_{i\in I}$$ and $$H^{(r)}:=\left\{u\in H:\operatorname{rank}u=r\right\}\;\;\;\text{for }r\in\mathbb N_0.$$

I'm struggling to understand the importance and implication of the following result: Let $$v\in H$$.

1. There is a $$u\in H$$ with $$\operatorname{rank}u=1$$ and $$\left\|u-v\right\|_H=\inf_{u\in H^{(1)}}\left\|u-v\right\|_H\tag1.$$
2. There is a $$u\in H^{(3)}$$ and a $$(u_n)_{n\in\mathbb N}\subseteq H^{(2)}$$ with $$\left\|u_n-u\right\|_H\xrightarrow{n\to\infty}0\tag2.$$

Okay, by 1., there is a (not necessarily unique) minimizer of $$H^{(1)}\to[0,\infty)\;,\;\;\;u\mapsto\left\|u-v\right\|_H\tag3.$$

Question 1: But why can we infer from 2. that the analogous problem of minimizing $$H^{(2)}\to[0,\infty)\;,\;\;\;u\mapsto\left\|u-v\right\|_H\tag4$$ may have no solution? I guess we need to take $$v=u$$ (with $$u$$ as in 2.), but why does the existence of $$(u_n)_{n\in\mathbb N}$$ imply that there is no solution?

Question 2: That we can only guarantee the existence of a minimizer of $$H^{(r)}\to[0,\infty)\;,\;\;\;u\mapsto\left\|u-v\right\|_H\tag5$$ for $$r=1$$ is unsatisfactory only if it would actually be beneficial to take $$r$$ as large as possible. I could imagine that the error $$\left\|u^{(r)}-v\right\|_H$$ of a hypothetical minimizer $$u^{(r)}$$ of $$(5)$$ is nonincreasing in $$r\in\mathbb N_0$$. Is this the case? If so, how can we show this?

Question 3: Can we infer from 2. that $$(5)$$ may have no minimizer for all $$r\ge2$$?

$$^1$$ If $$E_i$$ is a $$\mathbb R$$-vector space, I'm defining $$(x_1\otimes x_2)(B):=B(x_1,x_2)\;\;\;\text{for }B\in\mathcal B(E_1\times E_2)\text{ and }x_i\in E_i,$$ where $$\mathcal B(E_1\times E_2)$$ is the space of bilinear forms on $$E_1\times E_2$$, and $$E_1\otimes E_2:=\operatorname{span}\{x_1\otimes x_2:E_i\in E_i\}\subseteq{\mathcal B(E_1\times E_2)}^\ast.$$

• The rank strata are generally not closed so it should not be surprising that minimzers may not exist. It’s more natural to take the union of rank strata for ranks up to $r$ (sadly these unions are still generally not closed, except in the matrix case or when $r=1$), and being nested it’s then clear that errors are nonincreasing in $r$. Dec 15, 2021 at 13:53

I don't know about infinite-dimensional Hilbert spaces and arbitrary $$v \in H$$. For any $$I, J, K \in \mathbb{Z}_{\geq 1}$$ and for $$H_1 = \mathbb{R}^I, H_2 = \mathbb{R}^J, H_3 = \mathbb{R}^K$$, there is an example of a rank-3 $$v \in H_1 \otimes H_2 \otimes H_3$$, which can be approximated arbitrarily well by a rank-2 $$u \in H_1 \otimes H_2 \otimes H_3$$. You can find the example in http://www.kolda.net/publication/koba09/ page 469.
• Thank you for your answer. Yes, I know this particular example. That's the point of (2.). The question is, why the desired conclusions are possible. The existence of $u$ as in this example shows that $H^{(r)}$ is not closed though. Jul 23, 2020 at 10:12