# Generalized tensor-train decomposition

If $$U\in\bigotimes_{k=1}^p\mathbb R^{n_k}$$ and $$U^{(k)}\in\mathbb R^{r_{k-1}}\otimes\mathbb R^{n_k}\otimes\mathbb R^{r_k}$$ with$$^1$$ $$U_{i_1,\:\ldots\:,i_p}=\sum_{j_0=1}^{r_0}\cdots\sum_{j_p=1}^{r_p}U^{(1)}_{j_0i_1j_1}\cdots U^{(p)}_{j_{p-1}i_pj_p}\tag1,$$ then $$(1)$$ is called a tensor-train decomposition of $$U$$. $$(1)$$ has to be understood with respect to the isomorphisms $$\bigotimes_{k=1}^p\mathbb R^{n_k}\cong\mathbb R^{n_1\times\cdots\times n_p}$$ and $$\mathbb R^{r_{k-1}}\otimes\mathbb R^{n_k}\otimes\mathbb R^{r_k}\cong\mathbb R^{r_{k-1}\times n_k\times r_k}$$.

Is there a generalization of this decomposition type to general $$\mathbb R$$-vector spaces (or Banach/Hilbert spaces) $$E_i,F_j$$? Something like $$U\in\bigotimes_{k=1}^p E_k$$, $$U^{(k)}\in F_{k-1}\times E_k\times F_k$$, $$U=\iota\bigotimes_{k=1}^pU^{(k)}\tag2,$$ where some identification/transformation $$\iota$$ is applied to the right-hand side?

I'm quite sure that it should be possible to write $$(1)$$ in the form of $$(2)$$ using a suitable tensor contraction$$^2$$. I'm strongly interested in a way to express $$(1)$$ without the identification with matrices.

EDIT 1: Clearly, we have the canonical isomorphisms, $$$$\begin{split}\Xi_k:\mathbb R^{r_{k-1}\times n_k\times r_k}&\to\mathbb R^{r_{k-1}}\otimes\mathbb R^{n_k}\otimes\mathbb R^{r_k},\\\left(a_{j_{k-1}i_kj_k}\right)&\mapsto\sum_{j_{k-1}=1}^{r_{k-1}}\sum_{i_k=1}^{n_k}\sum_{j_k=1}^{r_k}a_{j_{k-1}i_kj_k}\left(f^{(k-1)}_{r_{k-1}}\otimes e^{(k)}_{i_k}\otimes f^{(k)}_{j_k}\right),\end{split}$$$$ where $$\left(e^{(k)}_1,\ldots,e^{(k)}_{n_k}\right)$$ and $$\left(f^{(k)}_1,\ldots,f^{(k)}_{r_k}\right)$$ denote the standard bases of $$\mathbb R^{n_k}$$ and $$\mathbb R^{r_k}$$, respectively, and $$\Lambda_k:\mathcal L(\mathbb R^{n_k},\mathbb R^{r_{k-1}}\otimes\mathbb R^{r_k})\to\mathbb R^{r_{k-1}}\otimes\mathbb R^{n_k}\otimes\mathbb R^{r_k}\;,\;\;\;A\mapsto\sum_{i_k=1}^{n_k}e^{(k)}_i\otimes Ae^{(k)}_i.$$ Now, it's easy to see that $$\Lambda_k^{-1}\left(U^{(k)}\right)=\sum_{i_k=1}^{n_k}\left\langle\;\cdot\;,e^{(k)}_{i_k}\right\rangle\operatorname{tr}_{21}\left(U^{(k)}\otimes e^{(k)}_{i_k}\right)\tag3.$$ So, $$\Lambda_k^{-1}\left(U^{(k)}\right)(x_k)=\operatorname{tr}_{21}\left(U^{(k)}\otimes x_k\right)\;\;\;\text{for all }x_k\in\mathbb R^{n_k}.\tag4$$ Does this allow us to rewrite $$(1)$$?

EDIT 2: I guess the natural generalization is the following: Let $$E_i$$ be a $$\mathbb R$$-vector space, $$I$$ be a finite nonempty set and $$U\in\bigotimes_{i\in I}E_i$$. Let $$\langle\;\cdot\;,\;\cdot\;\rangle_{\bigotimes_{i\in I}E_i,\:\bigotimes_{i\in I}E_i^\ast}$$ denote the canonical duality pairing between $$\bigotimes_{i\in I}E_i$$ and $$\bigotimes_{i\in I}E_i^\ast$$. Then we may look for a factorization of $$\left\langle U,\bigotimes_{i\in I}\varphi_i\right\rangle_{\bigotimes_{i\in I}E_i,\:\bigotimes_{i\in I}E_i^\ast},\tag5$$ where $$\varphi_i\in E_i^\ast$$ for $$i\in I$$. In the case of $$E_i=\mathbb R^{n_i}$$ we obtain the matrix entry $$U_{i_1,\:\ldots\:,i_p}$$ by inserting for $$\varphi_{i_k}$$ the basis functional corresponding to the $$i_k$$th standard basis vector of $$\mathbb R^{n_k}$$. Now, I guess the cores are naturally assumed to be some $$U^{(i)}\in F_{i-1}^\ast\otimes E_i\otimes F_i$$, where $$F_i$$ is another $$\mathbb R$$-vector space. I guess we need to build the trace $$\operatorname{tr}\left(\bigotimes_{i\in I}\varphi_i\otimes U^{(i)}\right)\in\bigotimes_{i\in I}F_{i-1}^\ast\otimes F_i\subseteq\bigotimes_{i\in I}\mathcal L(F_{i-1},F_i).$$ Now I think the operators induced by the embedded on the right-hand side only need to be concatenated (which might be express in terms of the tensor constraction as well; see my related question Tensor contraction (vector-valued trace) on $\bigotimes_{i=1}^k\mathcal L(E_{i-1},E_i)$) and we need to assume that $$\dim F_0=\dim F_p=1$$.

It would be great if someone could put these pieces together to formulate a generalization.

$$^1$$ $$r_0=r_p=1$$.

$$^2$$ Let $$I,J\subseteq\mathbb N$$ be finite and nonempty and $$E_i,F_j$$ be $$\mathbb R$$-vector spaces for $$(i,j)\in I\times J$$. If $$I_0\subseteq I$$ and $$J_0\subseteq J$$ with $$|I_0|=|J_0|\ge1$$, $$\phi:I_0\to J_0$$ denotes the canonical bijection and $$E_i=F_{\phi(i)}^\ast\;\;\;\text{for all }i\in I_0\tag6,$$ then the tensor contraction (or vector-valued trace) with respect to $$(I_0,J_0)$$ is given by $$$$\begin{split}\operatorname{tr}_{I_0J_0}:&\bigotimes_{k\in I}E_k\otimes\bigotimes_{l\in J}F_l\to\bigotimes_{k\in I\setminus I_0}E_k\otimes\bigotimes_{l\in J\setminus J_0}F_l,\\&\bigotimes_{k\in I}x_k\otimes\bigotimes_{l\in J}y_l\mapsto\prod_{i\in I_0}\left\langle x_i,y_{\phi(i)}\right\rangle_{F_{\phi(i)}}\bigotimes_{k\in I\setminus I_0}x_k\otimes\bigotimes_{l\in J\setminus J_0}y_l.\end{split}$$$$ If $$(i,j)\in I\times J$$, then $$\operatorname{tr}_{ij}:=\operatorname{tr}_{\{i\}\{j\}}$$.

• Sure. One way is to define the cores in the tensor train as matrix valued functions. The term to search for is "functional tensor train" – Nick Alger May 23 at 20:54
• @NickAlger Thank you for the search term. Do you have an explicit reference at hand which is particularly suitable for my purpose? – 0xbadf00d May 24 at 4:30
• @NickAlger Please note that I've edited the question once again. – 0xbadf00d May 24 at 14:19
• @0xbadf00d: your question is close to unreadable. To remedy this you need to use a graphical/diagrammatic notation as discussed in this MO question mathoverflow.net/questions/168888/… It's OK to write by hand, scan and insert the picture in the MO question. – Abdelmalek Abdesselam May 24 at 23:19

I'm not sure about your approach. The conventional way to extend tensor trains to Hilbert spaces (or more generally) is as follows. One may identify tensors with multilinear functions, $$T:H_1 \times H_2 \times \dots H_p \rightarrow \mathbb{R}$$, where $$H_i$$ are Hilbert spaces. A tensor train is then a sequence of functions $$G_i : H_i \rightarrow \mathbb{R}^{r_i \times r^{i+1}}, \quad i=1,\dots,p$$ where $$r_1=1$$ and $$r_p=1$$ such that $$T(u_1, u_2, \dots, u_p) = G(u_1)G(u_2) \dots G(u_p)$$ where the $$G_i(u_i)$$ are multiplied with usual matrix multiplication.
So the cores $$G_i$$ are infinite in the externally facing mode, but finite in the internally facing modes. If you want to allow the internal modes of the cores to be infinite, it would be pretty straightforward to replace the matrices $$G_i(u_i)$$ with linear operators between some suitable spaces.