# Cores in the tensor-train decomposition

Let $$d_i\in\mathbb N$$, $$I_i:=\{1,\ldots,d_i\}$$ and $$u\in\mathbb R^{d_1}\otimes\mathbb R^{d_2}\otimes\mathbb R^{d_3}$$. It's somehow clear to me that we may regard $$u$$ as a three-dimensional array (see Corollary 3 below) and hence consider its entry $$u_{i_1i_2i_3}$$ at the index $$(i_1,i_2,i_3)\in I_1\times I_2\times I_3$$. By fixing the middle index $$i_2\in I_2$$, we may define $$v^{(i_2)}_{i_1i_3}:=u_{i_1i_2i_3}\;\;\;\text{for }(i_1,i_3)\in I_1\times I_3$$ and I guess we can again consider $$v^{(i_2)}$$ as being an element of $$\mathbb R^{d_1}\otimes\mathbb R^{d_3}$$.

However, I'm really struggling to see that all the identifications involved are really legitimate. I'm used to think of tensor products in a more abstract fashion (see my definition of the tensor product below$$^1$$).

Is the process of "fixing the middle index", which is effectively a (surjective?) transformation from $$\mathbb R^{d_1}\otimes\mathbb R^{d_2}\otimes\mathbb R^{d_3}$$ to $$\mathbb R^{d_1}\otimes\mathbb R^{d_3}$$, a special case of a more general result of somehow "folding" a tensor product space $$E_1\otimes E_2\otimes E_3$$ to $$E_1\otimes E_3$$?

Remark: Tensors $$u$$ of order 3 like this are so-called cores in the tensor-train decomposition which I'm trying to understand.

$$^1$$ If $$E_i$$ is a $$\mathbb R$$-vector space, I'm defining $$(x_1\otimes x_2)(B):=B(x_1,x_2)\;\;\;\text{for }B\in\mathcal B(E_1\times E_2)\text{ and }x_i\in E_i,$$ where $$\mathcal B(E_1\times E_2)$$ is the space of bilinear forms on $$E_1\times E_2$$, and $$E_1\otimes E_2:=\operatorname{span}\{x_1\otimes x_2:E_i\in E_i\}\subseteq{\mathcal B(E_1\times E_2)}^\ast.$$

Lemma 1: If $$I$$ is a finite nonempty set, $$(e_i)_{i\in I}$$ is the standard basis of $$\mathbb R^I$$ and $$E$$ is a $$\mathbb R$$-vector space, then the linear extension $$\iota_1$$ of $$a\otimes x\mapsto(a_ix)_{i\in I}\tag1$$ is an isomorphism between $$\mathbb R^I\otimes E$$ and $$E^I$$.

Lemma 2: If $$I_i$$ is a finite nonempty set, then $$\iota_2:\left(\mathbb R^{I_2}\right)^{I_1}\to\mathbb R^{I_1\times I_2}\;,\;\;\;a\mapsto\left(a(i_1)(i_2)\right)_{(i_1,\:i_2)\in I_1\times I_2}$$ is an isomorphism.

Corollary 3: If $$k\in\mathbb N$$ and $$d_1,\ldots,d_k\in\mathbb N$$, then $$\bigotimes_{i=1}^k\mathbb R^{d_i}\cong\mathbb R^{d_1\times\cdots\times d_k}\tag2.$$ Proof: This follows by induction from Lemma 1 and Lemma 2.

## 1 Answer

Yes, mapping $$A \otimes B \otimes C$$ to $$A \otimes C$$ by choosing a coordinate of $$B$$ (equivalently, basis element of dual space $$B^*$$) is indeed a special case of choosing any element of $$B^*$$, considering it as a map $$B \to k$$ (where $$k$$ is the field), and then extending that map to $$A \otimes B \otimes C \to A \otimes k \otimes C \cong A \otimes C$$.

Your $$v^{(i_2)}$$ is sometimes called a slice, or more specifically a $$2$$-slice, since the index in the $$2$$nd position was fixed. This map, that sends a tensor to its $$i_2$$'th $$2$$-slice, is surjective: any matrix arises as the $$2$$-slice of a tensor given by sticking that matrix into the appropriate entries, and filling the rest with any old thing (zeros, whatever).

• Thank you for your answer. The isomorphism is $(a\otimes b\otimes c)(b^\ast):=b^\ast(b)(a\otimes b)$, right? Is there an established notation which lets me denote the evaluation of $(a\otimes b\otimes c)$ at a basis functional in a more convenient way? The introduction of the notation $v^{(i_2)}_{i_1i_3}$ is quite annoying. – 0xbadf00d May 21 at 4:55
• I'm not sure of any really great notation. Sometimes it's called a "contraction" (contracting by $b^*$) and they use a symbol like $\neg$ ($b^* \neg a \otimes b \otimes c = b^*(b)(a \otimes c)$), but I suspect that's far from universal. I think there are different conventions and different notations in geometry, physics, engineering, representation theory... :-( One problem with saying "oh we'll just write $b^*$ for the map contracting tensors by $b^*$ in the 2nd entry" is, what if $A=B=C$, so $b^*$ could act in any position? Ugh. Maybe someone else has a good idea, but I don't know, sorry. – Zach Teitler May 21 at 5:01
• The notion of contraction that I'm aware of is also called the vector-valued trace. It is the mapping $$(A\otimes B^\ast)\otimes(B\otimes C)\to A\otimes C\;,\;\;\;(a\otimes b^\ast)\otimes(b\otimes c)\mapsto b^\ast(b)a\otimes c.$$ Is this what you mean? – 0xbadf00d May 21 at 5:07
• Yeah, pretty much that. I guess I was thinking primarily of the situation where you have a fixed element $b^* \in B^*$, yielding a map $A \otimes B \otimes C \to A \otimes C$; but that's just a restriction or specialization of what you wrote. I suppose the identification behind this is $\operatorname{Hom}(B^*, \operatorname{Hom}(A \otimes B \otimes C, A \otimes C)) \cong \operatorname{Hom}((A \otimes B^*) \otimes (B \otimes C), A \otimes C)$. Your vector-valued trace is an element of the right-hand Hom. My version of contraction is a particular map in the left-hand Hom. – Zach Teitler May 21 at 5:24
• Okay, denoting the linearization of $$(E_1⊗ E_2^\ast)\times(E_2⊗ E_3)\to E_1⊗ E_3\;,\;\;\;(x_1⊗φ_2)\times(x_2⊗ x_3)\mapsto\langleφ_2,x_2\rangle_{E_2}x_1⊗ x_3$$ by $\operatorname{tr}_{E_2}$, we may note that for any $φ_2\in E_2^\ast$, the linear extension $\iota_{φ_2}$ of $$x_1⊗ x_2⊗ x_3\to x_1⊗φ_2⊗ x_2⊗ x_3$$ is an embedding of $E_1⊗ E_2⊗ E_3$ into $(E_1⊗ E_2^\ast)⊗(E_2⊗ E_3)$ and hence we could define $\operatorname{slice}_{φ_2}:=\operatorname{tr}_{E_2}\circ\:\iota_{φ_2}$. By choosing $φ_2$ to be a basis functional, we can express $v^{(i_2)}_{i_1i_3}$ in terms of this slice operator. – 0xbadf00d May 21 at 11:40