Decomposition of order-3 tensors over the complex numbers

Question

This is a question about decomposition of order-3 tensors. The survey Tensor Decompositions and Applications give a good account of recent developments in this area.

Let $T$ be an order-3 tensor, i.e., the number of indices for each entry is 3. For instance, an order-3 tensor can be defined as an operator $T:[n_1]\times [n_2] \times [n_3]\to \mathbb{C}$, where $[n_i]=\{1,\dots,n_i\}$. I'm particularly interested in tensors with entries defined over the complex numbers.

There are several ways to decompose a tensor. Two of the most popular are the CP decomposition and Tucker decomposition (see sections 3 and 4 in the paper above).

My questions are:

1. The paper above define its tensors with entries in $\mathbb{R}$. Does the CP and Tucker decompositions work the same for tensors with entries in $\mathbb{C}$?

2. In the paper above, page 475 about the Tucker decomposition, it reads "Most fitting algorithms assume that the factor matrices are columwise orthonormal....". Orthonormal in what sense? $\ell_1$ norm, $\ell_2$ norm? If the entries of the tensor are in $\mathbb{C}$, is it correct to assume that the Tucker decomposition always decomposes in 3 unitary matrices?

3. The same as in question 2, but with the CP decomposition. If your tensor decomposes in matrices $A,B,C$, in page 464 in the paper above, reads "It is often useful to assume that the columns of $A,B,C$ are normalized to length 1 with the weights absorbed in the vector $\lambda\in\mathbb{R}^R$ so that $T=\sum_{r=1}^R \lambda_r (a_r \otimes b_r \otimes c_r)$, where $R$ is the rank of $T$, $\otimes$ is the outer product, and $a_r,b_r,c_r$ are the $r$-th columns of $A,B,C$ respectively. Can we assume w.l.o.g. that $A,B,C$ are always unitary?2.

One important thing to note is that the rank can be different if the tensor is over $\mathbb{R}$ or $\mathbb{C}$. Also, if the number of terms in the summation of both decompositions is the rank, then the decompositions are exact.

Answer 1

1. Yes, both CP and Tucker decomposition are defined on complex spaces as well as on real. Note that a CP tensor rank (number of rank-one tensors $a_k \otimes b_k \otimes c_k$ in the decomposition) depends on the field (unlike the matrix rank). The Tucker ranks (mode ranks) do not depend on the field.
2. The Tucker factors, just like factors $U$ and $V$ of the SVD decomposition $A = U S V^*,$ can be always chosen unitary in a sense that $U^* U=I$ and $V^* V=I.$
3. The CP factors are in general non-orthogonal. A classical example is the decomposition of a Laplace-like operator $$ \def\eps{\varepsilon} A = D \otimes I \otimes I + I \otimes D \otimes I + I \otimes I \otimes D \approx \frac{(I + \eps D)^{\otimes 3} - I^{\otimes 3}}{\eps}. $$ The exact CP rank is $3,$ but it can be approximated with any precision $\eps$ by a rank-2 CP format. The factors of the CP format in the right-hand side are almost collinear, which also demonstrates the numerical instability of this approximation.