This is a question about decomposition of order-3 tensors. The survey Tensor Decompositions and Applications give a good account of recent developments in this area.

Let $T$ be an order-3 tensor, i.e., the number of indices for each entry is 3. For instance, an order-3 tensor can be defined as an operator $T:[n_1]\times [n_2] \times [n_3]\to \mathbb{C}$, where $[n_i]=\{1,\dots,n_i\}$. I'm particularly interested in tensors with entries defined over the complex numbers.

There are several ways to decompose a tensor. Two of the most popular are the CP decomposition and Tucker decomposition (see sections 3 and 4 in the paper above).

My questions are:

  1. The paper above define its tensors with entries in $\mathbb{R}$. Does the CP and Tucker decompositions work the same for tensors with entries in $\mathbb{C}$?

  2. In the paper above, page 475 about the Tucker decomposition, it reads "Most fitting algorithms assume that the factor matrices are columwise orthonormal....". Orthonormal in what sense? $\ell_1$ norm, $\ell_2$ norm? If the entries of the tensor are in $\mathbb{C}$, is it correct to assume that the Tucker decomposition always decomposes in 3 unitary matrices?

  3. The same as in question 2, but with the CP decomposition. If your tensor decomposes in matrices $A,B,C$, in page 464 in the paper above, reads "It is often useful to assume that the columns of $A,B,C$ are normalized to length 1 with the weights absorbed in the vector $\lambda\in\mathbb{R}^R$ so that $T=\sum_{r=1}^R \lambda_r (a_r \otimes b_r \otimes c_r)$, where $R$ is the rank of $T$, $\otimes$ is the outer product, and $a_r,b_r,c_r$ are the $r$-th columns of $A,B,C$ respectively. Can we assume w.l.o.g. that $A,B,C$ are always unitary?2.

One important thing to note is that the rank can be different if the tensor is over $\mathbb{R}$ or $\mathbb{C}$. Also, if the number of terms in the summation of both decompositions is the rank, then the decompositions are exact.

  • 1
    $\begingroup$ 1. I don't see why the decompositions wouldn't work if your tensor had complex entries. 2. Orthonormal must mean using $\ell_2$-norm 3. The columns are normalized to have $\ell_2$-norm equal to 1---why should that allow you have unitary matrices? $\endgroup$
    – Suvrit
    Oct 8, 2011 at 13:08
  • $\begingroup$ @suvrit, regarding 3, that's exactly my question. $\endgroup$ Oct 8, 2011 at 13:22
  • $\begingroup$ I think you cannot do the wlog; having column norms smaller than 1, or with some additional scaling, you can assume each of $A$, $B$, and $C$ to be contractions, and then write them as "sums of unitary" matrices, not just consider only unitary matrices instead. $\endgroup$
    – Suvrit
    Oct 8, 2011 at 13:28
  • $\begingroup$ I see, the "w.l.o.g." is the important part for me right now. $\endgroup$ Oct 8, 2011 at 13:43

1 Answer 1

  1. Yes, both CP and Tucker decomposition are defined on complex spaces as well as on real. Note that a CP tensor rank (number of rank-one tensors $a_k \otimes b_k \otimes c_k$ in the decomposition) depends on the field (unlike the matrix rank). The Tucker ranks (mode ranks) do not depend on the field.
  2. The Tucker factors, just like factors $U$ and $V$ of the SVD decomposition $A = U S V^*,$ can be always chosen unitary in a sense that $U^* U=I$ and $V^* V=I.$
  3. The CP factors are in general non-orthogonal. A classical example is the decomposition of a Laplace-like operator $$ \def\eps{\varepsilon} A = D \otimes I \otimes I + I \otimes D \otimes I + I \otimes I \otimes D \approx \frac{(I + \eps D)^{\otimes 3} - I^{\otimes 3}}{\eps}. $$ The exact CP rank is $3,$ but it can be approximated with any precision $\eps$ by a rank-2 CP format. The factors of the CP format in the right-hand side are almost collinear, which also demonstrates the numerical instability of this approximation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.