Diagonalizability, orthogonal diagonalizability of higher order tensors and their being or not being dense in some suitable topology

Question

For our discussion, we'll assume that we're working with $\mathbb{R}^m$ only, but much or all of the following discussion should be carried over immediately to any finite dimensional inner product space.

For completeness, we'll state a few definitions, that'll be used in the questions that follow.

1) Tensors: We'll call $T$ a tensor of order $k$ (or just a $k$-tensor) on $\mathbb{R}^m$ if $T:(\mathbb{R}^m)^k\to \mathbb{R}$ is a $k$-linear map.

2) Tensor product of two $1$-linear forms on $\mathbb{R}^m:$

Let $A, B: \mathbb{R}^m\to \mathbb{R}$ be two $1$-linear forms. Then we define their tensor product $A\otimes B$ to be the bilinear form: $A\otimes B: (v,w)\mapsto A(v)B(w).$

3) Constructing a $k$-tensor from just one vector: Identifying $\mathbb{R}^m$ with its dual thanks to the canonical inner product on $\mathbb{R}^m,$ we can think of any vector $v$ as a $1$-linear form, and $v\otimes v\otimes \dots \otimes v =:v^{\otimes k}$ as a $k$-linear form (i.e. a $k$-tensor).

4) Symmetry: We define $T$ to be symmetric if $T(\sigma(v_1),\dots \sigma(v_n))=T(v_1\dots v_n)$ for every permutation $\sigma.$

5) Diagonalizability and orthogonal diagonalizability : Let $\{v_1\dots v_m\}$ be a basis for $\mathbb{R}^m.$ We say that $T$ is diagonalizable if there exist $\lambda_i\in \mathbb{R}, 1\le i \le m,$ so that $T=\sum_{i=1}^{k}\lambda_iv_i^{\otimes k}.$ Furthermore, we define $T$ to be orthogonally diagonalizable (in short, odeco, following this paper) if $v_i's$ form an orthonormal basis for $\mathbb{R}^m.$ Note that $T$ is odeco implies that there exists $R\in O(n), R:=[v_1\dots v_m],$ so that $T(x\dots x)=\sum_{i=1}^{m}\lambda_i(Rx)_i^k.$

P.S. The paper linked above, as well this paper do not define diagonalizability of a tensor, although they do orthogonally diagonalizable. I made up the former definition just by dropping the assumption that the basis need not be orthonormal. Also they used the word "orthogonally decomposable" instead of "orthogonally diagonalizable."

Here are my questions:

1. By the definition above, all diagonalizable tensors are symmetric (but this is not the case for matrices, so I'm having trouble here - did I define diagonalizability of a tensor correctly?)
2. When $k=2,$ a tensor becomes a bilinear form, that correspond to a matrix after choosing a basis. We know, thanks to the spectral theorem, that symmetry of a matrix is equivalent to orthogonal diagonalizability. However, the papers linked above claim that there that this is not so for higher order tensors. What's an example of a symmetric tensor that is not orthogonally diagonalizable (with proof), please?
3. Density(?): What's an appropriate topology (ideally a metrizable one) on the space of $k$-tensors, and in this topology, can we approximate a symmetric tensor by a sequence of orthogonally diagonalizable tensors? If no, then how about a sequence of diagonalizable tensors?

Relevant references very highly appreciated! I'll take the liberty to add a few, as it appears that tensors have been of interest in the signal processing community more than the math community.

i) https://hal.science/hal-00561523/file/ifac94.pdf

Answer 1

1. This is indeed true, and this may be the reason why the paper uses decomposable instead of diagonalizable. In the context of general tensors, it may be better to define diagonalizability as $$T = \sum_{i = 1}^n \lambda_i v_{1i} \otimes v_{2i} \otimes \dots \otimes v_{ki}$$ for some bases $v_{p1}, \dots, v_{pn}$. If the tensor is symmetric, then this notion of diagonalizability is equivalent to the symmetric version from your post (Proof for $k \geq 3$: Take a basis $u_{11}, \dots, u_{1n}$ biorthogonal to $v_{11}, \dots, v_{1n}$. Then the contraction $$T \cdot u_{1i} = v_{2i} \otimes \dots \otimes v_{ki}$$ is a rank $1$ symmetric tensor, so $v_{2i},\dots,v_{ki}$ are proportional to each other. Now do the same on any other factor to also include $v_{1i}$).

2. Consider the $W$-tensor $$W = e_0 \otimes e_0 \otimes e_1 + e_0 \otimes e_1 \otimes e_0 + e_1 \otimes e_0 \otimes e_0.$$ As in the previous point, if $W$ is diagonalizable, then there exists a basis $u_0, u_1$ such that the contraction $W \cdot u_i$ has rank $1$. But for $u = \alpha_0 e_0 + \alpha_1 e_1$ we have $$W \cdot u = \begin{bmatrix} \alpha_1 & \alpha_0 \\ \alpha_0 & 0 \end{bmatrix},$$ so the only $u$ such that $W \cdot u$ has rank $1$ is proportional to $e_1$. Contradiction. (This is a well-known proof, usually used to prove the fact that $W$ has tensor rank greater than $2$).

3. Tensor product is a finite-dimensional vector space, so there is a unique topology compatible with vector space structure, and it comes from the usual norm. The set of orthogonally diagonalizable tensors is, I believe, closed (if we allow $\lambda_i = 0$), because you can always define a limit of a family of orthonormal bases. $W$ can be approximated by diagonalizable tensors (this is usually stated as follows: border rank $\underline{R}(W) = 2$), but there are larger examples which cannot be approximated. For example, $$H = \sum_{\sigma \in S_3} e_{\sigma 1} \otimes e_{\sigma 2} \otimes e_{\sigma 3}$$ has border rank $4$, that is, not approximable by diagonalizable tensors. The tensor $H$ is a special case of a small Coppersmith-Winograd tensor used in matrix multiplication algorithms.

In general, tensors diagonalizable in the sense I defined in the beginning have tensor rank $n$, and tensors approximated by diagonalizable have border rank $n$. If the tensor is concise (nondegenerate in some sense), it always has border rank at least $n$, so concise tensors of border rank $n$ are called tensors of minimal border rank. This is a complicated class, you can look at the paper Concise tensors of minimal border rank by Jelisiejew, Landsberg and Pal, and references therein for the information on them. (UPDATE: if we are talking about real tensors, things are even more complicated. The structure tensor of $\mathbb{C}$ as an algebra over $\mathbb{R}$ has rank $2$ over $\mathbb{C}$ but not over $\mathbb{R}$)

There is also literature on orthogonal decompositions, but I'm not very familiar with it.