Skip to main content
Source Link
CrabMan
  • 171
  • 6

SVD is often used to perform tensor decompositions in Tucker and Tensor Train formats. HOSVD (Higher-order SVD) is an algorithm that approximates a given tensor (a multidimensional array of real or complex numbers) with a smaller tensor and a bunch of matrices - this is called the Tucker format. Sometimes HOSVD is used to initialize such a decomposition which is then improved using ALS (Alternating Least Squares) - the resulting algorithm that combines these two parts is called HOOI (Higher Order Orthogonal Iteration). All these three algorithms as well as Tucker format are described in Tensor Decompositions and Applications by T. G. Kolda, B. W. Bader (http://www.kolda.net/publication/koba09/). Also, these algorithms are implemented in http://tensorly.org/stable/modules/generated/tensorly.decomposition.Tucker.html. Some applications of these decompositions are given in the aforementioned paper. Another application I would like to mention is making neural networks work faster by approximating their tensors in Tucker format using these decompositions, see Automated Multi-Stage Compression of Neural Networks by Gusak et al. (https://www.semanticscholar.org/paper/Automated-Multi-Stage-Compression-of-Neural-Gusak-Kholyavchenko/b8bbda9bc5e0861a64a54057af7f6a88b49498c7).

To approximate a tensor using Tensor Train format, one typically uses a bunch of SVDs and nothing else. The format and the algorithm are described in Tensor-train decomposition by Oseledets (http://pitt.edu/~sjh95/related_papers/tensor_train_decomposition.pdf). I don't know much about its applications. I know that quantum physicists call this format Matrix Product State when it's used to represent quantum states and Matrix Product Operator when it's used to represent a Hamiltonian, more can be read about this in Hand-waving and Interpretive Dance: An Introductory Course on Tensor Networks by Jacob C. Bridgeman, Christopher T. Chubb (https://arxiv.org/abs/1603.03039).

Post Made Community Wiki by CrabMan